{"id":8435,"date":"2015-04-09T19:06:02","date_gmt":"2015-04-09T18:06:02","guid":{"rendered":"https:\/\/wp.okra.host\/article\/understanding-storage-usage\/"},"modified":"2021-03-07T14:11:04","modified_gmt":"2021-03-07T13:11:04","slug":"understanding-storage-usage","status":"publish","type":"ht_kb","link":"https:\/\/kb.okra.host\/de\/article\/understanding-storage-usage\/","title":{"rendered":"Understanding storage usage"},"content":{"rendered":"<h2 id=\"overview\" >\u00dcbersicht<\/h2>\n<p>The <a title=\"Anmeldung am Bedienfeld\" href=\"https:\/\/kb.okra.host\/de\/control-panel\/logging-into-the-control-panel\/\">Bedienfeld<\/a> provides a comprehensive storage breakdown via <strong>Dateien<\/strong> &gt; <strong>Storage Usage<\/strong>. A birds eye view is available in graphical form under <em>Storage Usage\u00a0<\/em>as well as individual files via\u00a0<em>File Listing<\/em> &gt;\u00a0<em>Download File Listing<\/em>.<\/p>\n<div id=\"attachment_947\" style=\"width: 310px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/kb.okra.host\/wp-content\/uploads\/2015\/04\/sample-storage-usage.png\" rel=\"lightbox-0\"><img decoding=\"async\" aria-describedby=\"caption-attachment-947\" loading=\"lazy\" class=\"size-medium wp-image-947\" src=\"https:\/\/kb.okra.host\/wp-content\/uploads\/2015\/04\/sample-storage-usage-300x144.png\" alt=\"Sample storage usage overview\" width=\"300\" height=\"144\" \/><\/a><\/p>\n<p id=\"caption-attachment-947\" class=\"wp-caption-text\">Sample storage usage overview<\/p>\n<\/div>\n<h2 id=\"storage-contributors\" >Storage contributors<\/h2>\n<p>Any file created on your account contributes to storage utilization. This includes the following services:<\/p>\n<ul>\n<li>MySQL<\/li>\n<li>PostgreSQL<\/li>\n<li>FTP<\/li>\n<li>E-Mail<\/li>\n<li>Web server logs<\/li>\n<li>Files uploaded to your web site<\/li>\n<li>Files uploaded within the control panel<\/li>\n<\/ul>\n<p><strong><span style=\"color: #ff0000\">Special note:\u00a0<\/span><\/strong><span style=\"color: #ff0000\"><span style=\"color: #000000\">web <\/span><\/span><span style=\"color: #ff0000\"><span style=\"color: #000000\">server logs cannot be adjusted nor removed. These are used in circumstances where an account is hacked\/compromised. At the minimum, five days of log files must be retained to create an audit trail. These access logs are represented as <code>\/var\/log\/httpd\/access_log{,.1.gz,.2.gz,.3.gz,.4.gz}<\/code> in abbreviated form. <code>error_log<\/code> is the error log component and\u00a0<em>can<\/em> be removed on special request in egregious situations, per ticket, but facilitate recreating an audit trail. Errors generated by random attacks and PHP bugs are logged in this file and provide us with additional insight in postmortem investigations.<\/span><\/span><\/p>\n<h2 id=\"storage-allocation\" >Storage allocation<\/h2>\n<p>Each file created on the server is rounded up to the nearest 4 KB unit; this is the storage occupied on disk. For example, a 5 byte file would contribute towards 4 KB of storage usage. A 4097 byte file (4096 bytes = 4 KB) would therefore\u00a0be counted as 8 KB, and so on. This rounding mechanism is dependent upon the storage\u00a0<a href=\"http:\/\/en.wikipedia.org\/wiki\/Block_%28data_storage%29\">block size<\/a>; on our servers &#8211; <em>and most servers<\/em> &#8211; it is 4 KB.<\/p>\n<p>Even though an account may have 10,000 small files, the storage usage may appear inflated because of this behavior. For example, with block storage, 10,000 small files &#8211; each 1 KB in size &#8211; would occupy 40,000 KB in storage (~40 MB). E-mail, as a consequence, can have a profound impact on actual storage usage vs perceived storage usage.<\/p>\n<h2 id=\"apache-usage\" >&#8220;apache&#8221; usage<\/h2>\n<p>All PHP scripts run as a separate user, <a title=\"Schreiben in Dateien\" href=\"https:\/\/kb.okra.host\/de\/php\/writing-to-files\/\">for security<\/a>, that is distinct to your account. This user is a reserved system user called, &#8220;apache&#8221; (named after the <a href=\"http:\/\/httpd.apache.org\/\">web server<\/a>). Apache usage includes any file uploaded from a PHP application (WordPress, Drupal, Joomla, etc).<\/p>\n<h3 id=\"user-separation-benefits\" >User separation benefits<\/h3>\n<p>Although the separate user may seem counter-intuitive, there are three enormous\u00a0benefits by separating PHP from the account:<\/p>\n<ol>\n<li>We can quickly\u00a0track down malicious code uploaded as a consequence of running outdated PHP applications (owner is always &#8220;apache&#8221;)<\/li>\n<li>You can quickly track what storage is contributed by PHP uploads<\/li>\n<li>In the event of a break-in, the attacker cannot access your control panel, e-mail, SSH private keys, SSL certificates, and other confidential data since it operates on a different user<\/li>\n<\/ol>\n<h2 id=\"detailed-storage-usage\" >Detailed storage usage<\/h2>\n<p>A detailed listing of all files counted towards storage is available via\u00a0<strong>File Listing<\/strong> &gt; <b>Download File Listing<\/b>. Storage is presented in a CSV file that can be easily imported into a spreadsheet.<\/p>\n<p><span style=\"color: #000080\"><strong>Hinweis:<\/strong> depending upon account size and file count, it may take several minutes to generate this list and the list may be rather large.\u00a0<\/span><\/p>\n<p>Four fields are provided:<\/p>\n<ol>\n<li>quota: this is the amount, in kilobytes rounded to the nearest 4 KB unit (<em>4096 bytes<\/em>), consistent with quota utilization<\/li>\n<li>szdisk: size on disk, consistent with the actual size, in bytes, of the file on disk.\u00a0<em>quota<\/em> is equal to <b>\u2308<\/b><em>szdisk<\/em>\/1024<b>\u2309 + <\/b>4 &#8211; \u2308<em>szdisk<\/em>\/1024\u2309 mod 4<\/li>\n<li>username: user to which this is storage is credited<\/li>\n<li>path: location on the server where this file can be found<\/li>\n<\/ol>\n<h3 id=\"special-paths\" >Special paths<\/h3>\n<p>Certain files should <span style=\"color: #ff0000\"><strong>never be removed<\/strong><\/span> directly from the filesystem. These files include:<\/p>\n<ul>\n<li><code>\/var\/lib\/mysql<\/code>: database files used by MySQL. Removal of these files, directly, will result in localized database corruption. Use <strong>Databases<\/strong> &gt; <strong>MySQL Manager<\/strong> to drop the offending database.<\/li>\n<li><code>\/var\/lib\/pgsql<\/code>: database files used by PostgreSQL. Removal of these files, directly, will result in <strong>massive<\/strong> database corruption (all databases on account -&gt; *poof*). Use <strong>Databases<\/strong> &gt; <strong>PostgreSQL Manager<\/strong> to drop these databases.<\/li>\n<li><code>\/var\/log\/httpd<\/code>: audit trail HTTP request logs. These are the only logs counted towards your storage (excludes mail + FTP). These are used to create audit trails if your account is compromised.\u00a0Removal of these logs, in conjunction with a hacked account, will result in account termination &#8211; <em>don&#8217;t do it<\/em>!<\/li>\n<\/ul>\n<h3 id=\"removing-files\" >Removing files<\/h3>\n<p>Files may be removed by the owner (<em>username<\/em> field in &#8220;detailed storage user&#8221;). When removing a file via <a title=\"Accessing FTP server\" href=\"https:\/\/kb.okra.host\/de\/ftp\/accessing-ftp-server\/\">FTP<\/a> oder <a title=\"Zugriff auf das Terminal\" href=\"https:\/\/kb.okra.host\/de\/terminal\/accessing-terminal\/\">Terminal<\/a>, if the username doesn&#8217;t match your login, then access is denied. This can be overcome by either <a title=\"Benutzer umschalten\" href=\"https:\/\/kb.okra.host\/de\/terminal\/switching-users\/\">switching users<\/a> (available on <a title=\"Ermitteln der Plattformversion\" href=\"https:\/\/kb.okra.host\/de\/platform\/determining-platform-version\/\">v6+<\/a> platforms) or by using the control panel file manager (<strong>Dateien<\/strong> &gt; <strong>Datei-Manager<\/strong>).<\/p>\n<h3 id=\"incongruities-between-reported-and-used\" >Incongruities between reported and used<\/h3>\n<p>Although uncommon, there have been incidents in the past decade where the storage represented in the control panel doesn&#8217;t corroborate with the storage listed within the storage usage dump. This is caused by\u00a0accounting errors (<em>no book cooking<\/em>, promise!) and generally impacts more than 1 user on a server. Open up a ticket if you feel the storage represented by a user, other than apache, is off. In most cases, the actual storage consumed by a user is grossly off by at least an order of magnitude.<\/p>","protected":false},"excerpt":{"rendered":"<p>Overview The control panel provides a comprehensive storage breakdown via Files &gt; Storage Usage. A birds eye view is available in graphical form under Storage Usage\u00a0as well as individual files via\u00a0File Listing &gt;\u00a0Download File Listing. Sample storage usage overview Storage contributors Any file created on your account contributes to storage&#8230;<\/p>","protected":false},"author":1,"comment_status":"open","ping_status":"closed","template":"","format":"standard","meta":{"footnotes":""},"ht-kb-category":[56],"ht-kb-tag":[],"class_list":["post-8435","ht_kb","type-ht_kb","status-publish","format-standard","has-post-thumbnail","hentry","ht_kb_category-control-panel"],"_links":{"self":[{"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/ht-kb\/8435","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/ht-kb"}],"about":[{"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/types\/ht_kb"}],"author":[{"embeddable":true,"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/comments?post=8435"}],"version-history":[{"count":1,"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/ht-kb\/8435\/revisions"}],"predecessor-version":[{"id":8436,"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/ht-kb\/8435\/revisions\/8436"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/media\/8437"}],"wp:attachment":[{"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/media?parent=8435"}],"wp:term":[{"taxonomy":"ht_kb_category","embeddable":true,"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/ht-kb-category?post=8435"},{"taxonomy":"ht_kb_tag","embeddable":true,"href":"https:\/\/kb.okra.host\/de\/wp-json\/wp\/v2\/ht-kb-tag?post=8435"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}