[llvm] r329966 - [CachePruning] Clarify the per-directory entry limit on Linux ext4.

[Fangrui Song via llvm-commits]

Author: maskray
Date: Thu Apr 12 15:27:38 2018
New Revision: 329966

URL: http://llvm.org/viewvc/llvm-project?rev=329966&view=rev
Log:
[CachePruning] Clarify the per-directory entry limit on Linux ext4.

Summary:
508   root node entries (root_limit)
510   internal node entries (node_limit)

For a filename with 40 bytes, its sizeof(ext4_dir_entry_2) = 48, a linear directory can contain at most floor(4096/48)=85 of them.
The real per-directory entry limit should be 508*510*85 = 22021800
The limit varies with the average length of filenames.

However, the Linux ext4 code does not try rebalancing the htree, so we will not be able to create filenames in a full leaf node. This is demonstrated with the following example, certain filenames cannot be used while others can:

  % touch d/0000000000000000000000000000000000816a6f
  touch: cannot touch 'd/0000000000000000000000000000000000816a6f': No
  space left on device
  % touch d/0000000000000000000000000000000000816a70
  # succeeded

Reviewers: pcc

Subscribers: llvm-commits

Differential Revision: https://reviews.llvm.org/D45546

Modified:
    llvm/trunk/include/llvm/Support/CachePruning.h

Modified: llvm/trunk/include/llvm/Support/CachePruning.h
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Support/CachePruning.h?rev=329966&r1=329965&r2=329966&view=diff
==============================================================================
--- llvm/trunk/include/llvm/Support/CachePruning.h (original)
+++ llvm/trunk/include/llvm/Support/CachePruning.h Thu Apr 12 15:27:38 2018
@@ -52,8 +52,11 @@ struct CachePruningPolicy {
   /// the number of files based pruning.
   ///
   /// This defaults to 1000000 because with that many files there are
-  /// diminishing returns on the effectiveness of the cache, and file
-  /// systems have a limit on total number of files.
+  /// diminishing returns on the effectiveness of the cache. Some systems have a
+  /// limit on total number of files, and some also limit the number of files
+  /// per directory, such as Linux ext4, with the default setting (block size is
+  /// 4096 and large_dir disabled), there is a per-directory entry limit of
+  /// 508*510*floor(4096/(40+8))~=20M for average filename length of 40.
   uint64_t MaxSizeFiles = 1000000;
 };