[all-commits] [llvm/llvm-project] a033db: [Clang] Give Clang the ability to use a shared sta...

[fredriss via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: a033dbbe5c43247b60869b008e67ed86ed230eaa
      https://github.com/llvm/llvm-project/commit/a033dbbe5c43247b60869b008e67ed86ed230eaa
  Author: Fred Riss <friss at apple.com>
  Date:   2023-01-18 (Wed, 18 Jan 2023)

  Changed paths:
    M clang/include/clang/Basic/DiagnosticFrontendKinds.td
    M clang/include/clang/Driver/Options.td
    M clang/include/clang/Frontend/CompilerInvocation.h
    M clang/include/clang/Lex/HeaderSearchOptions.h
    M clang/lib/Frontend/ASTUnit.cpp
    M clang/lib/Frontend/CompilerInvocation.cpp
    M clang/test/CMakeLists.txt
    A clang/test/Driver/vfsstatcache.c
    A clang/test/clang-stat-cache/cache-effects.c
    A clang/test/clang-stat-cache/errors.test
    M clang/tools/CMakeLists.txt
    A clang/tools/clang-stat-cache/CMakeLists.txt
    A clang/tools/clang-stat-cache/clang-stat-cache.cpp
    A llvm/include/llvm/Support/StatCacheFileSystem.h
    M llvm/lib/Support/CMakeLists.txt
    A llvm/lib/Support/StatCacheFileSystem.cpp
    M llvm/unittests/Support/VirtualFileSystemTest.cpp

  Log Message:
  -----------
  [Clang] Give Clang the ability to use a shared stat cache

Every Clang instance uses an internal FileSystemStatCache to avoid
stating the same content multiple times. However, different instances
of Clang will contend for filesystem access for their initial stats
during HeaderSearch or module validation.

On some workloads, the time spent in the kernel in these concurrent
stat calls has been measured to be over 20% of the overall compilation
time. This is extremly wassteful when most of the stat calls target
mostly immutable content like a SDK.

This commit introduces a new tool `clang-stat-cache` able to generate
an OnDiskHashmap containing the stat data for a given filesystem
hierarchy.

The driver part of this has been modeled after -ivfsoverlay given
the similarities with what it influences. It introduces a new
-ivfsstatcache driver option to instruct Clang to use a stat cache
generated by `clang-stat-cache`. These stat caches are inserted at
the bottom of the VFS stack (right above the real filesystem).

Differential Revision: https://reviews.llvm.org/D136651