[PATCH] D102488: [clang][deps] Prune unused header search paths

[Jan Svoboda via Phabricator via cfe-commits]

jansvoboda11 created this revision.
jansvoboda11 added reviewers: Bigcheese, dexonsmith.
jansvoboda11 requested review of this revision.
Herald added a project: clang.
Herald added a subscriber: cfe-commits.

To reduce the number of explicit builds of a single module, we can try to squash multiple occurances of the module with different command-lines (and context hashes) by removing benign command-line options. The greatest contributors to benign differences between command-lines are the header search paths.

In this patch, the lookup cache in `HeaderSearch` is used to determine which paths were actually used when implicitly building the module during scanning. This information is serialized into the unhashed control block of the implicitly-built PCM. The dependency scanner then loads this and may use it to prune the header search paths before computing the context hash of the module and generating the command-line.

We could also prune the header search paths when serializing `HeaderSearchOptions` into the PCM. That way, we could do it only once instead of every load of the PCM file by dependency scanner. However, that would result in a PCM file whose contents don't produce the same context hash, which is probably highly surprising.

There is an alternative approach to storing extra information into the PCM: wire up preprocessor callbacks to capture the used header search paths on-the-fly during preprocessing of modularized headers (similar to what we currently do for the main source file and textual headers). Right now, that's not compatible with the fact that we do an actual implicit build producing PCM files during dependency scanning. The second run of dependency scanner loads the PCM from the first run, skipping the preprocessing altogether, which would result in different results between runs. We can revisit this approach when we


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D102488

Files:
  clang/include/clang/Lex/HeaderSearch.h
  clang/include/clang/Serialization/ASTBitCodes.h
  clang/include/clang/Serialization/ModuleFile.h
  clang/include/clang/Tooling/DependencyScanning/DependencyScanningService.h
  clang/include/clang/Tooling/DependencyScanning/DependencyScanningWorker.h
  clang/include/clang/Tooling/DependencyScanning/ModuleDepCollector.h
  clang/lib/Serialization/ASTReader.cpp
  clang/lib/Serialization/ASTWriter.cpp
  clang/lib/Tooling/DependencyScanning/DependencyScanningService.cpp
  clang/lib/Tooling/DependencyScanning/DependencyScanningWorker.cpp
  clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
  clang/test/ClangScanDeps/Inputs/header-search-pruning/a/a.h
  clang/test/ClangScanDeps/Inputs/header-search-pruning/b/b.h
  clang/test/ClangScanDeps/Inputs/header-search-pruning/cdb.json
  clang/test/ClangScanDeps/Inputs/header-search-pruning/mod.h
  clang/test/ClangScanDeps/Inputs/header-search-pruning/module.modulemap
  clang/test/ClangScanDeps/header-search-pruning.cpp
  clang/test/ClangScanDeps/modules-full.cpp
  clang/tools/clang-scan-deps/ClangScanDeps.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D102488.345409.patch
Type: text/x-patch
Size: 21815 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20210514/fb58af0b/attachment-0001.bin>