[PATCH] D39050: Add index-while-building support to Clang

[Nathan Hawes via Phabricator via cfe-commits]

nathawes created this revision.
Herald added a subscriber: mgorny.

Adds a new -index-store-path option that causes Clang to additionally collect and output source code indexing information to the supplied path.

This is done by wrapping the FrontendAction otherwise setup by the invocation with a WrappingIndexRecordAction. This action simply delegates to the wrapped action, but additionally multiplexes in its own IndexASTConsumer to collect symbol information from the AST and tracks the source file and module dependencies of the translation unit (via the IndexDependencyProvider class). When the action completes, it then writes this information out to the supplied index store path in the form of a unit file, which stores the dependency information of the translation unit, and record files, that store the symbol and symbol occurrences seen in each source file. These are written out in the LLVM Bitstream format.

This patch also adds a new IndexStore library under tools/ that abstracts away the structure and format of the index store so that clients (e.g. an IDE indexer service) can easily read the data it contains. It also notifies clients whenever data in the store is updated.

For a better (and more detailed) description of these changes, see the design document at: https://docs.google.com/document/d/1cH2sTpgSnJZCkZtJl1aY-rzy4uGPcrI-6RrUpdATO2Q/edit?usp=sharing

and the mailing list discussion 'RFC: Adding index-while-building support to Clang'


https://reviews.llvm.org/D39050

Files:
  include/clang/Basic/DiagnosticFrontendKinds.td
  include/clang/Basic/DiagnosticGroups.td
  include/clang/DirectoryWatcher/DirectoryWatcher.h
  include/clang/Driver/Job.h
  include/clang/Driver/Options.td
  include/clang/Frontend/CompilerInstance.h
  include/clang/Frontend/FrontendOptions.h
  include/clang/Index/IndexDataStore.h
  include/clang/Index/IndexDataStoreSymbolUtils.h
  include/clang/Index/IndexRecordReader.h
  include/clang/Index/IndexRecordWriter.h
  include/clang/Index/IndexUnitReader.h
  include/clang/Index/IndexUnitWriter.h
  include/clang/Index/IndexingAction.h
  include/indexstore/IndexStoreCXX.h
  include/indexstore/indexstore.h
  lib/CMakeLists.txt
  lib/DirectoryWatcher/CMakeLists.txt
  lib/DirectoryWatcher/DirectoryWatcher.cpp
  lib/Driver/Driver.cpp
  lib/Driver/Job.cpp
  lib/Driver/ToolChains/Clang.cpp
  lib/Driver/ToolChains/Darwin.cpp
  lib/Frontend/CompilerInstance.cpp
  lib/Frontend/CompilerInvocation.cpp
  lib/FrontendTool/CMakeLists.txt
  lib/FrontendTool/ExecuteCompilerInvocation.cpp
  lib/Index/BitstreamVisitor.h
  lib/Index/CMakeLists.txt
  lib/Index/ClangIndexRecordWriter.cpp
  lib/Index/ClangIndexRecordWriter.h
  lib/Index/FileIndexRecord.cpp
  lib/Index/FileIndexRecord.h
  lib/Index/IndexDataStore.cpp
  lib/Index/IndexDataStoreUtils.cpp
  lib/Index/IndexDataStoreUtils.h
  lib/Index/IndexRecordHasher.cpp
  lib/Index/IndexRecordHasher.h
  lib/Index/IndexRecordReader.cpp
  lib/Index/IndexRecordWriter.cpp
  lib/Index/IndexUnitReader.cpp
  lib/Index/IndexUnitWriter.cpp
  lib/Index/IndexingAction.cpp
  lib/Index/IndexingContext.cpp
  lib/Index/IndexingContext.h
  test/Index/Store/Inputs/head.h
  test/Index/Store/Inputs/json.c.json
  test/Index/Store/Inputs/module/ModDep.h
  test/Index/Store/Inputs/module/ModSystem.h
  test/Index/Store/Inputs/module/ModTop.h
  test/Index/Store/Inputs/module/ModTopSub1.h
  test/Index/Store/Inputs/module/ModTopSub2.h
  test/Index/Store/Inputs/module/module.modulemap
  test/Index/Store/Inputs/overlay.yaml
  test/Index/Store/Inputs/print-unit.h
  test/Index/Store/Inputs/sys/another.h
  test/Index/Store/Inputs/sys/syshead.h
  test/Index/Store/Inputs/test1.c
  test/Index/Store/Inputs/test2.c
  test/Index/Store/Inputs/test3.cpp
  test/Index/Store/Inputs/using-overlay.h
  test/Index/Store/assembly-invocation.c
  test/Index/Store/external-source-symbol-hash.m
  test/Index/Store/handle-prebuilt-module.m
  test/Index/Store/json-with-module.m
  test/Index/Store/json-with-module.m.json
  test/Index/Store/json-with-pch.c
  test/Index/Store/json-with-pch.c.json
  test/Index/Store/json.c
  test/Index/Store/print-record.mm
  test/Index/Store/print-unit.c
  test/Index/Store/print-units-with-modules.m
  test/Index/Store/print-units-with-pch.c
  test/Index/Store/record-hash-crash-invalid-name.cpp
  test/Index/Store/record-hash-crash.cpp
  test/Index/Store/record-hash-using.cpp
  test/Index/Store/record-hash.cpp
  test/Index/Store/relative-out-path.c
  test/Index/Store/syntax-only.c
  test/Index/Store/unit-with-vfs.c
  test/Index/Store/unit-workdir-prefix.c
  test/Index/Store/using-libstdcpp-arc.mm
  tools/CMakeLists.txt
  tools/IndexStore/CMakeLists.txt
  tools/IndexStore/IndexStore.cpp
  tools/IndexStore/IndexStore.exports
  tools/c-index-test/CMakeLists.txt
  tools/c-index-test/JSONAggregation.cpp
  tools/c-index-test/JSONAggregation.h
  tools/c-index-test/core_main.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D39050.118854.patch
Type: text/x-patch
Size: 339546 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20171018/0cce4395/attachment-0001.bin>