[PATCH] D27689: Module: hash the pcm content and use it as SIGNATURE for implicit modules.

[Manman Ren via Phabricator via cfe-commits]

manmanren created this revision.
manmanren added reviewers: rsmith, benlangmuir, aprantl.
manmanren added subscribers: bruno, cfe-commits, dexonsmith.

We change ASTFileSignature from a random 32-bit number to the actual SHA hash of the pcm content.

1> Definition of ASTFileSignature is moved to Basic/Module.h so Module and ASTSourceDescriptor can use it. Before this patch both are using uint64_t as the type of the signature.
2> Add DIAGNOSTIC_OPTIONS_BLOCK and make sure it is the last block of the pcm file. This block contains DIAGNOSTIC_OPTIONS and SIGNATURE.
We are using this patch to solve a PCH+Modules issue: PCH are handled by the build system and modules are handled by the compiler, when the compiler overwrites a pcm file with the same content except the diagnostic differences, we can still treat the PCH as valid and continue using it. This requires us not hashing the diagnostic differences. Moving the diagnostic differences to the end of the pcm makes the task easy:

  In a few records, we save bit positions in the pcm file, if the size of the diagnostic options is different, the bit positions can be off.
  We only need to hash the content till the start of the DIAGNOSTIC_OPTIONS_BLOCK.

3> When we hash the pcm content, there is no point in saving the file size and modification time for the imported module file. The patch will use 0 for both. And the validation will not check the size differences or the modification time differences.
4> In GlobalModuleIndexBuilder, when the signature is non-zero, we check the signature instead of file size/modification time for imported modules.
5> Debug info has a 64-bit slot for dwo id, this patch uses the lower 64 bits of the SHA hash.

I want to gather some feedback on the general direction of this. Given that this is a big patch, I am open to separate it into smaller patches.

Cheers,
Manman


https://reviews.llvm.org/D27689

Files:
  include/clang/AST/ExternalASTSource.h
  include/clang/Basic/Module.h
  include/clang/Frontend/PCHContainerOperations.h
  include/clang/Serialization/ASTBitCodes.h
  include/clang/Serialization/ASTReader.h
  include/clang/Serialization/ASTWriter.h
  include/clang/Serialization/Module.h
  lib/Basic/Module.cpp
  lib/CodeGen/CGDebugInfo.cpp
  lib/CodeGen/ObjectFilePCHContainerOperations.cpp
  lib/Frontend/ASTUnit.cpp
  lib/Serialization/ASTReader.cpp
  lib/Serialization/ASTWriter.cpp
  lib/Serialization/GeneratePCH.cpp
  lib/Serialization/GlobalModuleIndex.cpp
  lib/Serialization/Module.cpp
  lib/Serialization/ModuleManager.cpp
  test/Modules/diagnostic-options-out-of-date.m
  test/Modules/explicit-build.cpp
  test/Modules/module_file_info.m
  test/Modules/rebuild.m

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D27689.81145.patch
Type: text/x-patch
Size: 42527 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20161212/a6323a14/attachment-0001.bin>