[all-commits] [llvm/llvm-project] b9d78b: [clang][modules] Use relative offsets for input files

[Jan Svoboda via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: b9d78bdc730b2fcfe029a7579c24020536c3fa25
      https://github.com/llvm/llvm-project/commit/b9d78bdc730b2fcfe029a7579c24020536c3fa25
  Author: Jan Svoboda <jan_svoboda at apple.com>
  Date:   2023-08-24 (Thu, 24 Aug 2023)

  Changed paths:
    M clang/include/clang/Serialization/ModuleFile.h
    M clang/lib/Serialization/ASTReader.cpp
    M clang/lib/Serialization/ASTWriter.cpp

  Log Message:
  -----------
  [clang][modules] Use relative offsets for input files

This patch replaces absolute offsets into the input files block with offsets relative to the block start. This makes the whole section "relocatable". I confirmed all other uses of `GetCurrentBitNo()` are turned into relative offsets before being serialized into the AST file.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D158572


  Commit: 7d1565727dad3acb54fe76a908630843835d7bc8
      https://github.com/llvm/llvm-project/commit/7d1565727dad3acb54fe76a908630843835d7bc8
  Author: Jan Svoboda <jan_svoboda at apple.com>
  Date:   2023-08-24 (Thu, 24 Aug 2023)

  Changed paths:
    M clang/include/clang/Basic/Module.h
    M clang/include/clang/Serialization/ASTBitCodes.h
    M clang/include/clang/Serialization/ASTWriter.h
    M clang/lib/Serialization/ASTReader.cpp
    M clang/lib/Serialization/ASTWriter.cpp
    M clang/lib/Serialization/GlobalModuleIndex.cpp
    M clang/test/Modules/ASTSignature.c

  Log Message:
  -----------
  [clang][modules] Move `UNHASHED_CONTROL_BLOCK` up in the AST file

When loading (transitively) imported AST file, `ModuleManager::addModule()` first checks it has the expected signature via `readASTFileSignature()`. The signature is part of `UNHASHED_CONTROL_BLOCK`, which is placed at the end of the AST file. This means that just to verify signature of an AST file, we need to skip over all top-level blocks, paging in the whole AST file from disk. This is pretty slow.

This patch moves `UNHASHED_CONTROL_BLOCK` to the start of the AST file, so that it can be read more efficiently. To achieve this, we use dummy signature when first emitting the unhashed control block, and then backpatch the real signature at the end of the serialization process.

This speeds up dependency scanning by over 9% and significantly reduces run-to-run variability of my benchmarks.

Depends on D158572.

Reviewed By: benlangmuir

Differential Revision: https://reviews.llvm.org/D158573


Compare: https://github.com/llvm/llvm-project/compare/41db54380763...7d1565727dad