[all-commits] [llvm/llvm-project] 44ef5f: [clang][Serialization] Record SLoc dedup metadata ...

[Yuxuan Chen via All-commits]

  Branch: refs/heads/users/yuxuanchen1997/modules-sloc-dedup
  Home:   https://github.com/llvm/llvm-project
  Commit: 44ef5f54baab354d02f63f68d760723bb93e67b0
      https://github.com/llvm/llvm-project/commit/44ef5f54baab354d02f63f68d760723bb93e67b0
  Author: Yuxuan Chen <i at yuxuan.ch>
  Date:   2026-07-08 (Wed, 08 Jul 2026)

  Changed paths:
    M clang/include/clang/Lex/HeaderSearchOptions.h
    M clang/include/clang/Options/Options.td
    M clang/include/clang/Serialization/ASTBitCodes.h
    M clang/include/clang/Serialization/ModuleFile.h
    M clang/lib/Serialization/ASTReader.cpp
    M clang/lib/Serialization/ASTWriter.cpp
    A clang/test/Modules/cxx20-hu-sloc-dedup-info.cpp

  Log Message:
  -----------
  [clang][Serialization] Record SLoc dedup metadata in PCMs

Clang's loaded SourceLocation address space is finite: loaded AST files
reserve offsets from SourceManager's 32-bit loaded region. C++20 header
units make this easier to exhaust because independently-built PCMs can all
serialize SLocEntries for the same large textual include, such as a standard
library header.

This change adds the serialization-side metadata needed to recognize those
repeated file entries later. When the new cc1 flag is enabled, header-unit
PCMs now write an eager SLOC_ENTRY_DEDUP_INFO record parallel to
SOURCE_LOCATION_OFFSETS. Each serialized SLocEntry gets fixed-width metadata:
its local SLoc offset and length, an xxh3-128 content hash for dedupable file
entries, flags describing file properties that affect safe sharing, and the
input-file ID. Entries that are not currently dedupable keep zero hashes.

The record is intentionally module-kind agnostic, but emission is limited to
C++20 header units for the initial rollout. Readers cache the blob at module
load time without changing allocation behavior yet. The PCM major version is
bumped because this adds a new serialized AST record layout.


  Commit: 9cc537b921caa1c5f6fcac35f50a21e9fb002e88
      https://github.com/llvm/llvm-project/commit/9cc537b921caa1c5f6fcac35f50a21e9fb002e88
  Author: Yuxuan Chen <i at yuxuan.ch>
  Date:   2026-07-08 (Wed, 08 Jul 2026)

  Changed paths:
    M clang/include/clang/Serialization/ASTReader.h
    M clang/include/clang/Serialization/ModuleFile.h
    M clang/lib/Serialization/ASTReader.cpp
    A clang/test/Modules/cxx20-hu-sloc-dedup.cpp

  Log Message:
  -----------
  [clang][Serialization] Deduplicate loaded header-unit SLoc ranges

Header units that textually include the same large header used to consume
loaded SourceLocation address space independently, even when the serialized
file contents were identical. Importing many such header units could therefore
run out of the loaded SLoc region long before the translation unit was large in
any meaningful source sense.

Use the eager SLoc dedup metadata to share already-loaded file ranges when a
later header-unit PCM contains a matching dedupable file entry. The reader now
builds a per-module remapping from serialized local offsets to the global
SourceManager offsets actually used for the import. Fresh entries are packed
into the module's allocation, while duplicate file entries map back to the
canonical FileID and offset range from the first loaded PCM.

This changes the module-loading model from a single base-offset translation
for every loaded location to a segmented translation for PCMs that participate
in deduplication. Modules without dedup metadata keep the existing single-add
fast path. The SourceManager loaded-entry table only receives slots for entries
that are actually allocated, and serialized local FileIDs are remapped to their
canonical loaded FileIDs when a file range is shared.

The result is that repeated textual headers in header units no longer multiply
loaded SLoc address-space usage when the flag is enabled, while the default path
and PCMs without the new metadata continue to load as before.


Compare: https://github.com/llvm/llvm-project/compare/44ef5f54baab%5E...9cc537b921ca

To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications