[all-commits] [llvm/llvm-project] a7afb2: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: a7afb211dc460bd4cfb2542ad1f9b05876b57ba1
      https://github.com/llvm/llvm-project/commit/a7afb211dc460bd4cfb2542ad1f9b05876b57ba1
  Author: Dmitry Polukhin <dmitry.polukhin at gmail.com>
  Date:   2020-04-17 (Fri, 17 Apr 2020)

  Changed paths:
    M clang/include/clang/Serialization/ASTBitCodes.h
    M clang/include/clang/Serialization/ASTReader.h
    M clang/include/clang/Serialization/ASTWriter.h
    M clang/include/clang/Serialization/ModuleFile.h
    M clang/lib/Serialization/ASTReader.cpp
    M clang/lib/Serialization/ASTReaderDecl.cpp
    M clang/lib/Serialization/ASTWriter.cpp
    M clang/lib/Serialization/ASTWriterDecl.cpp

  Log Message:
  -----------
  [clang][AST] Support AST files larger than 512M

Summary:
Clang uses 32-bit integers for storing bit offsets from the beginning of
the file that results in 512M limit on AST file. This diff replaces
absolute offsets with relative offsets from the beginning of
corresponding data structure when it is possible. And uses 64-bit
offsets for DeclOffests and TypeOffssts because these coder AST
section may easily exceeds 512M alone.

This diff breaks AST file format compatibility so VERSION_MAJOR bumped.

Test Plan:
Existing clang AST serialization tests
Tested on clangd with ~700M and ~900M preamble files
check-clang with ubsan

Reviewers: rsmith, dexonsmith

Subscribers: ilya-biryukov, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76594