[all-commits] [llvm/llvm-project] 30d594: [clang][AST] Support AST files larger than 512M

[Dmitry Polukhin via All-commits]

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: 30d5946db95fa465d7ee6caceb2b1ff191e3727c
      https://github.com/llvm/llvm-project/commit/30d5946db95fa465d7ee6caceb2b1ff191e3727c
  Author: Dmitry Polukhin <dmitry.polukhin at gmail.com>
  Date:   2020-04-16 (Thu, 16 Apr 2020)

  Changed paths:
    M clang/include/clang/Serialization/ASTBitCodes.h
    M clang/include/clang/Serialization/ASTReader.h
    M clang/include/clang/Serialization/ASTWriter.h
    M clang/include/clang/Serialization/ModuleFile.h
    M clang/lib/Serialization/ASTReader.cpp
    M clang/lib/Serialization/ASTReaderDecl.cpp
    M clang/lib/Serialization/ASTWriter.cpp
    M clang/lib/Serialization/ASTWriterDecl.cpp

  Log Message:
  -----------
  [clang][AST] Support AST files larger than 512M

Summary:
Clang uses 32-bit integers for storing bit offsets from the beginning of
the file that results in 512M limit on AST file. This diff replaces
absolute offsets with relative offsets from the beginning of
corresponding data structure when it is possible. And uses 64-bit
offsets for DeclOffests and TypeOffssts because these coder AST
section may easily exceeds 512M alone.

This diff breaks AST file format compatibility so VERSION_MAJOR bumped.

Test Plan:
Existing clang AST serialization tests
Tested on clangd with ~700M and ~900M preamble files

Reviewers: rsmith, dexonsmith

Subscribers: ilya-biryukov, kadircet, usaxena95, cfe-commits

Tags: #clang

Differential Revision: https://reviews.llvm.org/D76594