[all-commits] [llvm/llvm-project] 2203a4: [NFC] [Serialization] Improve AST serialization by...

[Chuanqi Xu via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 2203a4e6e01ce6bfd69505420d304a81daf23dc9
      https://github.com/llvm/llvm-project/commit/2203a4e6e01ce6bfd69505420d304a81daf23dc9
  Author: Chuanqi Xu <yedeng.yd at linux.alibaba.com>
  Date:   2023-12-21 (Thu, 21 Dec 2023)

  Changed paths:
    M clang/lib/Serialization/ASTReaderDecl.cpp
    M clang/lib/Serialization/ASTReaderStmt.cpp
    M clang/lib/Serialization/ASTWriterDecl.cpp
    M clang/lib/Serialization/ASTWriterStmt.cpp

  Log Message:
  -----------
  [NFC] [Serialization] Improve AST serialization by reordering packed
bits and extract big bits from packed bits

Previously I tried to improve the size of .pcm files by introducing
packed bits. And I find we can improve it further by reordering the
bits.

The secret comes from the VBR format. We can find the formal definition
of VBR format in the doc of LLVM. The VBR format will be pretty
efficicent for small numbers.

For example, if we need to pack 8 bits into a value and the stored value
is 0xf0, the actual stored value will be 0b000111'110000, which takes 12
bits actually. However, if we changed the order to be 0x0f, then we
can store it as 0b001111, which takes 6 bits only now.

So we can improve the size by placing bits with lower probability to be
1 in the higher bits and extract bit bigs from the packed bits to make
it possible to be optimized by VBR.

After this patch, the size of std module becomes to 27.7MB from 28.1MB.