[all-commits] [llvm/llvm-project] a553c6: [clang][modules] Avoid allocations when reading bl...

[Jan Svoboda via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: a553c620b7d70dedd268aa2588e5e50e7dc6ccc8
      https://github.com/llvm/llvm-project/commit/a553c620b7d70dedd268aa2588e5e50e7dc6ccc8
  Author: Jan Svoboda <jan_svoboda at apple.com>
  Date:   2024-10-31 (Thu, 31 Oct 2024)

  Changed paths:
    M clang/include/clang/Serialization/ASTReader.h
    M clang/lib/Serialization/ASTReader.cpp

  Log Message:
  -----------
  [clang][modules] Avoid allocations when reading blob paths (#113984)

When reading a path from a bitstream blob, `ASTReader` performs up to
three allocations:
1. Conversion of the `StringRef` blob into `std::string` to conform to
the `ResolveImportedPath()` API that takes `std::string &`.
2. Concatenation of the module file prefix directory and the relative
path into a fresh `SmallString<128>` buffer in `ResolveImportedPath()`.
3. Propagating the result out of `ResolveImportedPath()` by calling
`std::string::assign()` on the out-parameter.

This patch makes is so that we avoid allocations altogether (amortized)
by:
1. Avoiding conversion of the `StringRef` blob into `std::string` and
changing the `ResolveImportedPath()` API.
 2. Using one "global" buffer to hold the concatenation.
3. Returning `StringRef` that points into the buffer and ensuring the
contents are not overwritten while it lives.

Note that in some places of the bitstream we don't store paths as blobs,
but rather as records that get VBR-encoded. This makes the allocation in
(1) unavoidable. I plan to fix this in a follow-up PR by changing the
PCM format.

Moreover, there are some data structures (e.g.
`serialization::InputFileInfo`) that store deserialized and resolved
paths as `std::string`. If we don't access them frequently, it would be
more efficient to store just the unresolved `StringRef` and resolve them
on demand (within some kind of shared buffer to prevent allocations).

This PR alone improves `clang-scan-deps` performance on my workload by
3.6%.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications