[all-commits] [llvm/llvm-project] b5bd19: [Serialization] Support load lazy specialization l...

[Chuanqi Xu via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: b5bd19211118c6d43bc525a4e3fb65d2c750d61e
      https://github.com/llvm/llvm-project/commit/b5bd19211118c6d43bc525a4e3fb65d2c750d61e
  Author: Chuanqi Xu <yedeng.yd at linux.alibaba.com>
  Date:   2024-12-06 (Fri, 06 Dec 2024)

  Changed paths:
    M clang/include/clang/AST/DeclTemplate.h
    M clang/include/clang/AST/ExternalASTSource.h
    M clang/include/clang/Sema/MultiplexExternalSemaSource.h
    M clang/include/clang/Serialization/ASTBitCodes.h
    M clang/include/clang/Serialization/ASTReader.h
    M clang/include/clang/Serialization/ASTWriter.h
    M clang/lib/AST/DeclTemplate.cpp
    M clang/lib/AST/ExternalASTSource.cpp
    M clang/lib/AST/ODRHash.cpp
    M clang/lib/Sema/MultiplexExternalSemaSource.cpp
    M clang/lib/Serialization/ASTCommon.h
    M clang/lib/Serialization/ASTReader.cpp
    M clang/lib/Serialization/ASTReaderDecl.cpp
    M clang/lib/Serialization/ASTReaderInternals.h
    M clang/lib/Serialization/ASTWriter.cpp
    M clang/lib/Serialization/ASTWriterDecl.cpp
    M clang/lib/Serialization/CMakeLists.txt
    A clang/lib/Serialization/TemplateArgumentHasher.cpp
    A clang/lib/Serialization/TemplateArgumentHasher.h
    M clang/test/Modules/odr_hash.cpp
    A clang/test/Modules/recursive-instantiations.cppm
    M clang/test/OpenMP/target_parallel_ast_print.cpp
    M clang/test/OpenMP/target_teams_ast_print.cpp
    M clang/test/OpenMP/task_ast_print.cpp
    M clang/test/OpenMP/teams_ast_print.cpp
    M clang/unittests/Serialization/CMakeLists.txt
    A clang/unittests/Serialization/LoadSpecLazilyTest.cpp

  Log Message:
  -----------
  [Serialization] Support load lazy specialization lazily

Currently all the specializations of a template (including
instantiation, specialization and partial specializations)  will be
loaded at once if we want to instantiate another instance for the
template, or find instantiation for the template, or just want to
complete the redecl chain.

This means basically we need to load every specializations for the
template once the template declaration got loaded. This is bad since
when we load a specialization, we need to load all of its template
arguments. Then we have to deserialize a lot of unnecessary
declarations.

For example,

```
// M.cppm
export module M;
export template <class T>
class A {};

export class ShouldNotBeLoaded {};

export class Temp {
   A<ShouldNotBeLoaded> AS;
};

// use.cpp
import M;
A<int> a;
```

We should a specialization ` A<ShouldNotBeLoaded>` in `M.cppm` and we
instantiate the template `A` in `use.cpp`. Then we will deserialize
`ShouldNotBeLoaded` surprisingly when compiling `use.cpp`. And this
patch tries to avoid that.

Given that the templates are heavily used in C++, this is a pain point
for the performance.

This patch adds MultiOnDiskHashTable for specializations in the
ASTReader. Then we will only deserialize the specializations with the
same template arguments. We made that by using ODRHash for the template
arguments as the key of the hash table.

To review this patch, I think `ASTReaderDecl::AddLazySpecializations`
may be a good entry point.

The patch was reviewed in
https://github.com/llvm/llvm-project/pull/83237 but that PR is a stacked
PR. But I feel the intention of the stacked PRs get lost during the
review process. So I feel it is better to merge the commits into a
single commit instead of merging them in the PR page. It is better for
us to cherry-pick and revert.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications