[PATCH] D156525: [ThinLTO} Use module hash instead of module ID for cache key

[Nikita Popov via Phabricator via llvm-commits]

nikic created this revision.
nikic added reviewers: tejohnson, akyrtzi.
Herald added subscribers: ormris, StephenFan, steven_wu, JDevlieghere, hiraditya, inglorion.
Herald added a project: All.
nikic requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

This is a followup to D151165 <https://reviews.llvm.org/D151165>. Instead of using the module ID, use the module hash for sorting the import list. The module hash is what will actually be included in the hash.

This has the advantage of being independent of the module order, which is something that Rust relies on.

A caveat here is that this doesn't quite work for linkonce_odr functions, because it seems like the function from the first encountered module gets picked (rather than say, the prevailing function). This doesn't really matter for Rust's purposes (because it does not use linkonce_odr linkage), but may still be worth addressing. For now I'm using a variant of the test using internal instead of linkonce_odr functions.


https://reviews.llvm.org/D156525

Files:
  llvm/lib/LTO/LTO.cpp
  llvm/test/ThinLTO/X86/Inputs/cache-import-lists3.ll
  llvm/test/ThinLTO/X86/Inputs/cache-import-lists4.ll
  llvm/test/ThinLTO/X86/cache-decoupled-from-order.ll


Index: llvm/test/ThinLTO/X86/cache-decoupled-from-order.ll
===================================================================
--- /dev/null
+++ llvm/test/ThinLTO/X86/cache-decoupled-from-order.ll
@@ -0,0 +1,24 @@
+; RUN: rm -rf %t && mkdir -p %t
+; RUN: opt -module-hash -module-summary %s -o %t/t.bc
+; RUN: opt -module-hash -module-summary %S/Inputs/cache-import-lists3.ll -o %t/a.bc
+; RUN: opt -module-hash -module-summary %S/Inputs/cache-import-lists4.ll -o %t/b.bc
+
+; Tests that the hash for t is insensitive to the order of the modules.
+
+; RUN: llvm-lto2 run -cache-dir %t/cache -o %t.o %t/t.bc %t/a.bc %t/b.bc -r=%t/t.bc,main,plx -r=%t/t.bc,f1,lx -r=%t/t.bc,f2,lx -r=%t/a.bc,f1,plx -r=%t/b.bc,f2,plx
+; RUN: ls %t/cache | count 3
+
+; RUN: llvm-lto2 run -cache-dir %t/cache -o %t.o %t/b.bc %t/a.bc %t/t.bc -r=%t/t.bc,main,plx -r=%t/t.bc,f1,lx -r=%t/t.bc,f2,lx -r=%t/a.bc,f1,plx -r=%t/b.bc,f2,plx
+; RUN: ls %t/cache | count 3
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @main() {
+  call void @f1()
+  call void @f2()
+  ret void
+}
+
+declare void @f1()
+declare void @f2()
Index: llvm/test/ThinLTO/X86/Inputs/cache-import-lists4.ll
===================================================================
--- /dev/null
+++ llvm/test/ThinLTO/X86/Inputs/cache-import-lists4.ll
@@ -0,0 +1,11 @@
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @f2() {
+  call void @internal()
+  ret void
+}
+
+define internal void @internal() {
+  ret void
+}
Index: llvm/test/ThinLTO/X86/Inputs/cache-import-lists3.ll
===================================================================
--- /dev/null
+++ llvm/test/ThinLTO/X86/Inputs/cache-import-lists3.ll
@@ -0,0 +1,11 @@
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @f1() {
+  call void @internal()
+  ret void
+}
+
+define internal void @internal() {
+  ret void
+}
Index: llvm/lib/LTO/LTO.cpp
===================================================================
--- llvm/lib/LTO/LTO.cpp
+++ llvm/lib/LTO/LTO.cpp
@@ -184,7 +184,6 @@
     }
 
     const ModuleHash &getHash() const { return ModInfo->second.second; }
-    uint64_t getId() const { return ModInfo->second.first; }
   };
 
   std::vector<ImportModule> ImportModulesVector;
@@ -194,11 +193,11 @@
        ++It) {
     ImportModulesVector.push_back({It, Index.getModule(It->getKey())});
   }
-  // Order using moduleId integer which is based on the order the module was
-  // added.
+  // Order using module hash, to be both independent of module name and
+  // module order.
   llvm::sort(ImportModulesVector,
              [](const ImportModule &Lhs, const ImportModule &Rhs) -> bool {
-               return Lhs.getId() < Rhs.getId();
+               return Lhs.getHash() < Rhs.getHash();
              });
   for (const ImportModule &Entry : ImportModulesVector) {
     auto ModHash = Entry.getHash();


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D156525.545106.patch
Type: text/x-patch
Size: 3134 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20230728/f7b15144/attachment.bin>