[lld] 83632c0 - [lld][BP] Order .Tgm symbols for startup (#126328)

[via llvm-commits]

Author: Ellis Hoag
Date: 2025-02-13T12:10:58-08:00
New Revision: 83632c039d17772769617b92a1658e8a98190c5e

URL: https://github.com/llvm/llvm-project/commit/83632c039d17772769617b92a1658e8a98190c5e
DIFF: https://github.com/llvm/llvm-project/commit/83632c039d17772769617b92a1658e8a98190c5e.diff

LOG: [lld][BP] Order .Tgm symbols for startup (#126328)

The Global Function Merger
(https://discourse.llvm.org/t/rfc-global-function-merging/82608) pass
optimistically creates merged instances of functions and suffixes their
names with `.Tgm`. Then in the linker, ICF will (hopefully) fold these
`.Tgm` functions. For example, a function `foo` might become a thunk
`foo` that calls a merged function `foo.Tgm`.

Since IRPGO runs before the global merger, we will only have a profile
for `foo`. We want to correlate this profile to both `foo` and `foo.Tgm`
so they can both be ordered to improve startup time.

I built a large binary and found that it increased the number of
functions ordered for startup, as expected.
```
Functions for startup: 12049 -> 12697
Functions for compression: 34733 -> 34707
```

The reason why we don't see a larger improvement is because there are
some cases where the code was accidentally working:
`getRootSymbol("foo.llvm.5555.Tgm")` already returns `foo`.

Added: 
    

Modified: 
    lld/include/lld/Common/BPSectionOrdererBase.inc
    lld/test/MachO/bp-section-orderer.s

Removed: 
    


################################################################################
diff  --git a/lld/include/lld/Common/BPSectionOrdererBase.inc b/lld/include/lld/Common/BPSectionOrdererBase.inc
index 83b87f4c7817d..b19d3670d34cc 100644
--- a/lld/include/lld/Common/BPSectionOrdererBase.inc
+++ b/lld/include/lld/Common/BPSectionOrdererBase.inc
@@ -147,11 +147,14 @@ static SmallVector<std::pair<unsigned, UtilityNodes>> getUnsForCompression(
   return sectionUns;
 }
 
-/// Symbols can be appended with "(.__uniq.xxxx)?.llvm.yyyy" where "xxxx" and
-/// "yyyy" are numbers that could change between builds. We need to use the
-/// root symbol name before this suffix so these symbols can be matched with
-/// profiles which may have 
diff erent suffixes.
+/// Symbols can be appended with "(.__uniq.xxxx)?(.llvm.yyyy)?(.Tgm)?" where
+/// "xxxx" and "yyyy" are numbers that could change between builds, and .Tgm is
+/// the global merge functions suffix
+/// (see GlobalMergeFunc::MergingInstanceSuffix). We need to use the root symbol
+/// name before this suffix so these symbols can be matched with profiles which
+/// may have 
diff erent suffixes.
 inline StringRef getRootSymbol(StringRef name) {
+  name.consume_back(".Tgm");
   auto [P0, S0] = name.rsplit(".llvm.");
   auto [P1, S1] = P0.rsplit(".__uniq.");
   return P1;

diff  --git a/lld/test/MachO/bp-section-orderer.s b/lld/test/MachO/bp-section-orderer.s
index f0ac1bd99f722..f3293b54dbcf1 100644
--- a/lld/test/MachO/bp-section-orderer.s
+++ b/lld/test/MachO/bp-section-orderer.s
@@ -5,11 +5,12 @@
 # RUN: llvm-profdata merge %t/a.proftext -o %t/a.profdata
 
 # RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile-sort=%t/a.profdata --verbose-bp-section-orderer 2>&1 | FileCheck %s --check-prefix=STARTUP
-# RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile-sort=%t/a.profdata --verbose-bp-section-orderer --icf=all --compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP
+# RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile-sort=%t/a.profdata --verbose-bp-section-orderer --icf=all --compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP-ICF
 
 # RUN: %lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile %t/a.profdata --bp-startup-sort=function --verbose-bp-section-orderer 2>&1 | FileCheck %s --check-prefix=STARTUP
-# RUN: %lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile=%t/a.profdata --bp-startup-sort=function --verbose-bp-section-orderer --icf=all --bp-compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP
-# STARTUP: Ordered 3 sections using balanced partitioning
+# RUN: %lld -arch arm64 -lSystem -e _main -o %t/a.out %t/a.o --irpgo-profile=%t/a.profdata --bp-startup-sort=function --verbose-bp-section-orderer --icf=all --bp-compression-sort=none 2>&1 | FileCheck %s --check-prefix=STARTUP-ICF
+# STARTUP: Ordered 5 sections using balanced partitioning
+# STARTUP-ICF: Ordered 4 sections using balanced partitioning
 
 # Check that orderfiles take precedence over BP
 # RUN: %no-fatal-warnings-lld -arch arm64 -lSystem -e _main -o - %t/a.o -order_file %t/a.orderfile --irpgo-profile-sort=%t/a.profdata  | llvm-nm --numeric-sort --format=just-symbols - | FileCheck %s --check-prefix=ORDERFILE
@@ -26,6 +27,8 @@
 # ORDERFILE-DAG: _main
 # ORDERFILE-DAG: _B
 # ORDERFILE-DAG: l_C
+# ORDERFILE-DAG: merged1.Tgm
+# ORDERFILE-DAG: merged2.Tgm
 
 # Data
 # ORDERFILE: s3
@@ -80,10 +83,10 @@ F:
   add w0, w0, #3
   bl l_C.__uniq.111111111111111111111111111111111111111.llvm.2222222222222222222
   ret
-merged1:
+merged1.Tgm:
   add w0, w0, #101
   ret
-merged2:
+merged2.Tgm:
   add w0, w0, #101
   ret
 
@@ -114,7 +117,7 @@ r4:
 1
 # Weight
 1
-A, B, C.__uniq.555555555555555555555555555555555555555.llvm.6666666666666666666
+A, B, C.__uniq.555555555555555555555555555555555555555.llvm.6666666666666666666, merged1, merged2
 
 A
 # Func Hash:
@@ -148,6 +151,22 @@ D
 # Counter Values:
 1
 
+merged1
+# Func Hash:
+5555
+# Num Counters:
+1
+# Counter Values:
+1
+
+merged2
+# Func Hash:
+6666
+# Num Counters:
+1
+# Counter Values:
+1
+
 #--- a.orderfile
 A
 F