[llvm] Implement bolt-align (PR #210634)
via llvm-commits
llvm-commits at lists.llvm.org
Mon Jul 20 03:30:33 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-bolt
Author: Ilija Tovilo (iluuu1994)
<details>
<summary>Changes</summary>
Closes GH-148042
Based on top of GH-207412 (commit 75415f80b26f7c7177283a1680bef44c744c4886) for personal testing purposes (BOLT is currently broken for my use-case).
>From the issue above:
> Binary layout can greatly influence the performance characteristics of programs. Sometimes, seemingly minuscule code changes can lead large performance regressions due to small layout changes.
This is a problem we've been struggling with for years, and we don't seem to be alone. [^1] I've tried many remediations with limited success. Particularly, `lld --randomize-section-padding` has been suggested. By testing a sufficient number of different versions of the same binary with different, randomized padding, one can filter out the layout effects. This turned out to be much too tedious for me personally. Another idea was to use PGO on both binaries, which didn't seem to sufficiently remove the layout effects in my testing.
The idea of `llvm-bolt-align` is simple, namely to take two binaries as input and create two new binaries with symbols aligned as much as possible.
The implementation is fairly straight-forward, though not particularly efficient.
- We pass both binaries to separate `llvm-bolt-align --generate-function-layout-file` invocations. This command records in a file the final output address of all functions relative to their corresponding section.
- We merge the two layout files into a new target layout file. Only functions present in both files with the same relative order persist. Enough padding is inserted for each symbol from either binary to fit.
- Finally, both binaries are generated using `llvm-bolt-align --function-layout-file`.
Feedback would be appreciated. I'm a C developer with limited C++ experience, so the code might not be idiomatic. My experience with LLVM and BOLT is also limited.
In a small test case, the tool has aligned a benchmark result with my intuitive expectation. Of course, this is hard to quantify and more confidence in its effectiveness can only be gained with time. If you have a use-case and would like to try it, your findings would be very helpful.
Open issues:
- [ ] Currently, we're only aligning functions. We're not aligning data, which might cause similar effects.
- [ ] We are also not aligning segments. If sections preceding the code section significantly differs in size between the two binaries, we might still end up with an entirely misaligned code segment.
- [ ] We're only aligning each function's main fragment. Is that enough?
[^1]: https://www.youtube.com/watch?v=IX16gcX4vDQ
---
Patch is 39.87 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/210634.diff
20 Files Affected:
- (modified) bolt/include/bolt/Core/BinaryFunction.h (+13)
- (added) bolt/include/bolt/Passes/AssignDesiredFunctionOffset.h (+41)
- (added) bolt/include/bolt/Rewrite/MergeFunctionLayouts.h (+31)
- (modified) bolt/include/bolt/Rewrite/RewriteInstance.h (+4)
- (modified) bolt/lib/Core/BinaryEmitter.cpp (+19-6)
- (added) bolt/lib/Passes/AssignDesiredFunctionOffset.cpp (+130)
- (modified) bolt/lib/Passes/CMakeLists.txt (+1)
- (modified) bolt/lib/Rewrite/BinaryPassManager.cpp (+3)
- (modified) bolt/lib/Rewrite/CMakeLists.txt (+1)
- (added) bolt/lib/Rewrite/MergeFunctionLayouts.cpp (+195)
- (modified) bolt/lib/Rewrite/RewriteInstance.cpp (+102-10)
- (modified) bolt/test/AArch64/runtime-relocs.test (+1-1)
- (added) bolt/test/X86/llvm-bolt-align.s (+85)
- (added) bolt/test/runtime/X86/rela-plt-order.c (+31)
- (modified) bolt/tools/driver/CMakeLists.txt (+1)
- (modified) bolt/tools/driver/llvm-bolt.cpp (+132)
- (modified) llvm/include/llvm/MC/MCObjectStreamer.h (+2)
- (modified) llvm/include/llvm/MC/MCSection.h (+9)
- (modified) llvm/lib/MC/MCAssembler.cpp (+2)
- (modified) llvm/lib/MC/MCObjectStreamer.cpp (+7-1)
``````````diff
diff --git a/bolt/include/bolt/Core/BinaryFunction.h b/bolt/include/bolt/Core/BinaryFunction.h
index b3af846c4eb57..5eee07c2748a8 100644
--- a/bolt/include/bolt/Core/BinaryFunction.h
+++ b/bolt/include/bolt/Core/BinaryFunction.h
@@ -271,6 +271,10 @@ class BinaryFunction {
/// Maximum number of bytes used for alignment of cold part of the function.
uint16_t MaxColdAlignmentBytes{0};
+ /// The desired output offset of the function's main fragment in bytes
+ /// relative to its code section.
+ std::optional<uint64_t> DesiredOffset;
+
const MCSymbol *PersonalityFunction{nullptr};
uint8_t PersonalityEncoding{dwarf::DW_EH_PE_sdata4 | dwarf::DW_EH_PE_pcrel};
@@ -1954,6 +1958,15 @@ class BinaryFunction {
uint16_t getMaxColdAlignmentBytes() const { return MaxColdAlignmentBytes; }
+ BinaryFunction &setDesiredOffset(uint64_t Offset) {
+ DesiredOffset = Offset;
+ return *this;
+ }
+
+ std::optional<uint64_t> getDesiredOffset() const {
+ return DesiredOffset;
+ }
+
BinaryFunction &setImageAddress(uint64_t Address) {
getLayout().getMainFragment().setImageAddress(Address);
return *this;
diff --git a/bolt/include/bolt/Passes/AssignDesiredFunctionOffset.h b/bolt/include/bolt/Passes/AssignDesiredFunctionOffset.h
new file mode 100644
index 0000000000000..0950ccd0edc4f
--- /dev/null
+++ b/bolt/include/bolt/Passes/AssignDesiredFunctionOffset.h
@@ -0,0 +1,41 @@
+//===- bolt/Passes/AssignDesiredFunctionOffset.h ----------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Pass to assign the desired output offsets from --function-layout-file to the
+// corresponding functions.
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef BOLT_PASSES_ASSIGNDESIREDFUNCTIONOFFSET_H
+#define BOLT_PASSES_ASSIGNDESIREDFUNCTIONOFFSET_H
+
+#include "bolt/Passes/BinaryPasses.h"
+#include "llvm/Support/CommandLine.h"
+
+using namespace llvm;
+
+namespace opts {
+extern cl::opt<std::string> FunctionLayoutFile;
+} // namespace opts
+
+namespace llvm {
+namespace bolt {
+
+class AssignDesiredFunctionOffset : public BinaryFunctionPass {
+public:
+ explicit AssignDesiredFunctionOffset() : BinaryFunctionPass(false) {}
+
+ const char *getName() const override { return "apply-function-layout"; }
+
+ Error runOnFunctions(BinaryContext &BC) override;
+};
+
+} // namespace bolt
+} // namespace llvm
+
+#endif
diff --git a/bolt/include/bolt/Rewrite/MergeFunctionLayouts.h b/bolt/include/bolt/Rewrite/MergeFunctionLayouts.h
new file mode 100644
index 0000000000000..fc3b182100c21
--- /dev/null
+++ b/bolt/include/bolt/Rewrite/MergeFunctionLayouts.h
@@ -0,0 +1,31 @@
+//===- bolt/Rewrite/MergeFunctionLayouts.h ----------------------*- C++ -*-===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef BOLT_REWRITE_MERGEFUNCTIONLAYOUTS_H
+#define BOLT_REWRITE_MERGEFUNCTIONLAYOUTS_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Error.h"
+
+namespace llvm {
+class raw_ostream;
+
+namespace bolt {
+
+/// Merge two layout files \p PathA and \p PathB into an aligned layout stored
+/// to \p OutputPath. Only common functions that occur in the same relative
+/// order are included.
+Error mergeFunctionLayouts(StringRef PathA,
+ StringRef PathB,
+ StringRef OutputPath,
+ raw_ostream &Log);
+
+} // namespace bolt
+} // namespace llvm
+
+#endif
diff --git a/bolt/include/bolt/Rewrite/RewriteInstance.h b/bolt/include/bolt/Rewrite/RewriteInstance.h
index 2e63ca93c3e57..2a27dd871b200 100644
--- a/bolt/include/bolt/Rewrite/RewriteInstance.h
+++ b/bolt/include/bolt/Rewrite/RewriteInstance.h
@@ -17,6 +17,7 @@
#include "bolt/Core/Linker.h"
#include "bolt/Rewrite/MetadataManager.h"
#include "bolt/Utils/NameResolver.h"
+#include "llvm/ADT/SmallVector.h"
#include "llvm/MC/StringTableBuilder.h"
#include "llvm/Object/ELFObjectFile.h"
#include "llvm/Object/ObjectFile.h"
@@ -519,6 +520,9 @@ class RewriteInstance {
/// True if relocation of specified type came from .rela.plt
DenseMap<uint64_t, bool> IsJmpRelocation;
+ /// Original dynamic relocation order.
+ SmallVector<uint64_t, 0> DynamicRelocationOrder;
+
/// Index of specified symbol in the dynamic symbol table. NOTE Currently it
/// is filled and used only with the relocations-related symbols.
std::unordered_map<const MCSymbol *, uint32_t> SymbolIndex;
diff --git a/bolt/lib/Core/BinaryEmitter.cpp b/bolt/lib/Core/BinaryEmitter.cpp
index a555c7185448e..571c659533c35 100644
--- a/bolt/lib/Core/BinaryEmitter.cpp
+++ b/bolt/lib/Core/BinaryEmitter.cpp
@@ -19,6 +19,8 @@
#include "bolt/Utils/CommandLineOpts.h"
#include "bolt/Utils/Utils.h"
#include "llvm/DebugInfo/DWARF/DWARFCompileUnit.h"
+#include "llvm/MC/MCExpr.h"
+#include "llvm/MC/MCObjectStreamer.h"
#include "llvm/MC/MCSection.h"
#include "llvm/MC/MCStreamer.h"
#include "llvm/Support/CommandLine.h"
@@ -309,12 +311,23 @@ bool BinaryEmitter::emitFunction(BinaryFunction &Function,
// tentative layout.
Section->ensureMinAlignment(Align(BC.AlignFunctions));
- Streamer.emitCodeAlignment(Function.getMinAlign(), *BC.STI);
- uint16_t MaxAlignBytes = FF.isSplitFragment()
- ? Function.getMaxColdAlignmentBytes()
- : Function.getMaxAlignmentBytes();
- if (MaxAlignBytes > 0)
- Streamer.emitCodeAlignment(Function.getAlign(), *BC.STI, MaxAlignBytes);
+ std::optional<uint64_t> DesiredOffset;
+ if (FF.isMainFragment())
+ DesiredOffset = Function.getDesiredOffset();
+
+ if (DesiredOffset) {
+ const MCExpr *OffsetExpr = MCConstantExpr::create(*DesiredOffset, *BC.Ctx);
+ const unsigned FillValue = BC.Ctx->getAsmInfo().getTextAlignFillValue();
+ static_cast<MCObjectStreamer &>(Streamer).emitValueToOffset(
+ OffsetExpr, FillValue, SMLoc(), /*AllowOmission=*/true);
+ } else {
+ Streamer.emitCodeAlignment(Function.getMinAlign(), *BC.STI);
+ uint16_t MaxAlignBytes = FF.isSplitFragment()
+ ? Function.getMaxColdAlignmentBytes()
+ : Function.getMaxAlignmentBytes();
+ if (MaxAlignBytes > 0)
+ Streamer.emitCodeAlignment(Function.getAlign(), *BC.STI, MaxAlignBytes);
+ }
} else {
Streamer.emitCodeAlignment(Function.getAlign(), *BC.STI);
}
diff --git a/bolt/lib/Passes/AssignDesiredFunctionOffset.cpp b/bolt/lib/Passes/AssignDesiredFunctionOffset.cpp
new file mode 100644
index 0000000000000..7e716e128c4a1
--- /dev/null
+++ b/bolt/lib/Passes/AssignDesiredFunctionOffset.cpp
@@ -0,0 +1,130 @@
+//===- bolt/Passes/AssignDesiredFunctionOffset.cpp ------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements the AssignDesiredFunctionOffset pass.
+//
+//===----------------------------------------------------------------------===//
+
+#include "bolt/Passes/AssignDesiredFunctionOffset.h"
+#include "bolt/Core/BinaryContext.h"
+#include "bolt/Core/BinaryData.h"
+#include "bolt/Core/BinaryFunction.h"
+#include "bolt/Utils/CommandLineOpts.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/LineIterator.h"
+#include "llvm/Support/MemoryBuffer.h"
+
+using namespace llvm;
+using namespace bolt;
+
+namespace opts {
+
+extern cl::OptionCategory BoltCategory;
+
+cl::opt<std::string> FunctionLayoutFile(
+ "function-layout-file",
+ cl::desc("file populating the functions' desired output offset. Requires "
+ "relocation mode."),
+ cl::value_desc("filename"), cl::Hidden, cl::cat(BoltCategory));
+
+} // namespace opts
+
+namespace llvm {
+namespace bolt {
+
+/// Mirror the lookup in ReorderFunctions.
+static BinaryFunction *lookupFunction(BinaryContext &BC, StringRef Name) {
+ BinaryData *BD = BC.getBinaryDataByName(Name);
+ if (!BD) {
+ for (uint32_t LocalID = 1;; ++LocalID) {
+ BD = BC.getBinaryDataByName((Name + "/" + Twine(LocalID)).str());
+ if (!BD)
+ break;
+ if (BinaryFunction *BF = BC.getFunctionForSymbol(BD->getSymbol()))
+ return BF;
+ }
+ return nullptr;
+ }
+ return BC.getFunctionForSymbol(BD->getSymbol());
+}
+
+Error AssignDesiredFunctionOffset::runOnFunctions(BinaryContext &BC) {
+ if (opts::FunctionLayoutFile.empty())
+ return Error::success();
+
+ if (!BC.HasRelocations) {
+ BC.errs() << "BOLT-ERROR: --function-layout-file is not supported in "
+ "non-relocation mode\n";
+ exit(1);
+ }
+
+ ErrorOr<std::unique_ptr<MemoryBuffer>> MB =
+ MemoryBuffer::getFile(opts::FunctionLayoutFile);
+ if (std::error_code EC = MB.getError())
+ return createStringError(EC, Twine("cannot open function layout file '") +
+ opts::FunctionLayoutFile +
+ "': " + EC.message());
+
+ uint64_t Applied = 0;
+ uint64_t NotFound = 0;
+ uint64_t Malformed = 0;
+ for (line_iterator LI(*MB.get(), /*SkipBlanks=*/true, /*CommentMarker=*/'#');
+ !LI.is_at_eof(); ++LI) {
+ StringRef Line = LI->trim();
+ if (Line.empty())
+ continue;
+
+ StringRef Name, OffsetStr;
+ std::tie(Name, OffsetStr) = Line.split(' ');
+ Name = Name.trim();
+ OffsetStr = OffsetStr.trim();
+
+ uint64_t Offset;
+ if (Name.empty() || OffsetStr.empty() ||
+ OffsetStr.getAsInteger(/*Radix=*/0, Offset)) {
+ BC.errs() << "BOLT-WARNING: --function-layout-file: malformed entry at "
+ << opts::FunctionLayoutFile << ":" << LI.line_number() << "\n";
+ ++Malformed;
+ continue;
+ }
+
+ BinaryFunction *BF = lookupFunction(BC, Name);
+ if (!BF) {
+ if (opts::Verbosity >= 1)
+ BC.errs() << "BOLT-WARNING: --function-layout-file: cannot find "
+ "function '" << Name << "'\n";
+ ++NotFound;
+ continue;
+ }
+
+ if (!isAligned(BF->getMinAlign(), Offset)) {
+ BC.errs() << "BOLT-WARNING: --function-layout-file: offset " << Offset
+ << " for function '" << Name
+ << "' is not aligned by the minimum function alignment ("
+ << BF->getMinAlign().value() << "), skipping\n";
+ ++Malformed;
+ continue;
+ }
+
+ BF->setDesiredOffset(Offset);
+ ++Applied;
+ }
+
+ BC.outs() << "BOLT-INFO: --function-layout-file: pinned " << Applied
+ << " functions";
+ if (NotFound)
+ BC.outs() << ", " << NotFound << " not found";
+ if (Malformed)
+ BC.outs() << ", " << Malformed << " malformed/skipped";
+ BC.outs() << "\n";
+
+ return Error::success();
+}
+
+} // namespace bolt
+} // namespace llvm
diff --git a/bolt/lib/Passes/CMakeLists.txt b/bolt/lib/Passes/CMakeLists.txt
index ec012f05cc498..616b269880d76 100644
--- a/bolt/lib/Passes/CMakeLists.txt
+++ b/bolt/lib/Passes/CMakeLists.txt
@@ -3,6 +3,7 @@ add_llvm_library(LLVMBOLTPasses
Aligner.cpp
AllocCombiner.cpp
AsmDump.cpp
+ AssignDesiredFunctionOffset.cpp
BinaryPasses.cpp
CMOVConversion.cpp
CacheMetrics.cpp
diff --git a/bolt/lib/Rewrite/BinaryPassManager.cpp b/bolt/lib/Rewrite/BinaryPassManager.cpp
index 6e3022c491a73..c61c404c67361 100644
--- a/bolt/lib/Rewrite/BinaryPassManager.cpp
+++ b/bolt/lib/Rewrite/BinaryPassManager.cpp
@@ -11,6 +11,7 @@
#include "bolt/Passes/Aligner.h"
#include "bolt/Passes/AllocCombiner.h"
#include "bolt/Passes/AsmDump.h"
+#include "bolt/Passes/AssignDesiredFunctionOffset.h"
#include "bolt/Passes/CMOVConversion.h"
#include "bolt/Passes/FixRISCVCallsPass.h"
#include "bolt/Passes/FixRelaxationPass.h"
@@ -525,6 +526,8 @@ Error BinaryFunctionPassManager::runAllPasses(BinaryContext &BC) {
Manager.registerPass(std::make_unique<AlignerPass>());
+ Manager.registerPass(std::make_unique<AssignDesiredFunctionOffset>());
+
// Perform reordering on data contained in one or more sections using
// memory profiling data.
Manager.registerPass(std::make_unique<ReorderData>());
diff --git a/bolt/lib/Rewrite/CMakeLists.txt b/bolt/lib/Rewrite/CMakeLists.txt
index bc1b2ed3c2e3c..a8a22bc6c468b 100644
--- a/bolt/lib/Rewrite/CMakeLists.txt
+++ b/bolt/lib/Rewrite/CMakeLists.txt
@@ -20,6 +20,7 @@ add_llvm_library(LLVMBOLTRewrite
JITLinkLinker.cpp
LinuxKernelRewriter.cpp
MachORewriteInstance.cpp
+ MergeFunctionLayouts.cpp
MetadataManager.cpp
BuildIDRewriter.cpp
PseudoProbeRewriter.cpp
diff --git a/bolt/lib/Rewrite/MergeFunctionLayouts.cpp b/bolt/lib/Rewrite/MergeFunctionLayouts.cpp
new file mode 100644
index 0000000000000..f44bd8fcf3079
--- /dev/null
+++ b/bolt/lib/Rewrite/MergeFunctionLayouts.cpp
@@ -0,0 +1,195 @@
+//===- bolt/Rewrite/MergeFunctionLayouts.cpp - Merge two function layouts -===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// This file implements mergeFunctionLayouts() for llvm-bolt-align.
+//
+//===----------------------------------------------------------------------===//
+
+#include "bolt/Rewrite/MergeFunctionLayouts.h"
+#include "bolt/Utils/Utils.h"
+#include "llvm/ADT/StringMap.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/LineIterator.h"
+#include "llvm/Support/MathExtras.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/raw_ostream.h"
+#include <algorithm>
+#include <limits>
+#include <vector>
+
+using namespace llvm;
+using namespace bolt;
+
+namespace {
+
+struct Entry {
+ std::string Name;
+ uint64_t Offset;
+};
+
+using Match = std::pair<const Entry *, const Entry *>;
+
+/// Keep some spare space between functions. This absorbs small layout changes
+/// in the final rewrite.
+constexpr uint64_t LayoutSlack = 64;
+constexpr uint64_t LayoutAlignment = 64;
+
+static Expected<std::vector<Entry>> parseFile(StringRef Path) {
+ ErrorOr<std::unique_ptr<MemoryBuffer>> MB = MemoryBuffer::getFile(Path);
+ if (std::error_code EC = MB.getError())
+ return createStringError(EC, Twine("cannot open layout file '") + Path +
+ "': " + EC.message());
+
+ std::vector<Entry> Entries;
+ for (line_iterator LI(*MB.get(), /*SkipBlanks=*/true, /*CommentMarker=*/'#');
+ !LI.is_at_eof(); ++LI) {
+ StringRef Line = LI->trim();
+ if (Line.empty())
+ continue;
+
+ StringRef Name, OffsetStr;
+ std::tie(Name, OffsetStr) = Line.split(' ');
+ Name = Name.trim();
+ OffsetStr = OffsetStr.trim();
+
+ uint64_t Offset;
+ if (Name.empty() || OffsetStr.empty() ||
+ OffsetStr.getAsInteger(/*Radix=*/0, Offset))
+ return createStringError(inconvertibleErrorCode(),
+ Twine("malformed entry at ") + Path + ":" +
+ Twine(LI.line_number()));
+
+ Entries.push_back({Name.str(), Offset});
+ }
+ return Entries;
+}
+
+/// Return all entries in both \p EntriesA and \p EntriesB in A's order.
+static std::vector<Match> findMatches(ArrayRef<Entry> EntriesA,
+ ArrayRef<Entry> EntriesB) {
+ StringMap<const Entry *> ExactB;
+ StringMap<const Entry *> CommonB;
+ StringMap<unsigned> CommonCountA;
+ StringMap<unsigned> CommonCountB;
+
+ for (const Entry &E : EntriesA)
+ if (std::optional<StringRef> Common = getLTOCommonName(E.Name))
+ ++CommonCountA[*Common];
+
+ for (const Entry &E : EntriesB) {
+ ExactB[E.Name] = &E;
+ if (std::optional<StringRef> Common = getLTOCommonName(E.Name)) {
+ ++CommonCountB[*Common];
+ CommonB[*Common] = &E;
+ }
+ }
+
+ std::vector<Match> Matches;
+ for (const Entry &E : EntriesA) {
+ const Entry *Match = ExactB.lookup(E.Name);
+ if (!Match) {
+ std::optional<StringRef> Common = getLTOCommonName(E.Name);
+ if (Common && CommonCountA.lookup(*Common) == 1 &&
+ CommonCountB.lookup(*Common) == 1)
+ Match = CommonB.lookup(*Common);
+ }
+ if (Match)
+ Matches.emplace_back(&E, Match);
+ }
+ return Matches;
+}
+
+static std::vector<Match> findLongestIncreasingSubsequence(ArrayRef<Match> Matches) {
+ if (Matches.empty())
+ return {};
+
+ const size_t NoIndex = std::numeric_limits<size_t>::max();
+ std::vector<size_t> Tails;
+ std::vector<size_t> Previous(Matches.size(), NoIndex);
+
+ for (size_t I = 0; I != Matches.size(); ++I) {
+ const uint64_t Offset = Matches[I].second->Offset;
+ auto It = std::lower_bound(Tails.begin(), Tails.end(), Offset,
+ [&](size_t Index, uint64_t Key) {
+ return Matches[Index].second->Offset < Key;
+ });
+ if (It != Tails.begin())
+ Previous[I] = It[-1];
+
+ if (It == Tails.end())
+ Tails.push_back(I);
+ else
+ *It = I;
+ }
+
+ std::vector<Match> Result;
+ for (size_t I = Tails.back(); I != NoIndex; I = Previous[I])
+ Result.push_back(Matches[I]);
+ std::reverse(Result.begin(), Result.end());
+ return Result;
+}
+
+} // namespace
+
+Error bolt::mergeFunctionLayouts(StringRef PathA,
+ StringRef PathB,
+ StringRef OutputPath,
+ raw_ostream &Log) {
+ Expected<std::vector<Entry>> EntriesA = parseFile(PathA);
+ if (!EntriesA)
+ return EntriesA.takeError();
+
+ Expected<std::vector<Entry>> EntriesB = parseFile(PathB);
+ if (!EntriesB)
+ return EntriesB.takeError();
+
+ const std::vector<Match> Matches =
+ findLongestIncreasingSubsequence(findMatches(*EntriesA, *EntriesB));
+
+ std::error_code EC;
+ raw_fd_ostream OS(OutputPath, EC, sys::fs::OpenFlags::OF_None);
+ if (EC)
+ return createStringError(EC, Twine("cannot open output layout file '") +
+ OutputPath + "': " + EC.message());
+
+ uint64_t Matched = 0;
+ bool First = true;
+ uint64_t PrevMergedOff = 0, PrevAOff = 0, PrevBOff = 0;
+ for (const Match &MatchPair : Matches) {
+ const Entry &A = *MatchPair.first;
+ const Entry &B = *MatchPair.second;
+ const uint64_t AOff = A.Offset;
+ const uint64_t BOff = B.Offset;
+
+ uint64_t MergedOff;
+ if (First) {
+ MergedOff = alignTo(std::max(AOff, BOff), LayoutAlignment);
+ } else {
+ const uint64_t RequiredGap = std::max(AOff - PrevAOff, BOff - PrevBOff);
+ MergedOff = alignTo(PrevMergedOff + RequiredGap + LayoutSlack, LayoutAlignment);
+ }
+
+ OS << A.Name << " 0x" << Twine::utohexstr(MergedOff) << "\n";
+ if (B.Name != A.Name)
+ OS << B.Name << " 0x" << Twine::utohexstr(MergedOff) << "\n";
+
+ ++Matched;
+ PrevMergedOff = MergedOff;
+ PrevAOff = AOff;
+ PrevBOff = BOff;
+ First = false;
+ }
+
+ const uint64_t PossibleMatches = std::min(EntriesA->size(), EntriesB->size());
+ const uint64_t MatchedPercent =
+ PossibleMatches ? Matched * 100 / PossibleMatches : 0;
+ Log << "BOLT-ALIGN: pinned " << Matched << " functions (" << MatchedPercent
+ << "%)\n";
+
+ return Error::success();
+}
diff --git a/bolt/lib/Rewrite/RewriteInstance.cpp b/bolt/lib/Rewrite/RewriteInstance.cpp
index a12b83859669b..23a268da71d52 100644
--- a/bolt/lib/Rewrite/RewriteInstance.cpp
+++ b/bolt/lib/Rewrite/RewriteInstance.cpp
@@ -17,6 +17,7 @@
#include "bolt/Core/MCPlusBuilder.h"
#include "bolt/Core/ParallelUtilities.h"
#include "bolt/Core/Relocation.h"
+#include "bolt/Passes/AssignDesiredFunctionOffset.h"
#include "bolt/Passes/BinaryPasses.h"
#include "bolt/Passes/CacheMetrics.h"
#include "bolt/Passes/IdenticalCodeFolding.h"
@@ -173,6 +174,12 @@ static cl::opt...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/210634
More information about the llvm-commits
mailing list