[llvm] [DWARFLinker] Make ODR uniquing deterministic (PR #194777)
Jonas Devlieghere via llvm-commits
llvm-commits at lists.llvm.org
Tue Apr 28 21:00:35 PDT 2026
https://github.com/JDevlieghere created https://github.com/llvm/llvm-project/pull/194777
The parallel DWARF linker deduplicates types across compile units using a shared TypePool. When multiple CUs define the same type, allocateTypeDie uses compare_exchange_strong to race for setting the canonical DIE. The first thread to succeed stores the DIE and clones its attributes, while subsequent threads use it the canonical one. Which thread wins depends on OS thread scheduling, making the output non-deterministic.
This PR fixes the non-determinism by assigning each CompileUnit a priority based on its position in the link order (object file index, CU index within the file). When a CU wants to mark DIE as canonical, it acquires the spinlock, and only stores its DIE if its priority is strictly lower than the current canonical DIE. This ensures that the canonical DIE is always the lowest-priority (i.e. first) CU that defines that type. The replaced DIE is leaked into the bump allocator and the existing DebugTypeDeclFilePatch and accelerator record filters skips the orphaned DIEs via getFinalDie() checks.
This PR also removes the AllowNonDeterministicOutput option, which was never set in the first place, and is now obsolete.
>From 5006b85993b1cf98a248c8cfc6b9eed5aed87ea9 Mon Sep 17 00:00:00 2001
From: Jonas Devlieghere <jonas at devlieghere.com>
Date: Tue, 28 Apr 2026 20:46:28 -0700
Subject: [PATCH] [DWARFLinker] Make ODR uniquing deterministic
The parallel DWARF linker deduplicates types across compile units using
a shared TypePool. When multiple CUs define the same type, allocateTypeDie
uses compare_exchange_strong to race for the type slot. The first thread
to succeed stores its DIE and clones its attributes, while subsequent
threads see the slot is taken and skip. Which thread wins depends on OS
thread scheduling, making the output non-deterministic.
Fix this by assigning each CompileUnit a deterministic priority based on
its position in the link order (object file index, CU index within the
file). Each TypeEntryBody gets a spinlock and a priority field.
When a CU wants to claim a type slot, it acquires the spinlock, and only
stores its DIE if its priority is strictly lower than the current
winner's. Because priority is a total order over CUs, the final winner
for each type slot is always the lowest-priority CU that defines that
type. A higher-priority CU that won early is replaced when the same
type is encountered in a lower-priority CU. The replaced DIE is leaked
into the bump allocator. The existing DebugTypeDeclFilePatch and
accelerator record filters already skip orphaned DIEs via getFinalDie()
checks.
This PR also removes the AllowNonDeterministicOutput option, which was
never set in the first place, and is now obsolete.
---
.../llvm/DWARFLinker/Classic/DWARFLinker.h | 4 -
.../llvm/DWARFLinker/DWARFLinkerBase.h | 3 -
.../DWARFLinker/Parallel/DIEAttributeCloner.h | 8 +-
.../Parallel/DWARFLinkerCompileUnit.cpp | 68 +++++-----
.../Parallel/DWARFLinkerCompileUnit.h | 13 ++
.../Parallel/DWARFLinkerGlobalData.h | 3 -
.../DWARFLinker/Parallel/DWARFLinkerImpl.cpp | 14 ++-
.../DWARFLinker/Parallel/DWARFLinkerImpl.h | 13 +-
.../Parallel/DWARFLinkerTypeUnit.cpp | 115 ++++++++---------
llvm/lib/DWARFLinker/Parallel/TypePool.h | 11 ++
.../odr-deterministic-type-die.test | 118 ++++++++++++++++++
.../dsymutil/X86/Inputs/odr-determinism/a.o | Bin 0 -> 2464 bytes
.../dsymutil/X86/Inputs/odr-determinism/b.o | Bin 0 -> 2464 bytes
13 files changed, 246 insertions(+), 124 deletions(-)
create mode 100644 llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/odr-deterministic-type-die.test
create mode 100644 llvm/test/tools/dsymutil/X86/Inputs/odr-determinism/a.o
create mode 100644 llvm/test/tools/dsymutil/X86/Inputs/odr-determinism/b.o
diff --git a/llvm/include/llvm/DWARFLinker/Classic/DWARFLinker.h b/llvm/include/llvm/DWARFLinker/Classic/DWARFLinker.h
index 23ce937d20b5b..81f4545dfbf4c 100644
--- a/llvm/include/llvm/DWARFLinker/Classic/DWARFLinker.h
+++ b/llvm/include/llvm/DWARFLinker/Classic/DWARFLinker.h
@@ -267,10 +267,6 @@ class LLVM_ABI DWARFLinker : public DWARFLinkerBase {
Options.Update = Update;
}
- /// Allow generating valid, but non-deterministic output.
- void setAllowNonDeterministicOutput(bool) override { /* Nothing to do. */
- }
-
/// Set whether to keep the enclosing function for a static variable.
void setKeepFunctionForStatic(bool KeepFunctionForStatic) override {
Options.KeepFunctionForStatic = KeepFunctionForStatic;
diff --git a/llvm/include/llvm/DWARFLinker/DWARFLinkerBase.h b/llvm/include/llvm/DWARFLinker/DWARFLinkerBase.h
index 99aeb0df076e6..30290cc77332b 100644
--- a/llvm/include/llvm/DWARFLinker/DWARFLinkerBase.h
+++ b/llvm/include/llvm/DWARFLinker/DWARFLinkerBase.h
@@ -119,9 +119,6 @@ class DWARFLinkerBase {
virtual void setNoODR(bool NoODR) = 0;
/// Update index tables only (do not modify rest of DWARF).
virtual void setUpdateIndexTablesOnly(bool Update) = 0;
- /// Allows generating non-deterministic output in exchange for more
- /// parallelism.
- virtual void setAllowNonDeterministicOutput(bool) = 0;
/// Set whether to keep the enclosing function for a static variable.
virtual void setKeepFunctionForStatic(bool KeepFunctionForStatic) = 0;
/// Use specified number of threads for parallel files linking.
diff --git a/llvm/lib/DWARFLinker/Parallel/DIEAttributeCloner.h b/llvm/lib/DWARFLinker/Parallel/DIEAttributeCloner.h
index 6a6bd08570d7d..10d809cc4ebe8 100644
--- a/llvm/lib/DWARFLinker/Parallel/DIEAttributeCloner.h
+++ b/llvm/lib/DWARFLinker/Parallel/DIEAttributeCloner.h
@@ -99,14 +99,12 @@ class DIEAttributeCloner {
InputDIEIdx = InUnit.getDIEIndex(InputDieEntry);
// Use DW_FORM_strp form for string attributes for DWARF version less than 5
- // or if output unit is type unit and we need to produce deterministic
- // result. (We can not generate deterministic results for debug_str_offsets
- // section when attributes are cloned parallelly).
+ // or if output unit is type unit and attributes are cloned in parallel
+ // (debug_str_offsets ordering would be non-deterministic otherwise).
Use_DW_FORM_strp =
(InUnit.getVersion() < 5) ||
(OutUnit.isTypeUnit() &&
- ((InUnit.getGlobalData().getOptions().Threads != 1) &&
- !InUnit.getGlobalData().getOptions().AllowNonDeterministicOutput));
+ InUnit.getGlobalData().getOptions().Threads != 1);
}
/// Clone string attribute.
diff --git a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp
index 8d065bf793f35..137787959bfd8 100644
--- a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp
+++ b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.cpp
@@ -1419,46 +1419,42 @@ DIE *CompileUnit::allocateTypeDie(TypeEntryBody *TypeDescriptor,
DIEGenerator &TypeDIEGenerator,
dwarf::Tag DieTag, bool IsDeclaration,
bool IsParentDeclaration) {
- DIE *DefinitionDie = TypeDescriptor->Die;
- // Do not allocate any new DIE if definition DIE is already met.
- if (DefinitionDie)
- return nullptr;
-
- DIE *DeclarationDie = TypeDescriptor->DeclarationDie;
- bool OldParentIsDeclaration = TypeDescriptor->ParentIsDeclaration;
-
- if (IsDeclaration && !DeclarationDie) {
- // Alocate declaration DIE.
- DIE *NewDie = TypeDIEGenerator.createDIE(DieTag, 0);
- if (TypeDescriptor->DeclarationDie.compare_exchange_strong(DeclarationDie,
- NewDie))
- return NewDie;
- } else if (IsDeclaration && !IsParentDeclaration && OldParentIsDeclaration) {
- // Overwrite existing declaration DIE if it's parent is also an declaration
- // while parent of current declaration DIE is a definition.
- if (TypeDescriptor->ParentIsDeclaration.compare_exchange_strong(
- OldParentIsDeclaration, false)) {
- DIE *NewDie = TypeDIEGenerator.createDIE(DieTag, 0);
- TypeDescriptor->DeclarationDie = NewDie;
- return NewDie;
- }
- } else if (!IsDeclaration && IsParentDeclaration && !DeclarationDie) {
- // Alocate declaration DIE since parent of current DIE is marked as
- // declaration.
- DIE *NewDie = TypeDIEGenerator.createDIE(DieTag, 0);
- if (TypeDescriptor->DeclarationDie.compare_exchange_strong(DeclarationDie,
- NewDie))
- return NewDie;
- } else if (!IsDeclaration && !IsParentDeclaration) {
- // Allocate definition DIE.
- DIE *NewDie = TypeDIEGenerator.createDIE(DieTag, 0);
- if (TypeDescriptor->Die.compare_exchange_strong(DefinitionDie, NewDie)) {
+ // Use a per-type spinlock and CU priority to ensure the earliest CU in link
+ // order always wins the type slot, producing deterministic output regardless
+ // of thread scheduling.
+ unsigned Priority = getDeterministicPriority();
+
+ // Speculatively allocate the DIE outside the lock so the critical section
+ // only covers the priority check and pointer store. If we lose, the DIE
+ // is leaked into the bump allocator.
+ DIE *NewDie = TypeDIEGenerator.createDIE(DieTag, 0);
+
+ while (TypeDescriptor->Lock.test_and_set(std::memory_order_acquire))
+ ; // spin
+
+ DIE *Result = nullptr;
+
+ if (!IsDeclaration && !IsParentDeclaration) {
+ // Definition: lowest priority wins.
+ if (Priority < TypeDescriptor->DiePriority) {
+ TypeDescriptor->DiePriority = Priority;
+ TypeDescriptor->Die = NewDie;
TypeDescriptor->ParentIsDeclaration = false;
- return NewDie;
+ Result = NewDie;
+ }
+ } else if (!TypeDescriptor->Die.load()) {
+ // Declaration (no definition exists yet): lowest priority wins.
+ if (Priority < TypeDescriptor->DeclarationDiePriority) {
+ TypeDescriptor->DeclarationDiePriority = Priority;
+ TypeDescriptor->DeclarationDie = NewDie;
+ if (!IsParentDeclaration)
+ TypeDescriptor->ParentIsDeclaration = false;
+ Result = NewDie;
}
}
- return nullptr;
+ TypeDescriptor->Lock.clear(std::memory_order_release);
+ return Result;
}
TypeEntry *CompileUnit::createTypeDIEandCloneAttributes(
diff --git a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.h b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.h
index 90c007ebe5f6c..b8344ec745e82 100644
--- a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.h
+++ b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerCompileUnit.h
@@ -11,6 +11,7 @@
#include "DWARFLinkerUnit.h"
#include "llvm/DWARFLinker/DWARFFile.h"
+#include <limits>
#include <optional>
namespace llvm {
@@ -110,6 +111,15 @@ class alignas(8) CompileUnit : public DwarfUnit {
/// Returns DWARFFile containing this compile unit.
const DWARFFile &getContaingFile() const { return File; }
+ /// Set deterministic priority for type DIE allocation ordering.
+ /// Lower priority values win when multiple CUs race to define the same type.
+ void setDeterministicPriority(unsigned ObjFileIdx, unsigned LocalIdx) {
+ assert(ObjFileIdx < (1u << 16) && LocalIdx < (1u << 16) &&
+ "priority encoding overflow");
+ DeterministicPriority = (ObjFileIdx << 16) | LocalIdx;
+ }
+ unsigned getDeterministicPriority() const { return DeterministicPriority; }
+
/// Load DIEs of input compilation unit. \returns true if input DIEs
/// successfully loaded.
bool loadInputDIEs();
@@ -702,6 +712,9 @@ class alignas(8) CompileUnit : public DwarfUnit {
/// Flag indicating whether type de-duplication is forbidden.
bool NoODR = true;
+ /// Deterministic priority for type DIE allocation (lower wins).
+ unsigned DeterministicPriority = std::numeric_limits<unsigned>::max();
+
/// The ranges in that map are the PC ranges for functions in this unit,
/// associated with the PC offset to apply to the addresses to get
/// the linked address.
diff --git a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerGlobalData.h b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerGlobalData.h
index 7ca81eb34f005..b47dd5ee58435 100644
--- a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerGlobalData.h
+++ b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerGlobalData.h
@@ -48,9 +48,6 @@ struct DWARFLinkerOptions {
/// function.
bool KeepFunctionForStatic = false;
- /// Allow to generate valid, but non deterministic output.
- bool AllowNonDeterministicOutput = false;
-
/// Number of threads.
unsigned Threads = 1;
diff --git a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp
index e9ee3ce1b8b27..4bdb65b3b0bd3 100644
--- a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp
+++ b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.cpp
@@ -27,11 +27,12 @@ DWARFLinkerImpl::DWARFLinkerImpl(MessageHandlerTy ErrorHandler,
}
DWARFLinkerImpl::LinkContext::LinkContext(LinkingGlobalData &GlobalData,
- DWARFFile &File,
+ DWARFFile &File, unsigned ObjFileIdx,
StringMap<uint64_t> &ClangModules,
std::atomic<size_t> &UniqueUnitID)
: OutputSections(GlobalData), InputDWARFFile(File),
- ClangModules(ClangModules), UniqueUnitID(UniqueUnitID) {
+ ObjectFileIdx(ObjFileIdx), ClangModules(ClangModules),
+ UniqueUnitID(UniqueUnitID) {
if (File.Dwarf) {
if (!File.Dwarf->compile_units().empty())
@@ -61,7 +62,7 @@ void DWARFLinkerImpl::LinkContext::addModulesCompileUnit(
void DWARFLinkerImpl::addObjectFile(DWARFFile &File, ObjFileLoaderTy Loader,
CompileUnitHandlerTy OnCUDieLoaded) {
ObjectContexts.emplace_back(std::make_unique<LinkContext>(
- GlobalData, File, ClangModules, UniqueUnitID));
+ GlobalData, File, ObjectContexts.size(), ClangModules, UniqueUnitID));
if (ObjectContexts.back()->InputDWARFFile.Dwarf) {
for (const std::unique_ptr<DWARFUnit> &CU :
@@ -452,6 +453,11 @@ Error DWARFLinkerImpl::LinkContext::link(TypeUnit *ArtificialTypeUnit) {
InputDWARFFile.Dwarf->getDebugMacinfo();
InputDWARFFile.Dwarf->getDebugMacro();
+ // Assign deterministic priorities to module CUs for type DIE allocation.
+ unsigned LocalCUIdx = 0;
+ for (auto &Mod : ModulesCompileUnits)
+ Mod.Unit->setDeterministicPriority(ObjectFileIdx, LocalCUIdx++);
+
// Link modules compile units first.
parallelForEach(ModulesCompileUnits, [&](RefModuleUnit &RefModule) {
linkSingleCompileUnit(*RefModule.Unit, ArtificialTypeUnit);
@@ -483,6 +489,8 @@ Error DWARFLinkerImpl::LinkContext::link(TypeUnit *ArtificialTypeUnit) {
CompileUnits.emplace_back(std::make_unique<CompileUnit>(
GlobalData, *OrigCU, UniqueUnitID.fetch_add(1), "", InputDWARFFile,
getUnitForOffset, OrigCU->getFormParams(), getEndianness()));
+ CompileUnits.back()->setDeterministicPriority(ObjectFileIdx,
+ LocalCUIdx++);
// Preload line table, as it can't be loaded asynchronously.
CompileUnits.back()->loadLineTable();
diff --git a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.h b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.h
index 597bb1b4da59a..65a553f1f3be6 100644
--- a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.h
+++ b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerImpl.h
@@ -78,13 +78,6 @@ class DWARFLinkerImpl : public DWARFLinker {
GlobalData.Options.UpdateIndexTablesOnly = UpdateIndexTablesOnly;
}
- /// Allow generating valid, but non-deterministic output.
- void
- setAllowNonDeterministicOutput(bool AllowNonDeterministicOutput) override {
- GlobalData.Options.AllowNonDeterministicOutput =
- AllowNonDeterministicOutput;
- }
-
/// Set to keep the enclosing function for a static variable.
void setKeepFunctionForStatic(bool KeepFunctionForStatic) override {
GlobalData.Options.KeepFunctionForStatic = KeepFunctionForStatic;
@@ -182,6 +175,10 @@ class DWARFLinkerImpl : public DWARFLinker {
/// Set of Compile Units for modules.
ModuleUnitListTy ModulesCompileUnits;
+ /// Index of this object file in the link order (used for deterministic
+ /// type DIE allocation).
+ unsigned ObjectFileIdx = 0;
+
/// Size of Debug info before optimizing.
uint64_t OriginalDebugInfoSize = 0;
@@ -202,7 +199,7 @@ class DWARFLinkerImpl : public DWARFLinker {
std::atomic<size_t> &UniqueUnitID;
LinkContext(LinkingGlobalData &GlobalData, DWARFFile &File,
- StringMap<uint64_t> &ClangModules,
+ unsigned ObjFileIdx, StringMap<uint64_t> &ClangModules,
std::atomic<size_t> &UniqueUnitID);
/// Check whether specified \p CUDie is a Clang module reference.
diff --git a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerTypeUnit.cpp b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerTypeUnit.cpp
index 07bbde70c77c4..1b0d9bac4c4f3 100644
--- a/llvm/lib/DWARFLinker/Parallel/DWARFLinkerTypeUnit.cpp
+++ b/llvm/lib/DWARFLinker/Parallel/DWARFLinkerTypeUnit.cpp
@@ -132,31 +132,26 @@ void TypeUnit::prepareDataForTreeCreation() {
getOrCreateSectionDescriptor(DebugSectionKind::DebugInfo);
// Type unit data created parallelly. So the order of data is not
- // deterministic. Order data here if we need deterministic output.
+ // deterministic. Sort data here to produce deterministic output.
llvm::parallel::TaskGroup TG;
- if (!GlobalData.getOptions().AllowNonDeterministicOutput) {
- TG.spawn([&]() {
- // Sort types to have a deterministic output.
- Types.sortTypes();
- });
- }
+ TG.spawn([&]() {
+ // Sort types to have a deterministic output.
+ Types.sortTypes();
+ });
TG.spawn([&]() {
- if (!GlobalData.getOptions().AllowNonDeterministicOutput) {
- // Sort decl type patches to have a deterministic output.
- std::function<bool(const DebugTypeDeclFilePatch &LHS,
- const DebugTypeDeclFilePatch &RHS)>
- PatchesComparator = [&](const DebugTypeDeclFilePatch &LHS,
- const DebugTypeDeclFilePatch &RHS) {
- return LHS.Directory->first() < RHS.Directory->first() ||
- (!(RHS.Directory->first() < LHS.Directory->first()) &&
- LHS.FilePath->first() < RHS.FilePath->first());
- };
- // Sort patches to have a deterministic output.
- DebugInfoSection.ListDebugTypeDeclFilePatch.sort(PatchesComparator);
- }
+ // Sort decl type patches to have a deterministic output.
+ std::function<bool(const DebugTypeDeclFilePatch &LHS,
+ const DebugTypeDeclFilePatch &RHS)>
+ PatchesComparator = [&](const DebugTypeDeclFilePatch &LHS,
+ const DebugTypeDeclFilePatch &RHS) {
+ return LHS.Directory->first() < RHS.Directory->first() ||
+ (!(RHS.Directory->first() < LHS.Directory->first()) &&
+ LHS.FilePath->first() < RHS.FilePath->first());
+ };
+ DebugInfoSection.ListDebugTypeDeclFilePatch.sort(PatchesComparator);
// Update DW_AT_decl_file attribute
dwarf::Form DeclFileForm =
@@ -189,51 +184,47 @@ void TypeUnit::prepareDataForTreeCreation() {
});
});
- if (!GlobalData.getOptions().AllowNonDeterministicOutput) {
- // Sort patches to have a deterministic output.
- TG.spawn([&]() {
- forEach([&](SectionDescriptor &OutSection) {
- std::function<bool(const DebugStrPatch &LHS, const DebugStrPatch &RHS)>
- StrPatchesComparator =
- [&](const DebugStrPatch &LHS, const DebugStrPatch &RHS) {
- return LHS.String->getKey() < RHS.String->getKey();
- };
- OutSection.ListDebugStrPatch.sort(StrPatchesComparator);
-
- std::function<bool(const DebugTypeStrPatch &LHS,
- const DebugTypeStrPatch &RHS)>
- TypeStrPatchesComparator = [&](const DebugTypeStrPatch &LHS,
- const DebugTypeStrPatch &RHS) {
- return LHS.String->getKey() < RHS.String->getKey();
- };
- OutSection.ListDebugTypeStrPatch.sort(TypeStrPatchesComparator);
- });
+ // Sort patches to have a deterministic output.
+ TG.spawn([&]() {
+ forEach([&](SectionDescriptor &OutSection) {
+ std::function<bool(const DebugStrPatch &LHS, const DebugStrPatch &RHS)>
+ StrPatchesComparator =
+ [&](const DebugStrPatch &LHS, const DebugStrPatch &RHS) {
+ return LHS.String->getKey() < RHS.String->getKey();
+ };
+ OutSection.ListDebugStrPatch.sort(StrPatchesComparator);
+
+ std::function<bool(const DebugTypeStrPatch &LHS,
+ const DebugTypeStrPatch &RHS)>
+ TypeStrPatchesComparator = [&](const DebugTypeStrPatch &LHS,
+ const DebugTypeStrPatch &RHS) {
+ return LHS.String->getKey() < RHS.String->getKey();
+ };
+ OutSection.ListDebugTypeStrPatch.sort(TypeStrPatchesComparator);
});
- }
+ });
- if (!GlobalData.getOptions().AllowNonDeterministicOutput) {
- // Sort patches to have a deterministic output.
- TG.spawn([&]() {
- forEach([&](SectionDescriptor &OutSection) {
- std::function<bool(const DebugLineStrPatch &LHS,
- const DebugLineStrPatch &RHS)>
- LineStrPatchesComparator = [&](const DebugLineStrPatch &LHS,
- const DebugLineStrPatch &RHS) {
- return LHS.String->getKey() < RHS.String->getKey();
- };
- OutSection.ListDebugLineStrPatch.sort(LineStrPatchesComparator);
-
- std::function<bool(const DebugTypeLineStrPatch &LHS,
- const DebugTypeLineStrPatch &RHS)>
- TypeLineStrPatchesComparator =
- [&](const DebugTypeLineStrPatch &LHS,
- const DebugTypeLineStrPatch &RHS) {
- return LHS.String->getKey() < RHS.String->getKey();
- };
- OutSection.ListDebugTypeLineStrPatch.sort(TypeLineStrPatchesComparator);
- });
+ // Sort patches to have a deterministic output.
+ TG.spawn([&]() {
+ forEach([&](SectionDescriptor &OutSection) {
+ std::function<bool(const DebugLineStrPatch &LHS,
+ const DebugLineStrPatch &RHS)>
+ LineStrPatchesComparator = [&](const DebugLineStrPatch &LHS,
+ const DebugLineStrPatch &RHS) {
+ return LHS.String->getKey() < RHS.String->getKey();
+ };
+ OutSection.ListDebugLineStrPatch.sort(LineStrPatchesComparator);
+
+ std::function<bool(const DebugTypeLineStrPatch &LHS,
+ const DebugTypeLineStrPatch &RHS)>
+ TypeLineStrPatchesComparator =
+ [&](const DebugTypeLineStrPatch &LHS,
+ const DebugTypeLineStrPatch &RHS) {
+ return LHS.String->getKey() < RHS.String->getKey();
+ };
+ OutSection.ListDebugTypeLineStrPatch.sort(TypeLineStrPatchesComparator);
});
- }
+ });
}
uint64_t TypeUnit::finalizeTypeEntryRec(uint64_t OutOffset, DIE *OutDIE,
diff --git a/llvm/lib/DWARFLinker/Parallel/TypePool.h b/llvm/lib/DWARFLinker/Parallel/TypePool.h
index 15a6428e1b8d6..5b8fd39df778b 100644
--- a/llvm/lib/DWARFLinker/Parallel/TypePool.h
+++ b/llvm/lib/DWARFLinker/Parallel/TypePool.h
@@ -15,6 +15,7 @@
#include "llvm/CodeGen/DIE.h"
#include "llvm/Support/Allocator.h"
#include <atomic>
+#include <limits>
namespace llvm {
namespace dwarf_linker {
@@ -64,6 +65,16 @@ class TypeEntryBody {
// True if parent type die is declaration.
std::atomic<bool> ParentIsDeclaration = {true};
+ // Priority of the CU that set Die (lower wins, for deterministic output).
+ std::atomic<unsigned> DiePriority = {std::numeric_limits<unsigned>::max()};
+
+ // Priority of the CU that set DeclarationDie (lower wins).
+ std::atomic<unsigned> DeclarationDiePriority = {
+ std::numeric_limits<unsigned>::max()};
+
+ // Spinlock for deterministic type DIE allocation.
+ std::atomic_flag Lock = {};
+
/// Children for current type.
ArrayList<TypeEntry *, 5> Children;
diff --git a/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/odr-deterministic-type-die.test b/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/odr-deterministic-type-die.test
new file mode 100644
index 0000000000000..370edd94f04d9
--- /dev/null
+++ b/llvm/test/tools/dsymutil/X86/DWARFLinkerParallel/odr-deterministic-type-die.test
@@ -0,0 +1,118 @@
+# RUN: dsymutil --linker=parallel -f -o %t1.o -oso-prepend-path=%p/../ \
+# RUN: -y %s --num-threads 4
+# RUN: dsymutil --linker=parallel -f -o %t2.o -oso-prepend-path=%p/../ \
+# RUN: -y %s --num-threads 4
+# RUN: dsymutil --linker=parallel -f -o %t3.o -oso-prepend-path=%p/../ \
+# RUN: -y %s --num-threads 4
+# RUN: diff %t1.o %t2.o
+# RUN: diff %t1.o %t3.o
+
+# This test checks that the parallel DWARF linker's type DIE allocation is
+# deterministic. The two .o files define the same C++ type (MyType) but were
+# compiled from different directories, giving them different DW_AT_decl_file
+# paths. Without deterministic type DIE allocation, the output varies between
+# runs because which CU "wins" the type slot depends on thread scheduling.
+#
+# The debug map duplicates each object file 16 times to maximize parallel
+# contention.
+#
+# Compiled from:
+#
+# include/types.h:
+# struct MyType { int x; int y; int getSum() { return x + y; } };
+#
+# dir_a/a.cpp:
+# #include "types.h"
+# int funcA(MyType *t) { return t->getSum(); }
+#
+# dir_b/b.cpp:
+# #include "types.h"
+# int funcB(MyType *t) { return t->getSum() + 1; }
+#
+# cd dir_a && clang++ -g -O0 -c -I../include a.cpp -o a.o
+# cd dir_b && clang++ -g -O0 -c -I../include b.cpp -o b.o
+
+---
+triple: 'x86_64-apple-darwin'
+objects:
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000000, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000020, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000100, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000120, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000200, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000220, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000300, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000320, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000400, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000420, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000500, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000520, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000600, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000620, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000700, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000720, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000800, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000820, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000900, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000920, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000A00, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000A20, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000B00, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000B20, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000C00, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000C20, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000D00, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000D20, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/a.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcAP6MyType, objAddr: 0x0, binAddr: 0x100000E00, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000E20, size: 0x20 }
+ - filename: 'Inputs/odr-determinism/b.o'
+ timestamp: 0
+ symbols:
+ - { sym: __Z5funcBP6MyType, objAddr: 0x0, binAddr: 0x100000F00, size: 0x20 }
+ - { sym: __ZN6MyType6getSumEv, objAddr: 0x20, binAddr: 0x100000F20, size: 0x20 }
+...
diff --git a/llvm/test/tools/dsymutil/X86/Inputs/odr-determinism/a.o b/llvm/test/tools/dsymutil/X86/Inputs/odr-determinism/a.o
new file mode 100644
index 0000000000000000000000000000000000000000..a5844199e9aea1a51ac9df12d6e29aa4a4ee6ce9
GIT binary patch
literal 2464
zcma)7OKclO82)G1>p0mYc49=51~qP#72%MzQ|DPCa70QHnxN2z5+PNKW3OW)XYJZ+
z<5Z~>2m(StqJo}KC4 at Lss6reO2ZU4}6)KNIC61gx9EwyS9-%zS|If}kiQ@`>>6`yK
z-;QVg8Gn82r+*p%m<%`|Jcv;mLI{*D#0QkTgM2}?D2qb7K5_y<r6h`B*k;9cY{y?X
zwz#n4yH->f#7C5Ujn2BzS$Atl8%EMxDlHrFrKN&dCV%pYi4zZ3OI)7IBC;jcS1kaL
zcG*?FjFrk!yq51fQG8zL5k(j~sxDvAE*QB~s#wdHr_Dx*@e$za|L5Z~YWdz5eAfh@
z&nt`_Rd;>yWU at ddwR~%$x>L-*F8CsN9wJqj&x)^@#X5X%i0W?y-wov3Lr~)yMk2S8
zk0)%SWIdCyl7=xmDW-Cj7Cjjg!H==oC=|Qn`pmSED&Xpz35Kz^|1Jr>wBQ4u!{@r#
z$JG?Y-2TgEEY4WFe*w{NuL^=R@{!ED{k-q at I{i$11~pAhKVKAl{SFQMD)@v;YR`5a
z!2q3|(-sFP&xML8z9{yog1QfVx~KDq-9oi{NOvdy!+v~v^ZV(I8-eM~bKBEf=eB>O
z`ss~N0;jL|d+;QAp6BMax0pQPIemq-F-;39vi97XR_F+HbZc$h?cQPsE<$Hy#yTxc
z){d(8t24m|@nMn`kUHG}J{LhYtje0IQU!q00<ty}Bxe*|ug-R6k9Q>x0Z}w4FcTa^
z!`F+|52z|uHu^CJ(BytJ8Jt2_bw1`)10AaZj2{AnL%<~WJY=A!hdc?oab%~ORFKkW
z$<p~sGe}!#f=6leF{XGZ&LmL0+>kxq1|Q?WhAfqVPf<LLF*KSOt^vuO#c>g#B3TQk
z;%OMinEQkfmxah9-bTEGxP*8Y at e$%p#LI~15uYMHM<lXwYdKUl3&l*%3LS~+5k0b;
zu|s`0#1QGyl8!g>Xf|733DeM at FXWy!6ShuOsmNGjEIK+iG%#eMX^ut*qJzU`WCV3`
zX&^Z^I6N|zibwB*xSq)8Aq-)AB_GZu3;4HCw8P0v!H7euWF;oRcyfpv^TTti3#)k(
zgsMQsvSAhTkBZFjvT2_ztsE<ZV}K2II#YzZ at b@VE2+%+Y at ezRDM%p9sRPrc2YXthA
z^C4zuPt8R}A`vvJb#(P;o_GS`=WAvER~IhdI&ksD+mZE`)GuGZ*?P0l`xfmd?JMmu
z?K$l;?Kug}ZAP>s=>8o<FkKFM at WWECqvfEZf%Rh69nX->8wlhW*^EcX+9lL#D7$|e
z4}|y#K;46~FHd_#!p@!mRtc~`bA#_StfCd%gBbnmHW$GUFwq)#L3*gg-3zLlL_|cL
z_|Cq%O{}3ap*Myx_GfDgDOwH!jUN8c&}w9~{mMb92W|N6_uyyR at mKO^e%Y9sGIJ+2
zny_~E+rQPyy?<Q2{=;M0r^<sKytCDJDyZ}{eW)DvKB9E0iyo$c_dTTxBi&wIX;;T3
z3BOja`E)&;u at c!*(iA;vWA7Dpx=SQ&2wZ7 at z|mgn74*F!p;2)^F7(}~{aD}%nSQno
OAva*u>thVoH2=S_2t=a*
literal 0
HcmV?d00001
diff --git a/llvm/test/tools/dsymutil/X86/Inputs/odr-determinism/b.o b/llvm/test/tools/dsymutil/X86/Inputs/odr-determinism/b.o
new file mode 100644
index 0000000000000000000000000000000000000000..0c00adae9648d359f4660aa03179f9b26bfb05c1
GIT binary patch
literal 2464
zcma)7OKclO82)G1yK%C4I58qG)VL at XsZeaE&a*_|Kw1)%pwNb5AytbX>)6OyyY|{R
zRRV#OM+r!&;D(S2;!ptxj)()QR1iJ1Du+rOIR$Yjs1ld(EdM_{n>dau_@!_D$Ncln
z?94y2H~;wMp9TOX12za3Vw at Tu0;L=AC3W6IZcr{tRA|RXP9SL1iK6S4QL${>_7{&w
z7x#R}3JZhyh;pFOUKcv+P7Y~ZPa5&kk{*l43r3mz>F1_Sp4d%scs>`AO|ib+1OTbZ
z-sQ`fsT{>?`F<417lj^Egt4dU@)fOuo=c^QwS0NnY?KIZ0j~XjK0czB?+d|qUGTY8
zVeF~8>x(6m1xl&qTNT+|V*D+^7sB%pU3K})*s at Wq!}q?({$B9iM!tOnHLk8Fa?ANx
z!qQ9TOBpk%>vPj$C|7AwlMxZTg2hIm$UWC*r1ew*S7#5<_5JmCS at 5L=AGmEk$Hh9{
z?V^}df7y)58Ef~?BkFBS5TuchWWlND1K(@cGx6zEG&S{nN$?HYG_WK1giC7Qb{@q5
zot<44J=D*Ik|-V(`&2<afIi*R1w^$_?mp7$&i}BVow=|*vvJ!qb75^~W^--lXUgBW
z!e%zU at tnDu+T#7ZZ+>T!$y2T~S6Msr`@M?ne{I!Y==5~<`rCUu)M6(tBEZOubeVqH
ze at uHsoAo}150k8b)a3;5x$v at 4P4;UVWdJBGAp2*%<cy=M2H4){$?oJ~Ac_VBX1&8`
zxcf2tQBA|l#x{%rG&z$dy-%X6*^}AX at GgFh0*oI5gTufi=R9Pfr-wWV4q?kqH)$ZH
z(UPT$m1dAO(FB*$=w?iDQJhJjsN9fUYJ;2cU_+MDz^y2*#t0fs4A+2UuVNcTXh_z=
zsdzU`V$69$h|5Ie5g#KyL0m?BiufAw0pe}MdBnGf?+}S>%v|!7jY2V#Gkr(Hflwf{
zl(BpR*hCQNQape+@>n)oUJg<-m at njBG!j;TvQnXm#6);}Vq|E<K+_lx4~2(EjnEkC
zWPB(&F+4gpk&1<n0J7xs5QLz$oDb%b1^g`(tza@!(BqIQnTf|ie}04;3#0QZiz|5p
zgsMQsv|t7MpB6o%ONMo-w0yh_wgDDc=}Zys!|ze}5ukzM<1GNajkHJNspL_5)(G^U
zb0g;FPS1zNLLoGH9 at RfR4RrlzEamom_~zZ;u9w?F>u+f{-n-L!r%}ChQ$3QSy`?>+
z{iZ#p{ib~;p>fTK4g}plUK#<}>%xhpep}04TLU|b8Nm9f<)s61gzV}#`Bg%EDAk|F
z13umYQ1zf3=%>9RVdqW)t9Y2Fxxu}KSv2<pi1EMfauNIhQ>~siq$gVFdq^m68W9qC
zVx9we?_ds{3B56lu|Hc|NYQc-X!P-ihE^k^-KHFp`p|~$ZRONw;n|*-&%XZA+OMy~
z{4`+o+z)?im4|-6cI&5SvM-c}U3h1A-zl%s*YuV0p!$^Za8&8iCS6Pc^}N!J!Cp0>
zbZC>3gcH`Qzd#_EF%#KR(hy~8VfhsUbe*7`CI%Sv*jlApL0|O=jf(p*q3=iCh6#?4
S*~Zqv=LGb6b&SE9;{O+_3`H#f
literal 0
HcmV?d00001
More information about the llvm-commits
mailing list