[llvm] [TSan] Add dominance-based redundant instrumentation elimination (PR #169897)
Alexey Paznikov via llvm-commits
llvm-commits at lists.llvm.org
Tue Jun 30 05:07:46 PDT 2026
https://github.com/apaznikov updated https://github.com/llvm/llvm-project/pull/169897
>From 54e36304d0dbd22b2bac4d1b9f02e8141f7ead03 Mon Sep 17 00:00:00 2001
From: Alexey Paznikov <apaznikov at gmail.com>
Date: Tue, 30 Jun 2026 20:03:46 +0800
Subject: [PATCH] [TSan] Add dominance-based redundant instrumentation
elimination
Adds DominanceBasedElimination, a new optimization that removes redundant
TSan instrumentation when one memory access dominates another to the same
memory location with no synchronization on any CFG path between them.
If access A dominates access B, both target the same memory location
(MustAlias, with A covering at least as many bytes as B), and no
synchronization (atomic, fence, or call without nosync) exists on any
CFG path from A to B:
- A write covers both read and write races; B is redundant regardless of
whether B is a read or a write.
- A read covers only read races; B is redundant only if B is also a read.
In our research prototype (which uses a more aggressive, inter-procedural
version of this analysis), we observed the following speedups from
dominance-based elimination alone:
* Redis: 1.35x, FFmpeg: 1.30x, SQLite: 1.21x
* MySQL Select: 1.15x, MySQL Write-only: 1.11x, Memcached: 1.03x
* Selected Chromium benchmarks: Speedometer 3.1 1.06x, Parser: HTML5
Render 1.19x, SVG: SvgCubics 1.20x, Paint: Transform Changes 1.35x,
Image Decoder: WebP 1.34x.
This patch implements a conservative, intra-procedural version; absolute
speedups may be lower than the prototype. Post-dominance elimination will
follow in a subsequent patch.
The optimization is enabled by default. Caching is intentionally omitted
to keep the implementation reviewable; it can be added as a follow-up if
compile-time profiling justifies it.
This work is part of a broader research effort on optimizing dynamic
race detectors. Preprint: https://arxiv.org/abs/2512.05555
Co-authored-by: Alexey Paznikov, Michael Schwarz, Andrey Kogutenko, Yaroslav Osipov, and Umang Mathur.
---
.../Instrumentation/ThreadSanitizer.cpp | 428 ++++++++-
.../ThreadSanitizer/capture-no-omit.ll | 2 +-
.../ThreadSanitizer/dominance-elimination.ll | 884 ++++++++++++++++++
.../ThreadSanitizer/read_before_write.ll | 8 +-
4 files changed, 1292 insertions(+), 30 deletions(-)
create mode 100644 llvm/test/Instrumentation/ThreadSanitizer/dominance-elimination.ll
diff --git a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
index f05efd863fb74..eb79e9ae37bd9 100644
--- a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
@@ -24,10 +24,12 @@
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/Statistic.h"
#include "llvm/ADT/StringExtras.h"
+#include "llvm/Analysis/AliasAnalysis.h"
#include "llvm/Analysis/CaptureTracking.h"
#include "llvm/Analysis/TargetLibraryInfo.h"
#include "llvm/Analysis/ValueTracking.h"
#include "llvm/IR/DataLayout.h"
+#include "llvm/IR/Dominators.h"
#include "llvm/IR/Function.h"
#include "llvm/IR/IRBuilder.h"
#include "llvm/IR/Instructions.h"
@@ -84,6 +86,22 @@ static cl::opt<bool>
ClOmitNonCaptured("tsan-omit-by-pointer-capturing", cl::init(true),
cl::desc("Omit accesses due to pointer capturing"),
cl::Hidden);
+// Removes redundant TSan instrumentation using CFG dominance analysis.
+// If access A dominates access B, both target the same memory location
+// (confirmed via MustAlias), A covers at least as many bytes as B, and no
+// synchronization (atomic, fence, or call without nosync) exists on any CFG
+// path from A to B, then B's instrumentation is redundant: any race
+// detectable at B would already be detected at A.
+// Coverage rule: a dominating write subsumes a subsequent read or write; a
+// dominating read subsumes only a subsequent read.
+// Post-dominance elimination (removing A when B post-dominates A) will be
+// added in a follow-up patch. See DominanceBasedElimination for the full
+// correctness argument.
+static cl::opt<bool>
+ ClUseDominanceAnalysis("tsan-use-dominance-analysis", cl::init(true),
+ cl::desc("Eliminate redundant instrumentation using "
+ "dominance analysis"),
+ cl::Hidden);
STATISTIC(NumInstrumentedReads, "Number of instrumented reads");
STATISTIC(NumInstrumentedWrites, "Number of instrumented writes");
@@ -96,11 +114,86 @@ STATISTIC(NumOmittedReadsFromConstantGlobals,
"Number of reads from constant globals");
STATISTIC(NumOmittedReadsFromVtable, "Number of vtable reads");
STATISTIC(NumOmittedNonCaptured, "Number of accesses ignored due to capturing");
+STATISTIC(NumOmittedByDominance, "Number of accesses ignored due to dominance");
const char kTsanModuleCtorName[] = "tsan.module_ctor";
const char kTsanInitName[] = "__tsan_init";
namespace {
+// Internal Instruction wrapper that contains more information about the
+// Instruction from prior analysis.
+struct InstructionInfo {
+ // Instrumentation emitted for this instruction is for a compounded set of
+ // read and write operations in the same basic block.
+ static constexpr unsigned kCompoundRW = (1U << 0);
+
+ explicit InstructionInfo(Instruction *Inst) : Inst(Inst) {}
+
+ bool isWriteOperation() const {
+ return isa<StoreInst>(Inst) || (Flags & kCompoundRW);
+ }
+
+ Instruction *Inst;
+ unsigned Flags = 0;
+};
+
+/// Eliminates redundant TSan instrumentation using dominance analysis.
+///
+/// An instrumented access B is redundant if there exists a prior access A such
+/// that all four conditions hold:
+///
+/// (1) A dominates B — A executes on every path that reaches B, so any
+/// execution that sees a race at B also sees A beforehand.
+/// (2) A and B MustAlias — they access the same concrete memory location,
+/// so a race partner of B would also race with A.
+/// (3) Coverage: if B is a write, A must also be a write. A write races with
+/// both reads and writes, so a dominating write covers either type. A
+/// read races only with writes, so a dominating read cannot cover a
+/// subsequent write (the write could introduce a new race with a third
+/// thread's read).
+/// (4) The path from A to B is synchronization-free — no atomic, fence, or
+/// call without `nosync` appears on any CFG path between them. If such an
+/// instruction existed, a second thread could synchronize with the
+/// current thread between A and B (establishing a new happens-before
+/// edge), making it possible for B to race with that thread while A does
+/// not.
+///
+/// When all four conditions hold, any race detectable at B is already
+/// detectable at A; removing B's instrumentation is sound.
+class DominanceBasedElimination {
+public:
+ DominanceBasedElimination(SmallVectorImpl<InstructionInfo> &AllInstr,
+ DominatorTree &DT, AAResults &AA)
+ : AllInstr(AllInstr), DT(DT), AA(AA) {}
+
+ void run() { eliminate(); }
+
+private:
+ /// Returns true if the instruction cannot cause inter-thread synchronization.
+ static bool isInstrSafe(const Instruction *Inst);
+
+ /// Returns true if no synchronization exists on any CFG path from DomInst
+ /// to CurrInst. CanReachEnd must be the reverse-reachability set of
+ /// CurrInst's block, pre-computed once per CurrInst by the caller.
+ static bool
+ isPathClear(Instruction *DomInst, Instruction *CurrInst,
+ const SmallPtrSetImpl<const BasicBlock *> &CanReachEnd);
+
+ DenseMap<Instruction *, size_t> createInstrToIndexMap() const;
+
+ /// Searches AllInstr for a dominating instruction that covers AllInstr[i].
+ /// Marks AllInstr[i] for removal and returns true if one is found.
+ bool findAndMarkDominatingInstr(
+ size_t i, const DenseMap<Instruction *, size_t> &InstrToIndexMap,
+ const SmallPtrSetImpl<const BasicBlock *> &CanReachEnd,
+ SmallVectorImpl<bool> &ToRemove);
+
+ void eliminate();
+
+ SmallVectorImpl<InstructionInfo> &AllInstr;
+ DominatorTree &DT;
+ AAResults &AA;
+};
/// ThreadSanitizer: instrument the code in module to find races.
///
@@ -109,7 +202,8 @@ namespace {
/// ensures the __tsan_init function is in the list of global constructors for
/// the module.
struct ThreadSanitizer {
- ThreadSanitizer() {
+ ThreadSanitizer(DominatorTree *DT = nullptr, AAResults *AA = nullptr)
+ : DT(DT), AA(AA) {
// Check options and warn user.
if (ClInstrumentReadBeforeWrite && ClCompoundReadBeforeWrite) {
errs()
@@ -121,19 +215,6 @@ struct ThreadSanitizer {
bool sanitizeFunction(Function &F, const TargetLibraryInfo &TLI);
private:
- // Internal Instruction wrapper that contains more information about the
- // Instruction from prior analysis.
- struct InstructionInfo {
- // Instrumentation emitted for this instruction is for a compounded set of
- // read and write operations in the same basic block.
- static constexpr unsigned kCompoundRW = (1U << 0);
-
- explicit InstructionInfo(Instruction *Inst) : Inst(Inst) {}
-
- Instruction *Inst;
- unsigned Flags = 0;
- };
-
void initialize(Module &M, const TargetLibraryInfo &TLI);
bool instrumentLoadOrStore(const InstructionInfo &II, const DataLayout &DL);
bool instrumentAtomic(Instruction *I, const DataLayout &DL);
@@ -145,6 +226,9 @@ struct ThreadSanitizer {
int getMemoryAccessFuncIndex(Type *OrigTy, Value *Addr, const DataLayout &DL);
void InsertRuntimeIgnores(Function &F);
+ DominatorTree *DT = nullptr;
+ AAResults *AA = nullptr;
+
Type *IntptrTy;
FunctionCallee TsanFuncEntry;
FunctionCallee TsanFuncExit;
@@ -174,6 +258,297 @@ struct ThreadSanitizer {
FunctionCallee MemmoveFn, MemcpyFn, MemsetFn;
};
+//-----------------------------------------------------------------------------
+// DominanceBasedElimination Implementation
+//-----------------------------------------------------------------------------
+
+static bool isTsanAtomic(const Instruction *I) {
+ // TODO: Ask TTI whether synchronization scope is between threads.
+ auto SSID = getAtomicSyncScopeID(I);
+ if (!SSID)
+ return false;
+ if (isa<LoadInst>(I) || isa<StoreInst>(I))
+ return *SSID != SyncScope::SingleThread;
+ return true;
+}
+
+bool DominanceBasedElimination::isInstrSafe(const Instruction *Inst) {
+ // Atomics with inter-thread scope establish happens-before between threads
+ // (e.g., a release store followed by an acquire load synchronizes the two
+ // threads). Any such operation on the path from A to B would allow another
+ // thread to "see" the state after A but before B, potentially racing with B
+ // while A does not observe the race.
+ if (isTsanAtomic(Inst))
+ return false;
+
+ // A function call may contain mutex unlocks, thread spawns, or other
+ // release-like operations that create new happens-before edges. A call is
+ // safe only if it is guaranteed not to synchronize with other threads in
+ // any way. hasFnAttr checks both the call-site attribute and the callee's
+ // function attributes, so it handles direct calls, indirect calls with a
+ // nosync call-site annotation, and intrinsics uniformly.
+ if (const auto *CB = dyn_cast<CallBase>(Inst))
+ return CB->hasFnAttr(Attribute::NoSync);
+
+ // Fences are handled by isTsanAtomic above. All remaining instructions
+ // (arithmetic, branches, GEPs, etc.) cannot establish inter-thread
+ // happens-before and are safe to cross.
+ return true;
+}
+
+DenseMap<Instruction *, size_t>
+DominanceBasedElimination::createInstrToIndexMap() const {
+ DenseMap<Instruction *, size_t> InstrToIndexMap;
+ InstrToIndexMap.reserve(AllInstr.size());
+ for (size_t i = 0; i < AllInstr.size(); ++i)
+ InstrToIndexMap[AllInstr[i].Inst] = i;
+ return InstrToIndexMap;
+}
+
+bool DominanceBasedElimination::isPathClear(
+ Instruction *DomInst, Instruction *CurrInst,
+ const SmallPtrSetImpl<const BasicBlock *> &CanReachEnd) {
+ BasicBlock *DomBB = DomInst->getParent();
+ BasicBlock *CurrBB = CurrInst->getParent();
+
+ // Intra-BB: only the instructions strictly between DomInst and CurrInst
+ // can introduce synchronization. Scan the open interval (DomInst, CurrInst).
+ // A loop around CurrBB is irrelevant here: DomInst and CurrInst share a
+ // block with DomInst first, so every dynamic execution of CurrInst is
+ // immediately preceded by DomInst within the same iteration.
+ if (DomBB == CurrBB) {
+ for (auto It = std::next(DomInst->getIterator());
+ It != CurrInst->getIterator(); ++It)
+ if (!isInstrSafe(&*It))
+ return false;
+ return true;
+ }
+
+ // Forward BFS that scans, in full, every block reachable from the given seed
+ // successors while staying inside the cone (CanReachEnd) and never passing
+ // back through CurrBB. Returns false if any scanned block synchronizes.
+ // Restricting traversal to CanReachEnd (the reverse-reachability set of
+ // CurrBB computed by the caller) ensures we only inspect blocks that lie on
+ // a path to CurrBB; blocks on diverging paths that never reach CurrBB must
+ // not veto the elimination.
+ auto ConeBlocksClear = [&](auto Seeds) {
+ SmallPtrSet<const BasicBlock *, 32> Visited;
+ SmallVector<const BasicBlock *, 16> Worklist;
+ auto Enqueue = [&](const BasicBlock *BB) {
+ if (CanReachEnd.count(BB) && BB != CurrBB && Visited.insert(BB).second)
+ Worklist.push_back(BB);
+ };
+ for (const BasicBlock *Succ : Seeds)
+ Enqueue(Succ);
+ while (!Worklist.empty()) {
+ const BasicBlock *BB = Worklist.pop_back_val();
+ for (const Instruction &I : *BB)
+ if (!isInstrSafe(&I))
+ return false;
+ for (const BasicBlock *Succ : successors(BB))
+ Enqueue(Succ);
+ }
+ return true;
+ };
+
+ // Inter-BB: every path from DomInst to CurrInst is covered by four regions.
+ //
+ // Region 1 — suffix of DomBB (instructions after DomInst in DomInst's block).
+ // These execute on every path from DomInst before leaving the block.
+ for (auto It = std::next(DomInst->getIterator()); It != DomBB->end(); ++It)
+ if (!isInstrSafe(&*It))
+ return false;
+
+ // Region 2 — prefix of CurrBB (instructions before CurrInst in CurrInst's
+ // block). These execute on every path that arrives at CurrInst.
+ for (auto It = CurrBB->begin(); It != CurrInst->getIterator(); ++It)
+ if (!isInstrSafe(&*It))
+ return false;
+
+ // Region 3 — intermediate blocks on a loop-free path from DomBB to CurrBB.
+ if (!ConeBlocksClear(successors(DomBB)))
+ return false;
+
+ // Region 4 — loop back-edge. If CurrInst can be reached again from itself,
+ // then between two consecutive dynamic executions of CurrInst the program
+ // runs CurrInst -> (suffix of CurrBB) -> ... -> (prefix of CurrBB) ->
+ // CurrInst. DomInst dominates CurrInst but, lying outside the loop, may
+ // execute only once and therefore cannot cover the later executions of
+ // CurrInst unless this back-edge path is itself synchronization-free. (When
+ // DomInst is inside the loop it re-executes every iteration and covers
+ // CurrInst directly; this check may then reject soundly-eliminable cases,
+ // but it is never unsound.) The cycle exists iff some successor of CurrBB can
+ // still reach CurrBB, i.e. is in CanReachEnd.
+ bool CurrInCycle = false;
+ for (const BasicBlock *Succ : successors(CurrBB))
+ if (CanReachEnd.count(Succ)) {
+ CurrInCycle = true;
+ break;
+ }
+ if (CurrInCycle) {
+ // Suffix of CurrBB (instructions after CurrInst), which executes before the
+ // back-edge re-enters CurrBB. CurrBB's prefix is already covered by
+ // Region 2, so the two scans together cover the whole block.
+ for (auto It = std::next(CurrInst->getIterator()); It != CurrBB->end();
+ ++It)
+ if (!isInstrSafe(&*It))
+ return false;
+ // Blocks on the CurrBB -> CurrBB back-edge path.
+ if (!ConeBlocksClear(successors(CurrBB)))
+ return false;
+ }
+
+ return true;
+}
+
+bool DominanceBasedElimination::findAndMarkDominatingInstr(
+ size_t i, const DenseMap<Instruction *, size_t> &InstrToIndexMap,
+ const SmallPtrSetImpl<const BasicBlock *> &CanReachEnd,
+ SmallVectorImpl<bool> &ToRemove) {
+ LLVM_DEBUG(dbgs() << "\nAnalyzing: " << *(AllInstr[i].Inst) << "\n");
+ const InstructionInfo &CurrII = AllInstr[i];
+ Instruction *CurrInst = CurrII.Inst;
+ const BasicBlock *CurrBB = CurrInst->getParent();
+
+ const DomTreeNode *CurrDTNode = DT.getNode(CurrBB);
+ if (!CurrDTNode)
+ return false;
+
+ // A dominating access A must be in a block that is an ancestor of CurrBB in
+ // the dominator tree (condition 1). Walking up from CurrBB's node to the
+ // tree root visits exactly those blocks, so we only scan instrumented
+ // accesses inside them. This is more efficient than scanning all of AllInstr
+ // and filtering by domination after the fact.
+ for (const DomTreeNode *IDomNode = CurrDTNode; IDomNode;
+ IDomNode = IDomNode->getIDom()) {
+ const BasicBlock *DomBB = IDomNode->getBlock();
+ if (!DomBB)
+ break;
+
+ // When DomBB == CurrBB, the dominating instruction must appear before
+ // CurrInst in program order; instructions after it do not dominate it.
+ auto EndIt = (DomBB == CurrBB) ? CurrInst->getIterator() : DomBB->end();
+
+ for (auto InstIt = DomBB->begin(); InstIt != EndIt; ++InstIt) {
+ LLVM_DEBUG(dbgs() << "Candidate: " << *InstIt << "\n");
+
+ const auto It = InstrToIndexMap.find(&*InstIt);
+ if (It == InstrToIndexMap.end() || ToRemove[It->second])
+ continue; // not an instrumented access, or already eliminated
+
+ const size_t DomIndex = It->second;
+ const InstructionInfo &DomII = AllInstr[DomIndex];
+ Instruction *DomInst = DomII.Inst;
+
+ auto IsVolatile = [](const Instruction *I) {
+ if (const auto *L = dyn_cast<LoadInst>(I))
+ return L->isVolatile();
+ if (const auto *S = dyn_cast<StoreInst>(I))
+ return S->isVolatile();
+ return false;
+ };
+ // With -tsan-distinguish-volatile, volatile and non-volatile accesses
+ // emit different runtime calls and must not be merged.
+ if (ClDistinguishVolatile &&
+ (IsVolatile(DomInst) || IsVolatile(CurrInst)))
+ continue;
+
+ // Condition (2): same memory location.
+ // isMustAlias checks that the base pointers are identical but does NOT
+ // compare access sizes — two MemoryLocations with the same pointer but
+ // different sizes both return MustAlias. We therefore check sizes
+ // separately: DomInst must cover at least as many bytes as CurrInst,
+ // otherwise races on the extra bytes in CurrInst's range would not be
+ // detected by DomInst's instrumentation call.
+ const MemoryLocation CurrLoc = MemoryLocation::get(CurrInst);
+ const MemoryLocation DomLoc = MemoryLocation::get(DomInst);
+ if (!AA.isMustAlias(CurrLoc, DomLoc))
+ continue;
+ // Require both sizes to be known, non-scalable, and DomSize >= CurrSize.
+ if (!DomLoc.Size.hasValue() || !CurrLoc.Size.hasValue() ||
+ DomLoc.Size.isScalable() || CurrLoc.Size.isScalable() ||
+ DomLoc.Size.getValue().getFixedValue() <
+ CurrLoc.Size.getValue().getFixedValue())
+ continue;
+
+ // Condition (3): write coverage rule.
+ // A write races with both reads and writes on other threads, so a
+ // dominating write makes CurrInst's check redundant regardless of
+ // whether CurrInst is a read or a write.
+ // A read races only with writes, so a dominating read cannot subsume
+ // a subsequent write: the write might race with a third thread's read
+ // that the dominating read would not catch.
+ if (!DomII.isWriteOperation() && CurrII.isWriteOperation())
+ continue;
+
+ // Condition (4): synchronization-free path (checked last because it
+ // involves CFG traversal and is the most expensive test).
+ if (isPathClear(DomInst, CurrInst, CanReachEnd)) {
+ LLVM_DEBUG(dbgs() << "TSAN: Omitting " << *CurrInst << " (dominated by "
+ << *DomInst << ")\n");
+ ToRemove[i] = true;
+ return true;
+ }
+ }
+ }
+ return false;
+}
+
+void DominanceBasedElimination::eliminate() {
+ LLVM_DEBUG(dbgs() << "Starting dominance-based elimination\n");
+ if (AllInstr.empty())
+ return;
+
+ SmallVector<bool, 16> ToRemove(AllInstr.size(), false);
+ unsigned RemovedCount = 0;
+ const DenseMap<Instruction *, size_t> InstrToIndexMap =
+ createInstrToIndexMap();
+
+ for (size_t i = 0; i < AllInstr.size(); ++i) {
+ if (ToRemove[i])
+ continue;
+
+ // Build the reverse-reachability set of CurrBB: the set of all blocks
+ // from which CurrBB is reachable. isPathClear uses this to restrict its
+ // forward BFS to blocks that actually lie on a path from DomBB to CurrBB.
+ //
+ // We compute it here — once per CurrInst — rather than inside isPathClear,
+ // because findAndMarkDominatingInstr may call isPathClear several times
+ // for different DomInst candidates that all share the same CurrBB.
+ // Pre-computing avoids repeating the reverse BFS for each candidate.
+ const BasicBlock *CurrBB = AllInstr[i].Inst->getParent();
+ SmallPtrSet<const BasicBlock *, 32> CanReachEnd;
+ SmallVector<const BasicBlock *, 16> RBFSWorklist;
+ CanReachEnd.insert(CurrBB);
+ RBFSWorklist.push_back(CurrBB);
+ while (!RBFSWorklist.empty()) {
+ const BasicBlock *BB = RBFSWorklist.pop_back_val();
+ for (const BasicBlock *Pred : predecessors(BB))
+ if (CanReachEnd.insert(Pred).second)
+ RBFSWorklist.push_back(Pred);
+ }
+
+ if (findAndMarkDominatingInstr(i, InstrToIndexMap, CanReachEnd, ToRemove))
+ RemovedCount++;
+ }
+
+ LLVM_DEBUG(dbgs() << "\nFinal instruction status:\n";
+ for (size_t i = 0; i < AllInstr.size(); ++i) dbgs()
+ << "[" << (ToRemove[i] ? "REMOVED" : "KEPT") << "]\t"
+ << *AllInstr[i].Inst << "\n");
+
+ if (RemovedCount > 0) {
+ auto It = ToRemove.begin();
+ erase_if(AllInstr, [&](const InstructionInfo &) { return *It++; });
+ NumOmittedByDominance += RemovedCount;
+ }
+ LLVM_DEBUG(dbgs() << "Dominance elimination complete\n");
+}
+
+//-----------------------------------------------------------------------------
+// ThreadSanitizer Implementation
+//-----------------------------------------------------------------------------
+
void insertModuleCtor(Module &M) {
getOrCreateSanitizerCtorAndInitFunctions(
M, kTsanModuleCtorName, kTsanInitName, /*InitArgTypes=*/{},
@@ -186,7 +561,15 @@ void insertModuleCtor(Module &M) {
PreservedAnalyses ThreadSanitizerPass::run(Function &F,
FunctionAnalysisManager &FAM) {
- ThreadSanitizer TSan;
+ DominatorTree *DT = nullptr;
+ AAResults *AA = nullptr;
+
+ if (ClUseDominanceAnalysis) {
+ DT = &FAM.getResult<DominatorTreeAnalysis>(F);
+ AA = &FAM.getResult<AAManager>(F);
+ }
+
+ ThreadSanitizer TSan(DT, AA);
if (TSan.sanitizeFunction(F, FAM.getResult<TargetLibraryAnalysis>(F)))
return PreservedAnalyses::none();
return PreservedAnalyses::all();
@@ -474,16 +857,6 @@ void ThreadSanitizer::chooseInstructionsToInstrument(
Local.clear();
}
-static bool isTsanAtomic(const Instruction *I) {
- // TODO: Ask TTI whether synchronization scope is between threads.
- auto SSID = getAtomicSyncScopeID(I);
- if (!SSID)
- return false;
- if (isa<LoadInst>(I) || isa<StoreInst>(I))
- return *SSID != SyncScope::SingleThread;
- return true;
-}
-
void ThreadSanitizer::InsertRuntimeIgnores(Function &F) {
InstrumentationIRBuilder IRB(&F.getEntryBlock(),
F.getEntryBlock().getFirstNonPHIIt());
@@ -545,6 +918,11 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
chooseInstructionsToInstrument(LocalLoadsAndStores, AllLoadsAndStores, DL);
}
+ if (ClUseDominanceAnalysis && DT && AA) {
+ DominanceBasedElimination DBE(AllLoadsAndStores, *DT, *AA);
+ DBE.run();
+ }
+
// We have collected all loads and stores.
// FIXME: many of these accesses do not need to be checked for races
// (e.g. variables that do not escape, etc).
diff --git a/llvm/test/Instrumentation/ThreadSanitizer/capture-no-omit.ll b/llvm/test/Instrumentation/ThreadSanitizer/capture-no-omit.ll
index cae04936002cd..f7d6eb56ba94f 100644
--- a/llvm/test/Instrumentation/ThreadSanitizer/capture-no-omit.ll
+++ b/llvm/test/Instrumentation/ThreadSanitizer/capture-no-omit.ll
@@ -1,4 +1,4 @@
-; RUN: opt < %s -passes=tsan -tsan-omit-by-pointer-capturing=0 -S | FileCheck %s
+; RUN: opt < %s -passes=tsan -tsan-omit-by-pointer-capturing=0 -tsan-use-dominance-analysis=0 -S | FileCheck %s
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
diff --git a/llvm/test/Instrumentation/ThreadSanitizer/dominance-elimination.ll b/llvm/test/Instrumentation/ThreadSanitizer/dominance-elimination.ll
new file mode 100644
index 0000000000000..037d6c3f88a6d
--- /dev/null
+++ b/llvm/test/Instrumentation/ThreadSanitizer/dominance-elimination.ll
@@ -0,0 +1,884 @@
+; RUN: opt < %s -passes=tsan -S | FileCheck %s
+; RUN: opt < %s -passes=tsan -tsan-use-dominance-analysis=false -S | FileCheck %s --check-prefix=NODOM
+; RUN: opt < %s -passes=tsan -tsan-distinguish-volatile -S | FileCheck %s --check-prefix=VOLATILE
+
+; Tests for TSan dominance-based redundant instrumentation elimination.
+; Redundant instrumentation is removed when one access dominates another to
+; the same location with no synchronization on any path between them.
+;
+; Check prefixes:
+; CHECK - default run (optimization enabled)
+; NODOM - optimization disabled; all accesses must remain instrumented
+; VOLATILE - -tsan-distinguish-volatile enabled; volatile and non-volatile
+; accesses emit different runtime calls and must not be merged
+
+target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
+
+ at g1 = global i32 0, align 4
+ at g2 = global i32 0, align 4
+ at arr = global [5 x i32] zeroinitializer, align 4
+
+; Unsafe call (no nosync): blocks dominance-based elimination.
+declare void @ext_call()
+; nosync call: safe to cross for dominance-based elimination.
+declare void @nosync_func() #0
+; Unsafe function returning i32, used in loop tests.
+declare i32 @ext_check(...)
+
+; ===========================================================================
+; Intra-block dominance
+; ===========================================================================
+
+; First write dominates second write to the same location.
+define void @intra_block_write_write() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @intra_block_write_write
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+;
+; NODOM-LABEL: define void @intra_block_write_write
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: ret void
+
+; Write dominates following read: write covers read, so read is removed.
+define void @intra_block_write_read() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ %val = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @intra_block_write_read
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK-NOT: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; First read dominates second read to the same location.
+define void @intra_block_read_read() nounwind uwtable sanitize_thread {
+entry:
+ %v1 = load i32, ptr @g1, align 4
+ %v2 = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @intra_block_read_read
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK-NOT: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; A dominating read does NOT eliminate a write on a dominated path.
+; The read-before-write elimination in chooseInstructionsToInstrument only
+; applies within the same basic block, so this uses an inter-block scenario.
+; The write is only on one branch so it is NOT the post-dominator of the
+; read; the dominance check therefore applies in isolation.
+define void @dom_read_does_not_cover_write(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ %v = load i32, ptr @g1, align 4
+ br i1 %cond, label %write.path, label %skip
+write.path:
+ store i32 1, ptr @g1, align 4
+ br label %end
+skip:
+ br label %end
+end:
+ ret void
+}
+; CHECK-LABEL: define void @dom_read_does_not_cover_write
+; The read dominates the write, but a read cannot cover write-write races.
+; Both must remain instrumented.
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: write.path:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Path safety
+; ===========================================================================
+
+; An inter-thread atomic on the path is a synchronization point: no elimination.
+; (isTsanAtomic returns true for cross-thread scope.)
+define void @path_dirty_atomic_interthread(ptr %p, ptr %flag) nounwind sanitize_thread {
+entry:
+ store i32 1, ptr %p, align 4
+ %v = load atomic i32, ptr %flag acquire, align 4
+ store i32 2, ptr %p, align 4
+ ret void
+}
+; CHECK-LABEL: define void @path_dirty_atomic_interthread
+; CHECK: call void @__tsan_write4(ptr %p)
+; CHECK: call void @__tsan_write4(ptr %p)
+; CHECK: ret void
+
+; A singlethread-scoped atomic on the path does NOT block elimination.
+; isTsanAtomic returns false for singlethread scope on load/store, so
+; it is treated as a plain access with no inter-thread synchronization.
+define void @path_clear_atomic_singlethread(ptr %p, ptr %q) nounwind sanitize_thread {
+entry:
+ store i32 1, ptr %p, align 4
+ store atomic i32 0, ptr %q syncscope("singlethread") seq_cst, align 4
+ store i32 2, ptr %p, align 4
+ ret void
+}
+; CHECK-LABEL: define void @path_clear_atomic_singlethread
+; CHECK: call void @__tsan_write4(ptr %p)
+; CHECK-NOT: call void @__tsan_write4(ptr %p)
+; CHECK: ret void
+
+; A singlethread fence IS treated as unsafe (conservative: isTsanAtomic
+; returns true for any non-load/store instruction with a sync scope,
+; regardless of whether the scope is singlethread or system). Both writes
+; remain instrumented.
+define void @path_dirty_fence_singlethread(ptr %p) nounwind sanitize_thread {
+entry:
+ store i32 1, ptr %p, align 4
+ fence syncscope("singlethread") seq_cst
+ store i32 2, ptr %p, align 4
+ ret void
+}
+; CHECK-LABEL: define void @path_dirty_fence_singlethread
+; CHECK: call void @__tsan_write4(ptr %p)
+; CHECK: call void @__tsan_write4(ptr %p)
+; CHECK: ret void
+
+; An unsafe call between two accesses makes the path dirty: no elimination.
+define void @path_dirty_call() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ call void @ext_call()
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @path_dirty_call
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; A nosync call does not block dominance elimination.
+define void @path_clear_nosync_call() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ call void @nosync_func()
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @path_clear_nosync_call
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; An indirect call with nosync at the call site does not block elimination.
+; #0 = { nosync } — the attribute group is defined at the bottom of this file.
+; This exercises the call-site attribute path in isInstrSafe (hasFnAttr).
+define void @path_clear_indirect_nosync_callsite(ptr %fn) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ call void %fn() #0
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @path_clear_indirect_nosync_callsite
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Inter-block dominance
+; ===========================================================================
+
+; A write in the entry block dominates writes in both branches of a diamond.
+define void @inter_block_dom(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ br i1 %cond, label %if.then, label %if.else
+if.then:
+ store i32 2, ptr @g1, align 4
+ br label %if.end
+if.else:
+ store i32 3, ptr @g1, align 4
+ br label %if.end
+if.end:
+ ret void
+}
+; CHECK-LABEL: define void @inter_block_dom
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: if.then:
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: if.else:
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+;
+; NODOM-LABEL: define void @inter_block_dom
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: ret void
+
+; ===========================================================================
+; Diamonds: an access at the merge point has no dominating same-location access
+;
+; Only dominance is exploited here, so an access whose only "relative" is in a
+; sibling branch or at the merge point (which post-dominates, but does not
+; dominate, the branch accesses) is never eliminated. These confirm the pass
+; does not over-eliminate in such shapes.
+; ===========================================================================
+
+; Writes in both branches and at the merge: none dominates another, since the
+; entry block has no write to @g1. All three remain.
+define void @diamond_branch_and_merge_writes(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ br i1 %cond, label %if.then, label %if.else
+if.then:
+ store i32 2, ptr @g1, align 4
+ br label %if.end
+if.else:
+ store i32 3, ptr @g1, align 4
+ br label %if.end
+if.end:
+ store i32 4, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @diamond_branch_and_merge_writes
+; CHECK: if.then:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: if.else:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: if.end:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+;
+; NODOM-LABEL: define void @diamond_branch_and_merge_writes
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: call void @__tsan_write4(ptr @g1)
+; NODOM: ret void
+
+; A write at the merge does not dominate the branch reads (entry has no access),
+; so the reads are not eliminated. All three accesses remain.
+define void @diamond_merge_write_branch_reads(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ br i1 %cond, label %if.then, label %if.else
+if.then:
+ %v1 = load i32, ptr @g1, align 4
+ br label %if.end
+if.else:
+ %v2 = load i32, ptr @g1, align 4
+ br label %if.end
+if.end:
+ store i32 0, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @diamond_merge_write_branch_reads
+; CHECK: if.then:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: if.else:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: if.end:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; A read at the merge does not dominate the branch reads. All three remain.
+define void @diamond_merge_read_branch_reads(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ br i1 %cond, label %if.then, label %if.else
+if.then:
+ %v1 = load i32, ptr @g1, align 4
+ br label %if.end
+if.else:
+ %v2 = load i32, ptr @g1, align 4
+ br label %if.end
+if.end:
+ %v3 = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @diamond_merge_read_branch_reads
+; CHECK: if.then:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: if.else:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: if.end:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; Writes in both branches with a read at the merge: the merge read does not
+; dominate the branch writes (and a read could not cover a write anyway). Both
+; writes remain instrumented.
+define void @diamond_merge_read_branch_writes(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ br i1 %cond, label %if.then, label %if.else
+if.then:
+ store i32 1, ptr @g1, align 4
+ br label %if.end
+if.else:
+ store i32 2, ptr @g1, align 4
+ br label %if.end
+if.end:
+ %v = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @diamond_merge_read_branch_writes
+; CHECK: if.then:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: if.else:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: if.end:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Multi-path: dirty vs clean
+; ===========================================================================
+
+; One branch carries an unsafe call: the read at the merge is not eliminated.
+define void @multi_path_dirty(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ br i1 %cond, label %then, label %else
+then:
+ call void @ext_call()
+ br label %merge
+else:
+ call void @nosync_func()
+ br label %merge
+merge:
+ %v = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @multi_path_dirty
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: merge:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; Both branches carry only nosync calls: the read at the merge is eliminated.
+define void @multi_path_clean(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ br i1 %cond, label %then, label %else
+then:
+ call void @nosync_func()
+ br label %merge
+else:
+ call void @nosync_func()
+ br label %merge
+merge:
+ %v = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @multi_path_clean
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: merge:
+; CHECK-NOT: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; Adjacent stores in the same block are eliminated; a dirty path to the merge
+; block keeps the final store instrumented.
+define void @mixed_intra_inter(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ store i32 2, ptr @g1, align 4
+ br i1 %cond, label %dirty, label %clean
+dirty:
+ call void @ext_call()
+ br label %merge
+clean:
+ call void @nosync_func()
+ br label %merge
+merge:
+ store i32 3, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @mixed_intra_inter
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: merge:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Path safety: dirty suffix / prefix / unrelated path
+; ===========================================================================
+
+; Unsafe call in the suffix of the dominating block (Region 1) blocks
+; elimination of the dominated read at the merge.
+define void @dom_dirty_dom_suffix(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ call void @ext_call()
+ br i1 %cond, label %path.then, label %path.else
+path.then:
+ br label %merge
+path.else:
+ br label %merge
+merge:
+ %v = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @dom_dirty_dom_suffix
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: merge:
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; Unsafe call in the prefix of the dominated block (Region 2) blocks
+; elimination.
+define void @dom_dirty_curr_prefix() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ br label %end
+end:
+ call void @ext_call()
+ %v = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @dom_dirty_curr_prefix
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; A dirty intermediate path that does not reach the dominated block must not
+; block elimination (Region 3 restricts the scan to the reverse-reachable cone).
+define void @dom_dirty_unrelated_path(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ br i1 %cond, label %to.end, label %to.dead
+to.end:
+ br label %end
+to.dead:
+ call void @ext_call()
+ br label %dead
+dead:
+ ret void
+end:
+ %v = load i32, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @dom_dirty_unrelated_path
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: end:
+; CHECK-NOT: call void @__tsan_read4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Loops: a store in the body and a store after the loop, with no dominance
+;
+; The loop may execute zero times (entry -> cond -> end), so the body store does
+; not dominate the exit store; neither store dominates the other and both remain.
+; ===========================================================================
+
+; The loop condition also contains an unsafe call.
+define void @loop_body_and_exit_writes_unsafe_cond() nounwind uwtable sanitize_thread {
+entry:
+ br label %while.cond
+while.cond:
+ %v = call i32 (...) @ext_check()
+ %tobool = icmp ne i32 %v, 0
+ br i1 %tobool, label %while.body, label %while.end
+while.body:
+ store i32 1, ptr @g1, align 4
+ br label %while.cond
+while.end:
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @loop_body_and_exit_writes_unsafe_cond
+; CHECK: while.body:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: while.end:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; Pure loop (only branch instructions, no calls or atomics).
+define void @loop_body_and_exit_writes_pure(i1 %cond) nounwind uwtable sanitize_thread {
+entry:
+ br label %while.cond
+while.cond:
+ br i1 %cond, label %while.body, label %while.end
+while.body:
+ store i32 1, ptr @g1, align 4
+ br label %while.cond
+while.end:
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @loop_body_and_exit_writes_pure
+; CHECK: while.body:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: while.end:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Loops: a dominator OUTSIDE the loop covering an access INSIDE the loop
+;
+; When the dominating access A is outside the loop, it executes only once, so
+; it can only cover the loop access B if the back-edge path B -> ... -> B is
+; also synchronization-free. A synchronization on that back-edge path means
+; later iterations of B are no longer covered by A and must stay instrumented.
+; ===========================================================================
+
+; Clean loop body: A in the entry block dominates B in the loop, and the whole
+; loop (including the back-edge) is sync-free, so B is eliminated.
+define void @loop_dom_clean(i1 %c) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %loop
+loop:
+ store i32 2, ptr @g1, align 4 ; B
+ br i1 %c, label %loop, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_clean
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: loop:
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; A nosync call on the back-edge is still safe: B is eliminated. This confirms
+; the loop handling does not over-conservatively reject sync-free loops.
+define void @loop_dom_clean_nosync(i1 %c) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %loop
+loop:
+ store i32 2, ptr @g1, align 4 ; B
+ call void @nosync_func()
+ br i1 %c, label %loop, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_clean_nosync
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: loop:
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; An inter-thread acquire AFTER B in the loop block sits on the back-edge path
+; between consecutive executions of B. A (entry) cannot cover those later
+; executions: both stores must remain instrumented.
+define void @loop_dom_dirty_atomic_tail(i1 %c) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %loop
+loop:
+ store i32 2, ptr @g1, align 4 ; B
+ %v = load atomic i32, ptr @g2 acquire, align 4
+ br i1 %c, label %loop, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_dirty_atomic_tail
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: loop:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; Same hazard, but the synchronization is in a separate latch (back-edge) block
+; rather than in B's own block. Both stores must remain instrumented.
+define void @loop_dom_dirty_atomic_latch(i1 %c) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %header
+header:
+ store i32 2, ptr @g1, align 4 ; B
+ br label %latch
+latch:
+ %v = load atomic i32, ptr @g2 acquire, align 4
+ br i1 %c, label %header, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_dirty_atomic_latch
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: header:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; An unsafe (non-nosync) call on the back-edge also blocks elimination.
+define void @loop_dom_dirty_call_tail(i1 %c) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %loop
+loop:
+ store i32 2, ptr @g1, align 4 ; B
+ call void @ext_call()
+ br i1 %c, label %loop, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_dirty_call_tail
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: loop:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; Nested loops with the synchronization in the OUTER latch: B in the inner loop
+; re-executes across outer iterations with the sync in between, and the
+; back-edge scan must reach the outer latch through the cone. Both stores remain.
+define void @loop_dom_nested_outer_sync(i1 %c1, i1 %c2) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %outer
+outer:
+ br label %inner
+inner:
+ store i32 2, ptr @g1, align 4 ; B
+ br i1 %c2, label %inner, label %outer.latch
+outer.latch:
+ %v = load atomic i32, ptr @g2 acquire, align 4
+ br i1 %c1, label %outer, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_nested_outer_sync
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: inner:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; A diamond on the back-edge path with synchronization on one of its branches:
+; the back-edge cone scan must visit both branches. Both stores remain.
+define void @loop_dom_diamond_backedge_sync(i1 %c, i1 %d) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %loop
+loop:
+ store i32 2, ptr @g1, align 4 ; B
+ br i1 %d, label %ba, label %bb
+ba:
+ %v = load atomic i32, ptr @g2 acquire, align 4
+ br label %latch
+bb:
+ br label %latch
+latch:
+ br i1 %c, label %loop, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_diamond_backedge_sync
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: loop:
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; Nested loops, fully sync-free: B is still eliminated (precision preserved
+; through the back-edge cone scan).
+define void @loop_dom_nested_clean(i1 %c1, i1 %c2) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4 ; A
+ br label %outer
+outer:
+ br label %inner
+inner:
+ store i32 2, ptr @g1, align 4 ; B
+ br i1 %c2, label %inner, label %outer.latch
+outer.latch:
+ br i1 %c1, label %outer, label %exit
+exit:
+ ret void
+}
+; CHECK-LABEL: define void @loop_dom_nested_clean
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: inner:
+; CHECK-NOT: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Synchronization barriers
+; ===========================================================================
+
+; An atomic RMW operation is a synchronization point: no elimination.
+define void @atomic_blocks_dom_elim() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ %old = atomicrmw add ptr @g1, i32 1 seq_cst
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @atomic_blocks_dom_elim
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; A seq_cst fence is a synchronization point: no elimination.
+define void @fence_blocks_dom_elim() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ fence seq_cst
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @fence_blocks_dom_elim
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: ret void
+
+; ===========================================================================
+; Access size compatibility
+; ===========================================================================
+
+; A dominating write of smaller size must NOT eliminate a larger write: the
+; 4-byte instrumentation does not cover bytes 4-7 of the 8-byte access.
+define void @size_mismatch_small_dom(ptr %p) nounwind sanitize_thread {
+entry:
+ store i32 1, ptr %p, align 4
+ store i64 2, ptr %p, align 8
+ ret void
+}
+; CHECK-LABEL: define void @size_mismatch_small_dom
+; CHECK: call void @__tsan_write4(ptr %p)
+; CHECK: call void @__tsan_write8(ptr %p)
+; CHECK: ret void
+
+; A dominating write of larger size CAN eliminate a smaller write: write8
+; covers the full [p, p+4) range that write4 would instrument.
+define void @size_mismatch_large_dom(ptr %p) nounwind sanitize_thread {
+entry:
+ store i64 1, ptr %p, align 8
+ store i32 2, ptr %p, align 4
+ ret void
+}
+; CHECK-LABEL: define void @size_mismatch_large_dom
+; CHECK: call void @__tsan_write8(ptr %p)
+; CHECK-NOT: call void @__tsan_write4(ptr %p)
+; CHECK: ret void
+
+; ===========================================================================
+; Alias analysis
+; ===========================================================================
+
+; Accesses to distinct globals are NoAlias: no elimination.
+define void @no_alias() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ store i32 2, ptr @g2, align 4
+ ret void
+}
+; CHECK-LABEL: define void @no_alias
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: call void @__tsan_write4(ptr @g2)
+; CHECK: ret void
+
+; Zero-index GEP is MustAlias with the base: elimination fires.
+define void @mustalias_gep0() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ %p = getelementptr i32, ptr @g1, i64 0
+ %v = load i32, ptr %p, align 4
+ ret void
+}
+; CHECK-LABEL: define void @mustalias_gep0
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK-NOT: call void @__tsan_read4(
+; CHECK: ret void
+
+; Different GEP offsets into the same array are NoAlias: no elimination.
+define void @noalias_different_offsets() nounwind uwtable sanitize_thread {
+entry:
+ %p0 = getelementptr [5 x i32], ptr @arr, i64 0, i64 0
+ %p1 = getelementptr [5 x i32], ptr @arr, i64 0, i64 1
+ store i32 1, ptr %p0, align 4
+ store i32 2, ptr %p1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @noalias_different_offsets
+; CHECK: call void @__tsan_write4(
+; CHECK: call void @__tsan_write4(
+; CHECK: ret void
+
+; Identical GEP offsets into the same array are MustAlias: elimination fires.
+define void @mustalias_same_offsets() nounwind uwtable sanitize_thread {
+entry:
+ %p0 = getelementptr [5 x i32], ptr @arr, i64 0, i64 1
+ %p1 = getelementptr [5 x i32], ptr @arr, i64 0, i64 1
+ store i32 1, ptr %p0, align 4
+ store i32 2, ptr %p1, align 4
+ ret void
+}
+; CHECK-LABEL: define void @mustalias_same_offsets
+; CHECK: call void @__tsan_write4(
+; CHECK-NOT: call void @__tsan_write4(
+; CHECK: ret void
+
+; phi selecting between two globals is MayAlias: no elimination.
+define void @mayalias_phi(i1 %c) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ br i1 %c, label %A, label %B
+A:
+ br label %join
+B:
+ br label %join
+join:
+ %p = phi ptr [ @g1, %A ], [ @g2, %B ]
+ %v = load i32, ptr %p, align 4
+ ret void
+}
+; CHECK-LABEL: define void @mayalias_phi
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: call void @__tsan_read4(
+; CHECK: ret void
+
+; ptrtoint/inttoptr round-trip breaks MustAlias: no elimination.
+define void @noalias_ptr_roundtrip() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ %i = ptrtoint ptr @g1 to i64
+ %p2 = inttoptr i64 %i to ptr
+ %v = load i32, ptr %p2, align 4
+ ret void
+}
+; CHECK-LABEL: define void @noalias_ptr_roundtrip
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK: call void @__tsan_read4(
+; CHECK: ret void
+
+; select with identical arms is MustAlias: elimination fires.
+define void @mustalias_select_same_ptr(i1 %c) nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ %p = select i1 %c, ptr @g1, ptr @g1
+ %v = load i32, ptr %p, align 4
+ ret void
+}
+; CHECK-LABEL: define void @mustalias_select_same_ptr
+; CHECK: call void @__tsan_write4(ptr @g1)
+; CHECK-NOT: call void @__tsan_read4(
+; CHECK: ret void
+
+; ===========================================================================
+; Volatile (VOLATILE check prefix only)
+;
+; When -tsan-distinguish-volatile is active, volatile and non-volatile accesses
+; to the same address emit different TSan calls and must never be merged.
+; ===========================================================================
+
+; Non-volatile write followed by volatile write: both must be kept.
+define void @write_then_volatile_write() nounwind uwtable sanitize_thread {
+entry:
+ store i32 1, ptr @g1, align 4
+ store volatile i32 2, ptr @g1, align 4
+ ret void
+}
+; VOLATILE-LABEL: define void @write_then_volatile_write
+; VOLATILE: call void @__tsan_write4(ptr @g1)
+; VOLATILE: call void @__tsan_volatile_write4(ptr @g1)
+; VOLATILE: ret void
+
+; Volatile write followed by non-volatile write: both must be kept.
+define void @volatile_write_then_write() nounwind uwtable sanitize_thread {
+entry:
+ store volatile i32 1, ptr @g1, align 4
+ store i32 2, ptr @g1, align 4
+ ret void
+}
+; VOLATILE-LABEL: define void @volatile_write_then_write
+; VOLATILE: call void @__tsan_volatile_write4(ptr @g1)
+; VOLATILE: call void @__tsan_write4(ptr @g1)
+; VOLATILE: ret void
+
+attributes #0 = { nosync }
diff --git a/llvm/test/Instrumentation/ThreadSanitizer/read_before_write.ll b/llvm/test/Instrumentation/ThreadSanitizer/read_before_write.ll
index 1ef283d8f8f4c..7c2de00640785 100644
--- a/llvm/test/Instrumentation/ThreadSanitizer/read_before_write.ll
+++ b/llvm/test/Instrumentation/ThreadSanitizer/read_before_write.ll
@@ -1,7 +1,7 @@
-; RUN: opt < %s -passes=tsan -S | FileCheck --check-prefixes=CHECK,CHECK-OPT %s
-; RUN: opt < %s -passes=tsan -tsan-instrument-read-before-write -S | FileCheck %s --check-prefixes=CHECK,CHECK-UNOPT
-; RUN: opt < %s -passes=tsan -tsan-compound-read-before-write -S | FileCheck %s --check-prefixes=CHECK,CHECK-COMPOUND
-; RUN: opt < %s -passes=tsan -tsan-distinguish-volatile -tsan-compound-read-before-write -S | FileCheck %s --check-prefixes=CHECK,CHECK-COMPOUND-VOLATILE
+; RUN: opt < %s -passes=tsan -tsan-use-dominance-analysis=0 -S | FileCheck --check-prefixes=CHECK,CHECK-OPT %s
+; RUN: opt < %s -passes=tsan -tsan-instrument-read-before-write -tsan-use-dominance-analysis=0 -S | FileCheck %s --check-prefixes=CHECK,CHECK-UNOPT
+; RUN: opt < %s -passes=tsan -tsan-compound-read-before-write -tsan-use-dominance-analysis=0 -S | FileCheck %s --check-prefixes=CHECK,CHECK-COMPOUND
+; RUN: opt < %s -passes=tsan -tsan-distinguish-volatile -tsan-compound-read-before-write -tsan-use-dominance-analysis=0 -S | FileCheck %s --check-prefixes=CHECK,CHECK-COMPOUND-VOLATILE
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
More information about the llvm-commits
mailing list