[clang] [llvm] [Sanitizers] Support sanitizer instrumentation regions in KMSAN, KASAN, KCOV and KCSAN (PR #231374)
Alexander Potapenko via cfe-commits
cfe-commits at lists.llvm.org
Fri Oct 23 07:52:02 PDT 2026
https://github.com/ramosian-glider updated https://github.com/llvm/llvm-project/pull/231374
>From 097a7dcbaffc8d19c9e3ae1b037cd574741cbf20 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Mon, 5 Oct 2026 14:04:11 +0000
Subject: [PATCH 1/6] [Sanitizers] Add sanitizer instrumentation region
intrinsics and builtins
Some code must not be instrumented as a whole but still has parts where
instrumentation is allowed. The Linux kernel marks such functions
`noinstr` (lowered to `disable_sanitizer_instrumentation` and/or the
absence of `sanitize_<x>`), and delimits the parts that may call
instrumented code with `instrumentation_begin()` and
`instrumentation_end()`. Sanitizers currently cannot see these parts:
they either skip the whole function or instrument all of it.
This patch adds the common infrastructure that lets sanitizers
instrument such regions:
- `llvm.sanitizer.instrumentation.begin` and
`llvm.sanitizer.instrumentation.end` delimit a region of an otherwise
uninstrumented function in which sanitizer instrumentation is enabled
again. What counts as uninstrumented and what gets instrumented inside
regions is up to each sanitizer; sanitizers that do not support
regions ignore the intrinsics. The intrinsics keep the default memory
effects so that accesses to memory that may be visible outside the
function do not move across region boundaries, and are `IntrNoMerge`.
SelectionDAG, FastISel and GlobalISel drop them if they reach codegen.
- `computeSanitizerInstrumentationRegions()` in Utils/Instrumentation
computes region membership for the passes. It is a must-analysis over
the nesting depth (meet = min over predecessors): regions nest, an
unmatched end is ignored, and an instruction is in a region only if
every path to it entered one, so it is always dominated by a begin.
- `__builtin_sanitizer_instrumentation_begin()` and
`__builtin_sanitizer_instrumentation_end()` expose the intrinsics to
C code, which can detect them with `__has_builtin()`. They are only
lowered to the intrinsics for sanitizers that consume them, so that
they do not act as optimization barriers elsewhere; no sanitizer does
yet, so for now they lower to nothing.
The semantics are documented in LangRef. Follow-up patches implement
regions in KernelMemorySanitizer, AddressSanitizer, SanitizerCoverage
and ThreadSanitizer.
Tests: `llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll`,
`llvm/test/CodeGen/{Generic,X86}/sanitizer-instrumentation-intrinsics.ll`,
`llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`.
---
clang/docs/LanguageExtensions.md | 45 ++++
clang/docs/ReleaseNotes.md | 7 +
clang/include/clang/Basic/Builtins.td | 18 ++
clang/lib/CodeGen/CGBuiltin.cpp | 14 ++
.../sanitizer-instrumentation-builtins.c | 27 +++
llvm/docs/LangRef.md | 42 ++++
llvm/docs/ReleaseNotes.md | 4 +
llvm/include/llvm/IR/Intrinsics.td | 18 ++
.../llvm/Transforms/Utils/Instrumentation.h | 24 +++
llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp | 5 +
llvm/lib/CodeGen/SelectionDAG/FastISel.cpp | 3 +
.../SelectionDAG/SelectionDAGBuilder.cpp | 2 +
llvm/lib/Transforms/Utils/Instrumentation.cpp | 94 +++++++++
.../sanitizer-instrumentation-intrinsics.ll | 19 ++
.../sanitizer-instrumentation-intrinsics.ll | 20 ++
.../sanitizer-instrumentation-markers.ll | 86 ++++++++
.../unittests/Transforms/Utils/CMakeLists.txt | 1 +
.../Transforms/Utils/SanitizerRegionsTest.cpp | 192 ++++++++++++++++++
18 files changed, 621 insertions(+)
create mode 100644 clang/test/CodeGen/sanitizer-instrumentation-builtins.c
create mode 100644 llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll
create mode 100644 llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll
create mode 100644 llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll
create mode 100644 llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
diff --git a/clang/docs/LanguageExtensions.md b/clang/docs/LanguageExtensions.md
index 3db8d083c40756..7ea54b04ada2d7 100644
--- a/clang/docs/LanguageExtensions.md
+++ b/clang/docs/LanguageExtensions.md
@@ -4317,6 +4317,51 @@ such as applying different hotness cutoffs to different check kinds.
Query for this feature with `__has_builtin(__builtin_allow_runtime_check)`.
+### `__builtin_sanitizer_instrumentation_begin` and `__builtin_sanitizer_instrumentation_end`
+
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` delimit a region of a function that
+is otherwise not instrumented by a sanitizer, inside which the sanitizer's
+instrumentation is enabled again.
+
+**Syntax**:
+
+```c
+void __builtin_sanitizer_instrumentation_begin(void);
+void __builtin_sanitizer_instrumentation_end(void);
+```
+
+**Example of use**:
+
+```c
+__attribute__((disable_sanitizer_instrumentation))
+void handle_irq(struct pt_regs *regs) {
+ enter_irq_context(regs); // not instrumented
+ __builtin_sanitizer_instrumentation_begin();
+ dispatch_irq(regs); // instrumented
+ __builtin_sanitizer_instrumentation_end();
+ exit_irq_context(regs); // not instrumented
+}
+```
+
+**Description**:
+
+The builtins are meant for code like the Linux kernel's `noinstr` functions,
+which must not be instrumented as a whole but call instrumented code between
+`instrumentation_begin()` and `instrumentation_end()`. Regions nest, and should
+be balanced on every path through the function. Which functions count as
+uninstrumented, and what gets instrumented inside regions, is up to each
+sanitizer; sanitizers that support regions describe this in their own
+documentation.
+
+The builtins are lowered to the
+[llvm.sanitizer.instrumentation.begin and llvm.sanitizer.instrumentation.end](https://llvm.org/docs/LangRef.html#llvm-sanitizer-instrumentation-begin-and-llvm-sanitizer-instrumentation-end-intrinsics)
+intrinsics only when a sanitizer that supports regions is enabled, and to
+nothing otherwise, so that they do not constrain optimization in other builds.
+
+Query for this feature with
+`__has_builtin(__builtin_sanitizer_instrumentation_begin)`.
+
### `__builtin_nondeterministic_value`
`__builtin_nondeterministic_value` returns a valid nondeterministic value of the same type as the provided argument.
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index 1723b7f78cc9d4..b27b3d2955bf4d 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -1316,6 +1316,13 @@ The `alpha.cplusplus.UseAfterLifetimeEnd` checker was renamed to `alpha.core.Use
modes for `-fsanitize=alloc-token`, selected with
`-Xclang -falloc-token-mode=<mode>`. These modes derive token IDs from both
the allocated type and the name of the function containing the allocation.
+- New builtins `__builtin_sanitizer_instrumentation_begin()` and
+ `__builtin_sanitizer_instrumentation_end()` delimit regions of an otherwise
+ uninstrumented function (e.g. a Linux kernel `noinstr` function) in which
+ sanitizer instrumentation is enabled again. AddressSanitizer,
+ KernelMemorySanitizer, ThreadSanitizer and SanitizerCoverage support them;
+ see [LanguageExtensions](LanguageExtensions.md) and the documentation of each
+ sanitizer.
### Python Binding Changes
diff --git a/clang/include/clang/Basic/Builtins.td b/clang/include/clang/Basic/Builtins.td
index ad9a58bb2ad2d3..8f34a25460df4e 100644
--- a/clang/include/clang/Basic/Builtins.td
+++ b/clang/include/clang/Basic/Builtins.td
@@ -1469,6 +1469,24 @@ def BuiltinAllowSanitizeCheck : Builtin {
let Prototype = "bool(char const*)";
}
+// Sanitizer instrumentation region markers. These delimit a region of an
+// otherwise uninstrumented function (e.g. one marked with
+// __attribute__((disable_sanitizer_instrumentation))) in which sanitizer
+// instrumentation is enabled. They must not be Const or Pure: they are
+// optimization barriers, and moving code across them changes which
+// instructions belong to the region.
+def SanitizerInstrumentationBegin : Builtin {
+ let Spellings = ["__builtin_sanitizer_instrumentation_begin"];
+ let Attributes = [NoThrow];
+ let Prototype = "void()";
+}
+
+def SanitizerInstrumentationEnd : Builtin {
+ let Spellings = ["__builtin_sanitizer_instrumentation_end"];
+ let Attributes = [NoThrow];
+ let Prototype = "void()";
+}
+
def ShuffleVector : Builtin {
let Spellings = ["__builtin_shufflevector"];
let Attributes = [NoThrow, Const, CustomTypeChecking, Constexpr];
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index c93375ff3a18ef..e7f938d63fe750 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3678,6 +3678,20 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
llvm::MetadataAsValue::get(Ctx, llvm::MDString::get(Ctx, Kind)));
return RValue::get(Allow);
}
+ case Builtin::BI__builtin_sanitizer_instrumentation_begin:
+ case Builtin::BI__builtin_sanitizer_instrumentation_end: {
+ // Only emit the region markers for sanitizers that honour them. Elsewhere
+ // they would just be optimization barriers that no pass ever removes. This
+ // lets code guard its uses on __has_builtin alone.
+ SanitizerMask Consumers;
+ if (getLangOpts().Sanitize.hasOneOf(Consumers))
+ Builder.CreateCall(CGM.getIntrinsic(
+ BuiltinIDIfNoAsmLabel ==
+ Builtin::BI__builtin_sanitizer_instrumentation_begin
+ ? Intrinsic::sanitizer_instrumentation_begin
+ : Intrinsic::sanitizer_instrumentation_end));
+ return RValue::get(nullptr);
+ }
case Builtin::BI__builtin_allow_sanitize_check: {
Intrinsic::ID IntrID = Intrinsic::not_intrinsic;
StringRef Name =
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
new file mode 100644
index 00000000000000..6fef449edd6815
--- /dev/null
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -0,0 +1,27 @@
+// Check that the sanitizer region markers are available, and that they lower to
+// nothing without a sanitizer that consumes them.
+
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
+// RUN: -o - %s | FileCheck %s
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
+
+#if !__has_builtin(__builtin_sanitizer_instrumentation_begin)
+#error "__builtin_sanitizer_instrumentation_begin not available"
+#endif
+#if !__has_builtin(__builtin_sanitizer_instrumentation_end)
+#error "__builtin_sanitizer_instrumentation_end not available"
+#endif
+
+extern void instrumented_callee(int);
+
+__attribute__((disable_sanitizer_instrumentation))
+void noinstr_function(int x) {
+ __builtin_sanitizer_instrumentation_begin();
+ instrumented_callee(x);
+ __builtin_sanitizer_instrumentation_end();
+}
+
+// CHECK-LABEL: define {{.*}}void @noinstr_function(
+// CHECK-NOT: llvm.sanitizer.instrumentation
+// CHECK: }
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 0099b7c0d63560..94b507584fcfa3 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28577,3 +28577,45 @@ taken. A late optimization pass will convert this intrinsic to either
`llvm.cond.loop(true)` or `llvm.cond.loop(pred)`, where `pred`
is a predicate for a conditional branch leading to the intrinsic call,
if possible.
+
+#### '`llvm.sanitizer.instrumentation.begin`' and '`llvm.sanitizer.instrumentation.end`' Intrinsics
+
+##### Syntax:
+
+```llvm
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+```
+
+##### Overview:
+
+The '`llvm.sanitizer.instrumentation.begin`' and
+'`llvm.sanitizer.instrumentation.end`' intrinsics delimit a region of a function
+that a sanitizer does not otherwise instrument, in which that sanitizer's
+instrumentation is enabled again.
+
+##### Semantics:
+
+Inside a region, a sanitizer instruments the code as if the function were
+instrumented, to the extent it supports instrumenting part of a function; code
+outside regions stays uninstrumented. Sanitizers that do not support regions
+ignore the intrinsics, as do all sanitizers in functions that they instrument
+anyway. Which functions count as uninstrumented, and what is instrumented inside
+regions, depends on the sanitizer.
+
+Regions nest: `llvm.sanitizer.instrumentation.begin` increments a region nesting
+depth and `llvm.sanitizer.instrumentation.end` decrements it, so an inner `end`
+does not terminate an enclosing region. An unmatched `end` is ignored rather
+than making the depth negative.
+
+The intrinsics have the default (unknown) memory effects, so that accesses to
+memory that may be visible outside the function are not moved across a region
+boundary, and are `nomerge`, so that two markers are never combined into one.
+If they reach code generation, they are dropped.
+
+An instruction is considered to be inside a region only if *every* path from the
+function entry to it is inside a region, i.e. region membership is an
+intersection over predecessors. This makes an in-region instruction always
+dominated by a `llvm.sanitizer.instrumentation.begin`, so any per-function
+sanitizer state set up there is available. A region that is entered on some
+paths but not others is therefore instrumented only where it is unambiguous.
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 302aa57396105e..33981839c657fa 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -171,6 +171,10 @@ Makes programs 10x faster by doing Special New Thing.
* Added the `bitinsert` and `bitextract` instructions for bit-range
manipulation on byte type values.
+* Added the `llvm.sanitizer.instrumentation.begin` and
+ `llvm.sanitizer.instrumentation.end` intrinsics, which delimit regions of an
+ otherwise uninstrumented function in which sanitizer instrumentation is
+ enabled again.
### Changes to LLVM infrastructure
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index 4494bf1e2a2048..c594a4e1a0a9ae 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -2554,6 +2554,24 @@ def int_load_relative: DefaultAttrsIntrinsic<[llvm_ptr_ty],
def int_asan_check_memaccess :
Intrinsic<[],[llvm_ptr_ty, llvm_i32_ty], [ImmArg<ArgIndex<1>>]>;
+// Sanitizer instrumentation regions in functions that are otherwise not
+// instrumented.
+//
+// These are markers consumed by sanitizer passes, but they deliberately
+// keep the default (read/write any memory) effects and so act as memory
+// barriers. Relaxing them, e.g. to IntrInaccessibleMemOnly, lets the optimizer
+// move memory operations across a region boundary: a load can be CSEd with an
+// identical load from before begin() and lose its shadow check, and a store can
+// sink past end() and leave a stale shadow behind. Both defeat the purpose of
+// the region. What the explicit attribute list buys us is the rest of the
+// modelling - these calls do not throw, do not free, do not call back into the
+// module and always return - plus IntrNoMerge, which stops identical markers in
+// sibling blocks from being merged and thereby moving code out of a region.
+def int_sanitizer_instrumentation_begin
+ : DefaultAttrsIntrinsic<[], [], [IntrNoMerge]>;
+def int_sanitizer_instrumentation_end
+ : DefaultAttrsIntrinsic<[], [], [IntrNoMerge]>;
+
// Spin in an infinite loop (using instructions specified by the target) iff the
// argument is true. Used to implement efficient conditional traps.
def int_cond_loop : Intrinsic<[], [llvm_i1_ty],
diff --git a/llvm/include/llvm/Transforms/Utils/Instrumentation.h b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
index 9945ef5665e9be..642ba946e9dbc6 100644
--- a/llvm/include/llvm/Transforms/Utils/Instrumentation.h
+++ b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
@@ -13,6 +13,7 @@
#ifndef LLVM_TRANSFORMS_INSTRUMENTATION_H
#define LLVM_TRANSFORMS_INSTRUMENTATION_H
+#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/IR/BasicBlock.h"
#include "llvm/IR/DebugInfoMetadata.h"
@@ -60,6 +61,29 @@ LLVM_ABI Comdat *getOrCreateFunctionComdat(Function &F, Triple &T);
LLVM_ABI void setGlobalVariableLargeSection(const Triple &TargetTriple,
GlobalVariable &GV);
+/// Returns true if \p I is a call to llvm.sanitizer.instrumentation.begin.
+LLVM_ABI bool isSanitizerInstrumentationBegin(const Instruction &I);
+/// Returns true if \p I is a call to llvm.sanitizer.instrumentation.end.
+LLVM_ABI bool isSanitizerInstrumentationEnd(const Instruction &I);
+
+/// Returns true if \p F calls llvm.sanitizer.instrumentation.begin or
+/// llvm.sanitizer.instrumentation.end.
+LLVM_ABI bool hasSanitizerInstrumentationRegions(const Function &F);
+
+/// Compute the instructions of \p F that lie inside an
+/// llvm.sanitizer.instrumentation.begin()/end() region, and add them to
+/// \p InRegion.
+///
+/// This is a *must* dataflow problem: an instruction is in a region only if
+/// *every* path from the function entry to it entered a region and did not
+/// leave it again, so an in-region instruction is always dominated by a
+/// begin(). Unbalanced regions are under-approximated, which agrees with
+/// objtool requiring them to be balanced on all paths. The lattice value is
+/// the nesting depth rather than a flag, so that in begin; begin; end; X; end
+/// the instruction X is still in a region.
+LLVM_ABI void computeSanitizerInstrumentationRegions(
+ const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion);
+
// Insert GCOV profiling instrumentation
struct GCOVOptions {
// Specify whether to emit .gcno files.
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index 36ed50c6baf473..a447bcbf8e224b 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -3292,6 +3292,11 @@ bool IRTranslatorImpl::translateKnownIntrinsic(const CallInst &CI,
case Intrinsic::experimental_noalias_scope_decl:
case Intrinsic::var_annotation:
case Intrinsic::sideeffect:
+ // The sanitizer markers are consumed by the sanitizer passes. They can still
+ // reach codegen when no such pass runs (or it does not look at a given
+ // function), in which case they have no effect.
+ case Intrinsic::sanitizer_instrumentation_begin:
+ case Intrinsic::sanitizer_instrumentation_end:
// Discard annotate attributes, assumptions, and artificial side-effects.
return true;
case Intrinsic::read_volatile_register:
diff --git a/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp b/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
index 3dd81d54773b41..e0fbd5d0b3a0e8 100644
--- a/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
@@ -1392,6 +1392,9 @@ bool FastISel::selectIntrinsicCall(const IntrinsicInst *II) {
case Intrinsic::donothing:
// Neither does the sideeffect intrinsic.
case Intrinsic::sideeffect:
+ // The sanitizer region markers are only meaningful to the sanitizer passes.
+ case Intrinsic::sanitizer_instrumentation_begin:
+ case Intrinsic::sanitizer_instrumentation_end:
// Neither does the assume intrinsic; it's also OK not to codegen its operand.
case Intrinsic::assume:
// Neither does the llvm.experimental.noalias.scope.decl intrinsic
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 30b1fee7410d76..2a29e1ff9eac35 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -8161,6 +8161,8 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
return;
}
case Intrinsic::donothing:
+ case Intrinsic::sanitizer_instrumentation_begin:
+ case Intrinsic::sanitizer_instrumentation_end:
case Intrinsic::seh_try_begin:
case Intrinsic::seh_scope_begin:
case Intrinsic::seh_try_end:
diff --git a/llvm/lib/Transforms/Utils/Instrumentation.cpp b/llvm/lib/Transforms/Utils/Instrumentation.cpp
index 5f71703725ba11..cfbe8ae90e6d44 100644
--- a/llvm/lib/Transforms/Utils/Instrumentation.cpp
+++ b/llvm/lib/Transforms/Utils/Instrumentation.cpp
@@ -12,6 +12,9 @@
//===----------------------------------------------------------------------===//
#include "llvm/Transforms/Utils/Instrumentation.h"
+#include "llvm/ADT/SetVector.h"
+#include "llvm/ADT/SmallVector.h"
+#include "llvm/IR/CFG.h"
#include "llvm/IR/DiagnosticInfo.h"
#include "llvm/IR/DiagnosticPrinter.h"
#include "llvm/IR/IntrinsicInst.h"
@@ -124,3 +127,94 @@ void llvm::setGlobalVariableLargeSection(const Triple &TargetTriple,
return;
GV.setCodeModel(CodeModel::Large);
}
+
+bool llvm::isSanitizerInstrumentationBegin(const Instruction &I) {
+ const auto *II = dyn_cast<IntrinsicInst>(&I);
+ return II &&
+ II->getIntrinsicID() == Intrinsic::sanitizer_instrumentation_begin;
+}
+
+bool llvm::isSanitizerInstrumentationEnd(const Instruction &I) {
+ const auto *II = dyn_cast<IntrinsicInst>(&I);
+ return II && II->getIntrinsicID() == Intrinsic::sanitizer_instrumentation_end;
+}
+
+bool llvm::hasSanitizerInstrumentationRegions(const Function &F) {
+ // Most modules never use the markers, so avoid scanning their functions.
+ const Module *M = F.getParent();
+ auto IsUsed = [M](Intrinsic::ID ID) {
+ const Function *Decl = Intrinsic::getDeclarationIfExists(M, ID);
+ return Decl && !Decl->use_empty();
+ };
+ if (!IsUsed(Intrinsic::sanitizer_instrumentation_begin) &&
+ !IsUsed(Intrinsic::sanitizer_instrumentation_end))
+ return false;
+
+ for (const BasicBlock &BB : F)
+ for (const Instruction &I : BB)
+ if (isSanitizerInstrumentationBegin(I) ||
+ isSanitizerInstrumentationEnd(I))
+ return true;
+ return false;
+}
+
+// Lattice top of the region analysis: this block has not been reached yet, so
+// it does not constrain the depth of its successors.
+static constexpr unsigned UnreachedDepth = ~0U;
+
+/// Return the region nesting depth after \p I, given the depth before it.
+static unsigned regionDepthAfter(const Instruction &I, unsigned Depth) {
+ if (isSanitizerInstrumentationBegin(I))
+ return Depth + 1;
+ // Saturate at zero: an unbalanced end() is malformed input, and wrapping
+ // around would spuriously instrument the rest of the function.
+ if (isSanitizerInstrumentationEnd(I) && Depth)
+ return Depth - 1;
+ return Depth;
+}
+
+void llvm::computeSanitizerInstrumentationRegions(
+ const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion) {
+ // Nesting depth at the start and at the end of each block, indexed by block
+ // number.
+ SmallVector<unsigned> DepthIn(F.getMaxBlockNumber(), UnreachedDepth);
+ SmallVector<unsigned> DepthOut(F.getMaxBlockNumber(), UnreachedDepth);
+
+ // Worklist iteration from the entry block. A block is queued when its input
+ // depth decreases, and depths are bounded below by zero, so this terminates;
+ // each block is scanned at most once per distinct input depth. (Iterating
+ // over the blocks in layout order instead is quadratic when the layout is
+ // far from a topological order.)
+ const BasicBlock *Entry = &F.getEntryBlock();
+ DepthIn[Entry->getNumber()] = 0;
+ SmallSetVector<const BasicBlock *, 32> Worklist;
+ Worklist.insert(Entry);
+ while (!Worklist.empty()) {
+ const BasicBlock *BB = Worklist.pop_back_val();
+ unsigned Depth = DepthIn[BB->getNumber()];
+ for (const Instruction &I : *BB)
+ Depth = regionDepthAfter(I, Depth);
+ if (Depth >= DepthOut[BB->getNumber()])
+ continue;
+ DepthOut[BB->getNumber()] = Depth;
+ // Must-join: the depth at the start of a successor is the minimum over
+ // its predecessors.
+ for (const BasicBlock *Succ : successors(BB)) {
+ if (Depth < DepthIn[Succ->getNumber()]) {
+ DepthIn[Succ->getNumber()] = Depth;
+ Worklist.insert(Succ);
+ }
+ }
+ }
+
+ for (const BasicBlock &BB : F) {
+ unsigned Depth = DepthIn[BB.getNumber()];
+ if (Depth == UnreachedDepth)
+ continue;
+ for (const Instruction &I : BB) {
+ Depth = regionDepthAfter(I, Depth);
+ if (Depth > 0)
+ InRegion.insert(&I);
+ }
+ }
+}
diff --git a/llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll b/llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll
new file mode 100644
index 00000000000000..e3de9022ad54f5
--- /dev/null
+++ b/llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll
@@ -0,0 +1,19 @@
+; RUN: llc -O0 < %s | FileCheck %s
+; RUN: llc -O2 < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel < %s | FileCheck %s
+
+; Verify that the backend lowers the sanitizer region intrinsics to nothing and
+; does not crash when compiling a module where no sanitizer pass ran.
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+define void @test_sanitizer_intrinsics() {
+; CHECK-LABEL: test_sanitizer_intrinsics:
+; CHECK: # %bb.0:
+; CHECK-NEXT: ret{{[ql]?}}
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
diff --git a/llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll b/llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll
new file mode 100644
index 00000000000000..b52aea91a03b1e
--- /dev/null
+++ b/llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll
@@ -0,0 +1,20 @@
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -O0 -fast-isel < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -O0 -global-isel -global-isel-abort=1 < %s | FileCheck %s
+
+; The sanitizer region intrinsics are consumed by sanitizer passes, but they can
+; still reach codegen when no such pass runs. All three instruction selectors
+; have to drop them instead of failing to select.
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+define void @test_sanitizer_intrinsics() {
+; CHECK-LABEL: test_sanitizer_intrinsics:
+; CHECK-NOT: call
+; CHECK: retq
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
diff --git a/llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll b/llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll
new file mode 100644
index 00000000000000..453d336a4494bc
--- /dev/null
+++ b/llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll
@@ -0,0 +1,86 @@
+; RUN: opt -passes='default<O2>' -S < %s | FileCheck %s
+
+; The sanitizer instrumentation region markers may access any memory that is
+; visible outside the function, so the optimizer must not move accesses to such
+; memory into or out of a region, and must not merge, hoist, sink or delete the
+; markers themselves.
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @callee(i32)
+
+define i32 @no_cse_across_markers(ptr %p) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define i32 @no_cse_across_markers(
+; CHECK: %a = load i32, ptr %p
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: %b = load i32, ptr %p
+; CHECK-NEXT: store i32 1, ptr %p
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 2, ptr %p
+ %a = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %b = load i32, ptr %p
+ store i32 1, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 2, ptr %p
+ %r = add i32 %a, %b
+ ret i32 %r
+}
+
+define void @no_licm_out_of_region(ptr %p, i32 %n) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @no_licm_out_of_region(
+; CHECK: loop:
+; CHECK: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: %v = load i32, ptr %p
+; CHECK-NEXT: tail call void @callee(i32 %v)
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+entry:
+ br label %loop
+loop:
+ %i = phi i32 [ 0, %entry ], [ %i.next, %loop ]
+ call void @llvm.sanitizer.instrumentation.begin()
+ %v = load i32, ptr %p
+ call void @callee(i32 %v)
+ call void @llvm.sanitizer.instrumentation.end()
+ %i.next = add i32 %i, 1
+ %c = icmp slt i32 %i.next, %n
+ br i1 %c, label %loop, label %exit
+exit:
+ ret void
+}
+
+define void @no_merge(i1 %c, i32 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @no_merge(
+; CHECK: a:
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: tail call void @callee(i32 %x)
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+; CHECK: b:
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: tail call void @callee(i32 %x)
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+entry:
+ br i1 %c, label %a, label %b
+a:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @callee(i32 %x)
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+b:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @callee(i32 %x)
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+exit:
+ ret void
+}
+
+define void @empty_region() disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @empty_region(
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
diff --git a/llvm/unittests/Transforms/Utils/CMakeLists.txt b/llvm/unittests/Transforms/Utils/CMakeLists.txt
index f4e4c817114c1b..fe325ca9e9dee6 100644
--- a/llvm/unittests/Transforms/Utils/CMakeLists.txt
+++ b/llvm/unittests/Transforms/Utils/CMakeLists.txt
@@ -29,6 +29,7 @@ add_llvm_unittest(UtilsTests
MemTransferLowering.cpp
ModuleUtilsTest.cpp
ScalarEvolutionExpanderTest.cpp
+ SanitizerRegionsTest.cpp
SizeOptsTest.cpp
SSAUpdaterBulkTest.cpp
UnrollLoopTest.cpp
diff --git a/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp b/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
new file mode 100644
index 00000000000000..e2fbe466f0ee55
--- /dev/null
+++ b/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
@@ -0,0 +1,192 @@
+//===- SanitizerRegionsTest.cpp - Unit tests for sanitizer regions --------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/AsmParser/Parser.h"
+#include "llvm/IR/InstIterator.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/Transforms/Utils/Instrumentation.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+
+namespace {
+
+// Every test function loads from %p into values named "in*" for instructions
+// expected to be in a region and "out*" for the rest.
+class SanitizerRegionsTest : public testing::Test {
+protected:
+ LLVMContext C;
+ std::unique_ptr<Module> M;
+
+ Function *parse(StringRef Body) {
+ std::string IR = (Twine(R"(
+ declare void @llvm.sanitizer.instrumentation.begin()
+ declare void @llvm.sanitizer.instrumentation.end()
+ declare void @callee()
+ define void @f(ptr %p, i1 %c) {
+ )") + Body + "}\n")
+ .str();
+ SMDiagnostic Err;
+ M = parseAssemblyString(IR, Err, C);
+ if (!M) {
+ Err.print("SanitizerRegionsTest", errs());
+ return nullptr;
+ }
+ return M->getFunction("f");
+ }
+
+ void expectRegions(StringRef Body, bool ExpectHasRegions = true) {
+ Function *F = parse(Body);
+ ASSERT_TRUE(F);
+ EXPECT_EQ(hasSanitizerInstrumentationRegions(*F), ExpectHasRegions);
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ computeSanitizerInstrumentationRegions(*F, InRegion);
+ unsigned Named = 0;
+ for (const Instruction &I : instructions(*F)) {
+ if (!I.hasName())
+ continue;
+ ++Named;
+ EXPECT_EQ(InRegion.contains(&I), I.getName().starts_with("in"))
+ << I.getName().str();
+ }
+ EXPECT_GT(Named, 0u);
+ }
+};
+
+TEST_F(SanitizerRegionsTest, NoRegions) {
+ expectRegions(R"(
+ %out = load i32, ptr %p
+ ret void
+ )",
+ /*ExpectHasRegions=*/false);
+}
+
+TEST_F(SanitizerRegionsTest, Straight) {
+ expectRegions(R"(
+ %out1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ call void @callee()
+ call void @llvm.sanitizer.instrumentation.end()
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+TEST_F(SanitizerRegionsTest, Nested) {
+ expectRegions(R"(
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %in2 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %out = load i32, ptr %p
+ ret void
+ )");
+}
+
+TEST_F(SanitizerRegionsTest, UnmatchedEnd) {
+ expectRegions(R"(
+ call void @llvm.sanitizer.instrumentation.end()
+ %out1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+// A block that can be reached without passing a begin() is not in a region.
+TEST_F(SanitizerRegionsTest, BeginOnOnePath) {
+ expectRegions(R"(
+ entry:
+ br i1 %c, label %then, label %join
+ then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ br label %join
+ join:
+ %out = load i32, ptr %p
+ ret void
+ )");
+}
+
+// Likewise for a block that can be reached after an end().
+TEST_F(SanitizerRegionsTest, EndOnOnePath) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in1 = load i32, ptr %p
+ br i1 %c, label %then, label %join
+ then:
+ call void @llvm.sanitizer.instrumentation.end()
+ %out1 = load i32, ptr %p
+ br label %join
+ join:
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+TEST_F(SanitizerRegionsTest, BalancedOnBothPaths) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %a, label %b
+ a:
+ %in1 = load i32, ptr %p
+ br label %join
+ b:
+ %in2 = load i32, ptr %p
+ br label %join
+ join:
+ %in3 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %out = load i32, ptr %p
+ ret void
+ )");
+}
+
+// A loop whose body leaves the region on the back edge.
+TEST_F(SanitizerRegionsTest, LoopLeavingRegion) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %loop
+ loop:
+ %out1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ br i1 %c, label %loop, label %exit
+ exit:
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+// A loop entirely inside a region.
+TEST_F(SanitizerRegionsTest, LoopInsideRegion) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %loop
+ loop:
+ %in1 = load i32, ptr %p
+ br i1 %c, label %loop, label %exit
+ exit:
+ %in2 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+ )");
+}
+
+} // namespace
>From 1443efa781943c8cd8b2852f9df37b7ec61fe678 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Mon, 5 Oct 2026 14:04:37 +0000
Subject: [PATCH 2/6] [MSan] Support sanitizer instrumentation regions in KMSAN
The Linux kernel marks functions that must not be instrumented as
`noinstr`, which under KMSAN means `disable_sanitizer_instrumentation`.
Such functions still call instrumented code between the kernel's
`instrumentation_begin()` and `instrumentation_end()` markers. Because
the caller is not instrumented, it never writes parameter shadow, and
the callee reads whatever the last instrumented caller left in the
per-context `param_tls`. This produces false positive KMSAN reports on
the kernel entry paths, e.g.
https://lore.kernel.org/all/69e7ee1f.a00a0220.17a17.001d.GAE@google.com/T/#u
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
MemorySanitizer in kernel mode: in `disable_sanitizer_instrumentation`
functions, fetch the context state at `begin`, and propagate shadow and
set up parameter shadow for calls inside regions, as in a fully
instrumented function. Code outside regions stays uninstrumented; values
computed there are treated as initialized when used inside a region,
and memory written there keeps its previous shadow. The function's own
locals are the exception: an alloca outside the regions is neither
poisoned nor does an out-of-region store update its shadow, so it would
carry whatever shadow the previous occupant of the stack slot left.
Static allocas outside the regions that a region can see (they reach an
in-region instruction through pointer arithmetic, casts, phis or selects,
or their address escapes) are therefore unpoisoned at every `begin` they
dominate, which also covers passing their address to an instrumented
callee inside the region. Memory intrinsics outside regions
are not lowered to `__msan_mem*`. Calls to variadic functions inside
regions publish the shadow of their variadic arguments like any other
in-region call; the callee side of varargs (`va_start`/`va_copy`) is
not handled, as the VarArgHelpers set it up in the function prologue.
The context state slot is null-initialized in the entry block; the
must-analysis in `computeSanitizerInstrumentationRegions()` guarantees
that every in-region use is dominated by a `begin` that stores to it. A
`begin` nested in an outer region does not fetch the context again, and
if no code in the regions uses the context (loads and stores go through
`__msan_metadata_ptr_for_*`), the fetches and the slot are removed.
The region markers are left in place for the sanitizer passes that
run after MemorySanitizer; codegen drops them.
Clang now lowers the region builtins to the intrinsics under
`-fsanitize=kernel-memory`. Userspace MSan ignores the intrinsics.
Tests: `llvm/test/Instrumentation/MemorySanitizer/kmsan-*.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/MemorySanitizer.md | 33 ++
clang/lib/CodeGen/CGBuiltin.cpp | 2 +-
.../sanitizer-instrumentation-builtins.c | 27 +-
...nitizer-instrumentation-regions-pipeline.c | 42 ++
llvm/docs/LangRef.md | 13 +
.../Instrumentation/MemorySanitizer.cpp | 407 +++++++++++++++---
.../MemorySanitizer/kmsan-region-basic.ll | 223 ++++++++++
.../MemorySanitizer/kmsan-region-dataflow.ll | 124 ++++++
.../kmsan-region-mem-intrinsics.ll | 41 ++
.../MemorySanitizer/kmsan-region-outside.ll | 136 ++++++
.../MemorySanitizer/kmsan-region-userspace.ll | 26 ++
.../MemorySanitizer/kmsan-region-varargs.ll | 64 +++
12 files changed, 1070 insertions(+), 68 deletions(-)
create mode 100644 clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-basic.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll
diff --git a/clang/docs/MemorySanitizer.md b/clang/docs/MemorySanitizer.md
index fdc7b7e0935f02..e282184ac7bc60 100644
--- a/clang/docs/MemorySanitizer.md
+++ b/clang/docs/MemorySanitizer.md
@@ -91,6 +91,39 @@ positives and therefore should be used with care, and only if absolutely
required; for example for certain code that cannot tolerate any instrumentation
and resulting side-effects. This attribute overrides `no_sanitize("memory")`.
+### Re-enabling Instrumentation in Regions
+
+In KernelMemorySanitizer (KMSAN), a function with
+`__attribute__((disable_sanitizer_instrumentation))` gets neither the function
+prologue nor any instrumentation. Inside such a function, a region delimited by
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` is instrumented again as if the
+function had `-fsanitize=kernel-memory`: shadow is propagated, memory accesses
+are instrumented, and calls publish the shadow of their arguments. Values
+computed outside a region are treated as initialized when used inside it, and
+memory written outside a region keeps its previous shadow, except for the
+function's own local variables that a region can see (they are used inside a
+region, or their address escapes), which are treated as initialized on entry
+to each region. `va_start` and `va_copy` are not instrumented in such a
+function.
+Regions nest, and code reachable without passing through a
+`__builtin_sanitizer_instrumentation_begin()` is not considered to be in a
+region; Clang warns about unbalanced regions with
+`-Wsanitizer-instrumentation-region`.
+
+```c
+__attribute__((disable_sanitizer_instrumentation))
+void handle_irq(struct pt_regs *regs) {
+ enter_irq_context(regs); // not instrumented
+ __builtin_sanitizer_instrumentation_begin();
+ dispatch_irq(regs); // instrumented, including the argument shadow
+ __builtin_sanitizer_instrumentation_end();
+ exit_irq_context(regs); // not instrumented
+}
+```
+
+Userspace MemorySanitizer ignores the builtins.
+
### Interaction of Inlining with Disabling Sanitizer Instrumentation
- A `no_sanitize` function will not be inlined heuristically by the compiler into a sanitized function.
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index e7f938d63fe750..1f4f2fb46a3b46 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3683,7 +3683,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
// Only emit the region markers for sanitizers that honour them. Elsewhere
// they would just be optimization barriers that no pass ever removes. This
// lets code guard its uses on __has_builtin alone.
- SanitizerMask Consumers;
+ SanitizerMask Consumers = SanitizerKind::KernelMemory;
if (getLangOpts().Sanitize.hasOneOf(Consumers))
Builder.CreateCall(CGM.getIntrinsic(
BuiltinIDIfNoAsmLabel ==
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index 6fef449edd6815..65f36048637cfb 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,10 +1,15 @@
-// Check that the sanitizer region markers are available, and that they lower to
-// nothing without a sanitizer that consumes them.
+// Check that the sanitizer region markers lower to their intrinsics under
+// -fsanitize=kernel-memory, and to nothing without a sanitizer that consumes
+// them.
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
-// RUN: -o - %s | FileCheck %s
+// RUN: -o - %s | FileCheck %s --check-prefix=NONE
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=address \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
#if !__has_builtin(__builtin_sanitizer_instrumentation_begin)
#error "__builtin_sanitizer_instrumentation_begin not available"
@@ -23,5 +28,15 @@ void noinstr_function(int x) {
}
// CHECK-LABEL: define {{.*}}void @noinstr_function(
-// CHECK-NOT: llvm.sanitizer.instrumentation
-// CHECK: }
+// CHECK: call void @llvm.sanitizer.instrumentation.begin()
+// CHECK: call void @instrumented_callee(
+// CHECK: call void @llvm.sanitizer.instrumentation.end()
+
+// The markers are optimization barriers, so they must not be marked readnone
+// or speculatable; only the default attributes plus nomerge are expected.
+// CHECK: declare void @llvm.sanitizer.instrumentation.begin() #[[ATTR:[0-9]+]]
+// CHECK: attributes #[[ATTR]] = { nocallback nofree nomerge nosync nounwind willreturn }
+
+// NONE-LABEL: define {{.*}}void @noinstr_function(
+// NONE-NOT: llvm.sanitizer.instrumentation
+// NONE: }
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
new file mode 100644
index 00000000000000..dde08578d614da
--- /dev/null
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -0,0 +1,42 @@
+// Check that, after the full -O2 pipeline, each sanitizer instruments the code
+// inside a region of a function it otherwise does not instrument, and nothing
+// else. The region markers come from an always_inline helper, as in the Linux
+// kernel. Debug info is enabled so that the verifier checks that the
+// instrumentation calls get debug locations.
+
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=kernel-memory -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN
+
+static inline __attribute__((always_inline)) void instrumentation_begin(void) {
+ __builtin_sanitizer_instrumentation_begin();
+}
+
+static inline __attribute__((always_inline)) void instrumentation_end(void) {
+ __builtin_sanitizer_instrumentation_end();
+}
+
+void instrumented_callee(int);
+
+__attribute__((noinline, disable_sanitizer_instrumentation))
+void noinstr_function(int *p, int x) {
+ int v = *p + (x == 42);
+ instrumentation_begin();
+ if (x == 7)
+ instrumented_callee(*p + v);
+ instrumentation_end();
+ *p = v;
+}
+
+// CHECK-LABEL: define {{.*}}void @noinstr_function(
+// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
+// CHECK: call void @llvm.sanitizer.instrumentation.begin()
+// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
+// The argument is checked eagerly (-fsanitize-memory-param-retval), so the
+// region never touches the context state and does not fetch it.
+// KMSAN: call void @__msan_warning(
+// KMSAN-NOT: @__msan_get_context_state
+// CHECK: call void @instrumented_callee(
+// CHECK: call void @llvm.sanitizer.instrumentation.end()
+// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
+// CHECK: ret void
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 94b507584fcfa3..e4ca5f0e20eb50 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28619,3 +28619,16 @@ intersection over predecessors. This makes an in-region instruction always
dominated by a `llvm.sanitizer.instrumentation.begin`, so any per-function
sanitizer state set up there is available. A region that is entered on some
paths but not others is therefore instrumented only where it is unambiguous.
+
+MemorySanitizer supports regions in kernel mode (KMSAN), in functions marked
+with `disable_sanitizer_instrumentation`. The function-level ABI prologue is
+skipped, the MSan context state is fetched at
+`llvm.sanitizer.instrumentation.begin`, and shadow propagation and parameter
+shadow setup are only performed inside regions. Memory written outside a region
+keeps whatever shadow it had, and values computed outside a region are treated
+as fully initialized when they are used inside one. The function's own static
+`alloca`s are the exception: those that are referenced inside a region, or
+whose address escapes, are unpoisoned at every
+`llvm.sanitizer.instrumentation.begin` they dominate. Calls to variadic
+functions inside regions publish the shadow of the variadic arguments;
+`va_start` and `va_copy` are not instrumented in such a function.
diff --git a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
index 0d710975b593ee..0aecb74df9edda 100644
--- a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
@@ -159,6 +159,7 @@
#include "llvm/ADT/SmallVector.h"
#include "llvm/ADT/StringExtras.h"
#include "llvm/ADT/StringRef.h"
+#include "llvm/Analysis/CaptureTracking.h"
#include "llvm/Analysis/GlobalsModRef.h"
#include "llvm/Analysis/TargetLibraryInfo.h"
#include "llvm/Analysis/ValueTracking.h"
@@ -1072,12 +1073,94 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
SmallVector<std::pair<IntrinsicInst *, AllocaInst *>, 16> LifetimeStartList;
SmallVector<StoreInst *, 16> StoreList;
int64_t SplittableBlocksCount = 0;
+ bool RegionOnly = false;
+ /// Static allocas of a region-instrumented function that lie outside the
+ /// regions but are referenced inside them. They are unpoisoned at every
+ /// non-nested begin() they dominate (see visitIntrinsicInst()).
+ SmallVector<AllocaInst *, 8> OutOfRegionAllocas;
+
+ Value *ContextStateAlloca = nullptr;
+ /// The __msan_get_context_state() calls emitted at the begin() markers of a
+ /// region-instrumented function, with the stores of their results into
+ /// ContextStateAlloca. Removed again if the context turns out to be unused.
+ SmallVector<std::pair<CallInst *, StoreInst *>, 4> ContextStateFetches;
+ /// begin() markers that are nested in an outer region. The outer begin() has
+ /// already fetched the context and unpoisoned the locals.
+ SmallPtrSet<const Instruction *, 4> NestedBegins;
+
+ /// The fields of MS.MsanContextStateTy (struct kmsan_context_state in the
+ /// kernel), in order.
+ enum KmsanContextStateField {
+ ParamShadow,
+ RetvalShadow,
+ VAArgShadow,
+ VAArgOrigin,
+ VAArgOverflowSize,
+ ParamOrigin,
+ RetvalOrigin,
+ };
+
+ static Value *getKmsanContextStateField(IRBuilderBase &IRB, Type *Ty,
+ Value *Ctx,
+ KmsanContextStateField Field,
+ const Twine &Name) {
+ return IRB.CreateGEP(Ty, Ctx, {IRB.getInt32(0), IRB.getInt32(Field)}, Name);
+ }
+
+ /// In a region-instrumented function, the context is only available in the
+ /// stack slot filled at begin(); elsewhere it is live from the prologue.
+ Value *getKmsanTLS(IRBuilderBase &IRB, KmsanContextStateField Field,
+ const Twine &Name) {
+ Value *Ctx = IRB.CreateLoad(MS.PtrTy, ContextStateAlloca);
+ return getKmsanContextStateField(IRB, MS.MsanContextStateTy, Ctx, Field,
+ Name);
+ }
+
+ Value *getParamTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, ParamShadow, "param_shadow")
+ : MS.ParamTLS;
+ }
+ Value *getRetvalTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, RetvalShadow, "retval_shadow")
+ : MS.RetvalTLS;
+ }
+ Value *getVAArgTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, VAArgShadow, "va_arg_shadow")
+ : MS.VAArgTLS;
+ }
+ Value *getVAArgOriginTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, VAArgOrigin, "va_arg_origin")
+ : MS.VAArgOriginTLS;
+ }
+ Value *getVAArgOverflowSizeTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca
+ ? getKmsanTLS(IRB, VAArgOverflowSize, "va_arg_overflow_size")
+ : MS.VAArgOverflowSizeTLS;
+ }
+ Value *getParamOriginTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, ParamOrigin, "param_origin")
+ : MS.ParamOriginTLS;
+ }
+ Value *getRetvalOriginTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, RetvalOrigin, "retval_origin")
+ : MS.RetvalOriginTLS;
+ }
+
+ SmallPtrSet<const Instruction *, 16> InRegion;
MemorySanitizerVisitor(Function &F, MemorySanitizer &MS,
const TargetLibraryInfo &TLI)
: Opts(MS.Opts), F(F), MS(MS), VAHelper(CreateVarArgHelper(F, MS, *this)),
TLI(&TLI) {
- bool SanitizeFunction = F.hasFnAttribute(Attribute::SanitizeMemory) &&
+ RegionOnly =
+ MS.CompileKernel &&
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation);
+
+ // Instructions outside the regions of a region-instrumented function are
+ // never visited (see visit()), so the function is handled as if it had
+ // sanitize_memory.
+ bool SanitizeFunction = (F.hasFnAttribute(Attribute::SanitizeMemory) ||
+ RegionOnly) &&
!Opts.msan_disable_checks;
InsertChecks = SanitizeFunction;
PropagateShadow = SanitizeFunction;
@@ -1098,7 +1181,24 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
if (MS.CompileKernel) {
IRBuilder<> IRB(FnPrologueEnd);
- insertKmsanPrologue(IRB);
+ if (MS.TargetTriple.getArch() == Triple::systemz)
+ MS.MsanMetadataAlloca = IRB.CreateAlloca(MS.MsanMetadata, 0u);
+ // A region-instrumented function must not call into the runtime outside
+ // a region, so it cannot fetch the context in the prologue. Each begin()
+ // fetches it instead and keeps it in a stack slot for the region.
+ if (!RegionOnly) {
+ insertKmsanPrologue(IRB);
+ } else {
+ ContextStateAlloca =
+ IRB.CreateAlloca(MS.PtrTy, nullptr, "msan_context_state");
+ // No context is available until the first begin().
+ // computeRegionMembership() guarantees that every instruction that
+ // loads this slot is dominated by a begin() that stored into it, so
+ // this store is dead and will be removed. Keep it anyway: should that
+ // invariant ever be broken, faulting on a null pointer is a far better
+ // failure mode than corrupting memory through an uninitialized one.
+ IRB.CreateStore(Constant::getNullValue(MS.PtrTy), ContextStateAlloca);
+ }
}
LLVM_DEBUG(if (!InsertChecks) dbgs()
@@ -1440,38 +1540,137 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
LLVM_DEBUG(dbgs() << "DONE:\n" << F);
}
- // Returns the last instruction in the new prologue
+ /// Fetch the current KMSAN context and cache it for the rest of the
+ /// function in SSA values.
void insertKmsanPrologue(IRBuilder<> &IRB) {
- Value *ContextState = IRB.CreateCall(MS.MsanGetContextStateFn, {});
- Constant *Zero = IRB.getInt32(0);
- MS.ParamTLS = IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
- {Zero, IRB.getInt32(0)}, "param_shadow");
- MS.RetvalTLS = IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
- {Zero, IRB.getInt32(1)}, "retval_shadow");
- MS.VAArgTLS = IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
- {Zero, IRB.getInt32(2)}, "va_arg_shadow");
- MS.VAArgOriginTLS = IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
- {Zero, IRB.getInt32(3)}, "va_arg_origin");
- MS.VAArgOverflowSizeTLS =
- IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
- {Zero, IRB.getInt32(4)}, "va_arg_overflow_size");
- MS.ParamOriginTLS = IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
- {Zero, IRB.getInt32(5)}, "param_origin");
+ Value *Ctx = IRB.CreateCall(MS.MsanGetContextStateFn, {});
+ Type *Ty = MS.MsanContextStateTy;
+ MS.ParamTLS =
+ getKmsanContextStateField(IRB, Ty, Ctx, ParamShadow, "param_shadow");
+ MS.RetvalTLS =
+ getKmsanContextStateField(IRB, Ty, Ctx, RetvalShadow, "retval_shadow");
+ MS.VAArgTLS =
+ getKmsanContextStateField(IRB, Ty, Ctx, VAArgShadow, "va_arg_shadow");
+ MS.VAArgOriginTLS =
+ getKmsanContextStateField(IRB, Ty, Ctx, VAArgOrigin, "va_arg_origin");
+ MS.VAArgOverflowSizeTLS = getKmsanContextStateField(
+ IRB, Ty, Ctx, VAArgOverflowSize, "va_arg_overflow_size");
+ MS.ParamOriginTLS =
+ getKmsanContextStateField(IRB, Ty, Ctx, ParamOrigin, "param_origin");
MS.RetvalOriginTLS =
- IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
- {Zero, IRB.getInt32(6)}, "retval_origin");
- if (MS.TargetTriple.getArch() == Triple::systemz)
- MS.MsanMetadataAlloca = IRB.CreateAlloca(MS.MsanMetadata, 0u);
+ getKmsanContextStateField(IRB, Ty, Ctx, RetvalOrigin, "retval_origin");
+ }
+
+ /// Fetch the current KMSAN context and cache it in ContextStateAlloca. Used
+ /// at every llvm.sanitizer.instrumentation.begin() of a region-instrumented
+ /// function.
+ void fetchKmsanContextState(IRBuilderBase &IRB) {
+ CallInst *Ctx = IRB.CreateCall(MS.MsanGetContextStateFn, {});
+ ContextStateFetches.emplace_back(Ctx, IRB.CreateStore(Ctx, ContextStateAlloca));
+ }
+
+ /// Drop the __msan_get_context_state() calls if nothing in the regions ended
+ /// up using the context (e.g. regions with only loads and stores, which go
+ /// through __msan_metadata_ptr_for_*). The call has side effects as far as
+ /// the optimizer knows, so it would not be removed otherwise.
+ void removeUnusedKmsanContextState() {
+ auto *Alloca = cast_or_null<AllocaInst>(ContextStateAlloca);
+ if (!Alloca || any_of(Alloca->users(), [](const User *U) {
+ return !isa<StoreInst>(U);
+ }))
+ return;
+ for (auto [Ctx, Store] : ContextStateFetches) {
+ Store->eraseFromParent();
+ Ctx->eraseFromParent();
+ }
+ ContextStateFetches.clear();
+ // Only the null store from the prologue is left.
+ for (User *U : make_early_inc_range(Alloca->users()))
+ cast<Instruction>(U)->eraseFromParent();
+ Alloca->eraseFromParent();
+ ContextStateAlloca = nullptr;
+ }
+
+ /// Compute, for every instruction of a region-instrumented function, whether
+ /// it belongs to an llvm.sanitizer.instrumentation.begin()/end() region.
+ ///
+ /// computeSanitizerInstrumentationRegions() solves this as a *must* dataflow
+ /// problem, which MSan relies on for the soundness of the cached context
+ /// state: every begin() stores the result of __msan_get_context_state() into
+ /// %msan_context_state, and in-region instrumentation loads the pointer back
+ /// from that slot. With a "may" (union) join, a block reachable both from
+ /// inside and from outside a region would be instrumented, and on the path
+ /// that never executed begin() the instrumentation would load an
+ /// uninitialized alloca and then read and *write* through the resulting wild
+ /// pointer. With the intersection join an in-region instruction is always
+ /// dominated by a begin(), so the slot is always initialized.
+ void computeRegionMembership() {
+ computeSanitizerInstrumentationRegions(F, InRegion);
+
+ // A begin() is nested if the code right before it is already in a region:
+ // the preceding instruction, or, at the start of a block, the terminators
+ // of *all* predecessors. (With the must-join, a block entered from inside
+ // a region on one path and from outside on another starts outside, and
+ // its begin() is the one that fetches the context on the latter path.)
+ for (BasicBlock &BB : F) {
+ for (Instruction &I : BB) {
+ if (!isSanitizerInstrumentationBegin(I))
+ continue;
+ bool Nested;
+ if (Instruction *Prev = I.getPrevNode())
+ Nested = InRegion.contains(Prev);
+ else
+ Nested = !pred_empty(&BB) &&
+ all_of(predecessors(&BB), [&](const BasicBlock *Pred) {
+ return InRegion.contains(Pred->getTerminator());
+ });
+ if (Nested)
+ NestedBegins.insert(&I);
+ }
+ }
}
/// Add MemorySanitizer instrumentation to a function.
bool runOnFunction() {
+ if (RegionOnly)
+ computeRegionMembership();
+
// Iterate all BBs in depth-first order and create shadow instructions
// for all instructions (where applicable).
// For PHI nodes we create dummy shadow PHIs which will be finalized later.
for (BasicBlock *BB : depth_first(FnPrologueEnd->getParent()))
visit(*BB);
+ // Only the locals that the regions can see need their shadow reset at
+ // begin(): those that reach an in-region instruction through pointer
+ // arithmetic, casts, phis or selects, and those whose address escapes (is
+ // stored to memory, cast to an integer, passed to a call, ...), as the
+ // regions may then reach them in ways the def-use chain does not show. A
+ // local that is neither can only be accessed outside the regions.
+ if (!OutOfRegionAllocas.empty()) {
+ // One backward walk from all in-region pointer operands, so that a
+ // pointer graph shared by several locals is traversed once.
+ SmallPtrSet<const Value *, 16> Visited;
+ SmallVector<const Value *, 16> Worklist;
+ for (const Instruction *I : InRegion)
+ for (const Value *Op : I->operands())
+ if (Op->getType()->isPointerTy() && Visited.insert(Op).second)
+ Worklist.push_back(Op);
+ while (!Worklist.empty()) {
+ const auto *I = dyn_cast<Instruction>(Worklist.pop_back_val());
+ if (!I || !isa<GetElementPtrInst, BitCastInst, AddrSpaceCastInst,
+ PHINode, SelectInst>(I))
+ continue;
+ for (const Value *Op : I->operands())
+ if (Op->getType()->isPointerTy() && Visited.insert(Op).second)
+ Worklist.push_back(Op);
+ }
+ erase_if(OutOfRegionAllocas, [&](const AllocaInst *AI) {
+ return !Visited.contains(AI) &&
+ !PointerMayBeCaptured(AI, /*ReturnCaptures=*/true);
+ });
+ }
+
// `visit` above only collects instructions. Process them after iterating
// CFG to avoid requirement on CFG transformations.
for (Instruction *I : Instructions)
@@ -1489,7 +1688,11 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
}
}
- VAHelper->finalizeInstrumentation();
+ // The callee side of varargs is set up in the prologue, outside every
+ // region of a region-instrumented function, where the context state is
+ // not available. Such a function has no instrumented va_start anyway.
+ if (!RegionOnly)
+ VAHelper->finalizeInstrumentation();
// Poison llvm.lifetime.start intrinsics, if we haven't fallen back to
// instrumenting only allocas.
@@ -1511,6 +1714,8 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
// This may not add new address checks.
materializeStores();
+ removeUnusedKmsanContextState();
+
return true;
}
@@ -1804,7 +2009,7 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
///
/// Shadow = ParamTLS+ArgOffset.
Value *getShadowPtrForArgument(IRBuilder<> &IRB, int ArgOffset) {
- return IRB.CreatePtrAdd(MS.ParamTLS,
+ return IRB.CreatePtrAdd(getParamTLS(IRB),
ConstantInt::get(MS.IntptrTy, ArgOffset), "_msarg");
}
@@ -1812,20 +2017,20 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
Value *getOriginPtrForArgument(IRBuilder<> &IRB, int ArgOffset) {
if (!MS.TrackOrigins)
return nullptr;
- return IRB.CreatePtrAdd(MS.ParamOriginTLS,
+ return IRB.CreatePtrAdd(getParamOriginTLS(IRB),
ConstantInt::get(MS.IntptrTy, ArgOffset),
"_msarg_o");
}
/// Compute the shadow address for a retval.
Value *getShadowPtrForRetval(IRBuilder<> &IRB) {
- return IRB.CreatePointerCast(MS.RetvalTLS, IRB.getPtrTy(0), "_msret");
+ return IRB.CreatePointerCast(getRetvalTLS(IRB), IRB.getPtrTy(0), "_msret");
}
/// Compute the origin address for a retval.
- Value *getOriginPtrForRetval() {
+ Value *getOriginPtrForRetval(IRBuilderBase &IRB) {
// We keep a single origin for the entire retval. Might be too optimistic.
- return MS.RetvalOriginTLS;
+ return getRetvalOriginTLS(IRB);
}
/// Set SV to be the shadow value for V.
@@ -1920,6 +2125,11 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
return AllOnes;
}
if (Argument *A = dyn_cast<Argument>(V)) {
+ if (RegionOnly) {
+ if (MS.TrackOrigins && !OriginMap.count(A))
+ setOrigin(A, getCleanOrigin());
+ return getCleanShadow(V);
+ }
// For arguments we compute the shadow on demand and store it in the map.
Value *&ShadowPtr = ShadowMap[V];
if (ShadowPtr)
@@ -2190,6 +2400,7 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
// Don't want to visit if we're in the prologue
if (isInPrologue(I))
return;
+
if (!DebugCounter::shouldExecute(DebugInstrumentInstruction)) {
LLVM_DEBUG(dbgs() << "Skipping instruction: " << I << "\n");
// We still need to set the shadow and origin to clean values.
@@ -2198,6 +2409,21 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
return;
}
+ // In a region-instrumented function, code outside the regions must be left
+ // exactly as it is: no runtime calls (which would leave .noinstr.text in
+ // the kernel), no changes to atomic orderings, no shadow computations.
+ // Values computed there are treated as initialized when used inside a
+ // region, and memory written there keeps whatever shadow it had. Locals
+ // are the exception: their shadow is whatever the previous occupant of
+ // the stack slot left, so they are unpoisoned at every begin() instead.
+ if (RegionOnly && !InRegion.contains(&I)) {
+ setShadow(&I, getCleanShadow(&I));
+ setOrigin(&I, getCleanOrigin());
+ if (auto *AI = dyn_cast<AllocaInst>(&I); AI && AI->isStaticAlloca())
+ OutOfRegionAllocas.push_back(AI);
+ return;
+ }
+
Instructions.push_back(&I);
}
@@ -3269,9 +3495,20 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
I.eraseFromParent();
}
- void visitVAStartInst(VAStartInst &I) { VAHelper->visitVAStartInst(I); }
+ // In a region-instrumented function only the caller side of varargs is
+ // handled (see visitCallBase()): the VarArgHelpers set up the callee side
+ // in the function prologue, which is outside every region.
+ void visitVAStartInst(VAStartInst &I) {
+ if (RegionOnly)
+ return;
+ VAHelper->visitVAStartInst(I);
+ }
- void visitVACopyInst(VACopyInst &I) { VAHelper->visitVACopyInst(I); }
+ void visitVACopyInst(VACopyInst &I) {
+ if (RegionOnly)
+ return;
+ VAHelper->visitVACopyInst(I);
+ }
/// Handle vector store-like intrinsics.
///
@@ -7579,6 +7816,40 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
}
void visitIntrinsicInst(IntrinsicInst &I) {
+ switch (I.getIntrinsicID()) {
+ case Intrinsic::sanitizer_instrumentation_begin: {
+ // Only meaningful in a region-instrumented function: elsewhere the
+ // context is already live from the function prologue. A nested begin()
+ // is dominated by the outer one, which has already done all this.
+ if (RegionOnly && !NestedBegins.contains(&I)) {
+ NextNodeIRBuilder IRB(&I);
+ fetchKmsanContextState(IRB);
+ // The locals allocated outside the regions were neither poisoned at
+ // their alloca nor had their shadow updated by the stores outside the
+ // regions, so their shadow is stale. Treat them like any other value
+ // coming from outside a region, i.e. as initialized. Static allocas
+ // live in the entry block, so they dominate this begin() unless they
+ // follow it in the same block.
+ // (Decide first, then emit: comesBefore() renumbers the block after
+ // every insertion.)
+ SmallVector<AllocaInst *, 8> ToUnpoison;
+ for (AllocaInst *AI : OutOfRegionAllocas)
+ if (AI->getParent() != I.getParent() || AI->comesBefore(&I))
+ ToUnpoison.push_back(AI);
+ for (AllocaInst *AI : ToUnpoison)
+ IRB.CreateCall(MS.MsanUnpoisonAllocaFn,
+ {AI, IRB.CreateAllocationSize(MS.IntptrTy, AI)});
+ }
+ // The region markers are shared by all sanitizers, so leave them for the
+ // passes that run after this one. They are dropped during codegen.
+ return;
+ }
+ case Intrinsic::sanitizer_instrumentation_end:
+ return;
+ default:
+ break;
+ }
+
if (maybeHandleCrossPlatformIntrinsic(I))
return;
@@ -7850,7 +8121,8 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
"_msret");
setShadow(&CB, RetvalShadow);
if (MS.TrackOrigins)
- setOrigin(&CB, IRBAfter.CreateLoad(MS.OriginTy, getOriginPtrForRetval()));
+ setOrigin(&CB, IRBAfter.CreateLoad(MS.OriginTy,
+ getOriginPtrForRetval(IRBAfter)));
}
bool isAMustTailRetVal(Value *RetVal) {
@@ -7891,7 +8163,7 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
if (StoreShadow) {
IRB.CreateAlignedStore(Shadow, ShadowPtr, kShadowTLSAlignment);
if (MS.TrackOrigins && StoreOrigin)
- IRB.CreateStore(getOrigin(RetVal), getOriginPtrForRetval());
+ IRB.CreateStore(getOrigin(RetVal), getOriginPtrForRetval(IRB));
}
}
@@ -8290,14 +8562,15 @@ struct VarArgHelperBase : public VarArgHelper {
: F(F), MS(MS), MSV(MSV), VAListTagSize(VAListTagSize) {}
Value *getShadowAddrForVAArgument(IRBuilder<> &IRB, unsigned ArgOffset) {
- Value *Base = IRB.CreatePointerCast(MS.VAArgTLS, MS.IntptrTy);
+ Value *Base = IRB.CreatePointerCast(MSV.getVAArgTLS(IRB), MS.IntptrTy);
return IRB.CreateAdd(Base, ConstantInt::get(MS.IntptrTy, ArgOffset));
}
/// Compute the shadow address for a given va_arg.
Value *getShadowPtrForVAArgument(IRBuilder<> &IRB, unsigned ArgOffset) {
- return IRB.CreatePtrAdd(
- MS.VAArgTLS, ConstantInt::get(MS.IntptrTy, ArgOffset), "_msarg_va_s");
+ return IRB.CreatePtrAdd(MSV.getVAArgTLS(IRB),
+ ConstantInt::get(MS.IntptrTy, ArgOffset),
+ "_msarg_va_s");
}
/// Compute the shadow address for a given va_arg.
@@ -8314,7 +8587,7 @@ struct VarArgHelperBase : public VarArgHelper {
// getOriginPtrForVAArgument() is always called after
// getShadowPtrForVAArgument(), so __msan_va_arg_origin_tls can never
// overflow.
- return IRB.CreatePtrAdd(MS.VAArgOriginTLS,
+ return IRB.CreatePtrAdd(MSV.getVAArgOriginTLS(IRB),
ConstantInt::get(MS.IntptrTy, ArgOffset),
"_msarg_va_o");
}
@@ -8505,7 +8778,7 @@ struct VarArgAMD64Helper : public VarArgHelperBase {
}
Constant *OverflowSize =
ConstantInt::get(IRB.getInt64Ty(), OverflowOffset - AMD64FpEndOffset);
- IRB.CreateStore(OverflowSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(OverflowSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
@@ -8516,7 +8789,7 @@ struct VarArgAMD64Helper : public VarArgHelperBase {
// va_arg_tls somewhere in the function entry block.
IRBuilder<> IRB(MSV.FnPrologueEnd);
VAArgOverflowSize =
- IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = IRB.CreateAdd(
ConstantInt::get(MS.IntptrTy, AMD64FpEndOffset), VAArgOverflowSize);
VAArgTLSCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
@@ -8527,13 +8800,14 @@ struct VarArgAMD64Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
if (MS.TrackOrigins) {
VAArgTLSOriginCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
VAArgTLSOriginCopy->setAlignment(kShadowTLSAlignment);
IRB.CreateMemCpy(VAArgTLSOriginCopy, kShadowTLSAlignment,
- MS.VAArgOriginTLS, kShadowTLSAlignment, SrcSize);
+ MSV.getVAArgOriginTLS(IRB), kShadowTLSAlignment,
+ SrcSize);
}
}
@@ -8682,7 +8956,7 @@ struct VarArgAArch64Helper : public VarArgHelperBase {
}
Constant *OverflowSize =
ConstantInt::get(IRB.getInt64Ty(), OverflowOffset - AArch64VAEndOffset);
- IRB.CreateStore(OverflowSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(OverflowSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
// Retrieve a va_list field of 'void*' size.
@@ -8708,7 +8982,7 @@ struct VarArgAArch64Helper : public VarArgHelperBase {
// va_arg_tls somewhere in the function entry block.
IRBuilder<> IRB(MSV.FnPrologueEnd);
VAArgOverflowSize =
- IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = IRB.CreateAdd(
ConstantInt::get(MS.IntptrTy, AArch64VAEndOffset), VAArgOverflowSize);
VAArgTLSCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
@@ -8719,7 +8993,7 @@ struct VarArgAArch64Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -8909,14 +9183,15 @@ struct VarArgPowerPC64Helper : public VarArgHelperBase {
ConstantInt::get(MS.IntptrTy, VAArgOffset - VAArgBase);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ VAArgSize =
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -8931,7 +9206,7 @@ struct VarArgPowerPC64Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(IRB.getInt64Ty(), kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9045,14 +9320,14 @@ struct VarArgPowerPC32Helper : public VarArgHelperBase {
ConstantInt::get(MS.IntptrTy, VAArgOffset - VAArgBase);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(MS.IntptrTy, MS.VAArgOverflowSizeTLS);
+ VAArgSize = IRB.CreateLoad(MS.IntptrTy, MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -9067,7 +9342,7 @@ struct VarArgPowerPC32Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9330,7 +9605,7 @@ struct VarArgSystemZHelper : public VarArgHelperBase {
}
Constant *OverflowSize = ConstantInt::get(
IRB.getInt64Ty(), OverflowOffset - SystemZOverflowOffset);
- IRB.CreateStore(OverflowSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(OverflowSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void copyRegSaveArea(IRBuilder<> &IRB, Value *VAListTag) {
@@ -9391,7 +9666,7 @@ struct VarArgSystemZHelper : public VarArgHelperBase {
// va_arg_tls somewhere in the function entry block.
IRBuilder<> IRB(MSV.FnPrologueEnd);
VAArgOverflowSize =
- IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize =
IRB.CreateAdd(ConstantInt::get(MS.IntptrTy, SystemZOverflowOffset),
VAArgOverflowSize);
@@ -9403,13 +9678,14 @@ struct VarArgSystemZHelper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
if (MS.TrackOrigins) {
VAArgTLSOriginCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
VAArgTLSOriginCopy->setAlignment(kShadowTLSAlignment);
IRB.CreateMemCpy(VAArgTLSOriginCopy, kShadowTLSAlignment,
- MS.VAArgOriginTLS, kShadowTLSAlignment, SrcSize);
+ MSV.getVAArgOriginTLS(IRB), kShadowTLSAlignment,
+ SrcSize);
}
}
@@ -9485,14 +9761,14 @@ struct VarArgI386Helper : public VarArgHelperBase {
Constant *TotalVAArgSize = ConstantInt::get(MS.IntptrTy, VAArgOffset);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(MS.IntptrTy, MS.VAArgOverflowSizeTLS);
+ VAArgSize = IRB.CreateLoad(MS.IntptrTy, MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -9506,7 +9782,7 @@ struct VarArgI386Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9570,14 +9846,14 @@ struct VarArgGenericHelper : public VarArgHelperBase {
Constant *TotalVAArgSize = ConstantInt::get(MS.IntptrTy, VAArgOffset);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(MS.IntptrTy, MS.VAArgOverflowSizeTLS);
+ VAArgSize = IRB.CreateLoad(MS.IntptrTy, MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -9591,7 +9867,7 @@ struct VarArgGenericHelper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9698,8 +9974,17 @@ bool MemorySanitizer::sanitizeFunction(Function &F, TargetLibraryInfo &TLI) {
if (!CompileKernel && F.getName() == kMsanModuleCtorName)
return false;
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
- return false;
+ // Functions marked disable_sanitizer_instrumentation are normally skipped
+ // entirely. They are still instrumented inside
+ // llvm.sanitizer.instrumentation.begin()/end() regions, so look for those
+ // first.
+ if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation)) {
+ // Region instrumentation is a KernelMemorySanitizer feature: it refers to
+ // the context state returned by __msan_get_context_state(), which only
+ // exists in the kernel runtime.
+ if (!CompileKernel || !hasSanitizerInstrumentationRegions(F))
+ return false;
+ }
MemorySanitizerVisitor Visitor(F, *this, TLI);
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-basic.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-basic.ll
new file mode 100644
index 00000000000000..6fb8ae5d09ba14
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-basic.ll
@@ -0,0 +1,223 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; Basic KMSAN handling of llvm.sanitizer.instrumentation.begin()/end() regions.
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare i64 @dummy_call(i64)
+
+define void @instrumented_function() sanitize_memory {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+define void @no_sanitize_function() {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; Outside disable_sanitizer_instrumentation functions the markers mean nothing
+; to KMSAN: the context is fetched once in the prologue, not at begin().
+; CHECK-LABEL: @instrumented_function(
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK-NOT: call ptr @__msan_get_context_state()
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+; CHECK-LABEL: @no_sanitize_function(
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK-NOT: call ptr @__msan_get_context_state()
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+; Only the accesses inside the region are instrumented. The region does not
+; use the context state (loads and stores go through
+; __msan_metadata_ptr_for_*), so it is not fetched at all.
+define void @noinstr_function(ptr %p, i64 %val) disable_sanitizer_instrumentation {
+entry:
+ %x = load i64, ptr %p
+ %add = add i64 %x, %val
+ call void @llvm.sanitizer.instrumentation.begin()
+ %y = load i64, ptr %p
+ %add2 = add i64 %y, %add
+ store i64 %add2, ptr %p
+ %z = load i64, ptr %p
+ store i64 %z, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %w = load i64, ptr %p
+ ret void
+}
+
+; CHECK-LABEL: @noinstr_function(
+; CHECK-NOT: msan_context_state
+; CHECK: %x = load i64, ptr %p
+; CHECK-NEXT: %add = add i64 %x, %val
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: %y = load i64, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: [[YS:%_msld.*]] = load i64, ptr
+; %add was computed outside the region, so its shadow is clean.
+; CHECK: [[ADD2S:%.*]] = or i64 [[YS]], 0
+; CHECK: %add2 = add i64 %y, %add
+; CHECK: [[SP1:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: [[SHP1:%.*]] = extractvalue { ptr, ptr } [[SP1]], 0
+; CHECK: store i64 [[ADD2S]], ptr [[SHP1]]
+; CHECK: store i64 %add2, ptr %p
+; CHECK: %z = load i64, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: [[ZS:%_msld.*]] = load i64, ptr
+; CHECK: [[SP2:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: [[SHP2:%.*]] = extractvalue { ptr, ptr } [[SP2]], 0
+; CHECK: store i64 [[ZS]], ptr [[SHP2]]
+; CHECK: store i64 %z, ptr %p
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: %w = load i64, ptr %p
+; CHECK-NEXT: ret void
+; CHECK-NOT: msan_context_state
+
+define void @noinstr_with_branch(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %cond, label %if.then, label %if.else
+
+if.then:
+ %a = load i64, ptr %p
+ store i64 %a, ptr %p
+ br label %if.end
+
+if.else:
+ %b = load i64, ptr %p
+ store i64 %b, ptr %p
+ br label %if.end
+
+if.end:
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @noinstr_with_branch(
+; CHECK: if.then:
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: if.else:
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+
+define void @noinstr_with_loop(ptr %p, i64 %n) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %loop
+
+loop:
+ %i = phi i64 [ 0, %entry ], [ %inc, %loop ]
+ %c = load i64, ptr %p
+ store i64 %c, ptr %p
+ %inc = add i64 %i, 1
+ %cmp = icmp ult i64 %inc, %n
+ br i1 %cmp, label %loop, label %exit
+
+exit:
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @noinstr_with_loop(
+; CHECK: loop:
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: exit:
+
+; A call inside the region needs the context state for the parameter and
+; return value shadow: it is fetched at begin() and kept in a stack slot.
+; Nothing outside the region is touched.
+define i64 @noinstr_with_various_insts(ptr %p, i64 %val) disable_sanitizer_instrumentation {
+entry:
+ %a = alloca i64
+ %rmw1 = atomicrmw add ptr %p, i64 %val seq_cst
+ %call1 = call i64 @dummy_call(i64 %rmw1)
+ store i64 %call1, ptr %a
+
+ call void @llvm.sanitizer.instrumentation.begin()
+ %rmw2 = atomicrmw add ptr %p, i64 %val seq_cst
+ %call2 = call i64 @dummy_call(i64 %rmw2)
+ store i64 %call2, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+
+ %rmw3 = atomicrmw add ptr %p, i64 %val seq_cst
+ %call3 = call i64 @dummy_call(i64 %rmw3)
+ store i64 %call3, ptr %a
+ ret i64 %call3
+}
+
+; CHECK-LABEL: @noinstr_with_various_insts(
+; CHECK: %msan_context_state = alloca ptr
+; CHECK-NEXT: store ptr null, ptr %msan_context_state
+; CHECK-NOT: __msan_metadata
+; CHECK-NOT: __msan_get_context_state
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: [[CTX:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT: store ptr [[CTX]], ptr %msan_context_state
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: load ptr, ptr %msan_context_state
+; CHECK: %param_shadow = getelementptr
+; CHECK: %call2 = call i64 @dummy_call(i64 %rmw2)
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NOT: __msan_metadata
+; CHECK-NOT: __msan_get_context_state
+; CHECK: ret i64 %call3
+
+; A nested begin() is dominated by the outer one, which has already fetched
+; the context; the context is fetched once.
+define void @nested_call(i64 %val) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ %call = call i64 @dummy_call(i64 %val)
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @nested_call(
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @__msan_get_context_state()
+; CHECK-NEXT: store ptr {{.*}}, ptr %msan_context_state
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NOT: __msan_get_context_state
+; CHECK: %call = call i64 @dummy_call(i64 %val)
+
+; A begin() at the start of a block that is entered from inside a region on
+; one path and from outside on the other is not nested: the block starts
+; outside the region (must-join), so this begin() has to fetch the context.
+; (The input is unbalanced and diagnosed, but must not miscompile.)
+define void @begin_after_partial_join(i64 %val, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ br i1 %c, label %a, label %b
+
+a:
+ br label %join
+
+b:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+
+join:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %call = call i64 @dummy_call(i64 %val)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @begin_after_partial_join(
+; CHECK: join:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @__msan_get_context_state()
+; CHECK-NEXT: store ptr {{.*}}, ptr %msan_context_state
+; CHECK: %call = call i64 @dummy_call(i64 %val)
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll
new file mode 100644
index 00000000000000..aee6d9fdb20ea8
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll
@@ -0,0 +1,124 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; Region membership is a "must" dataflow problem over the region nesting depth.
+; These tests cover the cases where an "any path" or flag-based analysis gets it
+; wrong.
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+; A block reachable both from inside and from outside a region must not be
+; instrumented: on the path that never executed begin(), %msan_context_state
+; holds no context state, and the instrumentation would load a wild pointer out
+; of it and then store through it.
+
+define i32 @unbalanced_join(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ br i1 %cond, label %left, label %right
+
+left:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %merge
+
+right:
+ br label %merge
+
+merge:
+ %v = load i32, ptr %p
+ ret i32 %v
+}
+
+; CHECK-LABEL: @unbalanced_join(
+; CHECK: merge:
+; CHECK-NOT: load ptr, ptr %msan_context_state
+; CHECK-NOT: __msan_metadata_ptr_for_load_4
+; CHECK: ret i32 %v
+
+; The same join is instrumented once every incoming path has entered a region.
+
+define i32 @balanced_join(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ br i1 %cond, label %left, label %right
+
+left:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %merge
+
+right:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %merge
+
+merge:
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; CHECK-LABEL: @balanced_join(
+; CHECK: merge:
+; CHECK: %v = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+
+; Regions nest: the inner end() must not terminate the outer region.
+
+define void @nested_regions(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ %a = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %b = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %c = load i32, ptr %p
+ ret void
+}
+
+; CHECK-LABEL: @nested_regions(
+; CHECK: %a = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+; CHECK: %b = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+; CHECK: %c = load i32, ptr %p
+; CHECK-NOT: __msan_metadata_ptr_for_load_4
+; CHECK: ret void
+
+; Nesting also has to survive a join: both predecessors of %merge are at depth
+; one, so %merge is still inside the outer region.
+
+define void @nested_regions_join(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %cond, label %inner, label %merge
+
+inner:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %merge
+
+merge:
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @nested_regions_join(
+; CHECK: merge:
+; CHECK: %v = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+
+; An end() without a matching begin() is malformed input. Saturate the depth at
+; zero rather than wrapping around and instrumenting the rest of the function.
+
+define void @unmatched_end(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.end()
+ %v = load i32, ptr %p
+ ret void
+}
+
+; CHECK-LABEL: @unmatched_end(
+; CHECK-NOT: __msan_metadata_ptr_for_load_4
+; CHECK: ret void
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll
new file mode 100644
index 00000000000000..b51a62abe33b69
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll
@@ -0,0 +1,41 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; Memory intrinsics are rewritten into KMSAN runtime calls. Outside an
+; instrumentation region they have to be left alone: emitting the call there
+; instruments code the region markers explicitly excluded, and in noinstr kernel
+; code it makes objtool report
+; "call to __msan_memcpy() leaves .noinstr.text section".
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.memcpy.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @llvm.memmove.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @llvm.memset.p0.i64(ptr, i8, i64, i1)
+
+define void @mem_intrinsics(ptr %dst, ptr %src) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memmove.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memset.p0.i64(ptr %dst, i8 0, i64 64, i1 false)
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memmove.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memset.p0.i64(ptr %dst, i8 0, i64 64, i1 false)
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ ret void
+}
+
+; CHECK-LABEL: @mem_intrinsics(
+; CHECK: call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+; CHECK: call void @llvm.memmove.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+; CHECK: call void @llvm.memset.p0.i64(ptr %dst, i8 0, i64 64, i1 false)
+; CHECK-NOT: __msan_get_context_state
+; CHECK: call ptr @__msan_memcpy(ptr %dst, ptr %src, i64 64)
+; CHECK: call ptr @__msan_memmove(ptr %dst, ptr %src, i64 64)
+; CHECK: call ptr @__msan_memset(ptr %dst, i32 0, i64 64)
+; CHECK: call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+; CHECK: ret void
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll
new file mode 100644
index 00000000000000..4bc119b157c0b6
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll
@@ -0,0 +1,136 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; In a disable_sanitizer_instrumentation function, code outside the regions is
+; left exactly as it is: no KMSAN runtime calls, no stronger atomic orderings,
+; no shadow computations. Inside a region the function is instrumented like a
+; sanitize_memory one, including poisoning undef and poison values.
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.masked.store.v4i32.p0(<4 x i32>, ptr, i32, <4 x i1>)
+
+define i32 @outside(ptr %p, <4 x i32> %v, <4 x i1> %m) disable_sanitizer_instrumentation {
+entry:
+ %a = load atomic i32, ptr %p monotonic, align 4
+ store atomic i32 1, ptr %p monotonic, align 4
+ %r = atomicrmw add ptr %p, i32 1 monotonic
+ %x = cmpxchg ptr %p, i32 0, i32 1 monotonic monotonic
+ call void @llvm.masked.store.v4i32.p0(<4 x i32> %v, ptr %p, i32 4, <4 x i1> %m)
+ %b = xor i32 %a, %r
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 poison, ptr %p, align 4
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %b
+}
+
+; CHECK-LABEL: define i32 @outside(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: call void @llvm.donothing()
+; CHECK-NEXT: %a = load atomic i32, ptr %p monotonic, align 4
+; CHECK-NEXT: store atomic i32 1, ptr %p monotonic, align 4
+; CHECK-NEXT: %r = atomicrmw add ptr %p, i32 1 monotonic
+; CHECK-NEXT: %x = cmpxchg ptr %p, i32 0, i32 1 monotonic monotonic
+; CHECK-NEXT: call void @llvm.masked.store.v4i32.p0(<4 x i32> %v, ptr align 4 %p, <4 x i1> %m)
+; CHECK-NEXT: %b = xor i32 %a, %r
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: [[SP:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_4(ptr %p)
+; CHECK-NEXT: [[SHP:%.*]] = extractvalue { ptr, ptr } [[SP]], 0
+; CHECK-NEXT: extractvalue
+; CHECK-NEXT: store i32 -1, ptr [[SHP]], align 4
+; CHECK: store i32 poison, ptr %p, align 4
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret i32 %b
+
+; Locals allocated outside the regions get no __msan_poison_alloca, and stores
+; to them outside the regions do not update their shadow, so their shadow is
+; whatever was left in the stack slot. Those that the regions can see (as the
+; underlying object of an in-region pointer operand, e.g. a load or a call
+; argument) are unpoisoned at every begin() they dominate, so that reading
+; them inside a region does not report; %unused is never referenced inside a
+; region and is left alone. Allocas inside a region are poisoned at the alloca
+; as usual; an alloca that follows the begin() in the entry block is not
+; unpoisoned at that begin().
+
+define void @outside_allocas(i64 %v, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ %a = alloca i64
+ %arr = alloca [4 x i32]
+ %unused = alloca i64
+ store i64 %v, ptr %a
+ store i64 %v, ptr %unused
+ call void @llvm.sanitizer.instrumentation.begin()
+ %x = load i64, ptr %a
+ call void @use(i64 %x)
+ %elt = getelementptr [4 x i32], ptr %arr, i64 0, i64 2
+ call void @use_ptr(ptr %elt)
+ %in = alloca i64
+ call void @llvm.sanitizer.instrumentation.end()
+ %late = alloca i64
+ br i1 %c, label %again, label %exit
+
+again:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %y = load i64, ptr %late
+ call void @use(i64 %y)
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+
+exit:
+ ret void
+}
+
+declare void @use(i64)
+declare void @use_ptr(ptr)
+
+; CHECK-LABEL: define void @outside_allocas(
+; CHECK: %a = alloca i64
+; CHECK-NEXT: %arr = alloca [4 x i32]
+; CHECK-NEXT: %unused = alloca i64
+; CHECK-NEXT: store i64 %v, ptr %a
+; CHECK-NEXT: store i64 %v, ptr %unused
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @__msan_get_context_state()
+; CHECK-NEXT: store ptr
+; CHECK-NEXT: call void @__msan_unpoison_alloca(ptr %a, i64 8)
+; CHECK-NEXT: call void @__msan_unpoison_alloca(ptr %arr, i64 16)
+; CHECK-NOT: __msan_unpoison_alloca
+; CHECK: %in = alloca i64
+; CHECK-NEXT: call void @__msan_poison_alloca(ptr %in, i64 8,
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: %late = alloca i64
+; CHECK-NEXT: br i1 %c
+; CHECK: again:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @__msan_get_context_state()
+; CHECK-NEXT: store ptr
+; CHECK-NEXT: call void @__msan_unpoison_alloca(ptr %a, i64 8)
+; CHECK-NEXT: call void @__msan_unpoison_alloca(ptr %arr, i64 16)
+; CHECK-NEXT: call void @__msan_unpoison_alloca(ptr %late, i64 8)
+; CHECK-NEXT: %y = load i64, ptr %late
+; CHECK-NEXT: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %late)
+
+; A local whose address escapes outside the regions (here: stored to memory)
+; may be reached inside them through that memory, so it is unpoisoned too.
+define void @escaped_alloca(ptr %slot) disable_sanitizer_instrumentation {
+entry:
+ %esc = alloca i64
+ %local = alloca i64
+ store ptr %esc, ptr %slot
+ store i64 1, ptr %local
+ call void @llvm.sanitizer.instrumentation.begin()
+ %p = load ptr, ptr %slot
+ %v = load i64, ptr %p
+ call void @use(i64 %v)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @escaped_alloca(
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @__msan_get_context_state()
+; CHECK-NEXT: store ptr
+; CHECK-NEXT: call void @__msan_unpoison_alloca(ptr %esc, i64 8)
+; CHECK-NEXT: %p = load ptr, ptr %slot
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll
new file mode 100644
index 00000000000000..8c1d79d8243bc7
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll
@@ -0,0 +1,26 @@
+; RUN: opt < %s -S -passes=msan 2>&1 | FileCheck %s
+
+; Region instrumentation is a KernelMemorySanitizer feature: it refers to the
+; context state returned by __msan_get_context_state(), which only exists in the
+; kernel runtime. Under userspace MemorySanitizer the markers must simply be
+; ignored - this used to crash the compiler on a null MsanGetContextStateFn.
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+define i32 @region_in_userspace_msan(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; CHECK-LABEL: @region_in_userspace_msan(
+; CHECK-NOT: __msan_get_context_state
+; CHECK-NOT: msan_context_state
+; CHECK: %v = load i32, ptr %p
+; CHECK: ret i32 %v
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll
new file mode 100644
index 00000000000000..17e79aebe50966
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll
@@ -0,0 +1,64 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; Varargs in a region-instrumented function.
+;
+; The callee side (va_start/va_copy) is not instrumented: the VarArgHelpers copy
+; va_arg_tls in the function prologue, which is outside every region, so
+; %msan_context_state has not been stored to yet and the emitted code would
+; load that slot and then read and write through the resulting pointer.
+;
+; The caller side is instrumented: a call to a variadic function inside a
+; region publishes the shadow of the variadic arguments and their total size,
+; like any other in-region call publishes param_shadow. Otherwise an
+; instrumented variadic callee would consume whatever the last variadic call
+; left in the context.
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.va_start.p0(ptr)
+declare void @llvm.va_end.p0(ptr)
+declare i32 @vprintf_like(ptr, ...)
+
+define i32 @va_noinstr(i32 %n, ...) disable_sanitizer_instrumentation {
+entry:
+ %ap = alloca [1 x { i32, i32, ptr, ptr }]
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.va_start.p0(ptr %ap)
+ %v = va_arg ptr %ap, i32
+ call void @llvm.va_end.p0(ptr %ap)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; Nothing in this function uses the context state, so it is not even fetched;
+; in particular no va_arg_tls is copied in the prologue.
+; CHECK-LABEL: @va_noinstr(
+; CHECK-NOT: msan_context_state
+; CHECK-NOT: va_arg_overflow_size
+; CHECK-NOT: va_arg_shadow
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NOT: msan_context_state
+; CHECK-NOT: va_arg_overflow_size
+; CHECK-NOT: va_arg_shadow
+; CHECK: ret i32
+
+define void @va_call(ptr %fmt, ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %x = load i64, ptr %p
+ call i32 (ptr, ...) @vprintf_like(ptr %fmt, i64 %x)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @va_call(
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK: [[XS:%_msld.*]] = load i64, ptr
+; CHECK: %va_arg_shadow = getelementptr {{.*}}, ptr {{%.*}}, i32 0, i32 2
+; CHECK: store i64 [[XS]], ptr {{%.*}}
+; CHECK: %va_arg_overflow_size = getelementptr {{.*}}, ptr {{%.*}}, i32 0, i32 4
+; CHECK: store i64 0, ptr %va_arg_overflow_size
+; CHECK: call i32 (ptr, ...) @vprintf_like(ptr %fmt, i64 %x)
>From e9c59e66c2e60ffdd970bde09c9c5ac501009062 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Mon, 5 Oct 2026 14:04:47 +0000
Subject: [PATCH 3/6] [ASan] Support sanitizer instrumentation regions
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
AddressSanitizer, in both user and kernel mode. Functions that lack
`sanitize_address` or have `disable_sanitizer_instrumentation` are
normally not instrumented at all; if they contain regions, memory
accesses and memory intrinsics inside the regions are now checked.
The Linux kernel uses this for `noinstr` functions, which may call
instrumented code between `instrumentation_begin()` and
`instrumentation_end()`, but whose own memory accesses in those
regions were never checked by KASAN.
Only memory accesses are instrumented in such functions. Their stack
frame is not laid out by ASan, so stack poisoning, use-after-return and
the `__asan_handle_no_return()` calls are skipped, and accesses to the
function's own allocas, which have no redzones, are not checked. A call
outside a region still resets the per-block deduplication of checks.
Clang now lowers the region builtins to the intrinsics under
`-fsanitize=address` and `-fsanitize=kernel-address`.
Tests: `llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/AddressSanitizer.md | 23 ++
clang/lib/CodeGen/CGBuiltin.cpp | 4 +-
.../sanitizer-instrumentation-builtins.c | 12 +-
...nitizer-instrumentation-regions-pipeline.c | 4 +
llvm/docs/LangRef.md | 6 +
.../Instrumentation/AddressSanitizer.cpp | 75 ++++--
.../sanitizer-instrumentation-regions.ll | 217 ++++++++++++++++++
7 files changed, 322 insertions(+), 19 deletions(-)
create mode 100644 llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll
diff --git a/clang/docs/AddressSanitizer.md b/clang/docs/AddressSanitizer.md
index 19276261b75620..3442ebb357ff78 100644
--- a/clang/docs/AddressSanitizer.md
+++ b/clang/docs/AddressSanitizer.md
@@ -296,6 +296,29 @@ AddressSanitizer also supports
works similarly to `__attribute__((no_sanitize("address")))`, but it also
prevents instrumentation performed by other sanitizers.
+### Re-enabling Instrumentation in Regions
+
+Inside a function that AddressSanitizer does not instrument, a region
+delimited by `__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` is instrumented again: memory
+accesses and memory intrinsics in it are checked. The function's stack frame
+is still not instrumented, and accesses to its own local variables are not
+checked. Regions nest, and code reachable without passing through a
+`__builtin_sanitizer_instrumentation_begin()` is not considered to be in a
+region. On targets where the shadow memory base is dynamic, it is still
+loaded at the function entry, outside the regions; this is a plain load, not
+a call into the runtime.
+
+```c
+__attribute__((no_sanitize("address")))
+void mostly_uninstrumented(int *p) {
+ ... // not checked
+ __builtin_sanitizer_instrumentation_begin();
+ *p = 0; // checked
+ __builtin_sanitizer_instrumentation_end();
+}
+```
+
### Interaction of Inlining with Disabling Sanitizer Instrumentation
- A `no_sanitize` function will not be inlined heuristically by the compiler into a sanitized function.
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 1f4f2fb46a3b46..40f32c324ead5b 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3683,7 +3683,9 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
// Only emit the region markers for sanitizers that honour them. Elsewhere
// they would just be optimization barriers that no pass ever removes. This
// lets code guard its uses on __has_builtin alone.
- SanitizerMask Consumers = SanitizerKind::KernelMemory;
+ SanitizerMask Consumers = SanitizerKind::KernelMemory |
+ SanitizerKind::Address |
+ SanitizerKind::KernelAddress;
if (getLangOpts().Sanitize.hasOneOf(Consumers))
Builder.CreateCall(CGM.getIntrinsic(
BuiltinIDIfNoAsmLabel ==
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index 65f36048637cfb..fdc1e3187bb069 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,13 +1,15 @@
// Check that the sanitizer region markers lower to their intrinsics under
-// -fsanitize=kernel-memory, and to nothing without a sanitizer that consumes
-// them.
+// -fsanitize=kernel-memory, kernel-address and address, and to nothing without
+// a sanitizer that consumes them.
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
// RUN: -o - %s | FileCheck %s --check-prefix=NONE
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=address \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
@@ -37,6 +39,10 @@ void noinstr_function(int x) {
// CHECK: declare void @llvm.sanitizer.instrumentation.begin() #[[ATTR:[0-9]+]]
// CHECK: attributes #[[ATTR]] = { nocallback nofree nomerge nosync nounwind willreturn }
+// ASAN-LABEL: define {{.*}}void @noinstr_function(
+// ASAN: call void @llvm.sanitizer.instrumentation.begin()
+// ASAN: call void @llvm.sanitizer.instrumentation.end()
+
// NONE-LABEL: define {{.*}}void @noinstr_function(
// NONE-NOT: llvm.sanitizer.instrumentation
// NONE: }
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
index dde08578d614da..3ba3b3868dbc91 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -7,6 +7,9 @@
// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
// RUN: -fsanitize=kernel-memory -emit-llvm -o - %s \
// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=kernel-address -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KASAN
static inline __attribute__((always_inline)) void instrumentation_begin(void) {
__builtin_sanitizer_instrumentation_begin();
@@ -32,6 +35,7 @@ void noinstr_function(int *p, int x) {
// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
// CHECK: call void @llvm.sanitizer.instrumentation.begin()
// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
+// KASAN: call void @__asan_report_load4(
// The argument is checked eagerly (-fsanitize-memory-param-retval), so the
// region never touches the context state and does not fetch it.
// KMSAN: call void @__msan_warning(
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index e4ca5f0e20eb50..1e2e8809851264 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28632,3 +28632,9 @@ whose address escapes, are unpoisoned at every
`llvm.sanitizer.instrumentation.begin` they dominate. Calls to variadic
functions inside regions publish the shadow of the variadic arguments;
`va_start` and `va_copy` are not instrumented in such a function.
+
+AddressSanitizer supports regions in functions that lack the `sanitize_address`
+attribute or are marked with `disable_sanitizer_instrumentation`. Inside
+regions, memory accesses and memory intrinsics are instrumented. The stack frame
+of such a function is not instrumented, accesses to its own `alloca`s are not
+checked, and no `__asan_handle_no_return` calls are inserted.
diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
index 70841c8324aa3f..fe544d6776bdb4 100644
--- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
@@ -2879,13 +2879,23 @@ bool AddressSanitizer::instrumentFunction(Function &F,
if (maybeInsertAsanInitAtFunctionEntry(F))
FunctionModified = true;
- // Leave if the function doesn't need instrumentation.
- if (!F.hasFnAttribute(Attribute::SanitizeAddress)) return FunctionModified;
-
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
- return FunctionModified;
-
- LLVM_DEBUG(dbgs() << "ASAN instrumenting:\n" << F << "\n");
+ // Leave if the function doesn't need instrumentation. Functions that are
+ // not instrumented as a whole are still instrumented inside
+ // llvm.sanitizer.instrumentation.begin()/end() regions. Only memory
+ // accesses are instrumented there: the stack frame of such a function is
+ // not laid out by ASan, so stack poisoning, use-after-return and the
+ // handling of noreturn calls (which unpoisons the stack) are left out.
+ bool RegionOnly = false;
+ if (!F.hasFnAttribute(Attribute::SanitizeAddress) ||
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation)) {
+ if (!hasSanitizerInstrumentationRegions(F))
+ return FunctionModified;
+ RegionOnly = true;
+ }
+
+ LLVM_DEBUG(dbgs() << "ASAN instrumenting" << (RegionOnly ? " regions" : "")
+ << ":\n"
+ << F << "\n");
initializeCallbacks(TLI);
@@ -2893,14 +2903,32 @@ bool AddressSanitizer::instrumentFunction(Function &F,
RuntimeCallInserter RTCI(F);
+ // On targets with a dynamic shadow base, the base is loaded in the entry
+ // block, i.e. outside the regions of a region-only function. That is a load
+ // of a global (or an empty inline asm), not a call into the runtime, so it
+ // is acceptable even in a function that must not call instrumented code.
FunctionModified |= maybeInsertDynamicShadowAtFunctionEntry(F);
- // We can't instrument allocas used with llvm.localescape. Only static allocas
- // can be passed to that intrinsic.
- markEscapedLocalAllocas(F);
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ if (RegionOnly) {
+ computeSanitizerInstrumentationRegions(F, InRegion);
+ } else {
+ // We can't instrument allocas used with llvm.localescape. Only static
+ // allocas can be passed to that intrinsic. (The locals of a region-only
+ // function are not instrumented at all, see below.)
+ markEscapedLocalAllocas(F);
- if (TargetTriple.isOSWindows())
- markCatchParametersAsUninteresting(F);
+ if (TargetTriple.isOSWindows())
+ markCatchParametersAsUninteresting(F);
+ }
+
+ // The locals of a function that is only instrumented in regions have no
+ // redzones, so checking them can only trip over stale stack poisoning.
+ auto IsLocalOnly = [](const Value *Ptr) {
+ SmallVector<const Value *, 4> Objects;
+ getUnderlyingObjects(Ptr, Objects);
+ return all_of(Objects, [](const Value *V) { return isa<AllocaInst>(V); });
+ };
// We want to instrument every address only once per basic block (unless there
// are calls between uses).
@@ -2922,12 +2950,20 @@ bool AddressSanitizer::instrumentFunction(Function &F,
// Skip instructions inserted by another instrumentation.
if (Inst.hasMetadata(LLVMContext::MD_nosanitize))
continue;
+ if (RegionOnly && !InRegion.contains(&Inst)) {
+ // A call between two regions can free memory like any other call.
+ if (isa<CallBase>(Inst))
+ TempsToInstrument.clear();
+ continue;
+ }
SmallVector<InterestingMemoryOperand, 1> InterestingOperands;
getInterestingMemoryOperands(&Inst, InterestingOperands, TTI);
AssumeInst *AI;
if (!InterestingOperands.empty()) {
for (auto &Operand : InterestingOperands) {
+ if (RegionOnly && IsLocalOnly(Operand.getPtr()))
+ continue;
if (Opts.asan_opt && Opts.asan_opt_same_temp) {
Value *Ptr = Operand.getPtr();
// If we have a mask, skip instrumentation if we've already
@@ -2952,6 +2988,10 @@ bool AddressSanitizer::instrumentFunction(Function &F,
isInterestingPointerSubtraction(&Inst))) {
PointerComparisonsOrSubtracts.push_back(&Inst);
} else if (MemIntrinsic *MI = dyn_cast<MemIntrinsic>(&Inst)) {
+ if (RegionOnly && IsLocalOnly(MI->getDest()) &&
+ (!isa<MemTransferInst>(MI) ||
+ IsLocalOnly(cast<MemTransferInst>(MI)->getSource())))
+ continue;
// ok, take it.
IntrinToInstrument.push_back(MI);
NumInsnsPerBB++;
@@ -2965,7 +3005,9 @@ bool AddressSanitizer::instrumentFunction(Function &F,
if (auto *CB = dyn_cast<CallBase>(&Inst)) {
// A call inside BB.
TempsToInstrument.clear();
- if (CB->doesNotReturn())
+ // The stack of a region-only function is not poisoned, so there is
+ // nothing to unpoison before a noreturn call.
+ if (CB->doesNotReturn() && !RegionOnly)
NoReturnCalls.push_back(CB);
}
if (CallInst *CI = dyn_cast<CallInst>(&Inst))
@@ -2996,8 +3038,11 @@ bool AddressSanitizer::instrumentFunction(Function &F,
FunctionModified = true;
}
- FunctionStackPoisoner FSP(F, *this, RTCI);
- bool ChangedStack = FSP.runOnFunction();
+ bool ChangedStack = false;
+ if (!RegionOnly) {
+ FunctionStackPoisoner FSP(F, *this, RTCI);
+ ChangedStack = FSP.runOnFunction();
+ }
// We must unpoison the stack before NoReturn calls (throw, _exit, etc).
// See e.g. https://github.com/google/sanitizers/issues/37
diff --git a/llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll b/llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll
new file mode 100644
index 00000000000000..f53278bb19cdb1
--- /dev/null
+++ b/llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll
@@ -0,0 +1,217 @@
+; Test that functions not instrumented by AddressSanitizer are still
+; instrumented inside llvm.sanitizer.instrumentation.begin()/end() regions, and
+; only there.
+;
+; RUN: opt < %s -passes=asan -asan-kernel=1 \
+; RUN: -asan-instrumentation-with-call-threshold=0 -S | FileCheck %s
+; RUN: opt < %s -passes=asan -asan-instrumentation-with-call-threshold=0 -S \
+; RUN: | FileCheck %s
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.memcpy.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @callee()
+
+; Only the access between begin() and end() is checked.
+define i32 @region(ptr %a, ptr %b, ptr %c) {
+entry:
+ %x = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ %y = load i32, ptr %b
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 %y, ptr %c
+ %s = add i32 %x, %y
+ ret i32 %s
+}
+
+; CHECK-LABEL: define i32 @region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %y = load i32, ptr %b
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 %y, ptr %c
+; CHECK-NOT: __asan_
+; CHECK: ret i32
+
+; The same for functions with disable_sanitizer_instrumentation.
+define i32 @region_disable(ptr %a, ptr %b) disable_sanitizer_instrumentation {
+entry:
+ %x = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 %x, ptr %b
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %x
+}
+
+; CHECK-LABEL: define i32 @region_disable(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 %x, ptr %b
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret i32 %x
+
+; A block reachable without passing begin() is not in a region.
+define i32 @diamond(i1 %cond, ptr %a) {
+entry:
+ br i1 %cond, label %then, label %join
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+join:
+ %x = load i32, ptr %a
+ ret i32 %x
+}
+
+; CHECK-LABEL: define i32 @diamond(
+; CHECK-NOT: __asan_
+; CHECK: ret i32
+
+; The inner end() of nested regions does not end the outer region.
+define void @nested(ptr %a) {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 0, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 1, ptr %a
+ ret void
+}
+
+; CHECK-LABEL: define void @nested(
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 0, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 1, ptr %a
+; CHECK-NEXT: ret void
+
+; The function's own locals have no redzones and are not checked; neither is
+; its stack frame laid out by ASan.
+define i32 @locals(ptr %p) {
+entry:
+ %local = alloca i32
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %local
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; CHECK-LABEL: define i32 @locals(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %local = alloca i32
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: store i32 1, ptr %local
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %v = load i32, ptr %p
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret i32 %v
+
+; Locals reached through a phi or select, and memory intrinsics that only touch
+; locals, are not checked either; a memcpy between a local and non-local memory
+; still is.
+define void @locals_indirect(ptr %p, i1 %c) {
+entry:
+ %a = alloca i32
+ %b = alloca [16 x i8]
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %then, label %join
+
+then:
+ br label %join
+
+join:
+ %phi = phi ptr [ %a, %entry ], [ %b, %then ]
+ %sel = select i1 %c, ptr %a, ptr %b
+ store i32 1, ptr %phi
+ store i32 2, ptr %sel
+ call void @llvm.memcpy.p0.p0.i64(ptr %b, ptr %a, i64 4, i1 false)
+ call void @llvm.memcpy.p0.p0.i64(ptr %b, ptr %p, i64 4, i1 false)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @locals_indirect(
+; CHECK: join:
+; CHECK-NEXT: %phi = phi ptr
+; CHECK-NEXT: %sel = select i1 %c, ptr %a, ptr %b
+; CHECK-NEXT: store i32 1, ptr %phi
+; CHECK-NEXT: store i32 2, ptr %sel
+; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr %b, ptr %a, i64 4, i1 false)
+; CHECK-NEXT: call ptr @{{(__asan_)?}}memcpy(ptr %b, ptr %p, i64 4)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+
+; A call between two regions may free memory, so the second access to the
+; same address is checked again.
+define void @call_between_regions(ptr %p) {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @callee()
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @call_between_regions(
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 0, ptr %p
+; CHECK: call void @callee()
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 1, ptr %p
+
+; Memory intrinsics are only replaced inside regions.
+define void @memcpy(ptr %d, ptr %s) {
+entry:
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %s, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %s, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @memcpy(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %s, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @{{(__asan_)?}}memcpy(ptr %d, ptr %s, i64 16)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+
+; Without regions, functions lacking sanitize_address are left alone.
+define i32 @no_region(ptr %a) {
+entry:
+ %x = load i32, ptr %a
+ ret i32 %x
+}
+
+; CHECK-LABEL: define i32 @no_region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK-NEXT: ret i32 %x
+
+; Fully instrumented functions are unaffected by regions.
+define i32 @sanitized(ptr %a, ptr %b) sanitize_address {
+entry:
+ %x = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ %y = load i32, ptr %b
+ call void @llvm.sanitizer.instrumentation.end()
+ %s = add i32 %x, %y
+ ret i32 %s
+}
+
+; CHECK-LABEL: define i32 @sanitized(
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %y = load i32, ptr %b
>From 4229bf7bfe15f91d4e33ec0c4cfd05252e5ce22a Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Tue, 6 Oct 2026 10:23:12 +0000
Subject: [PATCH 4/6] [SanitizerCoverage] Support sanitizer instrumentation
regions
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
SanitizerCoverage. Functions with `nosanitize_coverage` or
`disable_sanitizer_instrumentation` are normally not instrumented at
all; if they contain regions, the code inside the regions is now
instrumented.
The Linux kernel uses this for `noinstr` functions, which may call
instrumented code between `instrumentation_begin()` and
`instrumentation_end()`. Today KCOV does not see that code, so the
fuzzer gets no coverage or comparison feedback for it, e.g. for the
syscall and exception entry paths.
Each block that enters a region gets a coverage point right after the
point where it does, and comparisons, switches, divisions, GEPs, loads,
stores and indirect calls inside regions are traced as requested. A
block whose coverage point is not at its start, or that is the entry
block, is split there, so that the callback runs inside the region and
the PC table can refer to the block by address. Pruning is disabled in
such functions, as it assumes that all blocks are instrumented, and
function-level instrumentation (entry/exit callbacks, stack depth
tracing, control flow tables) is not emitted, as it would run outside
the regions.
Clang now lowers the region builtins to the intrinsics whenever
`-fsanitize-coverage` instrumentation is enabled.
Tests: `llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/SanitizerCoverage.md | 25 ++
clang/lib/CodeGen/CGBuiltin.cpp | 3 +-
.../sanitizer-instrumentation-builtins.c | 17 +-
...nitizer-instrumentation-regions-pipeline.c | 11 +
llvm/docs/LangRef.md | 9 +
.../Instrumentation/SanitizerCoverage.cpp | 108 ++++++-
.../sanitizer-instrumentation-regions.ll | 276 ++++++++++++++++++
7 files changed, 428 insertions(+), 21 deletions(-)
create mode 100644 llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll
diff --git a/clang/docs/SanitizerCoverage.md b/clang/docs/SanitizerCoverage.md
index ae074d13f5322a..05d1529e4aeec8 100644
--- a/clang/docs/SanitizerCoverage.md
+++ b/clang/docs/SanitizerCoverage.md
@@ -446,6 +446,31 @@ function attribute `__attribute__((no_sanitize("coverage")))`. Because this
attribute may not be supported by other compilers, it is recommended to use it
together with `__has_feature(coverage_sanitizer)`.
+### Re-enabling instrumentation in regions
+
+Inside a function that has `__attribute__((no_sanitize("coverage")))` or
+`__attribute__((disable_sanitizer_instrumentation))`, a region delimited by
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` is instrumented again: every basic
+block that enters a region gets a coverage point right after it does, and
+comparisons, switches, divisions, GEPs, loads, stores and indirect calls in
+regions are traced as requested. Coverage points are not pruned in such
+functions, and the function entry and exit callbacks, stack depth tracing and
+control flow tables are not emitted. Regions nest, and code reachable without
+passing through a `__builtin_sanitizer_instrumentation_begin()` is not
+considered to be in a region.
+
+```c
+__attribute__((no_sanitize("coverage")))
+void mostly_uninstrumented(int x) {
+ ... // not covered
+ __builtin_sanitizer_instrumentation_begin();
+ if (x) // covered, and traced with -fsanitize-coverage=trace-cmp
+ foo();
+ __builtin_sanitizer_instrumentation_end();
+}
+```
+
## Disabling instrumentation without source modification
It is sometimes useful to tell SanitizerCoverage to instrument only a subset of the
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 40f32c324ead5b..78e463a1869512 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3686,7 +3686,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
SanitizerMask Consumers = SanitizerKind::KernelMemory |
SanitizerKind::Address |
SanitizerKind::KernelAddress;
- if (getLangOpts().Sanitize.hasOneOf(Consumers))
+ if (getLangOpts().Sanitize.hasOneOf(Consumers) ||
+ CGM.getCodeGenOpts().hasSanitizeCoverage())
Builder.CreateCall(CGM.getIntrinsic(
BuiltinIDIfNoAsmLabel ==
Builtin::BI__builtin_sanitizer_instrumentation_begin
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index fdc1e3187bb069..36123ff4a4453f 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,15 +1,18 @@
// Check that the sanitizer region markers lower to their intrinsics under
-// -fsanitize=kernel-memory, kernel-address and address, and to nothing without
-// a sanitizer that consumes them.
+// -fsanitize=kernel-memory, kernel-address and address, and with
+// -fsanitize-coverage, and to nothing without a sanitizer that consumes them.
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
// RUN: -o - %s | FileCheck %s --check-prefix=NONE
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=MARKERS
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=address \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=MARKERS
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize-coverage-type=3 \
+// RUN: -fsanitize-coverage-trace-pc -emit-llvm -disable-llvm-passes -o - %s \
+// RUN: | FileCheck %s --check-prefix=MARKERS
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
@@ -39,9 +42,9 @@ void noinstr_function(int x) {
// CHECK: declare void @llvm.sanitizer.instrumentation.begin() #[[ATTR:[0-9]+]]
// CHECK: attributes #[[ATTR]] = { nocallback nofree nomerge nosync nounwind willreturn }
-// ASAN-LABEL: define {{.*}}void @noinstr_function(
-// ASAN: call void @llvm.sanitizer.instrumentation.begin()
-// ASAN: call void @llvm.sanitizer.instrumentation.end()
+// MARKERS-LABEL: define {{.*}}void @noinstr_function(
+// MARKERS: call void @llvm.sanitizer.instrumentation.begin()
+// MARKERS: call void @llvm.sanitizer.instrumentation.end()
// NONE-LABEL: define {{.*}}void @noinstr_function(
// NONE-NOT: llvm.sanitizer.instrumentation
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
index 3ba3b3868dbc91..2589a4f05c84a8 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -10,6 +10,15 @@
// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
// RUN: -fsanitize=kernel-address -emit-llvm -o - %s \
// RUN: | FileCheck %s --check-prefixes=CHECK,KASAN
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize-coverage-type=3 -fsanitize-coverage-trace-pc \
+// RUN: -fsanitize-coverage-trace-cmp -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KCOV
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=kernel-memory -fsanitize-coverage-type=3 \
+// RUN: -fsanitize-coverage-trace-pc -fsanitize-coverage-trace-cmp \
+// RUN: -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN,KCOV
static inline __attribute__((always_inline)) void instrumentation_begin(void) {
__builtin_sanitizer_instrumentation_begin();
@@ -34,6 +43,8 @@ void noinstr_function(int *p, int x) {
// CHECK-LABEL: define {{.*}}void @noinstr_function(
// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
// CHECK: call void @llvm.sanitizer.instrumentation.begin()
+// KCOV: call void @__sanitizer_cov_trace_pc()
+// KCOV: call void @__sanitizer_cov_trace_const_cmp4(i32 7,
// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
// KASAN: call void @__asan_report_load4(
// The argument is checked eagerly (-fsanitize-memory-param-retval), so the
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 1e2e8809851264..64ac4e7c592ff4 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28638,3 +28638,12 @@ attribute or are marked with `disable_sanitizer_instrumentation`. Inside
regions, memory accesses and memory intrinsics are instrumented. The stack frame
of such a function is not instrumented, accesses to its own `alloca`s are not
checked, and no `__asan_handle_no_return` calls are inserted.
+
+SanitizerCoverage supports regions in functions marked with
+`nosanitize_coverage` or `disable_sanitizer_instrumentation`. Each basic block
+that enters a region gets a coverage point right after the point where it does
+(splitting the block if necessary), without pruning, and the requested tracing
+of comparisons, switches, divisions, GEPs, loads, stores and indirect calls is
+performed inside regions. Function entry/exit callbacks, stack depth tracing and
+control
+flow tables are not emitted for such functions.
diff --git a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
index 88f39f904d803f..b8bab5ed0a565a 100644
--- a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
+++ b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
@@ -36,6 +36,7 @@
#include "llvm/TargetParser/Triple.h"
#include "llvm/Transforms/Utils/BasicBlockUtils.h"
#include "llvm/Transforms/Utils/EscapeEnumerator.h"
+#include "llvm/Transforms/Utils/Instrumentation.h"
#include "llvm/Transforms/Utils/ModuleUtils.h"
using namespace llvm;
@@ -209,6 +210,9 @@ class ModuleSanitizerCoverage {
Module &M;
DomTreeCallback DTCallback;
PostDomTreeCallback PDTCallback;
+ /// The function being instrumented is only instrumented inside
+ /// llvm.sanitizer.instrumentation.begin()/end() regions.
+ bool RegionOnly = false;
FunctionCallee SanCovStackDepthCallback;
FunctionCallee SanCovTracePCIndir;
@@ -552,6 +556,23 @@ static bool shouldInstrumentBlock(const Function &F, const BasicBlock *BB,
!(isFullPostDominator(BB, PDT) && !BB->getSinglePredecessor());
}
+// Returns the first instruction of \p BB inside a sanitizer instrumentation
+// region before which coverage can be inserted, or nullptr if there is none.
+// The region's begin() marker is skipped: it is the first instruction of the
+// region, but code inserted before it would run outside the region.
+static Instruction *
+getRegionCoveragePoint(BasicBlock &BB,
+ const SmallPtrSetImpl<const Instruction *> &InRegion) {
+ if (isa<UnreachableInst>(BB.getFirstNonPHIOrLifetime()))
+ return nullptr;
+ for (Instruction &I : make_range(BB.getFirstInsertionPt(), BB.end())) {
+ if (!InRegion.contains(&I) || isSanitizerInstrumentationBegin(I))
+ continue;
+ return &I;
+ }
+ return nullptr;
+}
+
// Returns true iff From->To is a backedge.
// A twist here is that we treat From->To as a backedge if
// * To dominates From or
@@ -612,14 +633,43 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
// Do not apply any instrumentation for naked functions.
if (F.hasFnAttribute(Attribute::Naked))
return;
- if (F.hasFnAttribute(Attribute::NoSanitizeCoverage))
- return;
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
- return;
+ // Functions that are not instrumented as a whole are still instrumented
+ // inside llvm.sanitizer.instrumentation.begin()/end() regions. There, every
+ // block that reaches a region gets a coverage point right after the point
+ // where it enters the region; function-level instrumentation (entry/exit
+ // callbacks, stack depth, control flow tables) is left out, as it would run
+ // outside of the regions.
+ RegionOnly = false;
+ if (F.hasFnAttribute(Attribute::NoSanitizeCoverage) ||
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation)) {
+ if (!hasSanitizerInstrumentationRegions(F))
+ return;
+ RegionOnly = true;
+ }
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ if (RegionOnly)
+ computeSanitizerInstrumentationRegions(F, InRegion);
if (Options.CoverageType >= SanitizerCoverageOptions::SCK_Edge) {
- SplitAllCriticalEdges(
- F, CriticalEdgeSplittingOptions().setIgnoreUnreachableDests());
+ CriticalEdgeSplittingOptions SplitOptions;
+ SplitOptions.setIgnoreUnreachableDests();
+ if (!RegionOnly) {
+ SplitAllCriticalEdges(F, SplitOptions);
+ } else {
+ // Only the critical edges leaving an in-region terminator matter for
+ // the coverage inside the regions; splitting the others would only
+ // churn the CFG of the code outside the regions. The split block
+ // inherits the region membership of the edge.
+ SmallVector<Instruction *, 16> InRegionTerminators;
+ for (BasicBlock &BB : F)
+ if (InRegion.contains(BB.getTerminator()))
+ InRegionTerminators.push_back(BB.getTerminator());
+ for (Instruction *TI : InRegionTerminators)
+ for (unsigned I = 0, E = TI->getNumSuccessors(); I != E; ++I)
+ if (BasicBlock *NewBB = SplitCriticalEdge(TI, I, SplitOptions))
+ InRegion.insert(NewBB->getTerminator());
+ }
}
+ SmallVector<Instruction *, 16> RegionCoveragePoints;
SmallVector<Instruction *, 8> IndirCalls;
SmallVector<BasicBlock *, 16> BlocksToInstrument;
SmallVector<Instruction *, 8> CmpTraceTargets;
@@ -629,14 +679,25 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
SmallVector<LoadInst *, 8> Loads;
SmallVector<StoreInst *, 8> Stores;
- const DominatorTree &DT = DTCallback(F);
- const PostDominatorTree &PDT = PDTCallback(F);
+ // Region-only functions need no post-dominators (no block pruning), and
+ // dominators only for the pruning of compares.
+ const DominatorTree *DT =
+ !RegionOnly || Options.TraceCmp ? &DTCallback(F) : nullptr;
+ const PostDominatorTree *PDT = !RegionOnly ? &PDTCallback(F) : nullptr;
bool IsLeafFunc = true;
for (auto &BB : F) {
- if (shouldInstrumentBlock(F, &BB, DT, PDT, Options))
+ if (RegionOnly) {
+ // Block pruning relies on every block being instrumented, so it is not
+ // applied to region-only functions.
+ if (Instruction *IP = getRegionCoveragePoint(BB, InRegion))
+ RegionCoveragePoints.push_back(IP);
+ } else if (shouldInstrumentBlock(F, &BB, *DT, *PDT, Options)) {
BlocksToInstrument.push_back(&BB);
+ }
for (auto &Inst : BB) {
+ if (RegionOnly && !InRegion.contains(&Inst))
+ continue;
if (Options.IndirectCalls) {
CallBase *CB = dyn_cast<CallBase>(&Inst);
if (CB && CB->isIndirectCall())
@@ -644,7 +705,7 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
}
if (Options.TraceCmp) {
if (ICmpInst *CMP = dyn_cast<ICmpInst>(&Inst))
- if (IsInterestingCmp(CMP, DT, Options))
+ if (IsInterestingCmp(CMP, *DT, Options))
CmpTraceTargets.push_back(&Inst);
if (isa<SwitchInst>(&Inst))
SwitchTraceTargets.push_back(&Inst);
@@ -670,7 +731,24 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
}
}
- if (Options.CollectControlFlow)
+ // Coverage is inserted at the start of a block, and the PC table refers to
+ // blocks by address. Start a new block at each region coverage point that is
+ // not already at the start of a block other than the entry block.
+ for (Instruction *IP : RegionCoveragePoints) {
+ BasicBlock *BB = IP->getParent();
+ BasicBlock::iterator SplitPt = IP->getIterator();
+ if (BB->isEntryBlock()) {
+ // Static allocas must stay in the entry block, or they become dynamic
+ // allocas. Move them (and llvm.localescape calls) above the split point.
+ SplitPt = PrepareToSplitEntryBlock(*BB, SplitPt);
+ BB = BB->splitBasicBlock(SplitPt, "sancov.region");
+ } else if (SplitPt != BB->getFirstInsertionPt()) {
+ BB = BB->splitBasicBlock(SplitPt, "sancov.region");
+ }
+ BlocksToInstrument.push_back(BB);
+ }
+
+ if (Options.CollectControlFlow && !RegionOnly)
createFunctionControlFlow(F);
Value *FunctionGateCmp = nullptr;
@@ -682,7 +760,7 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
InjectTraceForGep(F, GepTraceTargets);
InjectTraceForLoadsAndStores(F, Loads, Stores);
- if (Options.TracePCEntryExit)
+ if (Options.TracePCEntryExit && !RegionOnly)
InjectTraceForExits(F);
}
@@ -772,7 +850,11 @@ Value *ModuleSanitizerCoverage::CreateFunctionLocalGateCmp(IRBuilder<> &IRB) {
Instruction *ModuleSanitizerCoverage::CreateGateBranch(Function &F,
Value *&FunctionGateCmp,
Instruction *IP) {
- if (!FunctionGateCmp) {
+ if (RegionOnly) {
+ // The entry block is outside the regions; compute the gate at the use.
+ IRBuilder<> IRB(IP);
+ FunctionGateCmp = CreateFunctionLocalGateCmp(IRB);
+ } else if (!FunctionGateCmp) {
// Create this in the entry block
BasicBlock &BB = F.getEntryBlock();
BasicBlock::iterator IP = BB.getFirstInsertionPt();
diff --git a/llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll b/llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll
new file mode 100644
index 00000000000000..050e55a36a972d
--- /dev/null
+++ b/llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll
@@ -0,0 +1,276 @@
+; Test that functions that are not instrumented as a whole are instrumented
+; inside llvm.sanitizer.instrumentation.begin()/end() regions.
+; RUN: opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=4 \
+; RUN: -sanitizer-coverage-trace-pc -sanitizer-coverage-trace-compares \
+; RUN: -sanitizer-coverage-trace-loads -sanitizer-coverage-trace-stores \
+; RUN: -sanitizer-coverage-trace-divs \
+; RUN: -sanitizer-coverage-trace-pc-entry-exit -sanitizer-coverage-stack-depth \
+; RUN: -sanitizer-coverage-control-flow -S | FileCheck %s
+; RUN: opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=3 \
+; RUN: -sanitizer-coverage-inline-8bit-counters -sanitizer-coverage-pc-table \
+; RUN: -S | FileCheck %s --check-prefix=PCTABLE
+; RUN: opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=4 \
+; RUN: -sanitizer-coverage-trace-pc-guard -sanitizer-coverage-trace-compares \
+; RUN: -sanitizer-coverage-gated-trace-callbacks -S \
+; RUN: | FileCheck %s --check-prefix=GATED
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @foo()
+
+; PCTABLE: @__sancov_gen_{{.*}} = private constant {{.*}} [ptr blockaddress(@region_disable, %sancov.region), ptr null], section "__sancov_pcs"
+
+; Only the code between begin() and end() is instrumented. The block that
+; contains begin() is split right after it, so that the coverage callback runs
+; inside the region, and the function gets no entry/exit callbacks, no stack
+; depth tracking and no control flow table.
+define i32 @region(ptr %p, i32 %x, ptr %fp) nosanitize_coverage {
+entry:
+ %out.cmp = icmp eq i32 %x, 42
+ %out.load = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in.cmp = icmp eq i32 %x, 7
+ %in.load = load i32, ptr %p
+ store i32 %x, ptr %p
+ %in.div = udiv i32 %in.load, %x
+ call void %fp()
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 %out.load, ptr %p
+ call void %fp()
+ %r = select i1 %out.cmp, i32 %in.div, i32 0
+ ret i32 %r
+}
+
+; CHECK-LABEL: define i32 @region(
+; CHECK-NEXT: entry:
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_const_cmp4(i32 7, i32 %x)
+; CHECK-NEXT: %in.cmp = icmp eq i32 %x, 7
+; CHECK-NEXT: call void @__sanitizer_cov_load4(ptr %p)
+; CHECK-NEXT: %in.load = load i32, ptr %p
+; CHECK-NEXT: call void @__sanitizer_cov_store4(ptr %p)
+; CHECK-NEXT: store i32 %x, ptr %p
+; CHECK-NEXT: call void @__sanitizer_cov_trace_div4(i32 %x)
+; CHECK-NEXT: %in.div = udiv i32 %in.load, %x
+; CHECK-NEXT: %0 = ptrtoint ptr %fp to i64
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc_indir(i64 %0)
+; CHECK-NEXT: call void %fp()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: ret i32 %r
+
+; Every block of a region is covered, including the blocks inserted to split
+; critical edges: pruning does not apply, as it assumes that all blocks are
+; instrumented. A block whose only in-region instruction is end() is not.
+define void @region_blocks(i1 %c) nosanitize_coverage {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %then, label %join
+
+then:
+ call void @foo()
+ br label %join
+
+join:
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+
+exit:
+ ret void
+}
+
+; CHECK-LABEL: define void @region_blocks(
+; CHECK: entry:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: br i1 %c, label %then, label %entry.join_crit_edge
+; CHECK: entry.join_crit_edge:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK: then:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @foo()
+; CHECK: join:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK: exit:
+; CHECK-NEXT: ret void
+
+; disable_sanitizer_instrumentation functions are handled the same way.
+define void @region_disable(i32 %x) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %c = icmp eq i32 %x, 7
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @region_disable(
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_const_cmp4(i32 7, i32 %x)
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+; Static allocas after the region coverage point stay in the entry block when
+; it is split; otherwise they would become dynamic allocas.
+define void @entry_allocas(i32 %x) nosanitize_coverage {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @foo()
+ %a = alloca i32
+ %b = alloca i32, i32 %x
+ %c = alloca i64
+ store i32 %x, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @entry_allocas(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: %a = alloca i32
+; CHECK-NEXT: %c = alloca i64
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @foo()
+; CHECK-NEXT: %b = alloca i32, i32 %x
+; CHECK: store i32 %x, ptr %a
+
+; Only the critical edges inside the regions are split (see @region_blocks);
+; the ones outside are left alone.
+define void @outside_crit_edge(i1 %c) nosanitize_coverage {
+entry:
+ br i1 %c, label %then, label %join
+
+then:
+ call void @foo()
+ br label %join
+
+join:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @foo()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @outside_crit_edge(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: br i1 %c, label %then, label %join
+; CHECK-NOT: crit_edge
+; CHECK: join:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @foo()
+
+; With gated callbacks, the gate is read inside the region, not in the entry
+; block.
+; GATED-LABEL: define void @region_disable(
+; GATED-NEXT: entry:
+; GATED-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; GATED-NEXT: br label %sancov.region
+; GATED: sancov.region:
+; GATED-NEXT: load i64, ptr @__sancov_should_track
+; GATED-NOT: @llvm.sanitizer.instrumentation.end()
+; GATED: call void @__sanitizer_cov_trace_pc_guard(
+; GATED: call void @__sanitizer_cov_trace_const_cmp4(i32 7, i32 %x)
+; GATED: call void @llvm.sanitizer.instrumentation.end()
+
+; A block reachable without passing through begin() is not in a region.
+define void @diamond(i1 %c, i32 %x) nosanitize_coverage {
+entry:
+ br i1 %c, label %enter, label %join
+
+enter:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+
+join:
+ %cmp = icmp eq i32 %x, 7
+ call void @foo()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @diamond(
+; CHECK: enter:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK: join:
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: ret void
+
+; Regions nest: the code after an inner end() is still covered.
+define void @nested(i32 %x) nosanitize_coverage {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ %cmp = icmp eq i32 %x, 7
+ call void @llvm.sanitizer.instrumentation.end()
+ %cmp2 = icmp eq i32 %x, 8
+ ret void
+}
+
+; CHECK-LABEL: define void @nested(
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_const_cmp4(i32 7, i32 %x)
+; CHECK-NEXT: %cmp = icmp eq i32 %x, 7
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: %cmp2 = icmp eq i32 %x, 8
+; CHECK-NEXT: ret void
+
+; Without regions, nosanitize_coverage functions are left alone.
+define void @no_region(i32 %x) nosanitize_coverage {
+entry:
+ %cmp = icmp eq i32 %x, 7
+ call void @foo()
+ ret void
+}
+
+; CHECK-LABEL: define void @no_region(
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: ret void
+
+; Instrumented functions are instrumented as usual, regions or not.
+define void @instrumented(i32 %x) nounwind {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @foo()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @instrumented(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc_entry()
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @foo()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc_exit()
+
+; The PC table refers to the split-off region block, never to the entry block of
+; a region-only function.
+; PCTABLE-LABEL: define void @region_disable(
+; PCTABLE: call void @llvm.sanitizer.instrumentation.begin()
+; PCTABLE: sancov.region:
+; PCTABLE-NEXT: load i8, ptr @__sancov_gen_
+; PCTABLE: call void @llvm.sanitizer.instrumentation.end()
>From 790cd2f084f4f94e413159f8734398cfa6c6c5dc Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Tue, 6 Oct 2026 10:24:59 +0000
Subject: [PATCH 5/6] [TSan] Support sanitizer instrumentation regions
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
ThreadSanitizer.
Functions with `disable_sanitizer_instrumentation` are normally not
instrumented at all. If they contain regions, plain and atomic memory
accesses and memory intrinsics inside the regions are now instrumented.
They still get no function entry/exit hooks or runtime ignores, as
those would run outside the regions. A call outside a region still
separates the accesses on either side of it when deciding which reads
can be folded into a following write.
Functions without `sanitize_thread` (and without
`disable_sanitizer_instrumentation`) already get their atomics and
entry/exit hooks instrumented; regions now also enable plain memory
access and memory intrinsic instrumentation in them.
The Linux kernel uses this for `noinstr` functions, which KCSAN marks
`disable_sanitizer_instrumentation` (via `__no_kcsan`), and which may
call instrumented code between `instrumentation_begin()` and
`instrumentation_end()`. Data races on accesses in those regions, e.g.
in the syscall and exception entry code, are currently invisible to
KCSAN.
Clang now lowers the region builtins to the intrinsics under
`-fsanitize=thread`.
Tests: `llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/ThreadSanitizer.md | 23 +++
clang/lib/CodeGen/CGBuiltin.cpp | 6 +-
.../sanitizer-instrumentation-builtins.c | 4 +-
...nitizer-instrumentation-regions-pipeline.c | 4 +
llvm/docs/LangRef.md | 8 +
.../Instrumentation/ThreadSanitizer.cpp | 57 ++++--
.../sanitizer-instrumentation-regions.ll | 178 ++++++++++++++++++
7 files changed, 265 insertions(+), 15 deletions(-)
create mode 100644 llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll
diff --git a/clang/docs/ThreadSanitizer.md b/clang/docs/ThreadSanitizer.md
index 7f48a7470e2540..4a5abf5fc172a8 100644
--- a/clang/docs/ThreadSanitizer.md
+++ b/clang/docs/ThreadSanitizer.md
@@ -102,6 +102,29 @@ and only if absolutely required; for example for certain code that cannot
tolerate any instrumentation and resulting side-effects. This attribute
overrides `no_sanitize("thread")`.
+## Re-enabling Instrumentation in Regions
+
+Inside a function with `__attribute__((disable_sanitizer_instrumentation))` or
+`__attribute__((no_sanitize("thread")))`, a region delimited by
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` is instrumented again: plain
+memory accesses and memory intrinsics in it are instrumented. A
+`disable_sanitizer_instrumentation` function also gets its atomic accesses
+instrumented only inside regions, and no function entry/exit hooks. Regions
+nest, and code reachable without passing through a
+`__builtin_sanitizer_instrumentation_begin()` is not considered to be in a
+region.
+
+```c
+__attribute__((disable_sanitizer_instrumentation))
+void mostly_uninstrumented(int *p) {
+ ... // not instrumented
+ __builtin_sanitizer_instrumentation_begin();
+ *p = 0; // instrumented
+ __builtin_sanitizer_instrumentation_end();
+}
+```
+
## Interaction of Inlining with Disabling Sanitizer Instrumentation
- A `no_sanitize` function will not be inlined heuristically by the compiler into a sanitized function.
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 78e463a1869512..cec0e81d3eed01 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3683,9 +3683,9 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
// Only emit the region markers for sanitizers that honour them. Elsewhere
// they would just be optimization barriers that no pass ever removes. This
// lets code guard its uses on __has_builtin alone.
- SanitizerMask Consumers = SanitizerKind::KernelMemory |
- SanitizerKind::Address |
- SanitizerKind::KernelAddress;
+ SanitizerMask Consumers =
+ SanitizerKind::KernelMemory | SanitizerKind::Address |
+ SanitizerKind::KernelAddress | SanitizerKind::Thread;
if (getLangOpts().Sanitize.hasOneOf(Consumers) ||
CGM.getCodeGenOpts().hasSanitizeCoverage())
Builder.CreateCall(CGM.getIntrinsic(
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index 36123ff4a4453f..5fe7083ac668f8 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,5 +1,5 @@
// Check that the sanitizer region markers lower to their intrinsics under
-// -fsanitize=kernel-memory, kernel-address and address, and with
+// -fsanitize=kernel-memory, kernel-address, address and thread, and with
// -fsanitize-coverage, and to nothing without a sanitizer that consumes them.
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
@@ -14,6 +14,8 @@
// RUN: -fsanitize-coverage-trace-pc -emit-llvm -disable-llvm-passes -o - %s \
// RUN: | FileCheck %s --check-prefix=MARKERS
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=MARKERS
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=null \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
#if !__has_builtin(__builtin_sanitizer_instrumentation_begin)
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
index 2589a4f05c84a8..e9dde3787c34a9 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -19,6 +19,9 @@
// RUN: -fsanitize-coverage-trace-pc -fsanitize-coverage-trace-cmp \
// RUN: -emit-llvm -o - %s \
// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN,KCOV
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=thread -mllvm -tsan-instrument-func-entry-exit=0 \
+// RUN: -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,KCSAN
static inline __attribute__((always_inline)) void instrumentation_begin(void) {
__builtin_sanitizer_instrumentation_begin();
@@ -47,6 +50,7 @@ void noinstr_function(int *p, int x) {
// KCOV: call void @__sanitizer_cov_trace_const_cmp4(i32 7,
// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
// KASAN: call void @__asan_report_load4(
+// KCSAN: call void @__tsan_read4(
// The argument is checked eagerly (-fsanitize-memory-param-retval), so the
// region never touches the context state and does not fetch it.
// KMSAN: call void @__msan_warning(
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 64ac4e7c592ff4..da577c7c691a7f 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28647,3 +28647,11 @@ of comparisons, switches, divisions, GEPs, loads, stores and indirect calls is
performed inside regions. Function entry/exit callbacks, stack depth tracing and
control
flow tables are not emitted for such functions.
+
+ThreadSanitizer supports regions in functions marked with
+`disable_sanitizer_instrumentation`, and in functions that lack the
+`sanitize_thread` attribute. In the former, plain and atomic memory accesses and
+memory intrinsics are instrumented only inside regions, and no function
+entry/exit hooks are inserted. In the latter, which already get their atomic
+accesses and function entry/exit instrumented, plain memory accesses and memory
+intrinsics inside regions are instrumented as well.
diff --git a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
index 59e03878428e3d..4a8a981a8acb14 100644
--- a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
@@ -476,10 +476,23 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
if (F.hasFnAttribute(Attribute::Naked))
return false;
+ bool SanitizeFunction = F.hasFnAttribute(Attribute::SanitizeThread);
// __attribute__(disable_sanitizer_instrumentation) prevents all kinds of
- // instrumentation.
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
+ // instrumentation, except inside llvm.sanitizer.instrumentation.begin()/end()
+ // regions. Such a function is instrumented only there, and gets no
+ // function-level instrumentation (entry/exit hooks, runtime ignores), as that
+ // would run outside of the regions.
+ bool RegionOnly =
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation);
+ if (RegionOnly && !hasSanitizerInstrumentationRegions(F))
return false;
+ // Regions also enable memory access instrumentation in functions that lack
+ // the sanitize_thread attribute, which otherwise only get their atomics and
+ // entry/exit instrumented.
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ if (RegionOnly ||
+ (!SanitizeFunction && hasSanitizerInstrumentationRegions(F)))
+ computeSanitizerInstrumentationRegions(F, InRegion);
initialize(*F.getParent(), TLI);
SmallVector<InstructionInfo, 8> AllLoadsAndStores;
@@ -488,7 +501,6 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
SmallVector<Instruction*, 8> MemIntrinCalls;
bool Res = false;
bool HasCalls = false;
- bool SanitizeFunction = F.hasFnAttribute(Attribute::SanitizeThread);
const DataLayout &DL = F.getDataLayout();
// Traverse all instructions, collect loads/stores/returns, check for calls.
@@ -497,14 +509,32 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
// Skip instructions inserted by another instrumentation.
if (Inst.hasMetadata(LLVMContext::MD_nosanitize))
continue;
+ // The region markers are not calls: they neither need function entry
+ // and exit hooks nor separate the accesses around them.
+ if (isSanitizerInstrumentationBegin(Inst) ||
+ isSanitizerInstrumentationEnd(Inst))
+ continue;
+ // In a region-only function nothing outside the regions is instrumented,
+ // not even atomics.
+ if (RegionOnly && !InRegion.contains(&Inst)) {
+ // A call outside a region still separates the accesses around it.
+ if (isa<CallBase>(Inst))
+ chooseInstructionsToInstrument(LocalLoadsAndStores, AllLoadsAndStores,
+ DL);
+ continue;
+ }
+ // Plain accesses and memory intrinsics are instrumented in
+ // sanitize_thread functions and inside regions.
+ bool InstrumentAccesses = SanitizeFunction || InRegion.contains(&Inst);
if (isTsanAtomic(&Inst))
AtomicAccesses.push_back(&Inst);
- else if (isa<LoadInst>(Inst) || isa<StoreInst>(Inst))
- LocalLoadsAndStores.push_back(&Inst);
- else if (isa<CallInst>(Inst) || isa<InvokeInst>(Inst)) {
+ else if (isa<LoadInst>(Inst) || isa<StoreInst>(Inst)) {
+ if (InstrumentAccesses)
+ LocalLoadsAndStores.push_back(&Inst);
+ } else if (isa<CallInst>(Inst) || isa<InvokeInst>(Inst)) {
if (CallInst *CI = dyn_cast<CallInst>(&Inst))
maybeMarkSanitizerLibraryCallNoBuiltin(CI, &TLI);
- if (isa<MemIntrinsic>(Inst))
+ if (isa<MemIntrinsic>(Inst) && InstrumentAccesses)
MemIntrinCalls.push_back(&Inst);
HasCalls = true;
chooseInstructionsToInstrument(LocalLoadsAndStores, AllLoadsAndStores,
@@ -518,24 +548,29 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
// FIXME: many of these accesses do not need to be checked for races
// (e.g. variables that do not escape, etc).
- // Instrument memory accesses only if we want to report bugs in the function.
- if (Opts.tsan_instrument_memory_accesses && SanitizeFunction)
+ // Instrument memory accesses only if we want to report bugs in the function
+ // (or in the region they belong to).
+ if (Opts.tsan_instrument_memory_accesses)
for (const auto &II : AllLoadsAndStores) {
Res |= instrumentLoadOrStore(II, DL);
}
// Instrument atomic memory accesses in any case (they can be used to
- // implement synchronization).
+ // implement synchronization), except outside the regions of a region-only
+ // function.
if (Opts.tsan_instrument_atomics)
for (auto *Inst : AtomicAccesses) {
Res |= instrumentAtomic(Inst, DL);
}
- if (Opts.tsan_instrument_memintrinsics && SanitizeFunction)
+ if (Opts.tsan_instrument_memintrinsics)
for (auto *Inst : MemIntrinCalls) {
Res |= instrumentMemIntrinsic(Inst);
}
+ if (RegionOnly)
+ return Res;
+
if (F.hasFnAttribute("sanitize_thread_no_checking_at_run_time")) {
assert(!F.hasFnAttribute(Attribute::SanitizeThread));
if (HasCalls)
diff --git a/llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll b/llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll
new file mode 100644
index 00000000000000..5606e19768cc36
--- /dev/null
+++ b/llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll
@@ -0,0 +1,178 @@
+; Test that functions that are not instrumented as a whole are instrumented
+; inside llvm.sanitizer.instrumentation.begin()/end() regions.
+; RUN: opt < %s -passes='function(tsan),module(tsan-module)' -S | FileCheck %s
+; KCSAN disables the entry/exit hooks.
+; RUN: opt < %s -passes='function(tsan),module(tsan-module)' \
+; RUN: -tsan-instrument-func-entry-exit=0 -tsan-distinguish-volatile=1 -S \
+; RUN: | FileCheck %s --check-prefix=KCSAN
+
+target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.memcpy.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @foo()
+
+; In a disable_sanitizer_instrumentation function, plain and atomic accesses
+; and memory intrinsics are instrumented only inside regions, and there are no
+; entry/exit hooks.
+define i32 @region(ptr %a, ptr %b, ptr %d) disable_sanitizer_instrumentation {
+entry:
+ %out.load = load i32, ptr %a
+ %out.atomic = load atomic i32, ptr %b seq_cst, align 4
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %a, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in.load = load i32, ptr %a
+ store i32 %in.load, ptr %b
+ %in.atomic = load atomic i32, ptr %b seq_cst, align 4
+ store volatile i32 %in.atomic, ptr %d
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %a, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 %out.load, ptr %b
+ store atomic i32 %out.atomic, ptr %b seq_cst, align 4
+ ret i32 %in.load
+}
+
+; CHECK-LABEL: define i32 @region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %out.load = load i32, ptr %a
+; CHECK-NEXT: %out.atomic = load atomic i32, ptr %b seq_cst
+; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %a, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @__tsan_read4(ptr %a)
+; CHECK-NEXT: %in.load = load i32, ptr %a
+; CHECK-NEXT: call void @__tsan_write4(ptr %b)
+; CHECK-NEXT: store i32 %in.load, ptr %b
+; CHECK-NEXT: %0 = call i32 @__tsan_atomic32_load(ptr %b, i32 5)
+; CHECK: call void @__tsan_write4(ptr %d)
+; CHECK-NEXT: store volatile i32
+; CHECK-NEXT: call ptr @__tsan_memcpy(ptr %d, ptr %a, i64 16)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 %out.load, ptr %b
+; CHECK-NEXT: store atomic i32 %out.atomic, ptr %b seq_cst
+; CHECK-NEXT: ret i32 %in.load
+
+; KCSAN-LABEL: define i32 @region(
+; KCSAN: call void @llvm.sanitizer.instrumentation.begin()
+; KCSAN-NEXT: call void @__tsan_read4(ptr %a)
+; KCSAN: call void @__tsan_volatile_write4(ptr %d)
+; KCSAN: call void @llvm.sanitizer.instrumentation.end()
+; KCSAN-NOT: call {{.*}}@__tsan_
+; KCSAN: ret i32 %in.load
+
+; A block reachable without passing through begin() is not in a region.
+define void @diamond(i1 %c, ptr %a) disable_sanitizer_instrumentation {
+entry:
+ br i1 %c, label %enter, label %join
+
+enter:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %a
+ br label %join
+
+join:
+ store i32 2, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @diamond(
+; CHECK: enter:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @__tsan_write4(ptr %a)
+; CHECK-NEXT: store i32 1, ptr %a
+; CHECK: join:
+; CHECK-NEXT: store i32 2, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+; Regions nest.
+define void @nested(ptr %a) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 1, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 2, ptr %a
+ ret void
+}
+
+; CHECK-LABEL: define void @nested(
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call void @__tsan_write4(ptr %a)
+; CHECK-NEXT: store i32 1, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 2, ptr %a
+; CHECK-NEXT: ret void
+
+; A call outside the regions separates the accesses on either side of it, so
+; the read before the write is not folded into the write.
+define void @call_between_regions(ptr %a) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %v = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @foo()
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 %v, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @call_between_regions(
+; CHECK: call void @__tsan_read4(ptr %a)
+; CHECK: call void @foo()
+; CHECK: call void @__tsan_write4(ptr %a)
+
+; Functions without sanitize_thread still get their atomics and entry/exit
+; hooks instrumented everywhere; regions add plain accesses.
+define i32 @no_sanitize_thread(ptr %a) {
+entry:
+ %out.load = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in.load = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ %atomic = load atomic i32, ptr %a seq_cst, align 4
+ %r = add i32 %out.load, %in.load
+ ret i32 %r
+}
+
+; CHECK-LABEL: define i32 @no_sanitize_thread(
+; CHECK: call void @__tsan_func_entry(
+; CHECK-NEXT: %out.load = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @__tsan_read4(ptr %a)
+; CHECK-NEXT: %in.load = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call i32 @__tsan_atomic32_load(ptr %a, i32 5)
+; CHECK: call void @__tsan_func_exit()
+
+; Without regions, disable_sanitizer_instrumentation functions are left alone.
+define i32 @no_region(ptr %a) disable_sanitizer_instrumentation {
+entry:
+ %v = load i32, ptr %a
+ %atomic = load atomic i32, ptr %a seq_cst, align 4
+ ret i32 %v
+}
+
+; CHECK-LABEL: define i32 @no_region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %v = load i32, ptr %a
+; CHECK-NEXT: %atomic = load atomic i32, ptr %a seq_cst
+; CHECK-NEXT: ret i32 %v
+
+; Instrumented functions are instrumented as usual, regions or not.
+define i32 @sanitized(ptr %a) sanitize_thread {
+entry:
+ %out.load = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %out.load
+}
+
+; CHECK-LABEL: define i32 @sanitized(
+; CHECK: call void @__tsan_func_entry(
+; CHECK-NEXT: call void @__tsan_read4(ptr %a)
+; CHECK-NEXT: %out.load = load i32, ptr %a
>From 27e5d3191ebd36da4a37704e3f945135f95fccb8 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Tue, 13 Oct 2026 07:07:35 +0000
Subject: [PATCH 6/6] [Sanitizers] Warn about unbalanced sanitizer
instrumentation regions
`computeSanitizerInstrumentationRegions()` silently copes with malformed
region markers: an `end` without a matching `begin` is ignored, a region
left open extends to the end of the function, and code reached both from
inside and from outside a region is treated as outside. The result is
that instrumentation is quietly missing where the user expected it.
Report these cases as a warning from the region analysis, i.e. from
every sanitizer pass that implements regions:
- an `llvm.sanitizer.instrumentation.end` without a matching `begin`;
- a region that is still open at a `ret`;
- a block where paths with different region nesting depths merge.
Leaving a function without returning while a region is open is not
diagnosed: not closing a region before a `noreturn` call (e.g. `BUG()`
or `panic()` in the kernel) is common and harmless, and unwinding cannot
be checked consistently, because a call without a landing pad leaves no
terminator to look at.
With the "must" join, the analysis is exact as long as all paths into a
block agree on the depth, so the first disagreement shows up as a block
whose reached predecessors leave with different depths. Joins are
checked first, so that an `end` or a return that only looks unbalanced
because of an earlier bad join is not blamed instead.
`computeSanitizerInstrumentationRegions()` stays a pure analysis: it
returns the first problem it finds, and each sanitizer pass reports it
through `diagnoseSanitizerInstrumentationRegionProblem()`. The report is
a new `DiagnosticInfoSanitizerInstrumentationRegion` that carries the
kind of problem rather than a message, so that Clang owns the wording:
it maps it to `-Wsanitizer-instrumentation-region`, reports it at the
source location of the offending marker, return or merge point, and
warns at most once per function when several sanitizers (e.g. KMSAN and
KCOV) analyze it. Other drivers do not deduplicate: `opt` with several
sanitizer passes reports a function once per pass. Doing that in LLVM
would require tagging the IR; it can be revisited if it turns out to be
a problem (see the TODO at `diagnoseSanitizerInstrumentationRegionProblem()`).
In the Linux kernel, objtool already rejects unbalanced
`instrumentation_begin()`/`instrumentation_end()` when
`CONFIG_NOINSTR_VALIDATION` is enabled, but other users of the builtins
have no such check, and the kernel only gets it late, on the object file.
Tests: `llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll`,
`llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c`.
---
clang/docs/LanguageExtensions.md | 3 +-
clang/docs/ReleaseNotes.md | 4 +-
.../clang/Basic/DiagnosticFrontendKinds.td | 11 ++
clang/include/clang/Basic/DiagnosticGroups.td | 2 +
.../CodeGenUtils/BackendDiagnosticHandler.h | 9 +
.../CodeGenUtils/BackendDiagnosticHandler.cpp | 31 ++++
...izer-instrumentation-regions-diagnostics.c | 61 +++++++
llvm/docs/LangRef.md | 11 ++
llvm/include/llvm/IR/DiagnosticInfo.h | 45 +++++
.../llvm/Transforms/Utils/Instrumentation.h | 30 +++-
llvm/lib/IR/DiagnosticInfo.cpp | 37 ++++
.../Instrumentation/AddressSanitizer.cpp | 3 +-
.../Instrumentation/MemorySanitizer.cpp | 3 +-
.../Instrumentation/SanitizerCoverage.cpp | 3 +-
.../Instrumentation/ThreadSanitizer.cpp | 3 +-
llvm/lib/Transforms/Utils/Instrumentation.cpp | 71 +++++++-
...zer-instrumentation-regions-diagnostics.ll | 165 ++++++++++++++++++
.../Transforms/Utils/SanitizerRegionsTest.cpp | 87 ++++++++-
18 files changed, 565 insertions(+), 14 deletions(-)
create mode 100644 clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c
create mode 100644 llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll
diff --git a/clang/docs/LanguageExtensions.md b/clang/docs/LanguageExtensions.md
index 7ea54b04ada2d7..a5a0175044b57b 100644
--- a/clang/docs/LanguageExtensions.md
+++ b/clang/docs/LanguageExtensions.md
@@ -4349,7 +4349,8 @@ void handle_irq(struct pt_regs *regs) {
The builtins are meant for code like the Linux kernel's `noinstr` functions,
which must not be instrumented as a whole but call instrumented code between
`instrumentation_begin()` and `instrumentation_end()`. Regions nest, and should
-be balanced on every path through the function. Which functions count as
+be balanced on every path through the function; Clang warns about unbalanced
+regions with `-Wsanitizer-instrumentation-region`. Which functions count as
uninstrumented, and what gets instrumented inside regions, is up to each
sanitizer; sanitizers that support regions describe this in their own
documentation.
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index b27b3d2955bf4d..4ce0a911435b49 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -1322,7 +1322,9 @@ The `alpha.cplusplus.UseAfterLifetimeEnd` checker was renamed to `alpha.core.Use
sanitizer instrumentation is enabled again. AddressSanitizer,
KernelMemorySanitizer, ThreadSanitizer and SanitizerCoverage support them;
see [LanguageExtensions](LanguageExtensions.md) and the documentation of each
- sanitizer.
+ sanitizer. The new `-Wsanitizer-instrumentation-region` warning reports
+ unbalanced regions: an end without a matching begin, a region still open at
+ a return, and control flow merging from paths with different region nesting.
### Python Binding Changes
diff --git a/clang/include/clang/Basic/DiagnosticFrontendKinds.td b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
index 3df87215a0476d..3f4da4adb1706b 100644
--- a/clang/include/clang/Basic/DiagnosticFrontendKinds.td
+++ b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
@@ -410,6 +410,17 @@ def warn_profile_data_misexpect : Warning<
"potential performance regression from use of __builtin_expect(): "
"annotation was correct on %0 of profiled executions">,
BackendInfo, InGroup<MisExpect>;
+// The %select order must match llvm::SanitizerInstrumentationRegionIssue.
+def warn_fe_backend_sanitizer_instrumentation_region : Warning<
+ "in function '%1': "
+ "%select{sanitizer instrumentation region end without a matching begin"
+ "|sanitizer instrumentation region is not closed before return"
+ "|control flow merges from paths with different sanitizer instrumentation "
+ "region nesting; the code that follows is treated as outside the region"
+ "|control flow merges from paths with different sanitizer instrumentation "
+ "region nesting; the code that follows is treated as nested in the smaller "
+ "number of regions}0">,
+ BackendInfo, InGroup<SanitizerInstrumentationRegion>;
} // end of instrumentation issue category
def warn_cgs_no_proto : Warning<
diff --git a/clang/include/clang/Basic/DiagnosticGroups.td b/clang/include/clang/Basic/DiagnosticGroups.td
index b6c7ff7276260f..95d4380c1324e4 100644
--- a/clang/include/clang/Basic/DiagnosticGroups.td
+++ b/clang/include/clang/Basic/DiagnosticGroups.td
@@ -1836,6 +1836,8 @@ def ProfileInstrMissing : DiagGroup<"profile-instr-missing">;
def ProfileInstrOutOfDate : DiagGroup<"profile-instr-out-of-date">;
def ProfileInstrUnprofiled : DiagGroup<"profile-instr-unprofiled">;
def MisExpect : DiagGroup<"misexpect">;
+def SanitizerInstrumentationRegion
+ : DiagGroup<"sanitizer-instrumentation-region">;
// AddressSanitizer frontend instrumentation remarks.
def SanitizeAddressRemarks : DiagGroup<"sanitize-address">;
diff --git a/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h b/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h
index 484b5c68594951..ab0da5469eda9a 100644
--- a/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h
+++ b/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h
@@ -22,6 +22,7 @@
#include "clang/Basic/SourceLocation.h"
#include "llvm/ADT/Hashing.h"
#include "llvm/ADT/StringRef.h"
+#include "llvm/ADT/StringSet.h"
#include "llvm/IR/DiagnosticHandler.h"
#include <memory>
#include <optional>
@@ -35,6 +36,7 @@ class DiagnosticInfoMisExpect;
class DiagnosticInfoOptimizationBase;
class DiagnosticInfoOptimizationFailure;
class DiagnosticInfoResourceLimit;
+class DiagnosticInfoSanitizerInstrumentationRegion;
class DiagnosticInfoSrcMgr;
class DiagnosticInfoStackSize;
class DiagnosticInfoUnsupported;
@@ -134,6 +136,9 @@ class BackendDiagnosticConsumer {
/// Specialized handler for misexpect warnings.
/// Note that misexpect remarks are emitted through ORE
void MisExpectDiagHandler(const llvm::DiagnosticInfoMisExpect &D);
+ /// Specialized handler for unbalanced sanitizer instrumentation regions.
+ void SanitizerInstrumentationRegionDiagHandler(
+ const llvm::DiagnosticInfoSanitizerInstrumentationRegion &D);
DiagnosticsEngine &Diags;
const CodeGenOptions &CodeGenOpts;
@@ -149,6 +154,10 @@ class BackendDiagnosticConsumer {
// because we optimize for time building this map rather than time
// retrieving an entry, as backend diagnostics are uncommon.
std::vector<std::pair<llvm::hash_code, FullSourceLoc>> ManglingFullSourceLocs;
+
+ /// Functions for which an unbalanced sanitizer instrumentation region has
+ /// already been reported.
+ llvm::StringSet<> SanitizerRegionsDiagnosed;
};
} // namespace clang
diff --git a/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp b/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp
index 667a7948fa18f8..9c60746d618d8e 100644
--- a/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp
+++ b/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp
@@ -549,6 +549,33 @@ void BackendDiagnosticConsumer::MisExpectDiagHandler(
<< Filename << Line << Column;
}
+void BackendDiagnosticConsumer::SanitizerInstrumentationRegionDiagHandler(
+ const llvm::DiagnosticInfoSanitizerInstrumentationRegion &D) {
+ // Every sanitizer pass that implements regions analyzes the function and
+ // reports the same problem; warn only once per function.
+ if (!SanitizerRegionsDiagnosed.insert(D.getFunction().getName()).second)
+ return;
+
+ StringRef Filename;
+ unsigned Line, Column;
+ bool BadDebugInfo = false;
+ FullSourceLoc Loc =
+ getBestLocationFromDebugLoc(D, BadDebugInfo, Filename, Line, Column);
+
+ // The %select in the diagnostic enumerates the issues in enum order.
+ static_assert(
+ static_cast<unsigned>(
+ llvm::SanitizerInstrumentationRegionIssue::NumIssues) == 4,
+ "update warn_fe_backend_sanitizer_instrumentation_region");
+ Diags.Report(Loc, diag::warn_fe_backend_sanitizer_instrumentation_region)
+ << static_cast<unsigned>(D.getIssue())
+ << llvm::demangle(D.getFunction().getName());
+
+ if (BadDebugInfo)
+ Diags.Report(Loc, diag::note_fe_backend_invalid_loc)
+ << Filename << Line << Column;
+}
+
void BackendDiagnosticConsumer::handleDiagnostics(const DiagnosticInfo &DI) {
unsigned DiagID = diag::err_fe_inline_asm;
llvm::DiagnosticSeverity Severity = DI.getSeverity();
@@ -633,6 +660,10 @@ void BackendDiagnosticConsumer::handleDiagnostics(const DiagnosticInfo &DI) {
case llvm::DK_MisExpect:
MisExpectDiagHandler(cast<DiagnosticInfoMisExpect>(DI));
return;
+ case llvm::DK_SanitizerInstrumentationRegion:
+ SanitizerInstrumentationRegionDiagHandler(
+ cast<DiagnosticInfoSanitizerInstrumentationRegion>(DI));
+ return;
default:
// Plugin IDs are not bound to any value as they are set dynamically.
ComputeDiagRemarkID(Severity, backend_plugin, DiagID);
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c
new file mode 100644
index 00000000000000..597fa5a657ac86
--- /dev/null
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c
@@ -0,0 +1,61 @@
+// Check that unbalanced sanitizer instrumentation regions are diagnosed with a
+// source location, and that the warning can be disabled.
+
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
+// RUN: -debug-info-kind=line-tables-only -emit-llvm -o /dev/null %s -verify
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
+// RUN: -fsanitize-coverage-type=3 -fsanitize-coverage-trace-pc \
+// RUN: -debug-info-kind=line-tables-only -emit-llvm -o /dev/null %s -verify
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
+// RUN: -Wno-sanitizer-instrumentation-region -emit-llvm -o /dev/null %s \
+// RUN: -verify=none
+
+// none-no-diagnostics
+
+extern void callee(void);
+
+__attribute__((disable_sanitizer_instrumentation))
+void balanced(int c) {
+ __builtin_sanitizer_instrumentation_begin();
+ callee();
+ __builtin_sanitizer_instrumentation_end();
+ if (c) {
+ __builtin_sanitizer_instrumentation_begin();
+ callee();
+ __builtin_sanitizer_instrumentation_end();
+ }
+}
+
+// Not closing a region before a call that does not return is fine.
+__attribute__((noreturn)) extern void panic(void);
+
+__attribute__((disable_sanitizer_instrumentation))
+void open_at_noreturn(int c) {
+ __builtin_sanitizer_instrumentation_begin();
+ if (c)
+ panic();
+ if (c > 1)
+ __builtin_trap();
+ callee();
+ __builtin_sanitizer_instrumentation_end();
+}
+
+__attribute__((disable_sanitizer_instrumentation))
+void unmatched_end(void) {
+ callee();
+ __builtin_sanitizer_instrumentation_end(); // expected-warning {{in function 'unmatched_end': sanitizer instrumentation region end without a matching begin}}
+}
+
+__attribute__((disable_sanitizer_instrumentation))
+void not_closed(void) {
+ __builtin_sanitizer_instrumentation_begin();
+ callee();
+} // expected-warning {{in function 'not_closed': sanitizer instrumentation region is not closed before return}}
+
+__attribute__((disable_sanitizer_instrumentation))
+void conditional_begin(int c) {
+ if (c)
+ __builtin_sanitizer_instrumentation_begin();
+ callee(); // expected-warning {{in function 'conditional_begin': control flow merges from paths with different sanitizer instrumentation region nesting}}
+ __builtin_sanitizer_instrumentation_end();
+}
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index da577c7c691a7f..1d022118cd6fb3 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28620,6 +28620,17 @@ dominated by a `llvm.sanitizer.instrumentation.begin`, so any per-function
sanitizer state set up there is available. A region that is entered on some
paths but not others is therefore instrumented only where it is unambiguous.
+Sanitizer passes that implement regions warn about unbalanced markers: a
+`llvm.sanitizer.instrumentation.end` without a matching
+`llvm.sanitizer.instrumentation.begin`, a region that is still open at a `ret`,
+and a point where paths with different region nesting depths merge. Regions that
+are still open where the function is left without returning, i.e. at an
+`unreachable` (e.g. after a call to a `noreturn` function) or when an exception
+unwinds out of it, are not diagnosed. Markers in functions that are instrumented
+as a whole are ignored and not checked. In Clang, the warning is controlled by
+`-Wsanitizer-instrumentation-region` and emitted at most once per function;
+`opt` reports a function once per sanitizer pass that analyzes it.
+
MemorySanitizer supports regions in kernel mode (KMSAN), in functions marked
with `disable_sanitizer_instrumentation`. The function-level ABI prologue is
skipped, the MSan context state is fetched at
diff --git a/llvm/include/llvm/IR/DiagnosticInfo.h b/llvm/include/llvm/IR/DiagnosticInfo.h
index da62b62bd8c74a..72d60ab8098477 100644
--- a/llvm/include/llvm/IR/DiagnosticInfo.h
+++ b/llvm/include/llvm/IR/DiagnosticInfo.h
@@ -93,6 +93,7 @@ enum DiagnosticKind {
DK_SrcMgr,
DK_DontCall,
DK_MisExpect,
+ DK_SanitizerInstrumentationRegion,
DK_FirstPluginKind // Must be last value to work with
// getNextAvailablePluginDiagnosticKind
};
@@ -1151,6 +1152,50 @@ class LLVM_ABI DiagnosticInfoUnsupportedTargetIntrinsic
void print(DiagnosticPrinter &DP) const override;
};
+/// The kinds of problems with llvm.sanitizer.instrumentation.begin/end
+/// markers that the sanitizer passes report. Clang's
+/// warn_fe_backend_sanitizer_instrumentation_region selects its text by the
+/// numeric value, so keep the order in sync with that diagnostic.
+enum class SanitizerInstrumentationRegionIssue {
+ /// An end() without a matching begin().
+ UnmatchedEnd,
+ /// A region is still open at a return.
+ NotClosedAtReturn,
+ /// Paths with different region nesting depths merge, and the code that
+ /// follows is treated as outside any region.
+ MergeOutsideRegion,
+ /// Paths with different region nesting depths merge, and the code that
+ /// follows is treated as nested in the smaller number of regions.
+ MergeNestedRegions,
+ /// Not an issue; the number of issues above.
+ NumIssues
+};
+
+/// Diagnostic information for unbalanced sanitizer instrumentation regions
+/// (llvm.sanitizer.instrumentation.begin/end).
+class LLVM_ABI DiagnosticInfoSanitizerInstrumentationRegion
+ : public DiagnosticInfoWithLocationBase {
+public:
+ DiagnosticInfoSanitizerInstrumentationRegion(
+ const Function &Fn, const DebugLoc &Loc,
+ SanitizerInstrumentationRegionIssue Issue);
+
+ /// \see DiagnosticInfo::print.
+ void print(DiagnosticPrinter &DP) const override;
+
+ static bool classof(const DiagnosticInfo *DI) {
+ return DI->getKind() == DK_SanitizerInstrumentationRegion;
+ }
+
+ SanitizerInstrumentationRegionIssue getIssue() const { return Issue; }
+
+ /// The description of \p Issue used by print().
+ static StringRef getIssueText(SanitizerInstrumentationRegionIssue Issue);
+
+private:
+ SanitizerInstrumentationRegionIssue Issue;
+};
+
/// Diagnostic information for MisExpect analysis.
class LLVM_ABI DiagnosticInfoMisExpect : public DiagnosticInfoWithLocationBase {
public:
diff --git a/llvm/include/llvm/Transforms/Utils/Instrumentation.h b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
index 642ba946e9dbc6..89f6f685e01f89 100644
--- a/llvm/include/llvm/Transforms/Utils/Instrumentation.h
+++ b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
@@ -24,6 +24,7 @@
#include <cassert>
#include <cstdint>
#include <limits>
+#include <optional>
#include <string>
namespace llvm {
@@ -33,6 +34,7 @@ class OptimizationRemarkEmitter;
class Comdat;
class CallBase;
class Module;
+enum class SanitizerInstrumentationRegionIssue;
/// Check if module has flag attached, if not add the flag.
LLVM_ABI bool checkIfAlreadyInstrumented(Module &M, StringRef Flag);
@@ -70,6 +72,14 @@ LLVM_ABI bool isSanitizerInstrumentationEnd(const Instruction &I);
/// llvm.sanitizer.instrumentation.end.
LLVM_ABI bool hasSanitizerInstrumentationRegions(const Function &F);
+/// A problem with the llvm.sanitizer.instrumentation.begin()/end() markers of
+/// a function, found by computeSanitizerInstrumentationRegions().
+struct SanitizerInstrumentationRegionProblem {
+ SanitizerInstrumentationRegionIssue Issue;
+ /// The marker, return or first instruction of the merge block to blame.
+ const Instruction *Where;
+};
+
/// Compute the instructions of \p F that lie inside an
/// llvm.sanitizer.instrumentation.begin()/end() region, and add them to
/// \p InRegion.
@@ -81,9 +91,27 @@ LLVM_ABI bool hasSanitizerInstrumentationRegions(const Function &F);
/// objtool requiring them to be balanced on all paths. The lattice value is
/// the nesting depth rather than a flag, so that in begin; begin; end; X; end
/// the instruction X is still in a region.
-LLVM_ABI void computeSanitizerInstrumentationRegions(
+///
+/// Returns the first problem with the markers of \p F, if any: an end()
+/// without a matching begin(), a region still open at a return, or control
+/// flow merging from paths with different nesting depths. Leaving the
+/// function without returning (through an unreachable or by unwinding) with a
+/// region open is not reported. Callers that want the user to see the problem
+/// pass it to diagnoseSanitizerInstrumentationRegionProblem().
+LLVM_ABI std::optional<SanitizerInstrumentationRegionProblem>
+computeSanitizerInstrumentationRegions(
const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion);
+/// Report \p Problem in \p F as a DiagnosticInfoSanitizerInstrumentationRegion
+/// warning through the LLVMContext of \p F.
+///
+/// Every sanitizer pass that implements regions reports the problems it finds,
+/// so a function analyzed by several passes is reported several times. Clang
+/// deduplicates the reports; other drivers (e.g. opt) do not.
+/// TODO: Deduplicate here, without tagging the IR, if that becomes a problem.
+LLVM_ABI void diagnoseSanitizerInstrumentationRegionProblem(
+ const Function &F, const SanitizerInstrumentationRegionProblem &Problem);
+
// Insert GCOV profiling instrumentation
struct GCOVOptions {
// Specify whether to emit .gcno files.
diff --git a/llvm/lib/IR/DiagnosticInfo.cpp b/llvm/lib/IR/DiagnosticInfo.cpp
index a24b6d09350080..bcaebf7ca26a7b 100644
--- a/llvm/lib/IR/DiagnosticInfo.cpp
+++ b/llvm/lib/IR/DiagnosticInfo.cpp
@@ -495,6 +495,43 @@ void DiagnosticInfoMisExpect::print(DiagnosticPrinter &DP) const {
DP << getLocationStr() << ": " << getMsg();
}
+DiagnosticInfoSanitizerInstrumentationRegion::
+ DiagnosticInfoSanitizerInstrumentationRegion(
+ const Function &Fn, const DebugLoc &Loc,
+ SanitizerInstrumentationRegionIssue Issue)
+ : DiagnosticInfoWithLocationBase(DK_SanitizerInstrumentationRegion,
+ DS_Warning, Fn, Loc),
+ Issue(Issue) {}
+
+StringRef DiagnosticInfoSanitizerInstrumentationRegion::getIssueText(
+ SanitizerInstrumentationRegionIssue Issue) {
+ switch (Issue) {
+ case SanitizerInstrumentationRegionIssue::UnmatchedEnd:
+ return "sanitizer instrumentation region end without a matching begin";
+ case SanitizerInstrumentationRegionIssue::NotClosedAtReturn:
+ return "sanitizer instrumentation region is not closed before return";
+ case SanitizerInstrumentationRegionIssue::MergeOutsideRegion:
+ return "control flow merges from paths with different sanitizer "
+ "instrumentation region nesting; the code that follows is treated "
+ "as outside the region";
+ case SanitizerInstrumentationRegionIssue::MergeNestedRegions:
+ return "control flow merges from paths with different sanitizer "
+ "instrumentation region nesting; the code that follows is treated "
+ "as nested in the smaller number of regions";
+ case SanitizerInstrumentationRegionIssue::NumIssues:
+ llvm_unreachable("not an issue");
+ }
+ llvm_unreachable("unknown SanitizerInstrumentationRegionIssue");
+}
+
+void DiagnosticInfoSanitizerInstrumentationRegion::print(
+ DiagnosticPrinter &DP) const {
+ if (isLocationAvailable())
+ DP << getLocationStr() << ": ";
+ DP << "in function '" << getFunction().getName()
+ << "': " << getIssueText(Issue);
+}
+
void OptimizationRemarkAnalysisFPCommute::anchor() {}
void OptimizationRemarkAnalysisAliasing::anchor() {}
diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
index fe544d6776bdb4..bbe40ce9e512ec 100644
--- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
@@ -2911,7 +2911,8 @@ bool AddressSanitizer::instrumentFunction(Function &F,
SmallPtrSet<const Instruction *, 16> InRegion;
if (RegionOnly) {
- computeSanitizerInstrumentationRegions(F, InRegion);
+ if (auto Problem = computeSanitizerInstrumentationRegions(F, InRegion))
+ diagnoseSanitizerInstrumentationRegionProblem(F, *Problem);
} else {
// We can't instrument allocas used with llvm.localescape. Only static
// allocas can be passed to that intrinsic. (The locals of a region-only
diff --git a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
index 0aecb74df9edda..94f5dfce97eb42 100644
--- a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
@@ -1605,7 +1605,8 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
/// pointer. With the intersection join an in-region instruction is always
/// dominated by a begin(), so the slot is always initialized.
void computeRegionMembership() {
- computeSanitizerInstrumentationRegions(F, InRegion);
+ if (auto Problem = computeSanitizerInstrumentationRegions(F, InRegion))
+ diagnoseSanitizerInstrumentationRegionProblem(F, *Problem);
// A begin() is nested if the code right before it is already in a region:
// the preceding instruction, or, at the start of a block, the terminators
diff --git a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
index b8bab5ed0a565a..7538f4412bd398 100644
--- a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
+++ b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
@@ -648,7 +648,8 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
}
SmallPtrSet<const Instruction *, 16> InRegion;
if (RegionOnly)
- computeSanitizerInstrumentationRegions(F, InRegion);
+ if (auto Problem = computeSanitizerInstrumentationRegions(F, InRegion))
+ diagnoseSanitizerInstrumentationRegionProblem(F, *Problem);
if (Options.CoverageType >= SanitizerCoverageOptions::SCK_Edge) {
CriticalEdgeSplittingOptions SplitOptions;
SplitOptions.setIgnoreUnreachableDests();
diff --git a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
index 4a8a981a8acb14..a6f1e404776dc3 100644
--- a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
@@ -492,7 +492,8 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
SmallPtrSet<const Instruction *, 16> InRegion;
if (RegionOnly ||
(!SanitizeFunction && hasSanitizerInstrumentationRegions(F)))
- computeSanitizerInstrumentationRegions(F, InRegion);
+ if (auto Problem = computeSanitizerInstrumentationRegions(F, InRegion))
+ diagnoseSanitizerInstrumentationRegionProblem(F, *Problem);
initialize(*F.getParent(), TLI);
SmallVector<InstructionInfo, 8> AllLoadsAndStores;
diff --git a/llvm/lib/Transforms/Utils/Instrumentation.cpp b/llvm/lib/Transforms/Utils/Instrumentation.cpp
index cfbe8ae90e6d44..97973a0580d63c 100644
--- a/llvm/lib/Transforms/Utils/Instrumentation.cpp
+++ b/llvm/lib/Transforms/Utils/Instrumentation.cpp
@@ -12,6 +12,7 @@
//===----------------------------------------------------------------------===//
#include "llvm/Transforms/Utils/Instrumentation.h"
+#include "llvm/ADT/ArrayRef.h"
#include "llvm/ADT/SetVector.h"
#include "llvm/ADT/SmallVector.h"
#include "llvm/IR/CFG.h"
@@ -173,7 +174,73 @@ static unsigned regionDepthAfter(const Instruction &I, unsigned Depth) {
return Depth;
}
-void llvm::computeSanitizerInstrumentationRegions(
+/// Find the first problem with the region markers of \p F, given the fixed
+/// point of the region analysis.
+///
+/// With the "must" (minimum) join the analysis is exact as long as all paths
+/// to a block agree on the nesting depth, so the first point where they
+/// disagree is a block whose reached predecessors leave with different depths.
+/// Joins are checked first, so that an end() or a return that only looks
+/// unbalanced because of an earlier bad join is not blamed instead.
+static std::optional<SanitizerInstrumentationRegionProblem>
+findSanitizerInstrumentationRegionProblem(const Function &F,
+ ArrayRef<unsigned> DepthIn,
+ ArrayRef<unsigned> DepthOut) {
+ for (const BasicBlock &BB : F) {
+ if (&BB == &F.getEntryBlock() || DepthIn[BB.getNumber()] == UnreachedDepth)
+ continue;
+ unsigned Min = UnreachedDepth, Max = 0;
+ for (const BasicBlock *Pred : predecessors(&BB)) {
+ unsigned Out = DepthOut[Pred->getNumber()];
+ if (Out == UnreachedDepth)
+ continue;
+ Min = std::min(Min, Out);
+ Max = std::max(Max, Out);
+ }
+ if (Min == Max)
+ continue;
+ // Point at the first instruction that has a location, if any.
+ const Instruction *Where = &*BB.getFirstNonPHIOrAlloca();
+ for (const Instruction &I : BB)
+ if (I.getDebugLoc()) {
+ Where = &I;
+ break;
+ }
+ return SanitizerInstrumentationRegionProblem{
+ Min == 0 ? SanitizerInstrumentationRegionIssue::MergeOutsideRegion
+ : SanitizerInstrumentationRegionIssue::MergeNestedRegions,
+ Where};
+ }
+
+ for (const BasicBlock &BB : F) {
+ unsigned Depth = DepthIn[BB.getNumber()];
+ if (Depth == UnreachedDepth)
+ continue;
+ for (const Instruction &I : BB) {
+ if (isSanitizerInstrumentationEnd(I) && !Depth)
+ return SanitizerInstrumentationRegionProblem{
+ SanitizerInstrumentationRegionIssue::UnmatchedEnd, &I};
+ // Only returns are checked. Not closing a region before a call that
+ // does not return (BUG(), panic()) is common and harmless, and
+ // unwinding out of a function cannot be checked consistently: a call
+ // without a landing pad leaves no terminator to look at.
+ if (isa<ReturnInst>(I) && Depth)
+ return SanitizerInstrumentationRegionProblem{
+ SanitizerInstrumentationRegionIssue::NotClosedAtReturn, &I};
+ Depth = regionDepthAfter(I, Depth);
+ }
+ }
+ return std::nullopt;
+}
+
+void llvm::diagnoseSanitizerInstrumentationRegionProblem(
+ const Function &F, const SanitizerInstrumentationRegionProblem &Problem) {
+ F.getContext().diagnose(DiagnosticInfoSanitizerInstrumentationRegion(
+ F, Problem.Where->getDebugLoc(), Problem.Issue));
+}
+
+std::optional<SanitizerInstrumentationRegionProblem>
+llvm::computeSanitizerInstrumentationRegions(
const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion) {
// Nesting depth at the start and at the end of each block, indexed by block
// number.
@@ -217,4 +284,6 @@ void llvm::computeSanitizerInstrumentationRegions(
InRegion.insert(&I);
}
}
+
+ return findSanitizerInstrumentationRegionProblem(F, DepthIn, DepthOut);
}
diff --git a/llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll b/llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll
new file mode 100644
index 00000000000000..7a24bd738ba08a
--- /dev/null
+++ b/llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll
@@ -0,0 +1,165 @@
+; Unbalanced llvm.sanitizer.instrumentation.begin/end regions are reported as a
+; warning by every sanitizer pass that implements regions, once per function
+; and pass.
+
+; RUN: opt < %s -passes=msan -msan-kernel=1 -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+; RUN: opt < %s -passes=asan -asan-kernel=1 -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+; RUN: opt < %s -passes=tsan -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+; RUN: opt < %s -passes=sancov-module -sanitizer-coverage-level=3 \
+; RUN: -sanitizer-coverage-trace-pc -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+
+; Several passes analyzing the same function each report it. (Clang
+; deduplicates the reports; opt does not.)
+; RUN: opt < %s -passes='sancov-module,msan,tsan,asan' -msan-kernel=1 \
+; RUN: -asan-kernel=1 -sanitizer-coverage-level=3 -sanitizer-coverage-trace-pc \
+; RUN: -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --check-prefix=MULTI --implicit-check-not=warning:
+; Five malformed functions, reported by each of the four passes.
+; MULTI-COUNT-20: warning: {{.*}}in function '{{unmatched_end|not_closed|bad_join|nested_join|loop}}'
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+; Balanced, nested and conditional-but-balanced regions are fine.
+define void @balanced(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @llvm.sanitizer.instrumentation.end()
+ br i1 %c, label %then, label %exit
+
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+
+exit:
+ ret void
+}
+
+; Leaving the function without returning while a region is open is fine: after
+; a noreturn call, and when an exception unwinds out of the function.
+declare void @panic() noreturn
+declare void @may_throw()
+declare i32 @__gxx_personality_v0(...)
+
+define void @open_at_unreachable(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %die, label %ok
+
+die:
+ call void @panic()
+ unreachable
+
+ok:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+define void @open_at_resume(ptr %p) disable_sanitizer_instrumentation personality ptr @__gxx_personality_v0 {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ invoke void @may_throw()
+ to label %cont unwind label %lpad
+
+cont:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+
+lpad:
+ %lp = landingpad { ptr, i32 }
+ cleanup
+ resume { ptr, i32 } %lp
+}
+
+; CHECK: warning: {{.*}}regions.c:3:5: in function 'unmatched_end': sanitizer instrumentation region end without a matching begin
+define void @unmatched_end(ptr %p) disable_sanitizer_instrumentation !dbg !5 {
+entry:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end(), !dbg !8
+ ret void
+}
+
+; CHECK: warning: in function 'not_closed': sanitizer instrumentation region is not closed before return
+define void @not_closed(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 0, ptr %p
+ ret void
+}
+
+; The region is only entered on one side of the branch, so the join is blamed
+; rather than the end() or the return that follow it.
+; CHECK: warning: in function 'bad_join': control flow merges from paths with different sanitizer instrumentation region nesting; the code that follows is treated as outside the region
+define void @bad_join(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ br i1 %c, label %then, label %join
+
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+
+join:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; If all paths are inside a region but at different depths, the code that
+; follows stays instrumented.
+; CHECK: warning: in function 'nested_join': control flow merges from paths with different sanitizer instrumentation region nesting; the code that follows is treated as nested in the smaller number of regions
+define void @nested_join(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %then, label %join
+
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+
+join:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; A begin() in a loop body without an end() makes the loop header a bad join.
+; CHECK: warning: in function 'loop': control flow merges
+define void @loop(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ br label %header
+
+header:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %header, label %exit
+
+exit:
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3, !4}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C11, file: !1, producer: "clang", isOptimized: true, runtimeVersion: 0, emissionKind: LineTablesOnly)
+!1 = !DIFile(filename: "regions.c", directory: "/tmp")
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = !{i32 7, !"Dwarf Version", i32 5}
+!5 = distinct !DISubprogram(name: "unmatched_end", scope: !1, file: !1, line: 1, type: !6, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0)
+!6 = !DISubroutineType(types: !7)
+!7 = !{}
+!8 = !DILocation(line: 3, column: 5, scope: !5)
diff --git a/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp b/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
index e2fbe466f0ee55..4552302a4981c5 100644
--- a/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
+++ b/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
@@ -8,19 +8,26 @@
#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/AsmParser/Parser.h"
+#include "llvm/IR/DiagnosticInfo.h"
#include "llvm/IR/InstIterator.h"
+#include "llvm/IR/Instructions.h"
#include "llvm/IR/LLVMContext.h"
#include "llvm/IR/Module.h"
#include "llvm/Support/SourceMgr.h"
#include "llvm/Transforms/Utils/Instrumentation.h"
#include "gtest/gtest.h"
+#include <optional>
using namespace llvm;
namespace {
+using Issue = SanitizerInstrumentationRegionIssue;
+
// Every test function loads from %p into values named "in*" for instructions
-// expected to be in a region and "out*" for the rest.
+// expected to be in a region and "out*" for the rest. For malformed regions,
+// the expected problem is given by its kind and by the name of the instruction
+// it points at ("" for the unnamed marker or return itself).
class SanitizerRegionsTest : public testing::Test {
protected:
LLVMContext C;
@@ -43,12 +50,37 @@ class SanitizerRegionsTest : public testing::Test {
return M->getFunction("f");
}
- void expectRegions(StringRef Body, bool ExpectHasRegions = true) {
+ void expectRegions(StringRef Body, bool ExpectHasRegions = true,
+ std::optional<Issue> ExpectedIssue = std::nullopt,
+ StringRef ExpectedWhere = "") {
Function *F = parse(Body);
ASSERT_TRUE(F);
EXPECT_EQ(hasSanitizerInstrumentationRegions(*F), ExpectHasRegions);
SmallPtrSet<const Instruction *, 16> InRegion;
- computeSanitizerInstrumentationRegions(*F, InRegion);
+ std::optional<SanitizerInstrumentationRegionProblem> Problem =
+ computeSanitizerInstrumentationRegions(*F, InRegion);
+ ASSERT_EQ(Problem.has_value(), ExpectedIssue.has_value());
+ if (Problem) {
+ EXPECT_EQ(Problem->Issue, *ExpectedIssue);
+ ASSERT_TRUE(Problem->Where);
+ EXPECT_EQ(Problem->Where->getName(), ExpectedWhere);
+ switch (*ExpectedIssue) {
+ case Issue::UnmatchedEnd:
+ EXPECT_TRUE(isSanitizerInstrumentationEnd(*Problem->Where));
+ break;
+ case Issue::NotClosedAtReturn:
+ EXPECT_TRUE(isa<ReturnInst>(Problem->Where));
+ break;
+ case Issue::MergeOutsideRegion:
+ EXPECT_FALSE(InRegion.contains(Problem->Where));
+ break;
+ case Issue::MergeNestedRegions:
+ EXPECT_TRUE(InRegion.contains(Problem->Where));
+ break;
+ case Issue::NumIssues:
+ llvm_unreachable("not an issue");
+ }
+ }
unsigned Named = 0;
for (const Instruction &I : instructions(*F)) {
if (!I.hasName())
@@ -103,6 +135,26 @@ TEST_F(SanitizerRegionsTest, UnmatchedEnd) {
call void @llvm.sanitizer.instrumentation.end()
%out2 = load i32, ptr %p
ret void
+ )",
+ /*ExpectHasRegions=*/true, Issue::UnmatchedEnd);
+}
+
+TEST_F(SanitizerRegionsTest, NotClosedAtReturn) {
+ expectRegions(R"(
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ ret void
+ )",
+ /*ExpectHasRegions=*/true, Issue::NotClosedAtReturn);
+}
+
+// Leaving a function without returning with a region open is not reported
+// (see findSanitizerInstrumentationRegionProblem() for why).
+TEST_F(SanitizerRegionsTest, NotClosedAtUnreachable) {
+ expectRegions(R"(
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ unreachable
)");
}
@@ -118,7 +170,8 @@ TEST_F(SanitizerRegionsTest, BeginOnOnePath) {
join:
%out = load i32, ptr %p
ret void
- )");
+ )",
+ /*ExpectHasRegions=*/true, Issue::MergeOutsideRegion, "out");
}
// Likewise for a block that can be reached after an end().
@@ -135,7 +188,28 @@ TEST_F(SanitizerRegionsTest, EndOnOnePath) {
join:
%out2 = load i32, ptr %p
ret void
- )");
+ )",
+ /*ExpectHasRegions=*/true, Issue::MergeOutsideRegion, "out2");
+}
+
+// Paths that differ in nesting depth but are all inside a region: the join is
+// still in a region, at the smaller depth, and is reported as such.
+TEST_F(SanitizerRegionsTest, MergeNestedRegions) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %inner, label %join
+ inner:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in1 = load i32, ptr %p
+ br label %join
+ join:
+ %in2 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %out = load i32, ptr %p
+ ret void
+ )",
+ /*ExpectHasRegions=*/true, Issue::MergeNestedRegions, "in2");
}
TEST_F(SanitizerRegionsTest, BalancedOnBothPaths) {
@@ -170,7 +244,8 @@ TEST_F(SanitizerRegionsTest, LoopLeavingRegion) {
exit:
%out2 = load i32, ptr %p
ret void
- )");
+ )",
+ /*ExpectHasRegions=*/true, Issue::MergeOutsideRegion, "out1");
}
// A loop entirely inside a region.
More information about the cfe-commits
mailing list