[clang] [llvm] [Sanitizers] Support sanitizer instrumentation regions in KMSAN, KASAN, KCOV and KCSAN (PR #231374)
Alexander Potapenko via cfe-commits
cfe-commits at lists.llvm.org
Tue Oct 20 02:11:09 PDT 2026
https://github.com/ramosian-glider updated https://github.com/llvm/llvm-project/pull/231374
>From d5fc81910d92d17ba71caedd455a9371ee4f5376 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Mon, 5 Oct 2026 14:04:11 +0000
Subject: [PATCH 1/6] [Sanitizers] Add sanitizer instrumentation region
intrinsics and builtins
Some code must not be instrumented as a whole but still has parts where
instrumentation is allowed. The Linux kernel marks such functions
`noinstr` (lowered to `disable_sanitizer_instrumentation` and/or the
absence of `sanitize_<x>`), and delimits the parts that may call
instrumented code with `instrumentation_begin()` and
`instrumentation_end()`. Sanitizers currently cannot see these parts:
they either skip the whole function or instrument all of it.
This patch adds the common infrastructure that lets sanitizers
instrument such regions:
- `llvm.sanitizer.instrumentation.begin` and
`llvm.sanitizer.instrumentation.end` delimit a region of an otherwise
uninstrumented function in which sanitizer instrumentation is enabled
again. What counts as uninstrumented and what gets instrumented inside
regions is up to each sanitizer; sanitizers that do not support
regions ignore the intrinsics. The intrinsics keep the default memory
effects so that accesses to memory that may be visible outside the
function do not move across region boundaries, and are `IntrNoMerge`. SelectionDAG, FastISel and
GlobalISel drop them if they reach codegen.
- `computeSanitizerInstrumentationRegions()` in Utils/Instrumentation
computes region membership for the passes. It is a must-analysis over
the nesting depth (meet = min over predecessors): regions nest, an
unmatched end is ignored, and an instruction is in a region only if
every path to it entered one, so it is always dominated by a begin.
- `__builtin_sanitizer_instrumentation_begin()` and
`__builtin_sanitizer_instrumentation_end()` expose the intrinsics to
C code, which can detect them with `__has_builtin()`. They are only
lowered to the intrinsics for sanitizers that consume them, so that
they do not act as optimization barriers elsewhere; no sanitizer does
yet, so for now they lower to nothing.
The semantics are documented in LangRef. Follow-up patches implement
regions in KernelMemorySanitizer, AddressSanitizer, SanitizerCoverage
and ThreadSanitizer.
---
clang/docs/LanguageExtensions.md | 45 ++++
clang/docs/ReleaseNotes.md | 7 +
clang/include/clang/Basic/Builtins.td | 18 ++
clang/lib/CodeGen/CGBuiltin.cpp | 14 ++
.../sanitizer-instrumentation-builtins.c | 27 +++
llvm/docs/LangRef.md | 21 ++
llvm/docs/ReleaseNotes.md | 4 +
llvm/include/llvm/IR/Intrinsics.td | 18 ++
.../llvm/Transforms/Utils/Instrumentation.h | 19 ++
llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp | 5 +
llvm/lib/CodeGen/SelectionDAG/FastISel.cpp | 2 +
.../SelectionDAG/SelectionDAGBuilder.cpp | 2 +
llvm/lib/Transforms/Utils/Instrumentation.cpp | 100 +++++++++
.../sanitizer-instrumentation-intrinsics.ll | 19 ++
.../sanitizer-instrumentation-intrinsics.ll | 20 ++
.../sanitizer-instrumentation-markers.ll | 86 ++++++++
.../unittests/Transforms/Utils/CMakeLists.txt | 1 +
.../Transforms/Utils/SanitizerRegionsTest.cpp | 192 ++++++++++++++++++
18 files changed, 600 insertions(+)
create mode 100644 clang/test/CodeGen/sanitizer-instrumentation-builtins.c
create mode 100644 llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll
create mode 100644 llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll
create mode 100644 llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll
create mode 100644 llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
diff --git a/clang/docs/LanguageExtensions.md b/clang/docs/LanguageExtensions.md
index 3db8d083c40756..7ea54b04ada2d7 100644
--- a/clang/docs/LanguageExtensions.md
+++ b/clang/docs/LanguageExtensions.md
@@ -4317,6 +4317,51 @@ such as applying different hotness cutoffs to different check kinds.
Query for this feature with `__has_builtin(__builtin_allow_runtime_check)`.
+### `__builtin_sanitizer_instrumentation_begin` and `__builtin_sanitizer_instrumentation_end`
+
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` delimit a region of a function that
+is otherwise not instrumented by a sanitizer, inside which the sanitizer's
+instrumentation is enabled again.
+
+**Syntax**:
+
+```c
+void __builtin_sanitizer_instrumentation_begin(void);
+void __builtin_sanitizer_instrumentation_end(void);
+```
+
+**Example of use**:
+
+```c
+__attribute__((disable_sanitizer_instrumentation))
+void handle_irq(struct pt_regs *regs) {
+ enter_irq_context(regs); // not instrumented
+ __builtin_sanitizer_instrumentation_begin();
+ dispatch_irq(regs); // instrumented
+ __builtin_sanitizer_instrumentation_end();
+ exit_irq_context(regs); // not instrumented
+}
+```
+
+**Description**:
+
+The builtins are meant for code like the Linux kernel's `noinstr` functions,
+which must not be instrumented as a whole but call instrumented code between
+`instrumentation_begin()` and `instrumentation_end()`. Regions nest, and should
+be balanced on every path through the function. Which functions count as
+uninstrumented, and what gets instrumented inside regions, is up to each
+sanitizer; sanitizers that support regions describe this in their own
+documentation.
+
+The builtins are lowered to the
+[llvm.sanitizer.instrumentation.begin and llvm.sanitizer.instrumentation.end](https://llvm.org/docs/LangRef.html#llvm-sanitizer-instrumentation-begin-and-llvm-sanitizer-instrumentation-end-intrinsics)
+intrinsics only when a sanitizer that supports regions is enabled, and to
+nothing otherwise, so that they do not constrain optimization in other builds.
+
+Query for this feature with
+`__has_builtin(__builtin_sanitizer_instrumentation_begin)`.
+
### `__builtin_nondeterministic_value`
`__builtin_nondeterministic_value` returns a valid nondeterministic value of the same type as the provided argument.
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index 1723b7f78cc9d4..b27b3d2955bf4d 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -1316,6 +1316,13 @@ The `alpha.cplusplus.UseAfterLifetimeEnd` checker was renamed to `alpha.core.Use
modes for `-fsanitize=alloc-token`, selected with
`-Xclang -falloc-token-mode=<mode>`. These modes derive token IDs from both
the allocated type and the name of the function containing the allocation.
+- New builtins `__builtin_sanitizer_instrumentation_begin()` and
+ `__builtin_sanitizer_instrumentation_end()` delimit regions of an otherwise
+ uninstrumented function (e.g. a Linux kernel `noinstr` function) in which
+ sanitizer instrumentation is enabled again. AddressSanitizer,
+ KernelMemorySanitizer, ThreadSanitizer and SanitizerCoverage support them;
+ see [LanguageExtensions](LanguageExtensions.md) and the documentation of each
+ sanitizer.
### Python Binding Changes
diff --git a/clang/include/clang/Basic/Builtins.td b/clang/include/clang/Basic/Builtins.td
index ad9a58bb2ad2d3..8f34a25460df4e 100644
--- a/clang/include/clang/Basic/Builtins.td
+++ b/clang/include/clang/Basic/Builtins.td
@@ -1469,6 +1469,24 @@ def BuiltinAllowSanitizeCheck : Builtin {
let Prototype = "bool(char const*)";
}
+// Sanitizer instrumentation region markers. These delimit a region of an
+// otherwise uninstrumented function (e.g. one marked with
+// __attribute__((disable_sanitizer_instrumentation))) in which sanitizer
+// instrumentation is enabled. They must not be Const or Pure: they are
+// optimization barriers, and moving code across them changes which
+// instructions belong to the region.
+def SanitizerInstrumentationBegin : Builtin {
+ let Spellings = ["__builtin_sanitizer_instrumentation_begin"];
+ let Attributes = [NoThrow];
+ let Prototype = "void()";
+}
+
+def SanitizerInstrumentationEnd : Builtin {
+ let Spellings = ["__builtin_sanitizer_instrumentation_end"];
+ let Attributes = [NoThrow];
+ let Prototype = "void()";
+}
+
def ShuffleVector : Builtin {
let Spellings = ["__builtin_shufflevector"];
let Attributes = [NoThrow, Const, CustomTypeChecking, Constexpr];
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index c93375ff3a18ef..e7f938d63fe750 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3678,6 +3678,20 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
llvm::MetadataAsValue::get(Ctx, llvm::MDString::get(Ctx, Kind)));
return RValue::get(Allow);
}
+ case Builtin::BI__builtin_sanitizer_instrumentation_begin:
+ case Builtin::BI__builtin_sanitizer_instrumentation_end: {
+ // Only emit the region markers for sanitizers that honour them. Elsewhere
+ // they would just be optimization barriers that no pass ever removes. This
+ // lets code guard its uses on __has_builtin alone.
+ SanitizerMask Consumers;
+ if (getLangOpts().Sanitize.hasOneOf(Consumers))
+ Builder.CreateCall(CGM.getIntrinsic(
+ BuiltinIDIfNoAsmLabel ==
+ Builtin::BI__builtin_sanitizer_instrumentation_begin
+ ? Intrinsic::sanitizer_instrumentation_begin
+ : Intrinsic::sanitizer_instrumentation_end));
+ return RValue::get(nullptr);
+ }
case Builtin::BI__builtin_allow_sanitize_check: {
Intrinsic::ID IntrID = Intrinsic::not_intrinsic;
StringRef Name =
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
new file mode 100644
index 00000000000000..6fef449edd6815
--- /dev/null
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -0,0 +1,27 @@
+// Check that the sanitizer region markers are available, and that they lower to
+// nothing without a sanitizer that consumes them.
+
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
+// RUN: -o - %s | FileCheck %s
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
+
+#if !__has_builtin(__builtin_sanitizer_instrumentation_begin)
+#error "__builtin_sanitizer_instrumentation_begin not available"
+#endif
+#if !__has_builtin(__builtin_sanitizer_instrumentation_end)
+#error "__builtin_sanitizer_instrumentation_end not available"
+#endif
+
+extern void instrumented_callee(int);
+
+__attribute__((disable_sanitizer_instrumentation))
+void noinstr_function(int x) {
+ __builtin_sanitizer_instrumentation_begin();
+ instrumented_callee(x);
+ __builtin_sanitizer_instrumentation_end();
+}
+
+// CHECK-LABEL: define {{.*}}void @noinstr_function(
+// CHECK-NOT: llvm.sanitizer.instrumentation
+// CHECK: }
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 0099b7c0d63560..cf5f6c500afaff 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28577,3 +28577,24 @@ taken. A late optimization pass will convert this intrinsic to either
`llvm.cond.loop(true)` or `llvm.cond.loop(pred)`, where `pred`
is a predicate for a conditional branch leading to the intrinsic call,
if possible.
+
+#### '`llvm.sanitizer.instrumentation.begin`' and '`llvm.sanitizer.instrumentation.end`' Intrinsics
+
+##### Syntax:
+
+```llvm
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+```
+
+##### Overview:
+
+The '`llvm.sanitizer.instrumentation.begin`' and '`llvm.sanitizer.instrumentation.end`' intrinsics delimit a region of a function that a sanitizer does not otherwise instrument, in which that sanitizer's instrumentation is enabled again.
+
+##### Semantics:
+
+Inside a region, a sanitizer instruments the code as if the function were instrumented, to the extent it supports instrumenting part of a function; code outside regions stays uninstrumented. Sanitizers that do not support regions ignore the intrinsics, as do all sanitizers in functions that they instrument anyway. Which functions count as uninstrumented, and what is instrumented inside regions, depends on the sanitizer.
+
+Regions nest: `llvm.sanitizer.instrumentation.begin` increments a region nesting depth and `llvm.sanitizer.instrumentation.end` decrements it, so an inner `end` does not terminate an enclosing region. An unmatched `end` is ignored rather than making the depth negative.
+
+An instruction is considered to be inside a region only if *every* path from the function entry to it is inside a region, i.e. region membership is an intersection over predecessors. This makes an in-region instruction always dominated by a `llvm.sanitizer.instrumentation.begin`, so any per-function sanitizer state set up there is available. A region that is entered on some paths but not others is therefore instrumented only where it is unambiguous.
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 302aa57396105e..33981839c657fa 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -171,6 +171,10 @@ Makes programs 10x faster by doing Special New Thing.
* Added the `bitinsert` and `bitextract` instructions for bit-range
manipulation on byte type values.
+* Added the `llvm.sanitizer.instrumentation.begin` and
+ `llvm.sanitizer.instrumentation.end` intrinsics, which delimit regions of an
+ otherwise uninstrumented function in which sanitizer instrumentation is
+ enabled again.
### Changes to LLVM infrastructure
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index 4494bf1e2a2048..c594a4e1a0a9ae 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -2554,6 +2554,24 @@ def int_load_relative: DefaultAttrsIntrinsic<[llvm_ptr_ty],
def int_asan_check_memaccess :
Intrinsic<[],[llvm_ptr_ty, llvm_i32_ty], [ImmArg<ArgIndex<1>>]>;
+// Sanitizer instrumentation regions in functions that are otherwise not
+// instrumented.
+//
+// These are markers consumed by sanitizer passes, but they deliberately
+// keep the default (read/write any memory) effects and so act as memory
+// barriers. Relaxing them, e.g. to IntrInaccessibleMemOnly, lets the optimizer
+// move memory operations across a region boundary: a load can be CSEd with an
+// identical load from before begin() and lose its shadow check, and a store can
+// sink past end() and leave a stale shadow behind. Both defeat the purpose of
+// the region. What the explicit attribute list buys us is the rest of the
+// modelling - these calls do not throw, do not free, do not call back into the
+// module and always return - plus IntrNoMerge, which stops identical markers in
+// sibling blocks from being merged and thereby moving code out of a region.
+def int_sanitizer_instrumentation_begin
+ : DefaultAttrsIntrinsic<[], [], [IntrNoMerge]>;
+def int_sanitizer_instrumentation_end
+ : DefaultAttrsIntrinsic<[], [], [IntrNoMerge]>;
+
// Spin in an infinite loop (using instructions specified by the target) iff the
// argument is true. Used to implement efficient conditional traps.
def int_cond_loop : Intrinsic<[], [llvm_i1_ty],
diff --git a/llvm/include/llvm/Transforms/Utils/Instrumentation.h b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
index 9945ef5665e9be..b8ec84ae8d7dce 100644
--- a/llvm/include/llvm/Transforms/Utils/Instrumentation.h
+++ b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
@@ -13,6 +13,7 @@
#ifndef LLVM_TRANSFORMS_INSTRUMENTATION_H
#define LLVM_TRANSFORMS_INSTRUMENTATION_H
+#include "llvm/ADT/SmallPtrSet.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/IR/BasicBlock.h"
#include "llvm/IR/DebugInfoMetadata.h"
@@ -60,6 +61,24 @@ LLVM_ABI Comdat *getOrCreateFunctionComdat(Function &F, Triple &T);
LLVM_ABI void setGlobalVariableLargeSection(const Triple &TargetTriple,
GlobalVariable &GV);
+/// Returns true if \p F calls llvm.sanitizer.instrumentation.begin or
+/// llvm.sanitizer.instrumentation.end.
+LLVM_ABI bool hasSanitizerInstrumentationRegions(const Function &F);
+
+/// Compute the instructions of \p F that lie inside an
+/// llvm.sanitizer.instrumentation.begin()/end() region, and add them to
+/// \p InRegion.
+///
+/// This is a *must* dataflow problem: an instruction is in a region only if
+/// *every* path from the function entry to it entered a region and did not
+/// leave it again, so an in-region instruction is always dominated by a
+/// begin(). Unbalanced regions are under-approximated, which agrees with
+/// objtool requiring them to be balanced on all paths. The lattice value is
+/// the nesting depth rather than a flag, so that in begin; begin; end; X; end
+/// the instruction X is still in a region.
+LLVM_ABI void computeSanitizerInstrumentationRegions(
+ const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion);
+
// Insert GCOV profiling instrumentation
struct GCOVOptions {
// Specify whether to emit .gcno files.
diff --git a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
index 36ed50c6baf473..a447bcbf8e224b 100644
--- a/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/IRTranslator.cpp
@@ -3292,6 +3292,11 @@ bool IRTranslatorImpl::translateKnownIntrinsic(const CallInst &CI,
case Intrinsic::experimental_noalias_scope_decl:
case Intrinsic::var_annotation:
case Intrinsic::sideeffect:
+ // The sanitizer markers are consumed by the sanitizer passes. They can still
+ // reach codegen when no such pass runs (or it does not look at a given
+ // function), in which case they have no effect.
+ case Intrinsic::sanitizer_instrumentation_begin:
+ case Intrinsic::sanitizer_instrumentation_end:
// Discard annotate attributes, assumptions, and artificial side-effects.
return true;
case Intrinsic::read_volatile_register:
diff --git a/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp b/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
index 3dd81d54773b41..23dd0c7cff9fcf 100644
--- a/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
@@ -1390,6 +1390,8 @@ bool FastISel::selectIntrinsicCall(const IntrinsicInst *II) {
case Intrinsic::lifetime_end:
// The donothing intrinsic does, well, nothing.
case Intrinsic::donothing:
+ case Intrinsic::sanitizer_instrumentation_begin:
+ case Intrinsic::sanitizer_instrumentation_end:
// Neither does the sideeffect intrinsic.
case Intrinsic::sideeffect:
// Neither does the assume intrinsic; it's also OK not to codegen its operand.
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 30b1fee7410d76..2a29e1ff9eac35 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -8161,6 +8161,8 @@ void SelectionDAGBuilder::visitIntrinsicCall(const CallInst &I,
return;
}
case Intrinsic::donothing:
+ case Intrinsic::sanitizer_instrumentation_begin:
+ case Intrinsic::sanitizer_instrumentation_end:
case Intrinsic::seh_try_begin:
case Intrinsic::seh_scope_begin:
case Intrinsic::seh_try_end:
diff --git a/llvm/lib/Transforms/Utils/Instrumentation.cpp b/llvm/lib/Transforms/Utils/Instrumentation.cpp
index 5f71703725ba11..74300dadcef8df 100644
--- a/llvm/lib/Transforms/Utils/Instrumentation.cpp
+++ b/llvm/lib/Transforms/Utils/Instrumentation.cpp
@@ -12,6 +12,8 @@
//===----------------------------------------------------------------------===//
#include "llvm/Transforms/Utils/Instrumentation.h"
+#include "llvm/ADT/DenseMap.h"
+#include "llvm/IR/CFG.h"
#include "llvm/IR/DiagnosticInfo.h"
#include "llvm/IR/DiagnosticPrinter.h"
#include "llvm/IR/IntrinsicInst.h"
@@ -124,3 +126,101 @@ void llvm::setGlobalVariableLargeSection(const Triple &TargetTriple,
return;
GV.setCodeModel(CodeModel::Large);
}
+
+static bool isSanitizerInstrumentationBegin(const Instruction &I) {
+ const auto *II = dyn_cast<IntrinsicInst>(&I);
+ return II &&
+ II->getIntrinsicID() == Intrinsic::sanitizer_instrumentation_begin;
+}
+
+static bool isSanitizerInstrumentationEnd(const Instruction &I) {
+ const auto *II = dyn_cast<IntrinsicInst>(&I);
+ return II && II->getIntrinsicID() == Intrinsic::sanitizer_instrumentation_end;
+}
+
+bool llvm::hasSanitizerInstrumentationRegions(const Function &F) {
+ // Most modules never use the markers, so avoid scanning their functions.
+ const Module *M = F.getParent();
+ auto IsUsed = [M](Intrinsic::ID ID) {
+ const Function *Decl = Intrinsic::getDeclarationIfExists(M, ID);
+ return Decl && !Decl->use_empty();
+ };
+ if (!IsUsed(Intrinsic::sanitizer_instrumentation_begin) &&
+ !IsUsed(Intrinsic::sanitizer_instrumentation_end))
+ return false;
+
+ for (const BasicBlock &BB : F)
+ for (const Instruction &I : BB)
+ if (isSanitizerInstrumentationBegin(I) ||
+ isSanitizerInstrumentationEnd(I))
+ return true;
+ return false;
+}
+
+// Lattice top of the region analysis: this block has not been reached yet, so
+// it does not constrain the depth of its successors.
+static constexpr unsigned UnreachedDepth = ~0U;
+
+/// Return the region nesting depth after \p I, given the depth before it.
+static unsigned regionDepthAfter(const Instruction &I, unsigned Depth) {
+ if (isSanitizerInstrumentationBegin(I))
+ return Depth + 1;
+ // Saturate at zero: an unbalanced end() is malformed input, and wrapping
+ // around would spuriously instrument the rest of the function.
+ if (isSanitizerInstrumentationEnd(I) && Depth)
+ return Depth - 1;
+ return Depth;
+}
+
+void llvm::computeSanitizerInstrumentationRegions(
+ const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion) {
+ DenseMap<const BasicBlock *, unsigned> DepthIn;
+ DenseMap<const BasicBlock *, unsigned> DepthOut;
+ const BasicBlock *Entry = &F.getEntryBlock();
+ for (const BasicBlock &BB : F) {
+ DepthIn[&BB] = UnreachedDepth;
+ DepthOut[&BB] = UnreachedDepth;
+ }
+ DepthIn[Entry] = 0;
+
+ // Both maps only ever decrease and are bounded below by zero, so this
+ // fixed-point iteration terminates. A block is only rescanned when its input
+ // depth decreased, or when it has not been scanned yet.
+ bool Changed = true;
+ while (Changed) {
+ Changed = false;
+ for (const BasicBlock &BB : F) {
+ bool Rescan = DepthOut[&BB] == UnreachedDepth;
+ if (&BB != Entry) {
+ unsigned In = UnreachedDepth;
+ for (const BasicBlock *Pred : predecessors(&BB))
+ In = std::min(In, DepthOut[Pred]);
+ if (In < DepthIn[&BB]) {
+ DepthIn[&BB] = In;
+ Rescan = true;
+ }
+ }
+
+ unsigned Depth = DepthIn[&BB];
+ if (!Rescan || Depth == UnreachedDepth)
+ continue;
+ for (const Instruction &I : BB)
+ Depth = regionDepthAfter(I, Depth);
+ if (Depth < DepthOut[&BB]) {
+ DepthOut[&BB] = Depth;
+ Changed = true;
+ }
+ }
+ }
+
+ for (const BasicBlock &BB : F) {
+ unsigned Depth = DepthIn[&BB];
+ if (Depth == UnreachedDepth)
+ continue;
+ for (const Instruction &I : BB) {
+ Depth = regionDepthAfter(I, Depth);
+ if (Depth > 0)
+ InRegion.insert(&I);
+ }
+ }
+}
diff --git a/llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll b/llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll
new file mode 100644
index 00000000000000..e3de9022ad54f5
--- /dev/null
+++ b/llvm/test/CodeGen/Generic/sanitizer-instrumentation-intrinsics.ll
@@ -0,0 +1,19 @@
+; RUN: llc -O0 < %s | FileCheck %s
+; RUN: llc -O2 < %s | FileCheck %s
+; RUN: llc -O0 -fast-isel < %s | FileCheck %s
+
+; Verify that the backend lowers the sanitizer region intrinsics to nothing and
+; does not crash when compiling a module where no sanitizer pass ran.
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+define void @test_sanitizer_intrinsics() {
+; CHECK-LABEL: test_sanitizer_intrinsics:
+; CHECK: # %bb.0:
+; CHECK-NEXT: ret{{[ql]?}}
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
diff --git a/llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll b/llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll
new file mode 100644
index 00000000000000..b52aea91a03b1e
--- /dev/null
+++ b/llvm/test/CodeGen/X86/sanitizer-instrumentation-intrinsics.ll
@@ -0,0 +1,20 @@
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -O0 < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -O0 -fast-isel < %s | FileCheck %s
+; RUN: llc -mtriple=x86_64-unknown-linux-gnu -O0 -global-isel -global-isel-abort=1 < %s | FileCheck %s
+
+; The sanitizer region intrinsics are consumed by sanitizer passes, but they can
+; still reach codegen when no such pass runs. All three instruction selectors
+; have to drop them instead of failing to select.
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+define void @test_sanitizer_intrinsics() {
+; CHECK-LABEL: test_sanitizer_intrinsics:
+; CHECK-NOT: call
+; CHECK: retq
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
diff --git a/llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll b/llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll
new file mode 100644
index 00000000000000..453d336a4494bc
--- /dev/null
+++ b/llvm/test/Transforms/PhaseOrdering/sanitizer-instrumentation-markers.ll
@@ -0,0 +1,86 @@
+; RUN: opt -passes='default<O2>' -S < %s | FileCheck %s
+
+; The sanitizer instrumentation region markers may access any memory that is
+; visible outside the function, so the optimizer must not move accesses to such
+; memory into or out of a region, and must not merge, hoist, sink or delete the
+; markers themselves.
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @callee(i32)
+
+define i32 @no_cse_across_markers(ptr %p) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define i32 @no_cse_across_markers(
+; CHECK: %a = load i32, ptr %p
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: %b = load i32, ptr %p
+; CHECK-NEXT: store i32 1, ptr %p
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 2, ptr %p
+ %a = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %b = load i32, ptr %p
+ store i32 1, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 2, ptr %p
+ %r = add i32 %a, %b
+ ret i32 %r
+}
+
+define void @no_licm_out_of_region(ptr %p, i32 %n) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @no_licm_out_of_region(
+; CHECK: loop:
+; CHECK: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: %v = load i32, ptr %p
+; CHECK-NEXT: tail call void @callee(i32 %v)
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+entry:
+ br label %loop
+loop:
+ %i = phi i32 [ 0, %entry ], [ %i.next, %loop ]
+ call void @llvm.sanitizer.instrumentation.begin()
+ %v = load i32, ptr %p
+ call void @callee(i32 %v)
+ call void @llvm.sanitizer.instrumentation.end()
+ %i.next = add i32 %i, 1
+ %c = icmp slt i32 %i.next, %n
+ br i1 %c, label %loop, label %exit
+exit:
+ ret void
+}
+
+define void @no_merge(i1 %c, i32 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @no_merge(
+; CHECK: a:
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: tail call void @callee(i32 %x)
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+; CHECK: b:
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: tail call void @callee(i32 %x)
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+entry:
+ br i1 %c, label %a, label %b
+a:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @callee(i32 %x)
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+b:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @callee(i32 %x)
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+exit:
+ ret void
+}
+
+define void @empty_region() disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @empty_region(
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: tail call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
diff --git a/llvm/unittests/Transforms/Utils/CMakeLists.txt b/llvm/unittests/Transforms/Utils/CMakeLists.txt
index f4e4c817114c1b..fe325ca9e9dee6 100644
--- a/llvm/unittests/Transforms/Utils/CMakeLists.txt
+++ b/llvm/unittests/Transforms/Utils/CMakeLists.txt
@@ -29,6 +29,7 @@ add_llvm_unittest(UtilsTests
MemTransferLowering.cpp
ModuleUtilsTest.cpp
ScalarEvolutionExpanderTest.cpp
+ SanitizerRegionsTest.cpp
SizeOptsTest.cpp
SSAUpdaterBulkTest.cpp
UnrollLoopTest.cpp
diff --git a/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp b/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
new file mode 100644
index 00000000000000..e2fbe466f0ee55
--- /dev/null
+++ b/llvm/unittests/Transforms/Utils/SanitizerRegionsTest.cpp
@@ -0,0 +1,192 @@
+//===- SanitizerRegionsTest.cpp - Unit tests for sanitizer regions --------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/SmallPtrSet.h"
+#include "llvm/AsmParser/Parser.h"
+#include "llvm/IR/InstIterator.h"
+#include "llvm/IR/LLVMContext.h"
+#include "llvm/IR/Module.h"
+#include "llvm/Support/SourceMgr.h"
+#include "llvm/Transforms/Utils/Instrumentation.h"
+#include "gtest/gtest.h"
+
+using namespace llvm;
+
+namespace {
+
+// Every test function loads from %p into values named "in*" for instructions
+// expected to be in a region and "out*" for the rest.
+class SanitizerRegionsTest : public testing::Test {
+protected:
+ LLVMContext C;
+ std::unique_ptr<Module> M;
+
+ Function *parse(StringRef Body) {
+ std::string IR = (Twine(R"(
+ declare void @llvm.sanitizer.instrumentation.begin()
+ declare void @llvm.sanitizer.instrumentation.end()
+ declare void @callee()
+ define void @f(ptr %p, i1 %c) {
+ )") + Body + "}\n")
+ .str();
+ SMDiagnostic Err;
+ M = parseAssemblyString(IR, Err, C);
+ if (!M) {
+ Err.print("SanitizerRegionsTest", errs());
+ return nullptr;
+ }
+ return M->getFunction("f");
+ }
+
+ void expectRegions(StringRef Body, bool ExpectHasRegions = true) {
+ Function *F = parse(Body);
+ ASSERT_TRUE(F);
+ EXPECT_EQ(hasSanitizerInstrumentationRegions(*F), ExpectHasRegions);
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ computeSanitizerInstrumentationRegions(*F, InRegion);
+ unsigned Named = 0;
+ for (const Instruction &I : instructions(*F)) {
+ if (!I.hasName())
+ continue;
+ ++Named;
+ EXPECT_EQ(InRegion.contains(&I), I.getName().starts_with("in"))
+ << I.getName().str();
+ }
+ EXPECT_GT(Named, 0u);
+ }
+};
+
+TEST_F(SanitizerRegionsTest, NoRegions) {
+ expectRegions(R"(
+ %out = load i32, ptr %p
+ ret void
+ )",
+ /*ExpectHasRegions=*/false);
+}
+
+TEST_F(SanitizerRegionsTest, Straight) {
+ expectRegions(R"(
+ %out1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ call void @callee()
+ call void @llvm.sanitizer.instrumentation.end()
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+TEST_F(SanitizerRegionsTest, Nested) {
+ expectRegions(R"(
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %in2 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %out = load i32, ptr %p
+ ret void
+ )");
+}
+
+TEST_F(SanitizerRegionsTest, UnmatchedEnd) {
+ expectRegions(R"(
+ call void @llvm.sanitizer.instrumentation.end()
+ %out1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+// A block that can be reached without passing a begin() is not in a region.
+TEST_F(SanitizerRegionsTest, BeginOnOnePath) {
+ expectRegions(R"(
+ entry:
+ br i1 %c, label %then, label %join
+ then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in = load i32, ptr %p
+ br label %join
+ join:
+ %out = load i32, ptr %p
+ ret void
+ )");
+}
+
+// Likewise for a block that can be reached after an end().
+TEST_F(SanitizerRegionsTest, EndOnOnePath) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in1 = load i32, ptr %p
+ br i1 %c, label %then, label %join
+ then:
+ call void @llvm.sanitizer.instrumentation.end()
+ %out1 = load i32, ptr %p
+ br label %join
+ join:
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+TEST_F(SanitizerRegionsTest, BalancedOnBothPaths) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %a, label %b
+ a:
+ %in1 = load i32, ptr %p
+ br label %join
+ b:
+ %in2 = load i32, ptr %p
+ br label %join
+ join:
+ %in3 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %out = load i32, ptr %p
+ ret void
+ )");
+}
+
+// A loop whose body leaves the region on the back edge.
+TEST_F(SanitizerRegionsTest, LoopLeavingRegion) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %loop
+ loop:
+ %out1 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ br i1 %c, label %loop, label %exit
+ exit:
+ %out2 = load i32, ptr %p
+ ret void
+ )");
+}
+
+// A loop entirely inside a region.
+TEST_F(SanitizerRegionsTest, LoopInsideRegion) {
+ expectRegions(R"(
+ entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %loop
+ loop:
+ %in1 = load i32, ptr %p
+ br i1 %c, label %loop, label %exit
+ exit:
+ %in2 = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+ )");
+}
+
+} // namespace
>From 278a39d0fec0a0cfdda2804d643ae8ee2331e9ba Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Mon, 5 Oct 2026 14:04:37 +0000
Subject: [PATCH 2/6] [MSan] Support sanitizer instrumentation regions in KMSAN
The Linux kernel marks functions that must not be instrumented as
`noinstr`, which under KMSAN means `disable_sanitizer_instrumentation`.
Such functions still call instrumented code between the kernel's
`instrumentation_begin()` and `instrumentation_end()` markers. Because
the caller is not instrumented, it never writes parameter shadow, and
the callee reads whatever the last instrumented caller left in the
per-context `param_tls`. This produces false positive KMSAN reports on
the kernel entry paths, e.g.
https://lore.kernel.org/all/69e7ee1f.a00a0220.17a17.001d.GAE@google.com/T/#u
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
MemorySanitizer in kernel mode: in `disable_sanitizer_instrumentation`
functions, fetch the context state at `begin`, and propagate shadow and
set up parameter shadow for calls inside regions, as in a fully
instrumented function. Code outside regions stays uninstrumented; values
computed there are treated as initialized when used inside a region,
and memory written there keeps its previous shadow. Memory intrinsics
outside regions are not lowered to `__msan_mem*`, and vararg
bookkeeping is skipped in region-only functions. The context state slot
is null-initialized in the entry block; the must-analysis in
`computeSanitizerInstrumentationRegions()` guarantees that every
in-region use is dominated by a `begin` that stores to it.
The region markers are left in place for the sanitizer passes that
run after MemorySanitizer; codegen drops them.
Clang now lowers the region builtins to the intrinsics under
`-fsanitize=kernel-memory`. Userspace MSan ignores the intrinsics.
Tests: `llvm/test/Instrumentation/MemorySanitizer/kmsan-*.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/MemorySanitizer.md | 8 +
clang/lib/CodeGen/CGBuiltin.cpp | 2 +-
.../sanitizer-instrumentation-builtins.c | 27 +-
...nitizer-instrumentation-regions-pipeline.c | 39 +++
llvm/docs/LangRef.md | 2 +
.../Instrumentation/MemorySanitizer.cpp | 233 +++++++++++++++---
.../MemorySanitizer/kmsan-intrinsics.ll | 162 ++++++++++++
.../MemorySanitizer/kmsan-region-dataflow.ll | 124 ++++++++++
.../kmsan-region-mem-intrinsics.ll | 41 +++
.../MemorySanitizer/kmsan-region-outside.ll | 49 ++++
.../MemorySanitizer/kmsan-region-userspace.ll | 26 ++
.../MemorySanitizer/kmsan-region-varargs.ll | 36 +++
12 files changed, 701 insertions(+), 48 deletions(-)
create mode 100644 clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-intrinsics.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll
create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll
diff --git a/clang/docs/MemorySanitizer.md b/clang/docs/MemorySanitizer.md
index fdc7b7e0935f02..ca00735b632582 100644
--- a/clang/docs/MemorySanitizer.md
+++ b/clang/docs/MemorySanitizer.md
@@ -91,6 +91,14 @@ positives and therefore should be used with care, and only if absolutely
required; for example for certain code that cannot tolerate any instrumentation
and resulting side-effects. This attribute overrides `no_sanitize("memory")`.
+In the context of KernelMemorySanitizer (KMSAN), this attribute skips the standard
+function-level ABI prologue and MSan instrumentation. However,
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` can be used within a function marked
+with this attribute to selectively enable instrumentation for a specific region
+of code. Values computed outside such a region are treated as initialized when
+used inside it, and memory written outside a region keeps its previous shadow.
+
### Interaction of Inlining with Disabling Sanitizer Instrumentation
- A `no_sanitize` function will not be inlined heuristically by the compiler into a sanitized function.
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index e7f938d63fe750..1f4f2fb46a3b46 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3683,7 +3683,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
// Only emit the region markers for sanitizers that honour them. Elsewhere
// they would just be optimization barriers that no pass ever removes. This
// lets code guard its uses on __has_builtin alone.
- SanitizerMask Consumers;
+ SanitizerMask Consumers = SanitizerKind::KernelMemory;
if (getLangOpts().Sanitize.hasOneOf(Consumers))
Builder.CreateCall(CGM.getIntrinsic(
BuiltinIDIfNoAsmLabel ==
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index 6fef449edd6815..65f36048637cfb 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,10 +1,15 @@
-// Check that the sanitizer region markers are available, and that they lower to
-// nothing without a sanitizer that consumes them.
+// Check that the sanitizer region markers lower to their intrinsics under
+// -fsanitize=kernel-memory, and to nothing without a sanitizer that consumes
+// them.
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
-// RUN: -o - %s | FileCheck %s
+// RUN: -o - %s | FileCheck %s --check-prefix=NONE
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=address \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
#if !__has_builtin(__builtin_sanitizer_instrumentation_begin)
#error "__builtin_sanitizer_instrumentation_begin not available"
@@ -23,5 +28,15 @@ void noinstr_function(int x) {
}
// CHECK-LABEL: define {{.*}}void @noinstr_function(
-// CHECK-NOT: llvm.sanitizer.instrumentation
-// CHECK: }
+// CHECK: call void @llvm.sanitizer.instrumentation.begin()
+// CHECK: call void @instrumented_callee(
+// CHECK: call void @llvm.sanitizer.instrumentation.end()
+
+// The markers are optimization barriers, so they must not be marked readnone
+// or speculatable; only the default attributes plus nomerge are expected.
+// CHECK: declare void @llvm.sanitizer.instrumentation.begin() #[[ATTR:[0-9]+]]
+// CHECK: attributes #[[ATTR]] = { nocallback nofree nomerge nosync nounwind willreturn }
+
+// NONE-LABEL: define {{.*}}void @noinstr_function(
+// NONE-NOT: llvm.sanitizer.instrumentation
+// NONE: }
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
new file mode 100644
index 00000000000000..8ec56d40cb580b
--- /dev/null
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -0,0 +1,39 @@
+// Check that, after the full -O2 pipeline, each sanitizer instruments the code
+// inside a region of a function it otherwise does not instrument, and nothing
+// else. The region markers come from an always_inline helper, as in the Linux
+// kernel. Debug info is enabled so that the verifier checks that the
+// instrumentation calls get debug locations.
+
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=kernel-memory -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN
+
+static inline __attribute__((always_inline)) void instrumentation_begin(void) {
+ __builtin_sanitizer_instrumentation_begin();
+}
+
+static inline __attribute__((always_inline)) void instrumentation_end(void) {
+ __builtin_sanitizer_instrumentation_end();
+}
+
+void instrumented_callee(int);
+
+__attribute__((noinline, disable_sanitizer_instrumentation))
+void noinstr_function(int *p, int x) {
+ int v = *p + (x == 42);
+ instrumentation_begin();
+ if (x == 7)
+ instrumented_callee(*p + v);
+ instrumentation_end();
+ *p = v;
+}
+
+// CHECK-LABEL: define {{.*}}void @noinstr_function(
+// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
+// CHECK: call void @llvm.sanitizer.instrumentation.begin()
+// KMSAN: call ptr @__msan_get_context_state()
+// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
+// CHECK: call void @instrumented_callee(
+// CHECK: call void @llvm.sanitizer.instrumentation.end()
+// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
+// CHECK: ret void
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index cf5f6c500afaff..e28d35304aec84 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28598,3 +28598,5 @@ Inside a region, a sanitizer instruments the code as if the function were instru
Regions nest: `llvm.sanitizer.instrumentation.begin` increments a region nesting depth and `llvm.sanitizer.instrumentation.end` decrements it, so an inner `end` does not terminate an enclosing region. An unmatched `end` is ignored rather than making the depth negative.
An instruction is considered to be inside a region only if *every* path from the function entry to it is inside a region, i.e. region membership is an intersection over predecessors. This makes an in-region instruction always dominated by a `llvm.sanitizer.instrumentation.begin`, so any per-function sanitizer state set up there is available. A region that is entered on some paths but not others is therefore instrumented only where it is unambiguous.
+
+MemorySanitizer supports regions in kernel mode (KMSAN), in functions marked with `disable_sanitizer_instrumentation`. The function-level ABI prologue is skipped, the MSan context state is fetched at `llvm.sanitizer.instrumentation.begin`, and shadow propagation and parameter shadow setup are only performed inside regions. Memory written outside a region keeps whatever shadow it had, and values computed outside a region are treated as fully initialized when they are used inside one.
diff --git a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
index 0d710975b593ee..28dc6082a09bf5 100644
--- a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
@@ -1072,12 +1072,61 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
SmallVector<std::pair<IntrinsicInst *, AllocaInst *>, 16> LifetimeStartList;
SmallVector<StoreInst *, 16> StoreList;
int64_t SplittableBlocksCount = 0;
+ bool IsRegionInstrumentedOnly = false;
+
+ Value *ContextStateAlloca = nullptr;
+
+ Value *getKmsanTLS(IRBuilderBase &IRB, int Index, const Twine &Name) {
+ Value *Ctx = IRB.CreateLoad(MS.PtrTy, ContextStateAlloca);
+ Constant *Zero = IRB.getInt32(0);
+ return IRB.CreateGEP(MS.MsanContextStateTy, Ctx,
+ {Zero, IRB.getInt32(Index)}, Name);
+ }
+
+ Value *getParamTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, 0, "param_shadow")
+ : MS.ParamTLS;
+ }
+ Value *getRetvalTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, 1, "retval_shadow")
+ : MS.RetvalTLS;
+ }
+ Value *getVAArgTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, 2, "va_arg_shadow")
+ : MS.VAArgTLS;
+ }
+ Value *getVAArgOriginTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, 3, "va_arg_origin")
+ : MS.VAArgOriginTLS;
+ }
+ Value *getVAArgOverflowSizeTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, 4, "va_arg_overflow_size")
+ : MS.VAArgOverflowSizeTLS;
+ }
+ Value *getParamOriginTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, 5, "param_origin")
+ : MS.ParamOriginTLS;
+ }
+ Value *getRetvalOriginTLS(IRBuilderBase &IRB) {
+ return ContextStateAlloca ? getKmsanTLS(IRB, 6, "retval_origin")
+ : MS.RetvalOriginTLS;
+ }
+
+ SmallPtrSet<const Instruction *, 16> InRegion;
MemorySanitizerVisitor(Function &F, MemorySanitizer &MS,
const TargetLibraryInfo &TLI)
: Opts(MS.Opts), F(F), MS(MS), VAHelper(CreateVarArgHelper(F, MS, *this)),
TLI(&TLI) {
- bool SanitizeFunction = F.hasFnAttribute(Attribute::SanitizeMemory) &&
+ IsRegionInstrumentedOnly =
+ MS.CompileKernel &&
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation);
+
+ // Instructions outside the regions of a region-instrumented function are
+ // never visited (see visit()), so the function is handled as if it had
+ // sanitize_memory.
+ bool SanitizeFunction = (F.hasFnAttribute(Attribute::SanitizeMemory) ||
+ IsRegionInstrumentedOnly) &&
!Opts.msan_disable_checks;
InsertChecks = SanitizeFunction;
PropagateShadow = SanitizeFunction;
@@ -1098,7 +1147,24 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
if (MS.CompileKernel) {
IRBuilder<> IRB(FnPrologueEnd);
- insertKmsanPrologue(IRB);
+ if (MS.TargetTriple.getArch() == Triple::systemz)
+ MS.MsanMetadataAlloca = IRB.CreateAlloca(MS.MsanMetadata, 0u);
+ // A region-instrumented function must not call into the runtime outside
+ // a region, so it cannot fetch the context in the prologue. Each begin()
+ // fetches it instead and keeps it in a stack slot for the region.
+ if (!IsRegionInstrumentedOnly) {
+ insertKmsanPrologue(IRB);
+ } else {
+ ContextStateAlloca =
+ IRB.CreateAlloca(MS.PtrTy, nullptr, "msan_context_state");
+ // No context is available until the first begin().
+ // computeRegionMembership() guarantees that every instruction that
+ // loads this slot is dominated by a begin() that stored into it, so
+ // this store is dead and will be removed. Keep it anyway: should that
+ // invariant ever be broken, faulting on a null pointer is a far better
+ // failure mode than corrupting memory through an uninitialized one.
+ IRB.CreateStore(Constant::getNullValue(MS.PtrTy), ContextStateAlloca);
+ }
}
LLVM_DEBUG(if (!InsertChecks) dbgs()
@@ -1440,7 +1506,8 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
LLVM_DEBUG(dbgs() << "DONE:\n" << F);
}
- // Returns the last instruction in the new prologue
+ /// Fetch the current KMSAN context and cache it for the rest of the
+ /// function in SSA values.
void insertKmsanPrologue(IRBuilder<> &IRB) {
Value *ContextState = IRB.CreateCall(MS.MsanGetContextStateFn, {});
Constant *Zero = IRB.getInt32(0);
@@ -1460,12 +1527,38 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
MS.RetvalOriginTLS =
IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
{Zero, IRB.getInt32(6)}, "retval_origin");
- if (MS.TargetTriple.getArch() == Triple::systemz)
- MS.MsanMetadataAlloca = IRB.CreateAlloca(MS.MsanMetadata, 0u);
+ }
+
+ /// Fetch the current KMSAN context and cache it in ContextStateAlloca. Used
+ /// at every llvm.sanitizer.instrumentation.begin() of a region-instrumented
+ /// function.
+ void loadKmsanContextState(IRBuilderBase &IRB) {
+ IRB.CreateStore(IRB.CreateCall(MS.MsanGetContextStateFn, {}),
+ ContextStateAlloca);
+ }
+
+ /// Compute, for every instruction of a region-instrumented function, whether
+ /// it belongs to an llvm.sanitizer.instrumentation.begin()/end() region.
+ ///
+ /// computeSanitizerInstrumentationRegions() solves this as a *must* dataflow
+ /// problem, which MSan relies on for the soundness of the cached context
+ /// state: every begin() stores the result of __msan_get_context_state() into
+ /// %msan_context_state, and in-region instrumentation loads the pointer back
+ /// from that slot. With a "may" (union) join, a block reachable both from
+ /// inside and from outside a region would be instrumented, and on the path
+ /// that never executed begin() the instrumentation would load an
+ /// uninitialized alloca and then read and *write* through the resulting wild
+ /// pointer. With the intersection join an in-region instruction is always
+ /// dominated by a begin(), so the slot is always initialized.
+ void computeRegionMembership() {
+ computeSanitizerInstrumentationRegions(F, InRegion);
}
/// Add MemorySanitizer instrumentation to a function.
bool runOnFunction() {
+ if (IsRegionInstrumentedOnly)
+ computeRegionMembership();
+
// Iterate all BBs in depth-first order and create shadow instructions
// for all instructions (where applicable).
// For PHI nodes we create dummy shadow PHIs which will be finalized later.
@@ -1804,7 +1897,7 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
///
/// Shadow = ParamTLS+ArgOffset.
Value *getShadowPtrForArgument(IRBuilder<> &IRB, int ArgOffset) {
- return IRB.CreatePtrAdd(MS.ParamTLS,
+ return IRB.CreatePtrAdd(getParamTLS(IRB),
ConstantInt::get(MS.IntptrTy, ArgOffset), "_msarg");
}
@@ -1812,20 +1905,20 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
Value *getOriginPtrForArgument(IRBuilder<> &IRB, int ArgOffset) {
if (!MS.TrackOrigins)
return nullptr;
- return IRB.CreatePtrAdd(MS.ParamOriginTLS,
+ return IRB.CreatePtrAdd(getParamOriginTLS(IRB),
ConstantInt::get(MS.IntptrTy, ArgOffset),
"_msarg_o");
}
/// Compute the shadow address for a retval.
Value *getShadowPtrForRetval(IRBuilder<> &IRB) {
- return IRB.CreatePointerCast(MS.RetvalTLS, IRB.getPtrTy(0), "_msret");
+ return IRB.CreatePointerCast(getRetvalTLS(IRB), IRB.getPtrTy(0), "_msret");
}
/// Compute the origin address for a retval.
- Value *getOriginPtrForRetval() {
+ Value *getOriginPtrForRetval(IRBuilderBase &IRB) {
// We keep a single origin for the entire retval. Might be too optimistic.
- return MS.RetvalOriginTLS;
+ return getRetvalOriginTLS(IRB);
}
/// Set SV to be the shadow value for V.
@@ -1920,6 +2013,11 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
return AllOnes;
}
if (Argument *A = dyn_cast<Argument>(V)) {
+ if (IsRegionInstrumentedOnly) {
+ if (MS.TrackOrigins && !OriginMap.count(A))
+ setOrigin(A, getCleanOrigin());
+ return getCleanShadow(V);
+ }
// For arguments we compute the shadow on demand and store it in the map.
Value *&ShadowPtr = ShadowMap[V];
if (ShadowPtr)
@@ -2190,6 +2288,7 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
// Don't want to visit if we're in the prologue
if (isInPrologue(I))
return;
+
if (!DebugCounter::shouldExecute(DebugInstrumentInstruction)) {
LLVM_DEBUG(dbgs() << "Skipping instruction: " << I << "\n");
// We still need to set the shadow and origin to clean values.
@@ -2198,6 +2297,17 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
return;
}
+ // In a region-instrumented function, code outside the regions must be left
+ // exactly as it is: no runtime calls (which would leave .noinstr.text in
+ // the kernel), no changes to atomic orderings, no shadow computations.
+ // Values computed there are treated as initialized when used inside a
+ // region, and memory written there keeps whatever shadow it had.
+ if (IsRegionInstrumentedOnly && !InRegion.contains(&I)) {
+ setShadow(&I, getCleanShadow(&I));
+ setOrigin(&I, getCleanOrigin());
+ return;
+ }
+
Instructions.push_back(&I);
}
@@ -7579,6 +7689,24 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
}
void visitIntrinsicInst(IntrinsicInst &I) {
+ switch (I.getIntrinsicID()) {
+ case Intrinsic::sanitizer_instrumentation_begin: {
+ // Only meaningful in a region-instrumented function: elsewhere the
+ // context is already live from the function prologue.
+ if (IsRegionInstrumentedOnly) {
+ NextNodeIRBuilder IRB(&I);
+ loadKmsanContextState(IRB);
+ }
+ // The region markers are shared by all sanitizers, so leave them for the
+ // passes that run after this one. They are dropped during codegen.
+ return;
+ }
+ case Intrinsic::sanitizer_instrumentation_end:
+ return;
+ default:
+ break;
+ }
+
if (maybeHandleCrossPlatformIntrinsic(I))
return;
@@ -7850,7 +7978,8 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
"_msret");
setShadow(&CB, RetvalShadow);
if (MS.TrackOrigins)
- setOrigin(&CB, IRBAfter.CreateLoad(MS.OriginTy, getOriginPtrForRetval()));
+ setOrigin(&CB, IRBAfter.CreateLoad(MS.OriginTy,
+ getOriginPtrForRetval(IRBAfter)));
}
bool isAMustTailRetVal(Value *RetVal) {
@@ -7891,7 +8020,7 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
if (StoreShadow) {
IRB.CreateAlignedStore(Shadow, ShadowPtr, kShadowTLSAlignment);
if (MS.TrackOrigins && StoreOrigin)
- IRB.CreateStore(getOrigin(RetVal), getOriginPtrForRetval());
+ IRB.CreateStore(getOrigin(RetVal), getOriginPtrForRetval(IRB));
}
}
@@ -8290,14 +8419,15 @@ struct VarArgHelperBase : public VarArgHelper {
: F(F), MS(MS), MSV(MSV), VAListTagSize(VAListTagSize) {}
Value *getShadowAddrForVAArgument(IRBuilder<> &IRB, unsigned ArgOffset) {
- Value *Base = IRB.CreatePointerCast(MS.VAArgTLS, MS.IntptrTy);
+ Value *Base = IRB.CreatePointerCast(MSV.getVAArgTLS(IRB), MS.IntptrTy);
return IRB.CreateAdd(Base, ConstantInt::get(MS.IntptrTy, ArgOffset));
}
/// Compute the shadow address for a given va_arg.
Value *getShadowPtrForVAArgument(IRBuilder<> &IRB, unsigned ArgOffset) {
- return IRB.CreatePtrAdd(
- MS.VAArgTLS, ConstantInt::get(MS.IntptrTy, ArgOffset), "_msarg_va_s");
+ return IRB.CreatePtrAdd(MSV.getVAArgTLS(IRB),
+ ConstantInt::get(MS.IntptrTy, ArgOffset),
+ "_msarg_va_s");
}
/// Compute the shadow address for a given va_arg.
@@ -8314,7 +8444,7 @@ struct VarArgHelperBase : public VarArgHelper {
// getOriginPtrForVAArgument() is always called after
// getShadowPtrForVAArgument(), so __msan_va_arg_origin_tls can never
// overflow.
- return IRB.CreatePtrAdd(MS.VAArgOriginTLS,
+ return IRB.CreatePtrAdd(MSV.getVAArgOriginTLS(IRB),
ConstantInt::get(MS.IntptrTy, ArgOffset),
"_msarg_va_o");
}
@@ -8505,7 +8635,7 @@ struct VarArgAMD64Helper : public VarArgHelperBase {
}
Constant *OverflowSize =
ConstantInt::get(IRB.getInt64Ty(), OverflowOffset - AMD64FpEndOffset);
- IRB.CreateStore(OverflowSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(OverflowSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
@@ -8516,7 +8646,7 @@ struct VarArgAMD64Helper : public VarArgHelperBase {
// va_arg_tls somewhere in the function entry block.
IRBuilder<> IRB(MSV.FnPrologueEnd);
VAArgOverflowSize =
- IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = IRB.CreateAdd(
ConstantInt::get(MS.IntptrTy, AMD64FpEndOffset), VAArgOverflowSize);
VAArgTLSCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
@@ -8527,13 +8657,14 @@ struct VarArgAMD64Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
if (MS.TrackOrigins) {
VAArgTLSOriginCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
VAArgTLSOriginCopy->setAlignment(kShadowTLSAlignment);
IRB.CreateMemCpy(VAArgTLSOriginCopy, kShadowTLSAlignment,
- MS.VAArgOriginTLS, kShadowTLSAlignment, SrcSize);
+ MSV.getVAArgOriginTLS(IRB), kShadowTLSAlignment,
+ SrcSize);
}
}
@@ -8682,7 +8813,7 @@ struct VarArgAArch64Helper : public VarArgHelperBase {
}
Constant *OverflowSize =
ConstantInt::get(IRB.getInt64Ty(), OverflowOffset - AArch64VAEndOffset);
- IRB.CreateStore(OverflowSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(OverflowSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
// Retrieve a va_list field of 'void*' size.
@@ -8708,7 +8839,7 @@ struct VarArgAArch64Helper : public VarArgHelperBase {
// va_arg_tls somewhere in the function entry block.
IRBuilder<> IRB(MSV.FnPrologueEnd);
VAArgOverflowSize =
- IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = IRB.CreateAdd(
ConstantInt::get(MS.IntptrTy, AArch64VAEndOffset), VAArgOverflowSize);
VAArgTLSCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
@@ -8719,7 +8850,7 @@ struct VarArgAArch64Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -8909,14 +9040,15 @@ struct VarArgPowerPC64Helper : public VarArgHelperBase {
ConstantInt::get(MS.IntptrTy, VAArgOffset - VAArgBase);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ VAArgSize =
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -8931,7 +9063,7 @@ struct VarArgPowerPC64Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(IRB.getInt64Ty(), kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9045,14 +9177,14 @@ struct VarArgPowerPC32Helper : public VarArgHelperBase {
ConstantInt::get(MS.IntptrTy, VAArgOffset - VAArgBase);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(MS.IntptrTy, MS.VAArgOverflowSizeTLS);
+ VAArgSize = IRB.CreateLoad(MS.IntptrTy, MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -9067,7 +9199,7 @@ struct VarArgPowerPC32Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9330,7 +9462,7 @@ struct VarArgSystemZHelper : public VarArgHelperBase {
}
Constant *OverflowSize = ConstantInt::get(
IRB.getInt64Ty(), OverflowOffset - SystemZOverflowOffset);
- IRB.CreateStore(OverflowSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(OverflowSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void copyRegSaveArea(IRBuilder<> &IRB, Value *VAListTag) {
@@ -9391,7 +9523,7 @@ struct VarArgSystemZHelper : public VarArgHelperBase {
// va_arg_tls somewhere in the function entry block.
IRBuilder<> IRB(MSV.FnPrologueEnd);
VAArgOverflowSize =
- IRB.CreateLoad(IRB.getInt64Ty(), MS.VAArgOverflowSizeTLS);
+ IRB.CreateLoad(IRB.getInt64Ty(), MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize =
IRB.CreateAdd(ConstantInt::get(MS.IntptrTy, SystemZOverflowOffset),
VAArgOverflowSize);
@@ -9403,13 +9535,14 @@ struct VarArgSystemZHelper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
if (MS.TrackOrigins) {
VAArgTLSOriginCopy = IRB.CreateAlloca(Type::getInt8Ty(*MS.C), CopySize);
VAArgTLSOriginCopy->setAlignment(kShadowTLSAlignment);
IRB.CreateMemCpy(VAArgTLSOriginCopy, kShadowTLSAlignment,
- MS.VAArgOriginTLS, kShadowTLSAlignment, SrcSize);
+ MSV.getVAArgOriginTLS(IRB), kShadowTLSAlignment,
+ SrcSize);
}
}
@@ -9485,14 +9618,14 @@ struct VarArgI386Helper : public VarArgHelperBase {
Constant *TotalVAArgSize = ConstantInt::get(MS.IntptrTy, VAArgOffset);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(MS.IntptrTy, MS.VAArgOverflowSizeTLS);
+ VAArgSize = IRB.CreateLoad(MS.IntptrTy, MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -9506,7 +9639,7 @@ struct VarArgI386Helper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9570,14 +9703,14 @@ struct VarArgGenericHelper : public VarArgHelperBase {
Constant *TotalVAArgSize = ConstantInt::get(MS.IntptrTy, VAArgOffset);
// Here using VAArgOverflowSizeTLS as VAArgSizeTLS to avoid creation of
// a new class member i.e. it is the total size of all VarArgs.
- IRB.CreateStore(TotalVAArgSize, MS.VAArgOverflowSizeTLS);
+ IRB.CreateStore(TotalVAArgSize, MSV.getVAArgOverflowSizeTLS(IRB));
}
void finalizeInstrumentation() override {
assert(!VAArgSize && !VAArgTLSCopy &&
"finalizeInstrumentation called twice");
IRBuilder<> IRB(MSV.FnPrologueEnd);
- VAArgSize = IRB.CreateLoad(MS.IntptrTy, MS.VAArgOverflowSizeTLS);
+ VAArgSize = IRB.CreateLoad(MS.IntptrTy, MSV.getVAArgOverflowSizeTLS(IRB));
Value *CopySize = VAArgSize;
if (!VAStartInstrumentationList.empty()) {
@@ -9591,7 +9724,7 @@ struct VarArgGenericHelper : public VarArgHelperBase {
Value *SrcSize = IRB.CreateBinaryIntrinsic(
Intrinsic::umin, CopySize,
ConstantInt::get(MS.IntptrTy, kParamTLSSize));
- IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MS.VAArgTLS,
+ IRB.CreateMemCpy(VAArgTLSCopy, kShadowTLSAlignment, MSV.getVAArgTLS(IRB),
kShadowTLSAlignment, SrcSize);
}
@@ -9645,6 +9778,15 @@ struct VarArgNoOpHelper : public VarArgHelper {
static VarArgHelper *CreateVarArgHelper(Function &Func, MemorySanitizer &Msan,
MemorySanitizerVisitor &Visitor) {
+ // A region-instrumented function is only instrumented between
+ // llvm.sanitizer.instrumentation.begin() and end(). VarArgHelpers emit their
+ // setup into the function prologue, which is outside every region: the MSan
+ // context state has not been fetched there yet, so that code would read and
+ // write through an uninitialized pointer. Leave varargs alone instead.
+ // sanitizeFunction() only gets here for such a function in kernel mode.
+ if (Func.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
+ return new VarArgNoOpHelper(Func, Msan, Visitor);
+
// VarArg handling is only implemented on AMD64. False positives are possible
// on other platforms.
Triple TargetTriple(Func.getParent()->getTargetTriple());
@@ -9698,8 +9840,17 @@ bool MemorySanitizer::sanitizeFunction(Function &F, TargetLibraryInfo &TLI) {
if (!CompileKernel && F.getName() == kMsanModuleCtorName)
return false;
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
- return false;
+ // Functions marked disable_sanitizer_instrumentation are normally skipped
+ // entirely. They are still instrumented inside
+ // llvm.sanitizer.instrumentation.begin()/end() regions, so look for those
+ // first.
+ if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation)) {
+ // Region instrumentation is a KernelMemorySanitizer feature: it refers to
+ // the context state returned by __msan_get_context_state(), which only
+ // exists in the kernel runtime.
+ if (!CompileKernel || !hasSanitizerInstrumentationRegions(F))
+ return false;
+ }
MemorySanitizerVisitor Visitor(F, *this, TLI);
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-intrinsics.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-intrinsics.ll
new file mode 100644
index 00000000000000..ad7c004d707983
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-intrinsics.ll
@@ -0,0 +1,162 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+define void @instrumented_function() sanitize_memory {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+define void @no_sanitize_function() {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; Outside disable_sanitizer_instrumentation functions the markers mean nothing
+; to KMSAN: the context is fetched once in the prologue, not at begin().
+; CHECK-LABEL: @instrumented_function(
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK-NOT: call ptr @__msan_get_context_state()
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+; CHECK-LABEL: @no_sanitize_function(
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK-NOT: call ptr @__msan_get_context_state()
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+define void @noinstr_function(ptr %p, i64 %val) disable_sanitizer_instrumentation {
+entry:
+ %x = load i64, ptr %p
+ %add = add i64 %x, %val
+ call void @llvm.sanitizer.instrumentation.begin()
+ %y = load i64, ptr %p
+ %add2 = add i64 %y, %add
+ store i64 %add2, ptr %p
+ %z = load i64, ptr %p
+ store i64 %z, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %w = load i64, ptr %p
+ ret void
+}
+
+; CHECK-LABEL: @noinstr_function(
+; CHECK-NOT: __msan_get_context_state
+; CHECK: %x = load i64, ptr %p
+; CHECK: %add = add i64 %x, %val
+; CHECK: [[CTX1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK: store ptr [[CTX1]], ptr %msan_context_state
+; CHECK: %y = load i64, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: [[YS:%_msld.*]] = load i64, ptr
+; %add was computed outside the region, so its shadow is clean.
+; CHECK: [[ADD2S:%.*]] = or i64 [[YS]], 0
+; CHECK: %add2 = add i64 %y, %add
+; CHECK: [[SP1:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: [[SHP1:%.*]] = extractvalue { ptr, ptr } [[SP1]], 0
+; CHECK: store i64 [[ADD2S]], ptr [[SHP1]]
+; CHECK: store i64 %add2, ptr %p
+; CHECK-NOT: call ptr @__msan_get_context_state()
+; CHECK: %z = load i64, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: [[ZS:%_msld.*]] = load i64, ptr
+; CHECK: [[SP2:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: [[SHP2:%.*]] = extractvalue { ptr, ptr } [[SP2]], 0
+; CHECK: store i64 [[ZS]], ptr [[SHP2]]
+; CHECK: store i64 %z, ptr %p
+; CHECK-NOT: call ptr @__msan_get_context_state()
+; CHECK-NOT: call { ptr, ptr } @__msan_metadata_ptr_for_load_8
+; CHECK: %w = load i64, ptr %p
+; CHECK: ret void
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+define void @noinstr_with_branch(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %cond, label %if.then, label %if.else
+
+if.then:
+ %a = load i64, ptr %p
+ store i64 %a, ptr %p
+ br label %if.end
+
+if.else:
+ %b = load i64, ptr %p
+ store i64 %b, ptr %p
+ br label %if.end
+
+if.end:
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @noinstr_with_branch(
+; CHECK: [[CTX3:%.*]] = call ptr @__msan_get_context_state()
+; CHECK: store ptr [[CTX3]], ptr %msan_context_state
+; CHECK: if.then:
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: if.else:
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+
+define void @noinstr_with_loop(ptr %p, i64 %n) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %loop
+
+loop:
+ %i = phi i64 [ 0, %entry ], [ %inc, %loop ]
+ %c = load i64, ptr %p
+ store i64 %c, ptr %p
+ %inc = add i64 %i, 1
+ %cmp = icmp ult i64 %inc, %n
+ br i1 %cmp, label %loop, label %exit
+
+exit:
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @noinstr_with_loop(
+; CHECK: [[CTX4:%.*]] = call ptr @__msan_get_context_state()
+; CHECK: store ptr [[CTX4]], ptr %msan_context_state
+; CHECK: loop:
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_8(ptr %p)
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: exit:
+
+declare i64 @dummy_call(i64)
+
+define i64 @noinstr_with_various_insts(ptr %p, i64 %val) disable_sanitizer_instrumentation {
+entry:
+ %a = alloca i64
+ %rmw1 = atomicrmw add ptr %p, i64 %val seq_cst
+ %call1 = call i64 @dummy_call(i64 %rmw1)
+ store i64 %call1, ptr %a
+
+ call void @llvm.sanitizer.instrumentation.begin()
+ %rmw2 = atomicrmw add ptr %p, i64 %val seq_cst
+ %call2 = call i64 @dummy_call(i64 %rmw2)
+ store i64 %call2, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+
+ %rmw3 = atomicrmw add ptr %p, i64 %val seq_cst
+ %call3 = call i64 @dummy_call(i64 %rmw3)
+ store i64 %call3, ptr %a
+ ret i64 %call3
+}
+
+; CHECK-LABEL: @noinstr_with_various_insts(
+; CHECK-NOT: __msan_metadata
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_store_8(ptr %p)
+; CHECK: ret i64 %call3
\ No newline at end of file
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll
new file mode 100644
index 00000000000000..aee6d9fdb20ea8
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-dataflow.ll
@@ -0,0 +1,124 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; Region membership is a "must" dataflow problem over the region nesting depth.
+; These tests cover the cases where an "any path" or flag-based analysis gets it
+; wrong.
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+; A block reachable both from inside and from outside a region must not be
+; instrumented: on the path that never executed begin(), %msan_context_state
+; holds no context state, and the instrumentation would load a wild pointer out
+; of it and then store through it.
+
+define i32 @unbalanced_join(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ br i1 %cond, label %left, label %right
+
+left:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %merge
+
+right:
+ br label %merge
+
+merge:
+ %v = load i32, ptr %p
+ ret i32 %v
+}
+
+; CHECK-LABEL: @unbalanced_join(
+; CHECK: merge:
+; CHECK-NOT: load ptr, ptr %msan_context_state
+; CHECK-NOT: __msan_metadata_ptr_for_load_4
+; CHECK: ret i32 %v
+
+; The same join is instrumented once every incoming path has entered a region.
+
+define i32 @balanced_join(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ br i1 %cond, label %left, label %right
+
+left:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %merge
+
+right:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %merge
+
+merge:
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; CHECK-LABEL: @balanced_join(
+; CHECK: merge:
+; CHECK: %v = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+
+; Regions nest: the inner end() must not terminate the outer region.
+
+define void @nested_regions(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ %a = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %b = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ %c = load i32, ptr %p
+ ret void
+}
+
+; CHECK-LABEL: @nested_regions(
+; CHECK: %a = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+; CHECK: %b = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+; CHECK: %c = load i32, ptr %p
+; CHECK-NOT: __msan_metadata_ptr_for_load_4
+; CHECK: ret void
+
+; Nesting also has to survive a join: both predecessors of %merge are at depth
+; one, so %merge is still inside the outer region.
+
+define void @nested_regions_join(ptr %p, i1 %cond) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %cond, label %inner, label %merge
+
+inner:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %merge
+
+merge:
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: @nested_regions_join(
+; CHECK: merge:
+; CHECK: %v = load i32, ptr %p
+; CHECK: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(ptr %p)
+
+; An end() without a matching begin() is malformed input. Saturate the depth at
+; zero rather than wrapping around and instrumenting the rest of the function.
+
+define void @unmatched_end(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.end()
+ %v = load i32, ptr %p
+ ret void
+}
+
+; CHECK-LABEL: @unmatched_end(
+; CHECK-NOT: __msan_metadata_ptr_for_load_4
+; CHECK: ret void
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll
new file mode 100644
index 00000000000000..0643ce3331c95f
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-mem-intrinsics.ll
@@ -0,0 +1,41 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; Memory intrinsics are rewritten into KMSAN runtime calls. Outside an
+; instrumentation region they have to be left alone: emitting the call there
+; instruments code the region markers explicitly excluded, and in noinstr kernel
+; code it makes objtool report
+; "call to __msan_memcpy() leaves .noinstr.text section".
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.memcpy.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @llvm.memmove.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @llvm.memset.p0.i64(ptr, i8, i64, i1)
+
+define void @mem_intrinsics(ptr %dst, ptr %src) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memmove.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memset.p0.i64(ptr %dst, i8 0, i64 64, i1 false)
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memmove.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ call void @llvm.memset.p0.i64(ptr %dst, i8 0, i64 64, i1 false)
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+ ret void
+}
+
+; CHECK-LABEL: @mem_intrinsics(
+; CHECK: call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+; CHECK: call void @llvm.memmove.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+; CHECK: call void @llvm.memset.p0.i64(ptr %dst, i8 0, i64 64, i1 false)
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK: call ptr @__msan_memcpy(ptr %dst, ptr %src, i64 64)
+; CHECK: call ptr @__msan_memmove(ptr %dst, ptr %src, i64 64)
+; CHECK: call ptr @__msan_memset(ptr %dst, i32 0, i64 64)
+; CHECK: call void @llvm.memcpy.p0.p0.i64(ptr %dst, ptr %src, i64 64, i1 false)
+; CHECK: ret void
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll
new file mode 100644
index 00000000000000..73988e819c66b3
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-outside.ll
@@ -0,0 +1,49 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan | FileCheck %s
+
+; In a disable_sanitizer_instrumentation function, code outside the regions is
+; left exactly as it is: no KMSAN runtime calls, no stronger atomic orderings,
+; no shadow computations. Inside a region the function is instrumented like a
+; sanitize_memory one, including poisoning undef values.
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.masked.store.v4i32.p0(<4 x i32>, ptr, i32, <4 x i1>)
+
+define i32 @outside(ptr %p, <4 x i32> %v, <4 x i1> %m) disable_sanitizer_instrumentation {
+entry:
+ %a = load atomic i32, ptr %p monotonic, align 4
+ store atomic i32 1, ptr %p monotonic, align 4
+ %r = atomicrmw add ptr %p, i32 1 monotonic
+ %x = cmpxchg ptr %p, i32 0, i32 1 monotonic monotonic
+ call void @llvm.masked.store.v4i32.p0(<4 x i32> %v, ptr %p, i32 4, <4 x i1> %m)
+ %b = xor i32 %a, %r
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 undef, ptr %p, align 4
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %b
+}
+
+; CHECK-LABEL: define i32 @outside(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %msan_context_state = alloca ptr
+; CHECK-NEXT: store ptr null, ptr %msan_context_state
+; CHECK-NEXT: call void @llvm.donothing()
+; CHECK-NEXT: %a = load atomic i32, ptr %p monotonic, align 4
+; CHECK-NEXT: store atomic i32 1, ptr %p monotonic, align 4
+; CHECK-NEXT: %r = atomicrmw add ptr %p, i32 1 monotonic
+; CHECK-NEXT: %x = cmpxchg ptr %p, i32 0, i32 1 monotonic monotonic
+; CHECK-NEXT: call void @llvm.masked.store.v4i32.p0(<4 x i32> %v, ptr align 4 %p, <4 x i1> %m)
+; CHECK-NEXT: %b = xor i32 %a, %r
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @__msan_get_context_state()
+; CHECK-NEXT: store ptr
+; CHECK-NEXT: [[SP:%.*]] = call { ptr, ptr } @__msan_metadata_ptr_for_store_4(ptr %p)
+; CHECK-NEXT: [[SHP:%.*]] = extractvalue { ptr, ptr } [[SP]], 0
+; CHECK-NEXT: extractvalue
+; CHECK-NEXT: store i32 -1, ptr [[SHP]], align 4
+; CHECK: store i32 undef, ptr %p, align 4
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret i32 %b
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll
new file mode 100644
index 00000000000000..8c1d79d8243bc7
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-userspace.ll
@@ -0,0 +1,26 @@
+; RUN: opt < %s -S -passes=msan 2>&1 | FileCheck %s
+
+; Region instrumentation is a KernelMemorySanitizer feature: it refers to the
+; context state returned by __msan_get_context_state(), which only exists in the
+; kernel runtime. Under userspace MemorySanitizer the markers must simply be
+; ignored - this used to crash the compiler on a null MsanGetContextStateFn.
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+define i32 @region_in_userspace_msan(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; CHECK-LABEL: @region_in_userspace_msan(
+; CHECK-NOT: __msan_get_context_state
+; CHECK-NOT: msan_context_state
+; CHECK: %v = load i32, ptr %p
+; CHECK: ret i32 %v
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll
new file mode 100644
index 00000000000000..fa08752df90add
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-region-varargs.ll
@@ -0,0 +1,36 @@
+; RUN: opt < %s -msan-check-access-address=0 -msan-kernel=1 -S -passes=msan 2>&1 | FileCheck %s
+
+; VarArgHelpers emit their setup into the function prologue. In a
+; region-instrumented function the prologue is outside every region, so
+; %msan_context_state has not been stored to yet: the emitted code used to load
+; that slot and then read and write through the resulting pointer. Varargs must
+; be left alone in region-instrumented functions.
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.va_start.p0(ptr)
+declare void @llvm.va_end.p0(ptr)
+
+define i32 @va_noinstr(i32 %n, ...) disable_sanitizer_instrumentation {
+entry:
+ %ap = alloca [1 x { i32, i32, ptr, ptr }]
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.va_start.p0(ptr %ap)
+ %v = va_arg ptr %ap, i32
+ call void @llvm.va_end.p0(ptr %ap)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; The context state must not be read before the first begin() stores it. The
+; first thing touching the slot is the store from __msan_get_context_state().
+; CHECK-LABEL: @va_noinstr(
+; CHECK: %msan_context_state = alloca ptr
+; CHECK-NOT: load ptr, ptr %msan_context_state
+; CHECK-NOT: va_arg_overflow_size
+; CHECK-NOT: va_arg_shadow
+; CHECK: call ptr @__msan_get_context_state()
+; CHECK: store ptr {{.*}}, ptr %msan_context_state
>From 4a3f265971d15dd425300173488c057973fdc036 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Mon, 5 Oct 2026 14:04:47 +0000
Subject: [PATCH 3/6] [ASan] Support sanitizer instrumentation regions
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
AddressSanitizer, in both user and kernel mode. Functions that lack
`sanitize_address` or have `disable_sanitizer_instrumentation` are
normally not instrumented at all; if they contain regions, memory
accesses and memory intrinsics inside the regions are now checked.
The Linux kernel uses this for `noinstr` functions, which may call
instrumented code between `instrumentation_begin()` and
`instrumentation_end()`, but whose own memory accesses in those
regions were never checked by KASAN.
Only memory accesses are instrumented in such functions. Their stack
frame is not laid out by ASan, so stack poisoning, use-after-return and
the `__asan_handle_no_return()` calls are skipped, and accesses to the
function's own allocas, which have no redzones, are not checked. A call
outside a region still resets the per-block deduplication of checks.
Clang now lowers the region builtins to the intrinsics under
`-fsanitize=address` and `-fsanitize=kernel-address`.
Tests: `llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/AddressSanitizer.md | 21 ++
clang/lib/CodeGen/CGBuiltin.cpp | 4 +-
.../sanitizer-instrumentation-builtins.c | 12 +-
...nitizer-instrumentation-regions-pipeline.c | 4 +
llvm/docs/LangRef.md | 2 +
.../Instrumentation/AddressSanitizer.cpp | 49 ++++-
.../sanitizer-instrumentation-regions.ll | 183 ++++++++++++++++++
7 files changed, 262 insertions(+), 13 deletions(-)
create mode 100644 llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll
diff --git a/clang/docs/AddressSanitizer.md b/clang/docs/AddressSanitizer.md
index 19276261b75620..2920b122df1be5 100644
--- a/clang/docs/AddressSanitizer.md
+++ b/clang/docs/AddressSanitizer.md
@@ -296,6 +296,27 @@ AddressSanitizer also supports
works similarly to `__attribute__((no_sanitize("address")))`, but it also
prevents instrumentation performed by other sanitizers.
+### Re-enabling Instrumentation in Regions
+
+Inside a function that AddressSanitizer does not instrument, a region
+delimited by `__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` is instrumented again: memory
+accesses and memory intrinsics in it are checked. The function's stack frame
+is still not instrumented, and accesses to its own local variables are not
+checked. Regions nest, and code reachable without passing through a
+`__builtin_sanitizer_instrumentation_begin()` is not considered to be in a
+region.
+
+```c
+__attribute__((no_sanitize("address")))
+void mostly_uninstrumented(int *p) {
+ ... // not checked
+ __builtin_sanitizer_instrumentation_begin();
+ *p = 0; // checked
+ __builtin_sanitizer_instrumentation_end();
+}
+```
+
### Interaction of Inlining with Disabling Sanitizer Instrumentation
- A `no_sanitize` function will not be inlined heuristically by the compiler into a sanitized function.
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 1f4f2fb46a3b46..40f32c324ead5b 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3683,7 +3683,9 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
// Only emit the region markers for sanitizers that honour them. Elsewhere
// they would just be optimization barriers that no pass ever removes. This
// lets code guard its uses on __has_builtin alone.
- SanitizerMask Consumers = SanitizerKind::KernelMemory;
+ SanitizerMask Consumers = SanitizerKind::KernelMemory |
+ SanitizerKind::Address |
+ SanitizerKind::KernelAddress;
if (getLangOpts().Sanitize.hasOneOf(Consumers))
Builder.CreateCall(CGM.getIntrinsic(
BuiltinIDIfNoAsmLabel ==
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index 65f36048637cfb..fdc1e3187bb069 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,13 +1,15 @@
// Check that the sanitizer region markers lower to their intrinsics under
-// -fsanitize=kernel-memory, and to nothing without a sanitizer that consumes
-// them.
+// -fsanitize=kernel-memory, kernel-address and address, and to nothing without
+// a sanitizer that consumes them.
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
// RUN: -o - %s | FileCheck %s --check-prefix=NONE
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=address \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
@@ -37,6 +39,10 @@ void noinstr_function(int x) {
// CHECK: declare void @llvm.sanitizer.instrumentation.begin() #[[ATTR:[0-9]+]]
// CHECK: attributes #[[ATTR]] = { nocallback nofree nomerge nosync nounwind willreturn }
+// ASAN-LABEL: define {{.*}}void @noinstr_function(
+// ASAN: call void @llvm.sanitizer.instrumentation.begin()
+// ASAN: call void @llvm.sanitizer.instrumentation.end()
+
// NONE-LABEL: define {{.*}}void @noinstr_function(
// NONE-NOT: llvm.sanitizer.instrumentation
// NONE: }
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
index 8ec56d40cb580b..1a9229790441dd 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -7,6 +7,9 @@
// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
// RUN: -fsanitize=kernel-memory -emit-llvm -o - %s \
// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=kernel-address -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KASAN
static inline __attribute__((always_inline)) void instrumentation_begin(void) {
__builtin_sanitizer_instrumentation_begin();
@@ -33,6 +36,7 @@ void noinstr_function(int *p, int x) {
// CHECK: call void @llvm.sanitizer.instrumentation.begin()
// KMSAN: call ptr @__msan_get_context_state()
// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
+// KASAN: call void @__asan_report_load4(
// CHECK: call void @instrumented_callee(
// CHECK: call void @llvm.sanitizer.instrumentation.end()
// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index e28d35304aec84..ac3f4747a52d04 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28600,3 +28600,5 @@ Regions nest: `llvm.sanitizer.instrumentation.begin` increments a region nesting
An instruction is considered to be inside a region only if *every* path from the function entry to it is inside a region, i.e. region membership is an intersection over predecessors. This makes an in-region instruction always dominated by a `llvm.sanitizer.instrumentation.begin`, so any per-function sanitizer state set up there is available. A region that is entered on some paths but not others is therefore instrumented only where it is unambiguous.
MemorySanitizer supports regions in kernel mode (KMSAN), in functions marked with `disable_sanitizer_instrumentation`. The function-level ABI prologue is skipped, the MSan context state is fetched at `llvm.sanitizer.instrumentation.begin`, and shadow propagation and parameter shadow setup are only performed inside regions. Memory written outside a region keeps whatever shadow it had, and values computed outside a region are treated as fully initialized when they are used inside one.
+
+AddressSanitizer supports regions in functions that lack the `sanitize_address` attribute or are marked with `disable_sanitizer_instrumentation`. Inside regions, memory accesses and memory intrinsics are instrumented. The stack frame of such a function is not instrumented, accesses to its own `alloca`s are not checked, and no `__asan_handle_no_return` calls are inserted.
diff --git a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
index 70841c8324aa3f..ab638dfd98ed89 100644
--- a/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp
@@ -2879,13 +2879,23 @@ bool AddressSanitizer::instrumentFunction(Function &F,
if (maybeInsertAsanInitAtFunctionEntry(F))
FunctionModified = true;
- // Leave if the function doesn't need instrumentation.
- if (!F.hasFnAttribute(Attribute::SanitizeAddress)) return FunctionModified;
-
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
- return FunctionModified;
-
- LLVM_DEBUG(dbgs() << "ASAN instrumenting:\n" << F << "\n");
+ // Leave if the function doesn't need instrumentation. Functions that are
+ // not instrumented as a whole are still instrumented inside
+ // llvm.sanitizer.instrumentation.begin()/end() regions. Only memory
+ // accesses are instrumented there: the stack frame of such a function is
+ // not laid out by ASan, so stack poisoning, use-after-return and the
+ // handling of noreturn calls (which unpoisons the stack) are left out.
+ bool RegionOnly = false;
+ if (!F.hasFnAttribute(Attribute::SanitizeAddress) ||
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation)) {
+ if (!hasSanitizerInstrumentationRegions(F))
+ return FunctionModified;
+ RegionOnly = true;
+ }
+
+ LLVM_DEBUG(dbgs() << "ASAN instrumenting" << (RegionOnly ? " regions" : "")
+ << ":\n"
+ << F << "\n");
initializeCallbacks(TLI);
@@ -2902,6 +2912,10 @@ bool AddressSanitizer::instrumentFunction(Function &F,
if (TargetTriple.isOSWindows())
markCatchParametersAsUninteresting(F);
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ if (RegionOnly)
+ computeSanitizerInstrumentationRegions(F, InRegion);
+
// We want to instrument every address only once per basic block (unless there
// are calls between uses).
SmallPtrSet<Value *, 16> TempsToInstrument;
@@ -2922,12 +2936,24 @@ bool AddressSanitizer::instrumentFunction(Function &F,
// Skip instructions inserted by another instrumentation.
if (Inst.hasMetadata(LLVMContext::MD_nosanitize))
continue;
+ if (RegionOnly && !InRegion.contains(&Inst)) {
+ // A call between two regions can free memory like any other call.
+ if (isa<CallBase>(Inst))
+ TempsToInstrument.clear();
+ continue;
+ }
SmallVector<InterestingMemoryOperand, 1> InterestingOperands;
getInterestingMemoryOperands(&Inst, InterestingOperands, TTI);
AssumeInst *AI;
if (!InterestingOperands.empty()) {
for (auto &Operand : InterestingOperands) {
+ // The locals of a function that is only instrumented in regions
+ // have no redzones, so checking them can only trip over stale
+ // stack poisoning.
+ if (RegionOnly &&
+ isa<AllocaInst>(getUnderlyingObject(Operand.getPtr())))
+ continue;
if (Opts.asan_opt && Opts.asan_opt_same_temp) {
Value *Ptr = Operand.getPtr();
// If we have a mask, skip instrumentation if we've already
@@ -2996,8 +3022,13 @@ bool AddressSanitizer::instrumentFunction(Function &F,
FunctionModified = true;
}
- FunctionStackPoisoner FSP(F, *this, RTCI);
- bool ChangedStack = FSP.runOnFunction();
+ bool ChangedStack = false;
+ if (!RegionOnly) {
+ FunctionStackPoisoner FSP(F, *this, RTCI);
+ ChangedStack = FSP.runOnFunction();
+ } else {
+ NoReturnCalls.clear();
+ }
// We must unpoison the stack before NoReturn calls (throw, _exit, etc).
// See e.g. https://github.com/google/sanitizers/issues/37
diff --git a/llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll b/llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll
new file mode 100644
index 00000000000000..d82e185e962c6b
--- /dev/null
+++ b/llvm/test/Instrumentation/AddressSanitizer/sanitizer-instrumentation-regions.ll
@@ -0,0 +1,183 @@
+; Test that functions not instrumented by AddressSanitizer are still
+; instrumented inside llvm.sanitizer.instrumentation.begin()/end() regions, and
+; only there.
+;
+; RUN: opt < %s -passes=asan -asan-kernel=1 \
+; RUN: -asan-instrumentation-with-call-threshold=0 -S | FileCheck %s
+; RUN: opt < %s -passes=asan -asan-instrumentation-with-call-threshold=0 -S \
+; RUN: | FileCheck %s
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.memcpy.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @callee()
+
+; Only the access between begin() and end() is checked.
+define i32 @region(ptr %a, ptr %b, ptr %c) {
+entry:
+ %x = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ %y = load i32, ptr %b
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 %y, ptr %c
+ %s = add i32 %x, %y
+ ret i32 %s
+}
+
+; CHECK-LABEL: define i32 @region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %y = load i32, ptr %b
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 %y, ptr %c
+; CHECK-NOT: __asan_
+; CHECK: ret i32
+
+; The same for functions with disable_sanitizer_instrumentation.
+define i32 @region_disable(ptr %a, ptr %b) disable_sanitizer_instrumentation {
+entry:
+ %x = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 %x, ptr %b
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %x
+}
+
+; CHECK-LABEL: define i32 @region_disable(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 %x, ptr %b
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret i32 %x
+
+; A block reachable without passing begin() is not in a region.
+define i32 @diamond(i1 %cond, ptr %a) {
+entry:
+ br i1 %cond, label %then, label %join
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+join:
+ %x = load i32, ptr %a
+ ret i32 %x
+}
+
+; CHECK-LABEL: define i32 @diamond(
+; CHECK-NOT: __asan_
+; CHECK: ret i32
+
+; The inner end() of nested regions does not end the outer region.
+define void @nested(ptr %a) {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 0, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 1, ptr %a
+ ret void
+}
+
+; CHECK-LABEL: define void @nested(
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 0, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 1, ptr %a
+; CHECK-NEXT: ret void
+
+; The function's own locals have no redzones and are not checked; neither is
+; its stack frame laid out by ASan.
+define i32 @locals(ptr %p) {
+entry:
+ %local = alloca i32
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %local
+ %v = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %v
+}
+
+; CHECK-LABEL: define i32 @locals(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %local = alloca i32
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: store i32 1, ptr %local
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %v = load i32, ptr %p
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret i32 %v
+
+; A call between two regions may free memory, so the second access to the
+; same address is checked again.
+define void @call_between_regions(ptr %p) {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @callee()
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @call_between_regions(
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 0, ptr %p
+; CHECK: call void @callee()
+; CHECK: call void @__asan_store4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: store i32 1, ptr %p
+
+; Memory intrinsics are only replaced inside regions.
+define void @memcpy(ptr %d, ptr %s) {
+entry:
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %s, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %s, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @memcpy(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %s, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call ptr @{{(__asan_)?}}memcpy(ptr %d, ptr %s, i64 16)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+
+; Without regions, functions lacking sanitize_address are left alone.
+define i32 @no_region(ptr %a) {
+entry:
+ %x = load i32, ptr %a
+ ret i32 %x
+}
+
+; CHECK-LABEL: define i32 @no_region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK-NEXT: ret i32 %x
+
+; Fully instrumented functions are unaffected by regions.
+define i32 @sanitized(ptr %a, ptr %b) sanitize_address {
+entry:
+ %x = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ %y = load i32, ptr %b
+ call void @llvm.sanitizer.instrumentation.end()
+ %s = add i32 %x, %y
+ ret i32 %s
+}
+
+; CHECK-LABEL: define i32 @sanitized(
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %x = load i32, ptr %a
+; CHECK: call void @__asan_load4{{(_noabort)?}}(i64 %{{.*}})
+; CHECK-NEXT: %y = load i32, ptr %b
>From 9de82deb81a2b371ee516ad52b28f17c3b94bd4d Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Tue, 6 Oct 2026 10:23:12 +0000
Subject: [PATCH 4/6] [SanitizerCoverage] Support sanitizer instrumentation
regions
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
SanitizerCoverage. Functions with `nosanitize_coverage` or
`disable_sanitizer_instrumentation` are normally not instrumented at
all; if they contain regions, the code inside the regions is now
instrumented.
The Linux kernel uses this for `noinstr` functions, which may call
instrumented code between `instrumentation_begin()` and
`instrumentation_end()`. Today KCOV does not see that code, so the
fuzzer gets no coverage or comparison feedback for it, e.g. for the
syscall and exception entry paths.
Each block that enters a region gets a coverage point right after the
point where it does, and comparisons, switches, divisions, GEPs, loads,
stores and indirect calls inside regions are traced as requested. A
block whose coverage point is not at its start, or that is the entry
block, is split there, so that the callback runs inside the region and
the PC table can refer to the block by address. Pruning is disabled in
such functions, as it assumes that all blocks are instrumented, and
function-level instrumentation (entry/exit callbacks, stack depth
tracing, control flow tables) is not emitted, as it would run outside
the regions.
Clang now lowers the region builtins to the intrinsics whenever
`-fsanitize-coverage` instrumentation is enabled.
Tests: `llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/SanitizerCoverage.md | 25 +++
clang/lib/CodeGen/CGBuiltin.cpp | 3 +-
.../sanitizer-instrumentation-builtins.c | 17 +-
...nitizer-instrumentation-regions-pipeline.c | 11 +
llvm/docs/LangRef.md | 2 +
.../Instrumentation/SanitizerCoverage.cpp | 66 +++++-
.../sanitizer-instrumentation-regions.ll | 205 ++++++++++++++++++
7 files changed, 314 insertions(+), 15 deletions(-)
create mode 100644 llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll
diff --git a/clang/docs/SanitizerCoverage.md b/clang/docs/SanitizerCoverage.md
index ae074d13f5322a..4bd752e8122c08 100644
--- a/clang/docs/SanitizerCoverage.md
+++ b/clang/docs/SanitizerCoverage.md
@@ -446,6 +446,31 @@ function attribute `__attribute__((no_sanitize("coverage")))`. Because this
attribute may not be supported by other compilers, it is recommended to use it
together with `__has_feature(coverage_sanitizer)`.
+### Re-enabling instrumentation in regions
+
+Inside a function that has `__attribute__((no_sanitize("coverage")))` or
+`__attribute__((disable_sanitizer_instrumentation))`, a region delimited by
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` is instrumented again: every basic
+block that enters a region gets a coverage point right after it does, and
+comparisons, divisions, GEPs, loads, stores and indirect calls in regions are
+traced as requested. Coverage points are not pruned in such functions, and the
+function entry and exit callbacks, stack depth tracing and control flow tables
+are not emitted. Regions nest, and code reachable without passing through a
+`__builtin_sanitizer_instrumentation_begin()` is not considered to be in a
+region.
+
+```c
+__attribute__((no_sanitize("coverage")))
+void mostly_uninstrumented(int x) {
+ ... // not covered
+ __builtin_sanitizer_instrumentation_begin();
+ if (x) // covered, and traced with -fsanitize-coverage=trace-cmp
+ foo();
+ __builtin_sanitizer_instrumentation_end();
+}
+```
+
## Disabling instrumentation without source modification
It is sometimes useful to tell SanitizerCoverage to instrument only a subset of the
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 40f32c324ead5b..78e463a1869512 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3686,7 +3686,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
SanitizerMask Consumers = SanitizerKind::KernelMemory |
SanitizerKind::Address |
SanitizerKind::KernelAddress;
- if (getLangOpts().Sanitize.hasOneOf(Consumers))
+ if (getLangOpts().Sanitize.hasOneOf(Consumers) ||
+ CGM.getCodeGenOpts().hasSanitizeCoverage())
Builder.CreateCall(CGM.getIntrinsic(
BuiltinIDIfNoAsmLabel ==
Builtin::BI__builtin_sanitizer_instrumentation_begin
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index fdc1e3187bb069..36123ff4a4453f 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,15 +1,18 @@
// Check that the sanitizer region markers lower to their intrinsics under
-// -fsanitize=kernel-memory, kernel-address and address, and to nothing without
-// a sanitizer that consumes them.
+// -fsanitize=kernel-memory, kernel-address and address, and with
+// -fsanitize-coverage, and to nothing without a sanitizer that consumes them.
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s
// RUN: %clang_cc1 -triple x86_64-linux-gnu -emit-llvm -disable-llvm-passes \
// RUN: -o - %s | FileCheck %s --check-prefix=NONE
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=MARKERS
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=address \
-// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=ASAN
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=MARKERS
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize-coverage-type=3 \
+// RUN: -fsanitize-coverage-trace-pc -emit-llvm -disable-llvm-passes -o - %s \
+// RUN: | FileCheck %s --check-prefix=MARKERS
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
@@ -39,9 +42,9 @@ void noinstr_function(int x) {
// CHECK: declare void @llvm.sanitizer.instrumentation.begin() #[[ATTR:[0-9]+]]
// CHECK: attributes #[[ATTR]] = { nocallback nofree nomerge nosync nounwind willreturn }
-// ASAN-LABEL: define {{.*}}void @noinstr_function(
-// ASAN: call void @llvm.sanitizer.instrumentation.begin()
-// ASAN: call void @llvm.sanitizer.instrumentation.end()
+// MARKERS-LABEL: define {{.*}}void @noinstr_function(
+// MARKERS: call void @llvm.sanitizer.instrumentation.begin()
+// MARKERS: call void @llvm.sanitizer.instrumentation.end()
// NONE-LABEL: define {{.*}}void @noinstr_function(
// NONE-NOT: llvm.sanitizer.instrumentation
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
index 1a9229790441dd..d16868f2c8e161 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -10,6 +10,15 @@
// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
// RUN: -fsanitize=kernel-address -emit-llvm -o - %s \
// RUN: | FileCheck %s --check-prefixes=CHECK,KASAN
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize-coverage-type=3 -fsanitize-coverage-trace-pc \
+// RUN: -fsanitize-coverage-trace-cmp -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KCOV
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=kernel-memory -fsanitize-coverage-type=3 \
+// RUN: -fsanitize-coverage-trace-pc -fsanitize-coverage-trace-cmp \
+// RUN: -emit-llvm -o - %s \
+// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN,KCOV
static inline __attribute__((always_inline)) void instrumentation_begin(void) {
__builtin_sanitizer_instrumentation_begin();
@@ -35,6 +44,8 @@ void noinstr_function(int *p, int x) {
// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
// CHECK: call void @llvm.sanitizer.instrumentation.begin()
// KMSAN: call ptr @__msan_get_context_state()
+// KCOV: call void @__sanitizer_cov_trace_pc()
+// KCOV: call void @__sanitizer_cov_trace_const_cmp4(i32 7,
// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
// KASAN: call void @__asan_report_load4(
// CHECK: call void @instrumented_callee(
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index ac3f4747a52d04..b7440203f1e6e0 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28602,3 +28602,5 @@ An instruction is considered to be inside a region only if *every* path from the
MemorySanitizer supports regions in kernel mode (KMSAN), in functions marked with `disable_sanitizer_instrumentation`. The function-level ABI prologue is skipped, the MSan context state is fetched at `llvm.sanitizer.instrumentation.begin`, and shadow propagation and parameter shadow setup are only performed inside regions. Memory written outside a region keeps whatever shadow it had, and values computed outside a region are treated as fully initialized when they are used inside one.
AddressSanitizer supports regions in functions that lack the `sanitize_address` attribute or are marked with `disable_sanitizer_instrumentation`. Inside regions, memory accesses and memory intrinsics are instrumented. The stack frame of such a function is not instrumented, accesses to its own `alloca`s are not checked, and no `__asan_handle_no_return` calls are inserted.
+
+SanitizerCoverage supports regions in functions marked with `nosanitize_coverage` or `disable_sanitizer_instrumentation`. Each basic block that enters a region gets a coverage point right after the point where it does (splitting the block if necessary), without pruning, and the requested tracing of comparisons, divisions, GEPs, loads, stores and indirect calls is performed inside regions. Function entry/exit callbacks, stack depth tracing and control flow tables are not emitted for such functions.
diff --git a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
index 88f39f904d803f..c813bd81be07bd 100644
--- a/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
+++ b/llvm/lib/Transforms/Instrumentation/SanitizerCoverage.cpp
@@ -36,6 +36,7 @@
#include "llvm/TargetParser/Triple.h"
#include "llvm/Transforms/Utils/BasicBlockUtils.h"
#include "llvm/Transforms/Utils/EscapeEnumerator.h"
+#include "llvm/Transforms/Utils/Instrumentation.h"
#include "llvm/Transforms/Utils/ModuleUtils.h"
using namespace llvm;
@@ -552,6 +553,26 @@ static bool shouldInstrumentBlock(const Function &F, const BasicBlock *BB,
!(isFullPostDominator(BB, PDT) && !BB->getSinglePredecessor());
}
+// Returns the first instruction of \p BB inside a sanitizer instrumentation
+// region before which coverage can be inserted, or nullptr if there is none.
+// The region's begin() marker is skipped: it is the first instruction of the
+// region, but code inserted before it would run outside the region.
+static Instruction *
+getRegionCoveragePoint(BasicBlock &BB,
+ const SmallPtrSetImpl<const Instruction *> &InRegion) {
+ if (isa<UnreachableInst>(BB.getFirstNonPHIOrLifetime()))
+ return nullptr;
+ for (Instruction &I : make_range(BB.getFirstInsertionPt(), BB.end())) {
+ if (!InRegion.contains(&I))
+ continue;
+ if (auto *II = dyn_cast<IntrinsicInst>(&I))
+ if (II->getIntrinsicID() == Intrinsic::sanitizer_instrumentation_begin)
+ continue;
+ return &I;
+ }
+ return nullptr;
+}
+
// Returns true iff From->To is a backedge.
// A twist here is that we treat From->To as a backedge if
// * To dominates From or
@@ -612,14 +633,27 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
// Do not apply any instrumentation for naked functions.
if (F.hasFnAttribute(Attribute::Naked))
return;
- if (F.hasFnAttribute(Attribute::NoSanitizeCoverage))
- return;
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
- return;
+ // Functions that are not instrumented as a whole are still instrumented
+ // inside llvm.sanitizer.instrumentation.begin()/end() regions. There, every
+ // block that reaches a region gets a coverage point right after the point
+ // where it enters the region; function-level instrumentation (entry/exit
+ // callbacks, stack depth, control flow tables) is left out, as it would run
+ // outside of the regions.
+ bool RegionOnly = false;
+ if (F.hasFnAttribute(Attribute::NoSanitizeCoverage) ||
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation)) {
+ if (!hasSanitizerInstrumentationRegions(F))
+ return;
+ RegionOnly = true;
+ }
if (Options.CoverageType >= SanitizerCoverageOptions::SCK_Edge) {
SplitAllCriticalEdges(
F, CriticalEdgeSplittingOptions().setIgnoreUnreachableDests());
}
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ if (RegionOnly)
+ computeSanitizerInstrumentationRegions(F, InRegion);
+ SmallVector<Instruction *, 16> RegionCoveragePoints;
SmallVector<Instruction *, 8> IndirCalls;
SmallVector<BasicBlock *, 16> BlocksToInstrument;
SmallVector<Instruction *, 8> CmpTraceTargets;
@@ -634,9 +668,17 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
bool IsLeafFunc = true;
for (auto &BB : F) {
- if (shouldInstrumentBlock(F, &BB, DT, PDT, Options))
+ if (RegionOnly) {
+ // Block pruning relies on every block being instrumented, so it is not
+ // applied to region-only functions.
+ if (Instruction *IP = getRegionCoveragePoint(BB, InRegion))
+ RegionCoveragePoints.push_back(IP);
+ } else if (shouldInstrumentBlock(F, &BB, DT, PDT, Options)) {
BlocksToInstrument.push_back(&BB);
+ }
for (auto &Inst : BB) {
+ if (RegionOnly && !InRegion.contains(&Inst))
+ continue;
if (Options.IndirectCalls) {
CallBase *CB = dyn_cast<CallBase>(&Inst);
if (CB && CB->isIndirectCall())
@@ -670,7 +712,17 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
}
}
- if (Options.CollectControlFlow)
+ // Coverage is inserted at the start of a block, and the PC table refers to
+ // blocks by address. Start a new block at each region coverage point that is
+ // not already at the start of a block other than the entry block.
+ for (Instruction *IP : RegionCoveragePoints) {
+ BasicBlock *BB = IP->getParent();
+ if (BB->isEntryBlock() || IP != &*BB->getFirstInsertionPt())
+ BB = BB->splitBasicBlock(IP, "sancov.region");
+ BlocksToInstrument.push_back(BB);
+ }
+
+ if (Options.CollectControlFlow && !RegionOnly)
createFunctionControlFlow(F);
Value *FunctionGateCmp = nullptr;
@@ -682,7 +734,7 @@ void ModuleSanitizerCoverage::instrumentFunction(Function &F) {
InjectTraceForGep(F, GepTraceTargets);
InjectTraceForLoadsAndStores(F, Loads, Stores);
- if (Options.TracePCEntryExit)
+ if (Options.TracePCEntryExit && !RegionOnly)
InjectTraceForExits(F);
}
diff --git a/llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll b/llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll
new file mode 100644
index 00000000000000..d9fffa2c1f6d5a
--- /dev/null
+++ b/llvm/test/Instrumentation/SanitizerCoverage/sanitizer-instrumentation-regions.ll
@@ -0,0 +1,205 @@
+; Test that functions that are not instrumented as a whole are instrumented
+; inside llvm.sanitizer.instrumentation.begin()/end() regions.
+; RUN: opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=4 \
+; RUN: -sanitizer-coverage-trace-pc -sanitizer-coverage-trace-compares \
+; RUN: -sanitizer-coverage-trace-loads -sanitizer-coverage-trace-stores \
+; RUN: -sanitizer-coverage-trace-divs \
+; RUN: -sanitizer-coverage-trace-pc-entry-exit -sanitizer-coverage-stack-depth \
+; RUN: -sanitizer-coverage-control-flow -S | FileCheck %s
+; RUN: opt < %s -passes='module(sancov-module)' -sanitizer-coverage-level=3 \
+; RUN: -sanitizer-coverage-inline-8bit-counters -sanitizer-coverage-pc-table \
+; RUN: -S | FileCheck %s --check-prefix=PCTABLE
+
+target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @foo()
+
+; PCTABLE: @__sancov_gen_{{.*}} = private constant {{.*}} [ptr blockaddress(@region_disable, %sancov.region), ptr null], section "__sancov_pcs"
+
+; Only the code between begin() and end() is instrumented. The block that
+; contains begin() is split right after it, so that the coverage callback runs
+; inside the region, and the function gets no entry/exit callbacks, no stack
+; depth tracking and no control flow table.
+define i32 @region(ptr %p, i32 %x, ptr %fp) nosanitize_coverage {
+entry:
+ %out.cmp = icmp eq i32 %x, 42
+ %out.load = load i32, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in.cmp = icmp eq i32 %x, 7
+ %in.load = load i32, ptr %p
+ store i32 %x, ptr %p
+ %in.div = udiv i32 %in.load, %x
+ call void %fp()
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 %out.load, ptr %p
+ call void %fp()
+ %r = select i1 %out.cmp, i32 %in.div, i32 0
+ ret i32 %r
+}
+
+; CHECK-LABEL: define i32 @region(
+; CHECK-NEXT: entry:
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_const_cmp4(i32 7, i32 %x)
+; CHECK-NEXT: %in.cmp = icmp eq i32 %x, 7
+; CHECK-NEXT: call void @__sanitizer_cov_load4(ptr %p)
+; CHECK-NEXT: %in.load = load i32, ptr %p
+; CHECK-NEXT: call void @__sanitizer_cov_store4(ptr %p)
+; CHECK-NEXT: store i32 %x, ptr %p
+; CHECK-NEXT: call void @__sanitizer_cov_trace_div4(i32 %x)
+; CHECK-NEXT: %in.div = udiv i32 %in.load, %x
+; CHECK-NEXT: %0 = ptrtoint ptr %fp to i64
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc_indir(i64 %0)
+; CHECK-NEXT: call void %fp()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: ret i32 %r
+
+; Every block of a region is covered, including the blocks inserted to split
+; critical edges: pruning does not apply, as it assumes that all blocks are
+; instrumented. A block whose only in-region instruction is end() is not.
+define void @region_blocks(i1 %c) nosanitize_coverage {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %then, label %join
+
+then:
+ call void @foo()
+ br label %join
+
+join:
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+
+exit:
+ ret void
+}
+
+; CHECK-LABEL: define void @region_blocks(
+; CHECK: entry:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: br i1 %c, label %then, label %entry.join_crit_edge
+; CHECK: entry.join_crit_edge:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK: then:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @foo()
+; CHECK: join:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK: exit:
+; CHECK-NEXT: ret void
+
+; disable_sanitizer_instrumentation functions are handled the same way.
+define void @region_disable(i32 %x) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %c = icmp eq i32 %x, 7
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @region_disable(
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_const_cmp4(i32 7, i32 %x)
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+; A block reachable without passing through begin() is not in a region.
+define void @diamond(i1 %c, i32 %x) nosanitize_coverage {
+entry:
+ br i1 %c, label %enter, label %join
+
+enter:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+
+join:
+ %cmp = icmp eq i32 %x, 7
+ call void @foo()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @diamond(
+; CHECK: enter:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK: join:
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: ret void
+
+; Regions nest: the code after an inner end() is still covered.
+define void @nested(i32 %x) nosanitize_coverage {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ %cmp = icmp eq i32 %x, 7
+ call void @llvm.sanitizer.instrumentation.end()
+ %cmp2 = icmp eq i32 %x, 8
+ ret void
+}
+
+; CHECK-LABEL: define void @nested(
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: br label %sancov.region
+; CHECK: sancov.region:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_const_cmp4(i32 7, i32 %x)
+; CHECK-NEXT: %cmp = icmp eq i32 %x, 7
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: %cmp2 = icmp eq i32 %x, 8
+; CHECK-NEXT: ret void
+
+; Without regions, nosanitize_coverage functions are left alone.
+define void @no_region(i32 %x) nosanitize_coverage {
+entry:
+ %cmp = icmp eq i32 %x, 7
+ call void @foo()
+ ret void
+}
+
+; CHECK-LABEL: define void @no_region(
+; CHECK-NOT: call void @__sanitizer_cov
+; CHECK: ret void
+
+; Instrumented functions are instrumented as usual, regions or not.
+define void @instrumented(i32 %x) nounwind {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @foo()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @instrumented(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc_entry()
+; CHECK: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @foo()
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call void @__sanitizer_cov_trace_pc_exit()
+
+; The PC table refers to the split-off region block, never to the entry block of
+; a region-only function.
+; PCTABLE-LABEL: define void @region_disable(
+; PCTABLE: call void @llvm.sanitizer.instrumentation.begin()
+; PCTABLE: sancov.region:
+; PCTABLE-NEXT: load i8, ptr @__sancov_gen_
+; PCTABLE: call void @llvm.sanitizer.instrumentation.end()
>From 192e22265a2ec6251b55efb32611f04339380d40 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Tue, 6 Oct 2026 10:24:59 +0000
Subject: [PATCH 5/6] [TSan] Support sanitizer instrumentation regions
Implement `llvm.sanitizer.instrumentation.begin`/`end` regions in
ThreadSanitizer.
Functions with `disable_sanitizer_instrumentation` are normally not
instrumented at all. If they contain regions, plain and atomic memory
accesses and memory intrinsics inside the regions are now instrumented.
They still get no function entry/exit hooks or runtime ignores, as
those would run outside the regions. A call outside a region still
separates the accesses on either side of it when deciding which reads
can be folded into a following write.
Functions without `sanitize_thread` (and without
`disable_sanitizer_instrumentation`) already get their atomics and
entry/exit hooks instrumented; regions now also enable plain memory
access and memory intrinsic instrumentation in them.
The Linux kernel uses this for `noinstr` functions, which KCSAN marks
`disable_sanitizer_instrumentation` (via `__no_kcsan`), and which may
call instrumented code between `instrumentation_begin()` and
`instrumentation_end()`. Data races on accesses in those regions, e.g.
in the syscall and exception entry code, are currently invisible to
KCSAN.
Clang now lowers the region builtins to the intrinsics under
`-fsanitize=thread`.
Tests: `llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-builtins.c`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c`.
---
clang/docs/ThreadSanitizer.md | 23 +++
clang/lib/CodeGen/CGBuiltin.cpp | 6 +-
.../sanitizer-instrumentation-builtins.c | 4 +-
...nitizer-instrumentation-regions-pipeline.c | 4 +
llvm/docs/LangRef.md | 2 +
.../Instrumentation/ThreadSanitizer.cpp | 52 +++--
.../sanitizer-instrumentation-regions.ll | 178 ++++++++++++++++++
7 files changed, 254 insertions(+), 15 deletions(-)
create mode 100644 llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll
diff --git a/clang/docs/ThreadSanitizer.md b/clang/docs/ThreadSanitizer.md
index 7f48a7470e2540..4a5abf5fc172a8 100644
--- a/clang/docs/ThreadSanitizer.md
+++ b/clang/docs/ThreadSanitizer.md
@@ -102,6 +102,29 @@ and only if absolutely required; for example for certain code that cannot
tolerate any instrumentation and resulting side-effects. This attribute
overrides `no_sanitize("thread")`.
+## Re-enabling Instrumentation in Regions
+
+Inside a function with `__attribute__((disable_sanitizer_instrumentation))` or
+`__attribute__((no_sanitize("thread")))`, a region delimited by
+`__builtin_sanitizer_instrumentation_begin()` and
+`__builtin_sanitizer_instrumentation_end()` is instrumented again: plain
+memory accesses and memory intrinsics in it are instrumented. A
+`disable_sanitizer_instrumentation` function also gets its atomic accesses
+instrumented only inside regions, and no function entry/exit hooks. Regions
+nest, and code reachable without passing through a
+`__builtin_sanitizer_instrumentation_begin()` is not considered to be in a
+region.
+
+```c
+__attribute__((disable_sanitizer_instrumentation))
+void mostly_uninstrumented(int *p) {
+ ... // not instrumented
+ __builtin_sanitizer_instrumentation_begin();
+ *p = 0; // instrumented
+ __builtin_sanitizer_instrumentation_end();
+}
+```
+
## Interaction of Inlining with Disabling Sanitizer Instrumentation
- A `no_sanitize` function will not be inlined heuristically by the compiler into a sanitized function.
diff --git a/clang/lib/CodeGen/CGBuiltin.cpp b/clang/lib/CodeGen/CGBuiltin.cpp
index 78e463a1869512..cec0e81d3eed01 100644
--- a/clang/lib/CodeGen/CGBuiltin.cpp
+++ b/clang/lib/CodeGen/CGBuiltin.cpp
@@ -3683,9 +3683,9 @@ RValue CodeGenFunction::EmitBuiltinExpr(const GlobalDecl GD, unsigned BuiltinID,
// Only emit the region markers for sanitizers that honour them. Elsewhere
// they would just be optimization barriers that no pass ever removes. This
// lets code guard its uses on __has_builtin alone.
- SanitizerMask Consumers = SanitizerKind::KernelMemory |
- SanitizerKind::Address |
- SanitizerKind::KernelAddress;
+ SanitizerMask Consumers =
+ SanitizerKind::KernelMemory | SanitizerKind::Address |
+ SanitizerKind::KernelAddress | SanitizerKind::Thread;
if (getLangOpts().Sanitize.hasOneOf(Consumers) ||
CGM.getCodeGenOpts().hasSanitizeCoverage())
Builder.CreateCall(CGM.getIntrinsic(
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
index 36123ff4a4453f..5fe7083ac668f8 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-builtins.c
@@ -1,5 +1,5 @@
// Check that the sanitizer region markers lower to their intrinsics under
-// -fsanitize=kernel-memory, kernel-address and address, and with
+// -fsanitize=kernel-memory, kernel-address, address and thread, and with
// -fsanitize-coverage, and to nothing without a sanitizer that consumes them.
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
@@ -14,6 +14,8 @@
// RUN: -fsanitize-coverage-trace-pc -emit-llvm -disable-llvm-passes -o - %s \
// RUN: | FileCheck %s --check-prefix=MARKERS
// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=thread \
+// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=MARKERS
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=null \
// RUN: -emit-llvm -disable-llvm-passes -o - %s | FileCheck %s --check-prefix=NONE
#if !__has_builtin(__builtin_sanitizer_instrumentation_begin)
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
index d16868f2c8e161..69ecd6166d6e01 100644
--- a/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-pipeline.c
@@ -19,6 +19,9 @@
// RUN: -fsanitize-coverage-trace-pc -fsanitize-coverage-trace-cmp \
// RUN: -emit-llvm -o - %s \
// RUN: | FileCheck %s --check-prefixes=CHECK,KMSAN,KCOV
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -O2 -debug-info-kind=line-tables-only \
+// RUN: -fsanitize=thread -mllvm -tsan-instrument-func-entry-exit=0 \
+// RUN: -emit-llvm -o - %s | FileCheck %s --check-prefixes=CHECK,KCSAN
static inline __attribute__((always_inline)) void instrumentation_begin(void) {
__builtin_sanitizer_instrumentation_begin();
@@ -48,6 +51,7 @@ void noinstr_function(int *p, int x) {
// KCOV: call void @__sanitizer_cov_trace_const_cmp4(i32 7,
// KMSAN: call { ptr, ptr } @__msan_metadata_ptr_for_load_4(
// KASAN: call void @__asan_report_load4(
+// KCSAN: call void @__tsan_read4(
// CHECK: call void @instrumented_callee(
// CHECK: call void @llvm.sanitizer.instrumentation.end()
// CHECK-NOT: call {{.*}}@__{{msan|asan|tsan|sanitizer_cov}}
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index b7440203f1e6e0..06f182e20cd5fd 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28604,3 +28604,5 @@ MemorySanitizer supports regions in kernel mode (KMSAN), in functions marked wit
AddressSanitizer supports regions in functions that lack the `sanitize_address` attribute or are marked with `disable_sanitizer_instrumentation`. Inside regions, memory accesses and memory intrinsics are instrumented. The stack frame of such a function is not instrumented, accesses to its own `alloca`s are not checked, and no `__asan_handle_no_return` calls are inserted.
SanitizerCoverage supports regions in functions marked with `nosanitize_coverage` or `disable_sanitizer_instrumentation`. Each basic block that enters a region gets a coverage point right after the point where it does (splitting the block if necessary), without pruning, and the requested tracing of comparisons, divisions, GEPs, loads, stores and indirect calls is performed inside regions. Function entry/exit callbacks, stack depth tracing and control flow tables are not emitted for such functions.
+
+ThreadSanitizer supports regions in functions marked with `disable_sanitizer_instrumentation`, and in functions that lack the `sanitize_thread` attribute. In the former, plain and atomic memory accesses and memory intrinsics are instrumented only inside regions, and no function entry/exit hooks are inserted. In the latter, which already get their atomic accesses and function entry/exit instrumented, plain memory accesses and memory intrinsics inside regions are instrumented as well.
diff --git a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
index 59e03878428e3d..c9d11c86c3ecce 100644
--- a/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/ThreadSanitizer.cpp
@@ -476,10 +476,23 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
if (F.hasFnAttribute(Attribute::Naked))
return false;
+ bool SanitizeFunction = F.hasFnAttribute(Attribute::SanitizeThread);
// __attribute__(disable_sanitizer_instrumentation) prevents all kinds of
- // instrumentation.
- if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
+ // instrumentation, except inside llvm.sanitizer.instrumentation.begin()/end()
+ // regions. Such a function is instrumented only there, and gets no
+ // function-level instrumentation (entry/exit hooks, runtime ignores), as that
+ // would run outside of the regions.
+ bool RegionOnly =
+ F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation);
+ if (RegionOnly && !hasSanitizerInstrumentationRegions(F))
return false;
+ // Regions also enable memory access instrumentation in functions that lack
+ // the sanitize_thread attribute, which otherwise only get their atomics and
+ // entry/exit instrumented.
+ SmallPtrSet<const Instruction *, 16> InRegion;
+ if (RegionOnly ||
+ (!SanitizeFunction && hasSanitizerInstrumentationRegions(F)))
+ computeSanitizerInstrumentationRegions(F, InRegion);
initialize(*F.getParent(), TLI);
SmallVector<InstructionInfo, 8> AllLoadsAndStores;
@@ -488,7 +501,6 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
SmallVector<Instruction*, 8> MemIntrinCalls;
bool Res = false;
bool HasCalls = false;
- bool SanitizeFunction = F.hasFnAttribute(Attribute::SanitizeThread);
const DataLayout &DL = F.getDataLayout();
// Traverse all instructions, collect loads/stores/returns, check for calls.
@@ -497,14 +509,27 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
// Skip instructions inserted by another instrumentation.
if (Inst.hasMetadata(LLVMContext::MD_nosanitize))
continue;
+ // In a region-only function nothing outside the regions is instrumented,
+ // not even atomics.
+ if (RegionOnly && !InRegion.contains(&Inst)) {
+ // A call outside a region still separates the accesses around it.
+ if (isa<CallBase>(Inst))
+ chooseInstructionsToInstrument(LocalLoadsAndStores, AllLoadsAndStores,
+ DL);
+ continue;
+ }
+ // Plain accesses and memory intrinsics are instrumented in
+ // sanitize_thread functions and inside regions.
+ bool InstrumentAccesses = SanitizeFunction || InRegion.contains(&Inst);
if (isTsanAtomic(&Inst))
AtomicAccesses.push_back(&Inst);
- else if (isa<LoadInst>(Inst) || isa<StoreInst>(Inst))
- LocalLoadsAndStores.push_back(&Inst);
- else if (isa<CallInst>(Inst) || isa<InvokeInst>(Inst)) {
+ else if (isa<LoadInst>(Inst) || isa<StoreInst>(Inst)) {
+ if (InstrumentAccesses)
+ LocalLoadsAndStores.push_back(&Inst);
+ } else if (isa<CallInst>(Inst) || isa<InvokeInst>(Inst)) {
if (CallInst *CI = dyn_cast<CallInst>(&Inst))
maybeMarkSanitizerLibraryCallNoBuiltin(CI, &TLI);
- if (isa<MemIntrinsic>(Inst))
+ if (isa<MemIntrinsic>(Inst) && InstrumentAccesses)
MemIntrinCalls.push_back(&Inst);
HasCalls = true;
chooseInstructionsToInstrument(LocalLoadsAndStores, AllLoadsAndStores,
@@ -518,24 +543,29 @@ bool ThreadSanitizer::sanitizeFunction(Function &F,
// FIXME: many of these accesses do not need to be checked for races
// (e.g. variables that do not escape, etc).
- // Instrument memory accesses only if we want to report bugs in the function.
- if (Opts.tsan_instrument_memory_accesses && SanitizeFunction)
+ // Instrument memory accesses only if we want to report bugs in the function
+ // (or in the region they belong to).
+ if (Opts.tsan_instrument_memory_accesses)
for (const auto &II : AllLoadsAndStores) {
Res |= instrumentLoadOrStore(II, DL);
}
// Instrument atomic memory accesses in any case (they can be used to
- // implement synchronization).
+ // implement synchronization), except outside the regions of a region-only
+ // function.
if (Opts.tsan_instrument_atomics)
for (auto *Inst : AtomicAccesses) {
Res |= instrumentAtomic(Inst, DL);
}
- if (Opts.tsan_instrument_memintrinsics && SanitizeFunction)
+ if (Opts.tsan_instrument_memintrinsics)
for (auto *Inst : MemIntrinCalls) {
Res |= instrumentMemIntrinsic(Inst);
}
+ if (RegionOnly)
+ return Res;
+
if (F.hasFnAttribute("sanitize_thread_no_checking_at_run_time")) {
assert(!F.hasFnAttribute(Attribute::SanitizeThread));
if (HasCalls)
diff --git a/llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll b/llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll
new file mode 100644
index 00000000000000..5606e19768cc36
--- /dev/null
+++ b/llvm/test/Instrumentation/ThreadSanitizer/sanitizer-instrumentation-regions.ll
@@ -0,0 +1,178 @@
+; Test that functions that are not instrumented as a whole are instrumented
+; inside llvm.sanitizer.instrumentation.begin()/end() regions.
+; RUN: opt < %s -passes='function(tsan),module(tsan-module)' -S | FileCheck %s
+; KCSAN disables the entry/exit hooks.
+; RUN: opt < %s -passes='function(tsan),module(tsan-module)' \
+; RUN: -tsan-instrument-func-entry-exit=0 -tsan-distinguish-volatile=1 -S \
+; RUN: | FileCheck %s --check-prefix=KCSAN
+
+target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+declare void @llvm.memcpy.p0.p0.i64(ptr, ptr, i64, i1)
+declare void @foo()
+
+; In a disable_sanitizer_instrumentation function, plain and atomic accesses
+; and memory intrinsics are instrumented only inside regions, and there are no
+; entry/exit hooks.
+define i32 @region(ptr %a, ptr %b, ptr %d) disable_sanitizer_instrumentation {
+entry:
+ %out.load = load i32, ptr %a
+ %out.atomic = load atomic i32, ptr %b seq_cst, align 4
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %a, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in.load = load i32, ptr %a
+ store i32 %in.load, ptr %b
+ %in.atomic = load atomic i32, ptr %b seq_cst, align 4
+ store volatile i32 %in.atomic, ptr %d
+ call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %a, i64 16, i1 false)
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 %out.load, ptr %b
+ store atomic i32 %out.atomic, ptr %b seq_cst, align 4
+ ret i32 %in.load
+}
+
+; CHECK-LABEL: define i32 @region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %out.load = load i32, ptr %a
+; CHECK-NEXT: %out.atomic = load atomic i32, ptr %b seq_cst
+; CHECK-NEXT: call void @llvm.memcpy.p0.p0.i64(ptr %d, ptr %a, i64 16, i1 false)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @__tsan_read4(ptr %a)
+; CHECK-NEXT: %in.load = load i32, ptr %a
+; CHECK-NEXT: call void @__tsan_write4(ptr %b)
+; CHECK-NEXT: store i32 %in.load, ptr %b
+; CHECK-NEXT: %0 = call i32 @__tsan_atomic32_load(ptr %b, i32 5)
+; CHECK: call void @__tsan_write4(ptr %d)
+; CHECK-NEXT: store volatile i32
+; CHECK-NEXT: call ptr @__tsan_memcpy(ptr %d, ptr %a, i64 16)
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 %out.load, ptr %b
+; CHECK-NEXT: store atomic i32 %out.atomic, ptr %b seq_cst
+; CHECK-NEXT: ret i32 %in.load
+
+; KCSAN-LABEL: define i32 @region(
+; KCSAN: call void @llvm.sanitizer.instrumentation.begin()
+; KCSAN-NEXT: call void @__tsan_read4(ptr %a)
+; KCSAN: call void @__tsan_volatile_write4(ptr %d)
+; KCSAN: call void @llvm.sanitizer.instrumentation.end()
+; KCSAN-NOT: call {{.*}}@__tsan_
+; KCSAN: ret i32 %in.load
+
+; A block reachable without passing through begin() is not in a region.
+define void @diamond(i1 %c, ptr %a) disable_sanitizer_instrumentation {
+entry:
+ br i1 %c, label %enter, label %join
+
+enter:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %a
+ br label %join
+
+join:
+ store i32 2, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @diamond(
+; CHECK: enter:
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @__tsan_write4(ptr %a)
+; CHECK-NEXT: store i32 1, ptr %a
+; CHECK: join:
+; CHECK-NEXT: store i32 2, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: ret void
+
+; Regions nest.
+define void @nested(ptr %a) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 1, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ store i32 2, ptr %a
+ ret void
+}
+
+; CHECK-LABEL: define void @nested(
+; CHECK: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call void @__tsan_write4(ptr %a)
+; CHECK-NEXT: store i32 1, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: store i32 2, ptr %a
+; CHECK-NEXT: ret void
+
+; A call outside the regions separates the accesses on either side of it, so
+; the read before the write is not folded into the write.
+define void @call_between_regions(ptr %a) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ %v = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @foo()
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 %v, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; CHECK-LABEL: define void @call_between_regions(
+; CHECK: call void @__tsan_read4(ptr %a)
+; CHECK: call void @foo()
+; CHECK: call void @__tsan_write4(ptr %a)
+
+; Functions without sanitize_thread still get their atomics and entry/exit
+; hooks instrumented everywhere; regions add plain accesses.
+define i32 @no_sanitize_thread(ptr %a) {
+entry:
+ %out.load = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ %in.load = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.end()
+ %atomic = load atomic i32, ptr %a seq_cst, align 4
+ %r = add i32 %out.load, %in.load
+ ret i32 %r
+}
+
+; CHECK-LABEL: define i32 @no_sanitize_thread(
+; CHECK: call void @__tsan_func_entry(
+; CHECK-NEXT: %out.load = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.begin()
+; CHECK-NEXT: call void @__tsan_read4(ptr %a)
+; CHECK-NEXT: %in.load = load i32, ptr %a
+; CHECK-NEXT: call void @llvm.sanitizer.instrumentation.end()
+; CHECK-NEXT: call i32 @__tsan_atomic32_load(ptr %a, i32 5)
+; CHECK: call void @__tsan_func_exit()
+
+; Without regions, disable_sanitizer_instrumentation functions are left alone.
+define i32 @no_region(ptr %a) disable_sanitizer_instrumentation {
+entry:
+ %v = load i32, ptr %a
+ %atomic = load atomic i32, ptr %a seq_cst, align 4
+ ret i32 %v
+}
+
+; CHECK-LABEL: define i32 @no_region(
+; CHECK-NEXT: entry:
+; CHECK-NEXT: %v = load i32, ptr %a
+; CHECK-NEXT: %atomic = load atomic i32, ptr %a seq_cst
+; CHECK-NEXT: ret i32 %v
+
+; Instrumented functions are instrumented as usual, regions or not.
+define i32 @sanitized(ptr %a) sanitize_thread {
+entry:
+ %out.load = load i32, ptr %a
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.end()
+ ret i32 %out.load
+}
+
+; CHECK-LABEL: define i32 @sanitized(
+; CHECK: call void @__tsan_func_entry(
+; CHECK-NEXT: call void @__tsan_read4(ptr %a)
+; CHECK-NEXT: %out.load = load i32, ptr %a
>From b11cdb9f1a6aaaa3529f0693399ba86366357663 Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Tue, 13 Oct 2026 07:07:35 +0000
Subject: [PATCH 6/6] [Sanitizers] Warn about unbalanced sanitizer
instrumentation regions
`computeSanitizerInstrumentationRegions()` silently copes with malformed
region markers: an `end` without a matching `begin` is ignored, a region
left open extends to the end of the function, and code reached both from
inside and from outside a region is treated as outside. The result is
that instrumentation is quietly missing where the user expected it.
Report these cases as a warning from the region analysis, i.e. from every
sanitizer pass that implements regions:
- an `llvm.sanitizer.instrumentation.end` without a matching `begin`;
- a region that is still open at a `ret`;
- a block where paths with different region nesting depths merge.
Leaving a function without returning while a region is open is not
diagnosed: not closing a region before a `noreturn` call (e.g. `BUG()`
or `panic()` in the kernel) is common and harmless, and unwinding cannot
be checked consistently, because a call without a landing pad leaves no
terminator to look at.
With the "must" join, the analysis is exact as long as all paths into a
block agree on the depth, so the first disagreement shows up as a block
whose reached predecessors leave with different depths. Joins are
checked first, so that an `end` or a return that only looks unbalanced
because of an earlier bad join is not blamed instead. At most one warning
is emitted per function; a function-level metadata node records that the
function has been diagnosed, so that combining several sanitizers (e.g.
KMSAN and KCOV) does not repeat the warning.
The warning is a new `DiagnosticInfoSanitizerInstrumentationRegion`,
which Clang maps to `-Wsanitizer-instrumentation-region` and reports at
the source location of the offending marker, return or merge point.
In the Linux kernel, objtool already rejects unbalanced
`instrumentation_begin()`/`instrumentation_end()` when
`CONFIG_NOINSTR_VALIDATION` is enabled, but other users of the builtins
have no such check, and the kernel only gets it late, on the object file.
Tests: `llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll`,
`clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c`.
---
clang/docs/LanguageExtensions.md | 3 +-
clang/docs/ReleaseNotes.md | 4 +-
.../clang/Basic/DiagnosticFrontendKinds.td | 2 +
clang/include/clang/Basic/DiagnosticGroups.td | 2 +
.../CodeGenUtils/BackendDiagnosticHandler.h | 4 +
.../CodeGenUtils/BackendDiagnosticHandler.cpp | 20 +++
...izer-instrumentation-regions-diagnostics.c | 61 +++++++
llvm/docs/LangRef.md | 2 +
llvm/include/llvm/IR/DiagnosticInfo.h | 24 +++
.../llvm/Transforms/Utils/Instrumentation.h | 10 +-
llvm/lib/IR/DiagnosticInfo.cpp | 15 ++
llvm/lib/Transforms/Utils/Instrumentation.cpp | 87 +++++++++-
...zer-instrumentation-regions-diagnostics.ll | 161 ++++++++++++++++++
13 files changed, 391 insertions(+), 4 deletions(-)
create mode 100644 clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c
create mode 100644 llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll
diff --git a/clang/docs/LanguageExtensions.md b/clang/docs/LanguageExtensions.md
index 7ea54b04ada2d7..a5a0175044b57b 100644
--- a/clang/docs/LanguageExtensions.md
+++ b/clang/docs/LanguageExtensions.md
@@ -4349,7 +4349,8 @@ void handle_irq(struct pt_regs *regs) {
The builtins are meant for code like the Linux kernel's `noinstr` functions,
which must not be instrumented as a whole but call instrumented code between
`instrumentation_begin()` and `instrumentation_end()`. Regions nest, and should
-be balanced on every path through the function. Which functions count as
+be balanced on every path through the function; Clang warns about unbalanced
+regions with `-Wsanitizer-instrumentation-region`. Which functions count as
uninstrumented, and what gets instrumented inside regions, is up to each
sanitizer; sanitizers that support regions describe this in their own
documentation.
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index b27b3d2955bf4d..4ce0a911435b49 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -1322,7 +1322,9 @@ The `alpha.cplusplus.UseAfterLifetimeEnd` checker was renamed to `alpha.core.Use
sanitizer instrumentation is enabled again. AddressSanitizer,
KernelMemorySanitizer, ThreadSanitizer and SanitizerCoverage support them;
see [LanguageExtensions](LanguageExtensions.md) and the documentation of each
- sanitizer.
+ sanitizer. The new `-Wsanitizer-instrumentation-region` warning reports
+ unbalanced regions: an end without a matching begin, a region still open at
+ a return, and control flow merging from paths with different region nesting.
### Python Binding Changes
diff --git a/clang/include/clang/Basic/DiagnosticFrontendKinds.td b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
index 3df87215a0476d..5f08b83a76eda2 100644
--- a/clang/include/clang/Basic/DiagnosticFrontendKinds.td
+++ b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
@@ -410,6 +410,8 @@ def warn_profile_data_misexpect : Warning<
"potential performance regression from use of __builtin_expect(): "
"annotation was correct on %0 of profiled executions">,
BackendInfo, InGroup<MisExpect>;
+def warn_fe_backend_sanitizer_instrumentation_region : Warning<"%0">,
+ BackendInfo, InGroup<SanitizerInstrumentationRegion>;
} // end of instrumentation issue category
def warn_cgs_no_proto : Warning<
diff --git a/clang/include/clang/Basic/DiagnosticGroups.td b/clang/include/clang/Basic/DiagnosticGroups.td
index b6c7ff7276260f..95d4380c1324e4 100644
--- a/clang/include/clang/Basic/DiagnosticGroups.td
+++ b/clang/include/clang/Basic/DiagnosticGroups.td
@@ -1836,6 +1836,8 @@ def ProfileInstrMissing : DiagGroup<"profile-instr-missing">;
def ProfileInstrOutOfDate : DiagGroup<"profile-instr-out-of-date">;
def ProfileInstrUnprofiled : DiagGroup<"profile-instr-unprofiled">;
def MisExpect : DiagGroup<"misexpect">;
+def SanitizerInstrumentationRegion
+ : DiagGroup<"sanitizer-instrumentation-region">;
// AddressSanitizer frontend instrumentation remarks.
def SanitizeAddressRemarks : DiagGroup<"sanitize-address">;
diff --git a/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h b/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h
index 484b5c68594951..6b2b8000a68093 100644
--- a/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h
+++ b/clang/include/clang/CodeGenUtils/BackendDiagnosticHandler.h
@@ -35,6 +35,7 @@ class DiagnosticInfoMisExpect;
class DiagnosticInfoOptimizationBase;
class DiagnosticInfoOptimizationFailure;
class DiagnosticInfoResourceLimit;
+class DiagnosticInfoSanitizerInstrumentationRegion;
class DiagnosticInfoSrcMgr;
class DiagnosticInfoStackSize;
class DiagnosticInfoUnsupported;
@@ -134,6 +135,9 @@ class BackendDiagnosticConsumer {
/// Specialized handler for misexpect warnings.
/// Note that misexpect remarks are emitted through ORE
void MisExpectDiagHandler(const llvm::DiagnosticInfoMisExpect &D);
+ /// Specialized handler for unbalanced sanitizer instrumentation regions.
+ void SanitizerInstrumentationRegionDiagHandler(
+ const llvm::DiagnosticInfoSanitizerInstrumentationRegion &D);
DiagnosticsEngine &Diags;
const CodeGenOptions &CodeGenOpts;
diff --git a/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp b/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp
index 667a7948fa18f8..6e6bbb89509303 100644
--- a/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp
+++ b/clang/lib/CodeGenUtils/BackendDiagnosticHandler.cpp
@@ -549,6 +549,22 @@ void BackendDiagnosticConsumer::MisExpectDiagHandler(
<< Filename << Line << Column;
}
+void BackendDiagnosticConsumer::SanitizerInstrumentationRegionDiagHandler(
+ const llvm::DiagnosticInfoSanitizerInstrumentationRegion &D) {
+ StringRef Filename;
+ unsigned Line, Column;
+ bool BadDebugInfo = false;
+ FullSourceLoc Loc =
+ getBestLocationFromDebugLoc(D, BadDebugInfo, Filename, Line, Column);
+
+ Diags.Report(Loc, diag::warn_fe_backend_sanitizer_instrumentation_region)
+ << D.getMsg().str();
+
+ if (BadDebugInfo)
+ Diags.Report(Loc, diag::note_fe_backend_invalid_loc)
+ << Filename << Line << Column;
+}
+
void BackendDiagnosticConsumer::handleDiagnostics(const DiagnosticInfo &DI) {
unsigned DiagID = diag::err_fe_inline_asm;
llvm::DiagnosticSeverity Severity = DI.getSeverity();
@@ -633,6 +649,10 @@ void BackendDiagnosticConsumer::handleDiagnostics(const DiagnosticInfo &DI) {
case llvm::DK_MisExpect:
MisExpectDiagHandler(cast<DiagnosticInfoMisExpect>(DI));
return;
+ case llvm::DK_SanitizerInstrumentationRegion:
+ SanitizerInstrumentationRegionDiagHandler(
+ cast<DiagnosticInfoSanitizerInstrumentationRegion>(DI));
+ return;
default:
// Plugin IDs are not bound to any value as they are set dynamically.
ComputeDiagRemarkID(Severity, backend_plugin, DiagID);
diff --git a/clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c b/clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c
new file mode 100644
index 00000000000000..597fa5a657ac86
--- /dev/null
+++ b/clang/test/CodeGen/sanitizer-instrumentation-regions-diagnostics.c
@@ -0,0 +1,61 @@
+// Check that unbalanced sanitizer instrumentation regions are diagnosed with a
+// source location, and that the warning can be disabled.
+
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
+// RUN: -debug-info-kind=line-tables-only -emit-llvm -o /dev/null %s -verify
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-memory \
+// RUN: -fsanitize-coverage-type=3 -fsanitize-coverage-trace-pc \
+// RUN: -debug-info-kind=line-tables-only -emit-llvm -o /dev/null %s -verify
+// RUN: %clang_cc1 -triple x86_64-linux-gnu -fsanitize=kernel-address \
+// RUN: -Wno-sanitizer-instrumentation-region -emit-llvm -o /dev/null %s \
+// RUN: -verify=none
+
+// none-no-diagnostics
+
+extern void callee(void);
+
+__attribute__((disable_sanitizer_instrumentation))
+void balanced(int c) {
+ __builtin_sanitizer_instrumentation_begin();
+ callee();
+ __builtin_sanitizer_instrumentation_end();
+ if (c) {
+ __builtin_sanitizer_instrumentation_begin();
+ callee();
+ __builtin_sanitizer_instrumentation_end();
+ }
+}
+
+// Not closing a region before a call that does not return is fine.
+__attribute__((noreturn)) extern void panic(void);
+
+__attribute__((disable_sanitizer_instrumentation))
+void open_at_noreturn(int c) {
+ __builtin_sanitizer_instrumentation_begin();
+ if (c)
+ panic();
+ if (c > 1)
+ __builtin_trap();
+ callee();
+ __builtin_sanitizer_instrumentation_end();
+}
+
+__attribute__((disable_sanitizer_instrumentation))
+void unmatched_end(void) {
+ callee();
+ __builtin_sanitizer_instrumentation_end(); // expected-warning {{in function 'unmatched_end': sanitizer instrumentation region end without a matching begin}}
+}
+
+__attribute__((disable_sanitizer_instrumentation))
+void not_closed(void) {
+ __builtin_sanitizer_instrumentation_begin();
+ callee();
+} // expected-warning {{in function 'not_closed': sanitizer instrumentation region is not closed before return}}
+
+__attribute__((disable_sanitizer_instrumentation))
+void conditional_begin(int c) {
+ if (c)
+ __builtin_sanitizer_instrumentation_begin();
+ callee(); // expected-warning {{in function 'conditional_begin': control flow merges from paths with different sanitizer instrumentation region nesting}}
+ __builtin_sanitizer_instrumentation_end();
+}
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index 06f182e20cd5fd..1643a6006b4cc9 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -28599,6 +28599,8 @@ Regions nest: `llvm.sanitizer.instrumentation.begin` increments a region nesting
An instruction is considered to be inside a region only if *every* path from the function entry to it is inside a region, i.e. region membership is an intersection over predecessors. This makes an in-region instruction always dominated by a `llvm.sanitizer.instrumentation.begin`, so any per-function sanitizer state set up there is available. A region that is entered on some paths but not others is therefore instrumented only where it is unambiguous.
+Sanitizer passes that implement regions warn about unbalanced markers: a `llvm.sanitizer.instrumentation.end` without a matching `llvm.sanitizer.instrumentation.begin`, a region that is still open at a `ret`, and a point where paths with different region nesting depths merge. Regions that are still open where the function is left without returning, i.e. at an `unreachable` (e.g. after a call to a `noreturn` function) or when an exception unwinds out of it, are not diagnosed. At most one warning is emitted per function. In Clang, the warning is controlled by `-Wsanitizer-instrumentation-region`.
+
MemorySanitizer supports regions in kernel mode (KMSAN), in functions marked with `disable_sanitizer_instrumentation`. The function-level ABI prologue is skipped, the MSan context state is fetched at `llvm.sanitizer.instrumentation.begin`, and shadow propagation and parameter shadow setup are only performed inside regions. Memory written outside a region keeps whatever shadow it had, and values computed outside a region are treated as fully initialized when they are used inside one.
AddressSanitizer supports regions in functions that lack the `sanitize_address` attribute or are marked with `disable_sanitizer_instrumentation`. Inside regions, memory accesses and memory intrinsics are instrumented. The stack frame of such a function is not instrumented, accesses to its own `alloca`s are not checked, and no `__asan_handle_no_return` calls are inserted.
diff --git a/llvm/include/llvm/IR/DiagnosticInfo.h b/llvm/include/llvm/IR/DiagnosticInfo.h
index da62b62bd8c74a..4451c7a5052c5a 100644
--- a/llvm/include/llvm/IR/DiagnosticInfo.h
+++ b/llvm/include/llvm/IR/DiagnosticInfo.h
@@ -93,6 +93,7 @@ enum DiagnosticKind {
DK_SrcMgr,
DK_DontCall,
DK_MisExpect,
+ DK_SanitizerInstrumentationRegion,
DK_FirstPluginKind // Must be last value to work with
// getNextAvailablePluginDiagnosticKind
};
@@ -1151,6 +1152,29 @@ class LLVM_ABI DiagnosticInfoUnsupportedTargetIntrinsic
void print(DiagnosticPrinter &DP) const override;
};
+/// Diagnostic information for unbalanced sanitizer instrumentation regions
+/// (llvm.sanitizer.instrumentation.begin/end).
+class LLVM_ABI DiagnosticInfoSanitizerInstrumentationRegion
+ : public DiagnosticInfoWithLocationBase {
+public:
+ DiagnosticInfoSanitizerInstrumentationRegion(
+ const Function &Fn, const DebugLoc &Loc,
+ const Twine &Msg LLVM_LIFETIME_BOUND);
+
+ /// \see DiagnosticInfo::print.
+ void print(DiagnosticPrinter &DP) const override;
+
+ static bool classof(const DiagnosticInfo *DI) {
+ return DI->getKind() == DK_SanitizerInstrumentationRegion;
+ }
+
+ const Twine &getMsg() const { return Msg; }
+
+private:
+ /// Message to report.
+ const Twine &Msg;
+};
+
/// Diagnostic information for MisExpect analysis.
class LLVM_ABI DiagnosticInfoMisExpect : public DiagnosticInfoWithLocationBase {
public:
diff --git a/llvm/include/llvm/Transforms/Utils/Instrumentation.h b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
index b8ec84ae8d7dce..2744464a9ffbc2 100644
--- a/llvm/include/llvm/Transforms/Utils/Instrumentation.h
+++ b/llvm/include/llvm/Transforms/Utils/Instrumentation.h
@@ -76,8 +76,16 @@ LLVM_ABI bool hasSanitizerInstrumentationRegions(const Function &F);
/// objtool requiring them to be balanced on all paths. The lattice value is
/// the nesting depth rather than a flag, so that in begin; begin; end; X; end
/// the instruction X is still in a region.
+///
+/// Unbalanced regions are also reported as a warning through the
+/// LLVMContext diagnostic handler: an end() without a matching begin(), a
+/// region still open at a return, and control flow merging from paths with
+/// different nesting depths. Leaving the function without returning (through
+/// an unreachable or by unwinding) with a region open is not diagnosed. At
+/// most one warning is emitted per function,
+/// even if several sanitizer passes analyze it.
LLVM_ABI void computeSanitizerInstrumentationRegions(
- const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion);
+ Function &F, SmallPtrSetImpl<const Instruction *> &InRegion);
// Insert GCOV profiling instrumentation
struct GCOVOptions {
diff --git a/llvm/lib/IR/DiagnosticInfo.cpp b/llvm/lib/IR/DiagnosticInfo.cpp
index a24b6d09350080..a6505eabc32803 100644
--- a/llvm/lib/IR/DiagnosticInfo.cpp
+++ b/llvm/lib/IR/DiagnosticInfo.cpp
@@ -495,6 +495,21 @@ void DiagnosticInfoMisExpect::print(DiagnosticPrinter &DP) const {
DP << getLocationStr() << ": " << getMsg();
}
+DiagnosticInfoSanitizerInstrumentationRegion::
+ DiagnosticInfoSanitizerInstrumentationRegion(const Function &Fn,
+ const DebugLoc &Loc,
+ const Twine &Msg)
+ : DiagnosticInfoWithLocationBase(DK_SanitizerInstrumentationRegion,
+ DS_Warning, Fn, Loc),
+ Msg(Msg) {}
+
+void DiagnosticInfoSanitizerInstrumentationRegion::print(
+ DiagnosticPrinter &DP) const {
+ if (isLocationAvailable())
+ DP << getLocationStr() << ": ";
+ DP << getMsg();
+}
+
void OptimizationRemarkAnalysisFPCommute::anchor() {}
void OptimizationRemarkAnalysisAliasing::anchor() {}
diff --git a/llvm/lib/Transforms/Utils/Instrumentation.cpp b/llvm/lib/Transforms/Utils/Instrumentation.cpp
index 74300dadcef8df..71fb5734dbc1c3 100644
--- a/llvm/lib/Transforms/Utils/Instrumentation.cpp
+++ b/llvm/lib/Transforms/Utils/Instrumentation.cpp
@@ -172,8 +172,91 @@ static unsigned regionDepthAfter(const Instruction &I, unsigned Depth) {
return Depth;
}
+static const char *const SanitizerRegionsDiagnosedMD =
+ "sanitizer.regions.diagnosed";
+
+/// Find the first problem with the region markers of \p F, given the fixed
+/// point of the region analysis, and report it.
+///
+/// With the "must" (minimum) join the analysis is exact as long as all paths
+/// to a block agree on the nesting depth, so the first point where they
+/// disagree is a block whose reached predecessors leave with different depths.
+/// Joins are checked first, so that an end() or a return that only looks
+/// unbalanced because of an earlier bad join is not blamed instead.
+static void diagnoseSanitizerInstrumentationRegions(
+ Function &F, const DenseMap<const BasicBlock *, unsigned> &DepthIn,
+ const DenseMap<const BasicBlock *, unsigned> &DepthOut) {
+ if (F.getMetadata(SanitizerRegionsDiagnosedMD))
+ return;
+
+ const Instruction *Where = nullptr;
+ const char *Msg = nullptr;
+ for (const BasicBlock &BB : F) {
+ if (&BB == &F.getEntryBlock() || DepthIn.lookup(&BB) == UnreachedDepth)
+ continue;
+ unsigned Min = UnreachedDepth, Max = 0;
+ for (const BasicBlock *Pred : predecessors(&BB)) {
+ unsigned Out = DepthOut.lookup(Pred);
+ if (Out == UnreachedDepth)
+ continue;
+ Min = std::min(Min, Out);
+ Max = std::max(Max, Out);
+ }
+ if (Min != Max) {
+ // Point at the first instruction that has a location, if any.
+ Where = &*BB.getFirstNonPHIOrAlloca();
+ for (const Instruction &I : BB)
+ if (I.getDebugLoc()) {
+ Where = &I;
+ break;
+ }
+ Msg = Min == 0
+ ? "control flow merges from paths with different sanitizer "
+ "instrumentation region nesting; the code that follows is "
+ "treated as outside the region"
+ : "control flow merges from paths with different sanitizer "
+ "instrumentation region nesting; the code that follows is "
+ "treated as nested in the smaller number of regions";
+ break;
+ }
+ }
+
+ for (const BasicBlock &BB : F) {
+ if (Msg)
+ break;
+ unsigned Depth = DepthIn.lookup(&BB);
+ if (Depth == UnreachedDepth)
+ continue;
+ for (const Instruction &I : BB) {
+ if (isSanitizerInstrumentationEnd(I) && !Depth) {
+ Where = &I;
+ Msg = "sanitizer instrumentation region end without a matching begin";
+ break;
+ }
+ if (isa<ReturnInst>(I) && Depth) {
+ // Only returns are checked. Not closing a region before a call that
+ // does not return (BUG(), panic()) is common and harmless, and
+ // unwinding out of a function cannot be checked consistently: a call
+ // without a landing pad leaves no terminator to look at.
+ Where = &I;
+ Msg = "sanitizer instrumentation region is not closed before return";
+ break;
+ }
+ Depth = regionDepthAfter(I, Depth);
+ }
+ }
+
+ if (!Msg)
+ return;
+ F.getContext().diagnose(DiagnosticInfoSanitizerInstrumentationRegion(
+ F, Where->getDebugLoc(),
+ Twine("in function '") + F.getName() + "': " + Msg));
+ // Several sanitizer passes may analyze the same function; warn only once.
+ F.setMetadata(SanitizerRegionsDiagnosedMD, MDNode::get(F.getContext(), {}));
+}
+
void llvm::computeSanitizerInstrumentationRegions(
- const Function &F, SmallPtrSetImpl<const Instruction *> &InRegion) {
+ Function &F, SmallPtrSetImpl<const Instruction *> &InRegion) {
DenseMap<const BasicBlock *, unsigned> DepthIn;
DenseMap<const BasicBlock *, unsigned> DepthOut;
const BasicBlock *Entry = &F.getEntryBlock();
@@ -223,4 +306,6 @@ void llvm::computeSanitizerInstrumentationRegions(
InRegion.insert(&I);
}
}
+
+ diagnoseSanitizerInstrumentationRegions(F, DepthIn, DepthOut);
}
diff --git a/llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll b/llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll
new file mode 100644
index 00000000000000..89156cde7e1fe1
--- /dev/null
+++ b/llvm/test/Instrumentation/sanitizer-instrumentation-regions-diagnostics.ll
@@ -0,0 +1,161 @@
+; Unbalanced llvm.sanitizer.instrumentation.begin/end regions are reported as a
+; warning by every sanitizer pass that implements regions, at most once per
+; function.
+
+; RUN: opt < %s -passes=msan -msan-kernel=1 -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+; RUN: opt < %s -passes=asan -asan-kernel=1 -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+; RUN: opt < %s -passes=tsan -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+; RUN: opt < %s -passes=sancov-module -sanitizer-coverage-level=3 \
+; RUN: -sanitizer-coverage-trace-pc -S -o /dev/null 2>&1 \
+; RUN: | FileCheck %s --implicit-check-not=warning:
+
+; Several passes analyzing the same function still produce only one warning.
+; RUN: opt < %s -passes='sancov-module,msan,tsan,asan' -msan-kernel=1 \
+; RUN: -asan-kernel=1 -sanitizer-coverage-level=3 -sanitizer-coverage-trace-pc \
+; RUN: -S -o /dev/null 2>&1 | FileCheck %s --implicit-check-not=warning:
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @llvm.sanitizer.instrumentation.begin()
+declare void @llvm.sanitizer.instrumentation.end()
+
+; Balanced, nested and conditional-but-balanced regions are fine.
+define void @balanced(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ call void @llvm.sanitizer.instrumentation.end()
+ br i1 %c, label %then, label %exit
+
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 1, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ br label %exit
+
+exit:
+ ret void
+}
+
+; Leaving the function without returning while a region is open is fine: after
+; a noreturn call, and when an exception unwinds out of the function.
+declare void @panic() noreturn
+declare void @may_throw()
+declare i32 @__gxx_personality_v0(...)
+
+define void @open_at_unreachable(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %die, label %ok
+
+die:
+ call void @panic()
+ unreachable
+
+ok:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+define void @open_at_resume(ptr %p) disable_sanitizer_instrumentation personality ptr @__gxx_personality_v0 {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ invoke void @may_throw()
+ to label %cont unwind label %lpad
+
+cont:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+
+lpad:
+ %lp = landingpad { ptr, i32 }
+ cleanup
+ resume { ptr, i32 } %lp
+}
+
+; CHECK: warning: {{.*}}regions.c:3:5: in function 'unmatched_end': sanitizer instrumentation region end without a matching begin
+define void @unmatched_end(ptr %p) disable_sanitizer_instrumentation !dbg !5 {
+entry:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end(), !dbg !8
+ ret void
+}
+
+; CHECK: warning: in function 'not_closed': sanitizer instrumentation region is not closed before return
+define void @not_closed(ptr %p) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ store i32 0, ptr %p
+ ret void
+}
+
+; The region is only entered on one side of the branch, so the join is blamed
+; rather than the end() or the return that follow it.
+; CHECK: warning: in function 'bad_join': control flow merges from paths with different sanitizer instrumentation region nesting; the code that follows is treated as outside the region
+define void @bad_join(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ br i1 %c, label %then, label %join
+
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+
+join:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; If all paths are inside a region but at different depths, the code that
+; follows stays instrumented.
+; CHECK: warning: in function 'nested_join': control flow merges from paths with different sanitizer instrumentation region nesting; the code that follows is treated as nested in the smaller number of regions
+define void @nested_join(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %then, label %join
+
+then:
+ call void @llvm.sanitizer.instrumentation.begin()
+ br label %join
+
+join:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+; A begin() in a loop body without an end() makes the loop header a bad join.
+; CHECK: warning: in function 'loop': control flow merges
+define void @loop(ptr %p, i1 %c) disable_sanitizer_instrumentation {
+entry:
+ br label %header
+
+header:
+ store i32 0, ptr %p
+ call void @llvm.sanitizer.instrumentation.begin()
+ br i1 %c, label %header, label %exit
+
+exit:
+ call void @llvm.sanitizer.instrumentation.end()
+ ret void
+}
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3, !4}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C11, file: !1, producer: "clang", isOptimized: true, runtimeVersion: 0, emissionKind: LineTablesOnly)
+!1 = !DIFile(filename: "regions.c", directory: "/tmp")
+!3 = !{i32 2, !"Debug Info Version", i32 3}
+!4 = !{i32 7, !"Dwarf Version", i32 5}
+!5 = distinct !DISubprogram(name: "unmatched_end", scope: !1, file: !1, line: 1, type: !6, scopeLine: 1, spFlags: DISPFlagDefinition | DISPFlagOptimized, unit: !0)
+!6 = !DISubroutineType(types: !7)
+!7 = !{}
+!8 = !DILocation(line: 3, column: 5, scope: !5)
More information about the cfe-commits
mailing list