[clang] [llvm] [KMSAN] Pass clean parameter shadow to the calls in noinstr instrumentation regions (PR #240784)

Alexander Potapenko via cfe-commits cfe-commits at lists.llvm.org
Tue Nov 10 02:54:40 PST 2026


https://github.com/ramosian-glider created https://github.com/llvm/llvm-project/pull/240784

The Linux kernel marks the functions that must not be instrumented
(entry code, NMI handlers and the like) as `noinstr`, which under KMSAN
means `disable_sanitizer_instrumentation`. Such functions still call
instrumented code between `instrumentation_begin()` and
`instrumentation_end()`. Because the caller is not instrumented, it does
not write the shadow of the arguments, and the callee reads whatever the
last instrumented caller left in the per-context parameter TLS. This
produces false positive KMSAN reports on the kernel entry paths, e.g.
https://lore.kernel.org/all/69e7ee1f.a00a0220.17a17.001d.GAE@google.com/T/#u

Teach KMSAN about these regions. Under KMSAN, the kernel's markers
additionally expand to volatile asm statements that contain nothing but
an assembler comment:

  asm volatile("# kmsan.instrumentation.begin");
  asm volatile("# kmsan.instrumentation.end");

MemorySanitizer recognizes the tokens in the asm template and, in a
`disable_sanitizer_instrumentation` function, instruments the calls
between the markers the way it does in a `no_sanitize("kernel-memory")`
function: it fetches the context state and writes the shadow of the
call arguments, including the variadic ones, as initialized. Arguments
that the callee checks eagerly (`noundef` with
`-fsanitize-memory-param-retval`) need no shadow, and a call with only
such arguments is left as it is. Nothing else is instrumented: no
shadow is propagated, no checks are inserted, memory accesses and the
code outside the markers are not touched, and the function gets no
prologue.

No new intrinsics or builtins are needed. The markers assemble to
nothing whether or not the pass sees them, have exactly the semantics
of the volatile asm statements the kernel already uses for objtool, and
are left in place.

The context state is fetched before each call rather than once per
region, because `__msan_get_context_state()` returns different contexts
for task and interrupt contexts, and noinstr functions switch between
them inside a region (e.g. `irq_enter_rcu()` in
`run_irq_on_irqstack_cond()`).

Regions nest (`WARN_ON()` inside a region opens another one), so
membership is computed over the nesting depth. It is a must-analysis
with the minimum over the predecessors as the join: a call is in a
region only if every path from the function entry to it has entered a
region and not left it. Unbalanced markers thus err on the side of not
instrumenting, and an unmatched end is ignored.

Userspace MSan ignores the markers. The handling can be switched off
with `-msan-kernel-instrumentation-regions=0`.

Assisted-by: Claude


>From 81bde4ad5069e5bb6ee406e4b1dcb956a033525b Mon Sep 17 00:00:00 2001
From: Alexander Potapenko <glider at google.com>
Date: Tue, 10 Nov 2026 09:58:23 +0000
Subject: [PATCH] [KMSAN] Pass clean parameter shadow to the calls in noinstr
 instrumentation regions

The Linux kernel marks the functions that must not be instrumented
(entry code, NMI handlers and the like) as `noinstr`, which under KMSAN
means `disable_sanitizer_instrumentation`. Such functions still call
instrumented code between `instrumentation_begin()` and
`instrumentation_end()`. Because the caller is not instrumented, it does
not write the shadow of the arguments, and the callee reads whatever the
last instrumented caller left in the per-context parameter TLS. This
produces false positive KMSAN reports on the kernel entry paths, e.g.
https://lore.kernel.org/all/69e7ee1f.a00a0220.17a17.001d.GAE@google.com/T/#u

Teach KMSAN about these regions. Under KMSAN, the kernel's markers
additionally expand to volatile asm statements that contain nothing but
an assembler comment:

  asm volatile("# kmsan.instrumentation.begin");
  asm volatile("# kmsan.instrumentation.end");

MemorySanitizer recognizes the tokens in the asm template and, in a
`disable_sanitizer_instrumentation` function, instruments the calls
between the markers the way it does in a `no_sanitize("kernel-memory")`
function: it fetches the context state and writes the shadow of the
call arguments, including the variadic ones, as initialized. Arguments
that the callee checks eagerly (`noundef` with
`-fsanitize-memory-param-retval`) need no shadow, and a call with only
such arguments is left as it is. Nothing else is instrumented: no
shadow is propagated, no checks are inserted, memory accesses and the
code outside the markers are not touched, and the function gets no
prologue.

No new intrinsics or builtins are needed. The markers assemble to
nothing whether or not the pass sees them, have exactly the semantics
of the volatile asm statements the kernel already uses for objtool, and
are left in place.

The context state is fetched before each call rather than once per
region, because `__msan_get_context_state()` returns different contexts
for task and interrupt contexts, and noinstr functions switch between
them inside a region (e.g. `irq_enter_rcu()` in
`run_irq_on_irqstack_cond()`).

Regions nest (`WARN_ON()` inside a region opens another one), so
membership is computed over the nesting depth. It is a must-analysis
with the minimum over the predecessors as the join: a call is in a
region only if every path from the function entry to it has entered a
region and not left it. Unbalanced markers thus err on the side of not
instrumenting, and an unmatched end is ignored.

Userspace MSan ignores the markers. The handling can be switched off
with `-msan-kernel-instrumentation-regions=0`.

Assisted-by: Claude
---
 clang/docs/MemorySanitizer.md                 |  24 +
 .../CodeGen/kmsan-instrumentation-regions.c   | 103 +++
 .../Instrumentation/InstrumentationOptions.td |   6 +
 .../Instrumentation/MemorySanitizer.cpp       | 226 +++++-
 .../SystemZ/kmsan-instrumentation-regions.ll  | 103 +++
 .../kmsan-instrumentation-regions-cfg.ll      | 744 ++++++++++++++++++
 .../kmsan-instrumentation-regions-disabled.ll |  29 +
 .../kmsan-instrumentation-regions-eager.ll    | 105 +++
 .../kmsan-instrumentation-regions.ll          | 557 +++++++++++++
 9 files changed, 1885 insertions(+), 12 deletions(-)
 create mode 100644 clang/test/CodeGen/kmsan-instrumentation-regions.c
 create mode 100644 llvm/test/Instrumentation/MemorySanitizer/SystemZ/kmsan-instrumentation-regions.ll
 create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-cfg.ll
 create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-disabled.ll
 create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-eager.ll
 create mode 100644 llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions.ll

diff --git a/clang/docs/MemorySanitizer.md b/clang/docs/MemorySanitizer.md
index d9ba74641c7787..9d93ca2d3c3bff 100644
--- a/clang/docs/MemorySanitizer.md
+++ b/clang/docs/MemorySanitizer.md
@@ -91,6 +91,30 @@ positives and therefore should be used with care, and only if absolutely
 required; for example for certain code that cannot tolerate any instrumentation
 and resulting side-effects. This attribute overrides `no_sanitize("memory")`.
 
+With `-fsanitize=kernel-memory`, such a function can still tell
+KernelMemorySanitizer which of its calls go to instrumented code, so that the
+callees do not report false positives on their arguments. The calls are
+bracketed with inline assembly markers, which the Linux kernel emits from
+`instrumentation_begin()` and `instrumentation_end()`:
+
+```c
+__attribute__((disable_sanitizer_instrumentation))
+void handle_irq(struct pt_regs *regs) {
+  enter_irq_context(regs);       // left alone
+  asm volatile("# kmsan.instrumentation.begin");
+  dispatch_irq(regs);            // the arguments are passed as initialized
+  asm volatile("# kmsan.instrumentation.end");
+  exit_irq_context(regs);        // left alone
+}
+```
+
+The markers are recognized by the `kmsan.instrumentation.begin` and
+`kmsan.instrumentation.end` tokens in the assembly template; placed in an
+assembler comment, they generate no code. Regions nest, and a call is in a
+region only if every path to it has passed a begin marker that is not closed
+yet. Nothing but the calls inside the regions is instrumented, and no checks
+are inserted. Userspace MemorySanitizer ignores the markers.
+
 ### Interaction of Inlining with Disabling Sanitizer Instrumentation
 
 - A `no_sanitize` function will not be inlined heuristically by the compiler into a sanitized function.
diff --git a/clang/test/CodeGen/kmsan-instrumentation-regions.c b/clang/test/CodeGen/kmsan-instrumentation-regions.c
new file mode 100644
index 00000000000000..170c44942b7c69
--- /dev/null
+++ b/clang/test/CodeGen/kmsan-instrumentation-regions.c
@@ -0,0 +1,103 @@
+// REQUIRES: x86-registered-target
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -O2 -fsanitize=kernel-memory \
+// RUN:   -emit-llvm -o - %s | FileCheck %s
+// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -O2 -fsanitize=kernel-memory \
+// RUN:   -S -o - %s | FileCheck %s --check-prefix=ASM
+
+// The way the Linux kernel uses KMSAN instrumentation regions: the markers are
+// volatile asm statements that hold nothing but a comment. They survive the
+// optimization pipeline with the calls staying between them, and
+// KernelMemorySanitizer passes clean parameter shadow to those calls, in a
+// function it otherwise must not touch.
+
+#define instrumentation_begin() __asm__ volatile("# kmsan.instrumentation.begin")
+#define instrumentation_end() __asm__ volatile("# kmsan.instrumentation.end")
+#define noinstr __attribute__((disable_sanitizer_instrumentation))
+
+struct pt_regs {
+  long ip, sp;
+};
+struct pair {
+  long a, b;
+};
+
+void enter(struct pt_regs *regs);
+void leave(struct pt_regs *regs);
+void handle(struct pair p);
+int printk(const char *fmt, ...);
+
+// WARN_ON() opens a region of its own, which nests when it is used in one.
+static inline __attribute__((always_inline)) void warn_on(long cond) {
+  instrumentation_begin();
+  if (cond)
+    printk("warning\n");
+  instrumentation_end();
+}
+
+// CHECK-LABEL: define {{.*}}void @entry(
+// CHECK-NOT:     __msan
+// CHECK:         call void @enter(
+// CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin"
+// CHECK:         call void asm sideeffect "# kmsan.instrumentation.begin"
+//
+// The printk() in warn_on() has no variadic arguments in the overflow area:
+// CHECK:         [[CTX1:%.*]] = call ptr @__msan_get_context_state()
+// CHECK:         store i64 0, ptr
+// CHECK-NEXT:    call {{.*}}@printk(
+//
+// CHECK:         call void asm sideeffect "# kmsan.instrumentation.end"
+//
+// handle() takes the struct in two registers, which are not noundef:
+// CHECK-NEXT:    [[CTX2:%.*]] = call ptr @__msan_get_context_state()
+// CHECK-NEXT:    store i64 0, ptr [[CTX2]]
+// CHECK:         store i64 0, ptr
+// CHECK-NEXT:    call void @handle(
+//
+// CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end"
+// CHECK-NEXT:    call void @leave(
+// CHECK-NEXT:    ret void
+noinstr void entry(struct pt_regs *regs, struct pair p) {
+  enter(regs);
+  instrumentation_begin();
+  warn_on(regs->ip);
+  handle(p);
+  instrumentation_end();
+  leave(regs);
+}
+
+// A function that is not noinstr is instrumented as a whole; the markers do
+// not change that.
+// CHECK-LABEL: define {{.*}}void @instrumented(
+// CHECK:         call ptr @__msan_get_context_state()
+// CHECK-NOT:     call ptr @__msan_get_context_state()
+// CHECK:         ret void
+void instrumented(struct pt_regs *regs, struct pair p) {
+  instrumentation_begin();
+  handle(p);
+  instrumentation_end();
+  leave(regs);
+}
+
+// The markers generate no code, and neither does the function get a KMSAN
+// prologue.
+// ASM-LABEL: entry:
+// ASM-NOT:     __msan
+// ASM:         callq enter
+// ASM-NEXT:    #APP
+// ASM-NEXT:    # kmsan.instrumentation.begin
+// ASM-NEXT:    #NO_APP
+// ASM:         #APP
+// ASM-NEXT:    # kmsan.instrumentation.begin
+// ASM-NEXT:    #NO_APP
+// ASM:         callq __msan_get_context_state
+// ASM:         callq printk
+// ASM:         #APP
+// ASM-NEXT:    # kmsan.instrumentation.end
+// ASM-NEXT:    #NO_APP
+// ASM:         callq __msan_get_context_state
+// ASM:         callq handle
+// ASM:         #APP
+// ASM-NEXT:    # kmsan.instrumentation.end
+// ASM-NEXT:    #NO_APP
+// ASM-NOT:     __msan
+// ASM:         jmp leave
diff --git a/llvm/lib/Transforms/Instrumentation/InstrumentationOptions.td b/llvm/lib/Transforms/Instrumentation/InstrumentationOptions.td
index 046d9d28041100..8ccb4ced6c0de4 100644
--- a/llvm/lib/Transforms/Instrumentation/InstrumentationOptions.td
+++ b/llvm/lib/Transforms/Instrumentation/InstrumentationOptions.td
@@ -479,6 +479,12 @@ defm : OptionalBoolField<"msan-keep-going",
   "Keep going after reporting a UMR">;
 defm : OptionalBoolField<"msan-kernel",
   "Enable KernelMemorySanitizer instrumentation">;
+// The Linux kernel's noinstr functions (disable_sanitizer_instrumentation) may
+// call instrumented code between instrumentation_begin() and
+// instrumentation_end(). Under KMSAN these expand to inline asm markers, see
+// "KMSAN instrumentation regions" in MemorySanitizer.cpp.
+defm : BoolField<"msan-kernel-instrumentation-regions", "true",
+  "KMSAN: pass clean parameter shadow to the calls made between kmsan.instrumentation.begin/end inline asm markers in functions with disable_sanitizer_instrumentation">;
 defm : OptionalValueField<"msan-origin-base", "uint64_t",
   "Define custom MSan OriginBase">;
 defm : BoolField<"msan-poison-stack", "true",
diff --git a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
index f7a263d3c7a929..1f097c437d4d89 100644
--- a/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
+++ b/llvm/lib/Transforms/Instrumentation/MemorySanitizer.cpp
@@ -1032,6 +1032,134 @@ class NextNodeIRBuilder : public IRBuilder<> {
   }
 };
 
+//===----------------------------------------------------------------------===//
+// KMSAN instrumentation regions
+//
+// The Linux kernel marks the functions that must not be instrumented (entry
+// code, NMI handlers and the like) as noinstr, which under KMSAN means
+// disable_sanitizer_instrumentation: the function gets no prologue and no
+// instrumentation at all. Such functions still call instrumented code, which
+// the kernel brackets with instrumentation_begin() and instrumentation_end().
+// Since the caller does not write the shadow of the arguments, the callee
+// reads whatever the last instrumented caller left in the per-context
+// parameter TLS, and reports uninitialized values that are not there.
+//
+// To let KMSAN see these calls, the kernel's markers additionally expand to
+//
+//   asm volatile("# kmsan.instrumentation.begin");
+//   asm volatile("# kmsan.instrumentation.end");
+//
+// The token lives in an assembler comment, so a marker assembles to nothing,
+// whether or not this pass sees it. No new IR constructs are involved, and
+// the markers have the semantics of the volatile asm statements the kernel
+// already uses for objtool: they are neither removed nor reordered with calls.
+// They are recognized by the token being a substring of the asm template and
+// are left in place.
+//
+// In a disable_sanitizer_instrumentation function, a call between the markers
+// is instrumented the way it would be in a no_sanitize("kernel-memory")
+// function: the context state is fetched and the shadow of the call arguments
+// (including the variadic ones) is written as initialized. Nothing else is
+// instrumented: no shadow is propagated, no checks are inserted, memory
+// accesses and the code outside the markers are left alone.
+//
+// The context state is fetched right before each call rather than once per
+// region: __msan_get_context_state() returns a different context for task and
+// interrupt contexts, and noinstr functions do switch between them inside a
+// region (e.g. irq_enter_rcu() in run_irq_on_irqstack_cond()).
+//
+// Regions nest (WARN_ON() inside a region opens another one), so membership is
+// computed over the nesting depth. It is a must-analysis: a call is in a region
+// only if every path from the function entry to it has entered a region and
+// not left it, i.e. the depth at a join point is the minimum over the
+// predecessors. Unbalanced markers thus err on the side of not instrumenting,
+// and an unmatched end is ignored. (objtool, which validates the kernel's
+// markers, tracks them with a counter as well.)
+//===----------------------------------------------------------------------===//
+
+static constexpr StringLiteral kKmsanRegionBegin =
+    "kmsan.instrumentation.begin";
+static constexpr StringLiteral kKmsanRegionEnd = "kmsan.instrumentation.end";
+
+static bool isKmsanRegionMarker(const Instruction &I, StringRef Token) {
+  const auto *CB = dyn_cast<CallBase>(&I);
+  if (!CB)
+    return false;
+  const auto *IA = dyn_cast<InlineAsm>(CB->getCalledOperand());
+  return IA && StringRef(IA->getAsmString()).contains(Token);
+}
+
+/// Returns true if \p F opens a KMSAN instrumentation region.
+static bool hasKmsanRegions(const Function &F) {
+  for (const BasicBlock &BB : F)
+    for (const Instruction &I : BB)
+      if (isKmsanRegionMarker(I, kKmsanRegionBegin))
+        return true;
+  return false;
+}
+
+/// Returns the region nesting depth after \p I, given the depth before it.
+static unsigned kmsanRegionDepthAfter(const Instruction &I, unsigned Depth) {
+  if (isKmsanRegionMarker(I, kKmsanRegionBegin))
+    return Depth + 1;
+  // Saturate at zero: wrapping around on an unmatched end would put the rest
+  // of the function into a region.
+  if (isKmsanRegionMarker(I, kKmsanRegionEnd) && Depth)
+    return Depth - 1;
+  return Depth;
+}
+
+/// Collects the calls of \p F that are inside a KMSAN instrumentation region
+/// and need their parameter shadow set up.
+static void collectKmsanRegionCalls(Function &F,
+                                    SmallVectorImpl<CallBase *> &Calls) {
+  // The depth of a block that has not been reached from the entry (yet).
+  constexpr unsigned Unreached = ~0U;
+  // Nesting depth at the start and at the end of each block.
+  SmallVector<unsigned> DepthIn(F.getMaxBlockNumber(), Unreached);
+  SmallVector<unsigned> DepthOut(F.getMaxBlockNumber(), Unreached);
+
+  // A block is queued when its input depth decreases, and depths are bounded
+  // below by zero, so this terminates.
+  BasicBlock *Entry = &F.getEntryBlock();
+  DepthIn[Entry->getNumber()] = 0;
+  SmallSetVector<BasicBlock *, 16> Worklist;
+  Worklist.insert(Entry);
+  while (!Worklist.empty()) {
+    BasicBlock *BB = Worklist.pop_back_val();
+    unsigned Depth = DepthIn[BB->getNumber()];
+    for (const Instruction &I : *BB)
+      Depth = kmsanRegionDepthAfter(I, Depth);
+    if (Depth >= DepthOut[BB->getNumber()])
+      continue;
+    DepthOut[BB->getNumber()] = Depth;
+    for (BasicBlock *Succ : successors(BB)) {
+      if (Depth < DepthIn[Succ->getNumber()]) {
+        DepthIn[Succ->getNumber()] = Depth;
+        Worklist.insert(Succ);
+      }
+    }
+  }
+
+  for (BasicBlock &BB : F) {
+    unsigned Depth = DepthIn[BB.getNumber()];
+    if (Depth == Unreached)
+      continue;
+    for (Instruction &I : BB) {
+      Depth = kmsanRegionDepthAfter(I, Depth);
+      if (!Depth)
+        continue;
+      // Inline asm (the markers included) and intrinsics take no parameter
+      // shadow.
+      auto *CB = dyn_cast<CallBase>(&I);
+      if (!CB || CB->isInlineAsm() || isa<IntrinsicInst>(CB) ||
+          CB->hasMetadata(LLVMContext::MD_nosanitize))
+        continue;
+      Calls.push_back(CB);
+    }
+  }
+}
+
 /// This class does all the work for a given function. Store and Load
 /// instructions store and load corresponding shadow and origin
 /// values. Most instructions propagate shadow from arguments to their
@@ -1056,6 +1184,9 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
   bool PoisonStack;
   bool PoisonUndef;
   bool PoisonUndefVectors;
+  /// The function is not instrumented as a whole; only the calls inside KMSAN
+  /// instrumentation regions are, see runOnRegions().
+  bool RegionOnly;
 
   struct ShadowOriginAndInsertPoint {
     Value *Shadow;
@@ -1073,17 +1204,27 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
   int64_t SplittableBlocksCount = 0;
 
   MemorySanitizerVisitor(Function &F, MemorySanitizer &MS,
-                         const TargetLibraryInfo &TLI)
+                         const TargetLibraryInfo &TLI, bool RegionOnly = false)
       : Opts(MS.Opts), F(F), MS(MS), VAHelper(CreateVarArgHelper(F, MS, *this)),
-        TLI(&TLI) {
+        TLI(&TLI), RegionOnly(RegionOnly) {
     bool SanitizeFunction = F.hasFnAttribute(Attribute::SanitizeMemory) &&
-                            !Opts.msan_disable_checks;
+                            !Opts.msan_disable_checks && !RegionOnly;
     InsertChecks = SanitizeFunction;
     PropagateShadow = SanitizeFunction;
     PoisonStack = SanitizeFunction && Opts.msan_poison_stack;
     PoisonUndef = SanitizeFunction && Opts.msan_poison_undef;
     PoisonUndefVectors = SanitizeFunction && Opts.msan_poison_undef_vectors;
 
+    if (RegionOnly) {
+      // No prologue: the function must stay as it is outside of the regions.
+      // Nothing in a region looks up the shadow of memory, so the s390x
+      // metadata slot is not needed either.
+      MS.initializeCallbacks(*F.getParent(), TLI);
+      FnPrologueEnd = nullptr;
+      MS.MsanMetadataAlloca = nullptr;
+      return;
+    }
+
     // In the presence of unreachable blocks, we may see Phi nodes with
     // incoming nodes from such blocks. Since InstVisitor skips unreachable
     // blocks, such nodes will not have any shadow value associated with them.
@@ -1439,9 +1580,10 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
     LLVM_DEBUG(dbgs() << "DONE:\n" << F);
   }
 
-  // Returns the last instruction in the new prologue
-  void insertKmsanPrologue(IRBuilder<> &IRB) {
-    Value *ContextState = IRB.CreateCall(MS.MsanGetContextStateFn, {});
+  /// Fetch the KMSAN context state and point the TLS accessors at it. Returns
+  /// the call to __msan_get_context_state().
+  CallInst *insertKmsanContextFetch(IRBuilder<> &IRB) {
+    CallInst *ContextState = IRB.CreateCall(MS.MsanGetContextStateFn, {});
     Constant *Zero = IRB.getInt32(0);
     MS.ParamTLS = IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
                                 {Zero, IRB.getInt32(0)}, "param_shadow");
@@ -1459,10 +1601,52 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
     MS.RetvalOriginTLS =
         IRB.CreateGEP(MS.MsanContextStateTy, ContextState,
                       {Zero, IRB.getInt32(6)}, "retval_origin");
+    return ContextState;
+  }
+
+  void insertKmsanPrologue(IRBuilder<> &IRB) {
+    insertKmsanContextFetch(IRB);
     if (MS.TargetTriple.getArch() == Triple::systemz)
       MS.MsanMetadataAlloca = IRB.CreateAlloca(MS.MsanMetadata, 0u);
   }
 
+  /// Instrument the calls inside the KMSAN instrumentation regions of a
+  /// function that is otherwise not instrumented, see "KMSAN instrumentation
+  /// regions" above.
+  bool runOnRegions() {
+    assert(RegionOnly && MS.CompileKernel);
+    SmallVector<CallBase *, 16> Calls;
+    collectKmsanRegionCalls(F, Calls);
+    if (Calls.empty())
+      return false;
+
+    for (CallBase *CB : Calls) {
+      IRBuilder<> IRB(CB);
+      CallInst *ContextState = insertKmsanContextFetch(IRB);
+      // PropagateShadow is off, so this writes clean shadow for the arguments
+      // the callee is going to read from the TLS.
+      visitCallBase(*CB);
+      // Drop what turned out to be unused, in particular the context fetch if
+      // the callee reads nothing from the TLS because it checks all of its
+      // arguments eagerly (noundef). Everything between the fetch and the call
+      // has just been inserted.
+      SmallVector<Instruction *, 32> Inserted;
+      for (Instruction &I :
+           make_range(ContextState->getIterator(), CB->getIterator()))
+        Inserted.push_back(&I);
+      for (Instruction *I : reverse(Inserted))
+        if (isInstructionTriviallyDead(I) ||
+            (I == ContextState && I->use_empty()))
+          I->eraseFromParent();
+      // The next call fetches the context again.
+      MS.ParamTLS = MS.RetvalTLS = MS.VAArgTLS = MS.VAArgOriginTLS =
+          MS.VAArgOverflowSizeTLS = MS.ParamOriginTLS = MS.RetvalOriginTLS =
+              nullptr;
+    }
+
+    return true;
+  }
+
   /// Add MemorySanitizer instrumentation to a function.
   bool runOnFunction() {
     // Iterate all BBs in depth-first order and create shadow instructions
@@ -1897,6 +2081,10 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
   /// This function either returns the value set earlier with setShadow,
   /// or extracts if from ParamTLS (for function arguments).
   Value *getShadow(Value *V) {
+    // There is no prologue to load the shadow of the arguments in, and nothing
+    // else has shadow.
+    if (RegionOnly)
+      return getCleanShadow(V);
     if (Instruction *I = dyn_cast<Instruction>(V)) {
       if (!PropagateShadow || I->getMetadata(LLVMContext::MD_nosanitize))
         return getCleanShadow(V);
@@ -7818,10 +8006,13 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
           MaybeAlign Alignment = std::nullopt;
           if (ParamAlignment)
             Alignment = std::min(*ParamAlignment, kShadowTLSAlignment);
-          Value *AShadowPtr, *AOriginPtr;
-          std::tie(AShadowPtr, AOriginPtr) =
-              getShadowOriginPtr(A, IRB, IRB.getInt8Ty(), Alignment,
-                                 /*isStore*/ false);
+          Value *AShadowPtr = nullptr, *AOriginPtr = nullptr;
+          // The shadow of the argument is not going to be read below if
+          // RegionOnly, and looking it up is a call into the KMSAN runtime.
+          if (!RegionOnly)
+            std::tie(AShadowPtr, AOriginPtr) =
+                getShadowOriginPtr(A, IRB, IRB.getInt8Ty(), Alignment,
+                                   /*isStore*/ false);
           if (!PropagateShadow) {
             Store = IRB.CreateMemSet(ArgShadowBase,
                                      Constant::getNullValue(IRB.getInt8Ty()),
@@ -7881,6 +8072,10 @@ struct MemorySanitizerVisitor : public InstVisitor<MemorySanitizerVisitor> {
       return;
     }
 
+    // Nobody is going to look at the shadow of the return value.
+    if (RegionOnly)
+      return;
+
     IRBuilder<> IRBBefore(&CB);
     // Until we have full dynamic coverage, make sure the retval shadow is 0.
     Value *Base = getShadowPtrForRetval(IRBBefore);
@@ -9758,8 +9953,15 @@ bool MemorySanitizer::sanitizeFunction(Function &F, TargetLibraryInfo &TLI) {
   if (!CompileKernel && F.getName() == kMsanModuleCtorName)
     return false;
 
-  if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation))
-    return false;
+  if (F.hasFnAttribute(Attribute::DisableSanitizerInstrumentation)) {
+    // Such a function is not instrumented, except for the calls inside KMSAN
+    // instrumentation regions.
+    if (!CompileKernel || !Opts.msan_kernel_instrumentation_regions ||
+        !hasKmsanRegions(F))
+      return false;
+    return MemorySanitizerVisitor(F, *this, TLI, /*RegionOnly=*/true)
+        .runOnRegions();
+  }
 
   MemorySanitizerVisitor Visitor(F, *this, TLI);
 
diff --git a/llvm/test/Instrumentation/MemorySanitizer/SystemZ/kmsan-instrumentation-regions.ll b/llvm/test/Instrumentation/MemorySanitizer/SystemZ/kmsan-instrumentation-regions.ll
new file mode 100644
index 00000000000000..ab42c0bdabeff5
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/SystemZ/kmsan-instrumentation-regions.ll
@@ -0,0 +1,103 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 6
+; RUN: opt < %s -msan-kernel=1 -S -passes=msan | FileCheck %s
+
+; KMSAN instrumentation regions on s390x. Unlike in an instrumented function,
+; there is no prologue, and so no stack slot for the KMSAN runtime to return
+; shadow pointers through: the calls in a region do not look up any shadow.
+
+target datalayout = "E-m:e-i1:8:16-i8:8:16-i64:64-f128:64-a:8:16-n32:64"
+target triple = "s390x-unknown-linux-gnu"
+
+%struct.pair = type { i64, i64 }
+
+declare void @callee(i64)
+declare void @callee_byval(ptr byval(%struct.pair))
+declare void @callee_varargs(ptr, ...)
+
+define void @basic(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @basic(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void @callee(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", ""()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @callee(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", ""()
+; CHECK-NEXT:    call void @callee(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void @callee(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", ""()
+  call void @callee(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", ""()
+  call void @callee(i64 %x)
+  ret void
+}
+
+define void @byval_arg(ptr %p) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @byval_arg(
+; CHECK-SAME: ptr [[P:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", ""()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr [[_MSARG]], i8 0, i64 16, i1 false)
+; CHECK-NEXT:    call void @callee_byval(ptr byval([[STRUCT_PAIR:%.*]]) [[P]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", ""()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", ""()
+  call void @callee_byval(ptr byval(%struct.pair) %p)
+  call void asm sideeffect "# kmsan.instrumentation.end", ""()
+  ret void
+}
+
+define void @varargs_call(ptr %fmt, i64 %x, i32 signext %y) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @varargs_call(
+; CHECK-SAME: ptr [[FMT:%.*]], i64 [[X:%.*]], i32 signext [[Y:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", ""()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[VA_ARG_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 2
+; CHECK-NEXT:    [[VA_ARG_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 3
+; CHECK-NEXT:    [[VA_ARG_OVERFLOW_SIZE:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 4
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 16
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    [[TMP1:%.*]] = ptrtoint ptr [[VA_ARG_SHADOW]] to i64
+; CHECK-NEXT:    [[TMP2:%.*]] = add i64 [[TMP1]], 24
+; CHECK-NEXT:    [[_MSARG_VA_O:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 24
+; CHECK-NEXT:    [[_MSARG_VA_S:%.*]] = inttoptr i64 [[TMP2]] to ptr
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S]], align 8
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG_VA_O]], align 4
+; CHECK-NEXT:    [[TMP3:%.*]] = getelementptr i32, ptr [[_MSARG_VA_O]], i32 1
+; CHECK-NEXT:    store i32 0, ptr [[TMP3]], align 4
+; CHECK-NEXT:    [[TMP4:%.*]] = ptrtoint ptr [[VA_ARG_SHADOW]] to i64
+; CHECK-NEXT:    [[TMP5:%.*]] = add i64 [[TMP4]], 32
+; CHECK-NEXT:    [[_MSARG_VA_O3:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 32
+; CHECK-NEXT:    [[_MSARG_VA_S4:%.*]] = inttoptr i64 [[TMP5]] to ptr
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S4]], align 8
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG_VA_O3]], align 4
+; CHECK-NEXT:    [[TMP6:%.*]] = getelementptr i32, ptr [[_MSARG_VA_O3]], i32 1
+; CHECK-NEXT:    store i32 0, ptr [[TMP6]], align 4
+; CHECK-NEXT:    store i64 0, ptr [[VA_ARG_OVERFLOW_SIZE]], align 8
+; CHECK-NEXT:    call void (ptr, ...) @callee_varargs(ptr [[FMT]], i64 [[X]], i32 signext [[Y]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", ""()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", ""()
+  call void (ptr, ...) @callee_varargs(ptr %fmt, i64 %x, i32 signext %y)
+  call void asm sideeffect "# kmsan.instrumentation.end", ""()
+  ret void
+}
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-cfg.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-cfg.ll
new file mode 100644
index 00000000000000..e74e13a0cdd52b
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-cfg.ll
@@ -0,0 +1,744 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 6
+; RUN: opt < %s -msan-kernel=1 -S -passes=msan | FileCheck %s
+
+; Which calls are inside a KMSAN instrumentation region when the markers are
+; spread over several basic blocks. Membership is a must-analysis over the
+; nesting depth: a call is in a region only if every path from the function
+; entry to it has entered a region and not left it.
+;
+; The calls that are expected to be inside a region go to @in(), the ones that
+; are not go to @out(). Only the calls to @in() get the context fetch and the
+; parameter shadow.
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @in(i64)
+declare void @out(i64)
+declare i1 @cond()
+
+; Regions nest: the inner end does not close the outer region. This is what
+; WARN_ON() inside a region looks like.
+define void @nested(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @nested(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; A nested region that is only entered on one path does not change the depth
+; of the outer one at the join.
+define void @nested_in_branch(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @nested_in_branch(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br i1 [[C]], label %[[WARN:.*]], label %[[JOIN:.*]]
+; CHECK:       [[WARN]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br i1 %c, label %warn, label %join
+
+warn:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  br label %join
+
+join:
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; An end without a begin is ignored; it does not cancel the begin that follows.
+define void @unmatched_end(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @unmatched_end(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; A region that is never closed extends to the end of the function.
+define void @open_at_ret(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @open_at_ret(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void @out(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  ret void
+}
+
+; A region spanning an if/else: one begin, one end, control flow in between.
+define void @diamond_inside(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @diamond_inside(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br i1 [[C]], label %[[THEN:.*]], label %[[ELSE:.*]]
+; CHECK:       [[THEN]]:
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 1)
+; CHECK-NEXT:    br label %[[JOIN:.*]]
+; CHECK:       [[ELSE]]:
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @in(i64 2)
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    [[TMP2:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW3:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP2]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG4:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW3]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG4]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br i1 %c, label %then, label %else
+
+then:
+  call void @in(i64 1)
+  br label %join
+
+else:
+  call void @in(i64 2)
+  br label %join
+
+join:
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; The region is closed in both arms of a branch, by two different markers, as
+; in xen_pv_evtchn_do_upcall() or ct_nmi_exit().
+define void @end_in_both_arms(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @end_in_both_arms(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    br i1 [[C]], label %[[THEN:.*]], label %[[ELSE:.*]]
+; CHECK:       [[THEN]]:
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @in(i64 1)
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 1)
+; CHECK-NEXT:    br label %[[JOIN:.*]]
+; CHECK:       [[ELSE]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 2)
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  br i1 %c, label %then, label %else
+
+then:
+  call void @in(i64 1)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 1)
+  br label %join
+
+else:
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 2)
+  br label %join
+
+join:
+  call void @out(i64 %x)
+  ret void
+}
+
+; The region is opened in every arm and closed after the join, as in
+; ct_nmi_enter().
+define void @begin_in_all_arms(i64 %x, i32 %sel) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @begin_in_all_arms(
+; CHECK-SAME: i64 [[X:%.*]], i32 [[SEL:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    switch i32 [[SEL]], label %[[C:.*]] [
+; CHECK-NEXT:      i32 0, label %[[A:.*]]
+; CHECK-NEXT:      i32 1, label %[[B:.*]]
+; CHECK-NEXT:    ]
+; CHECK:       [[A]]:
+; CHECK-NEXT:    call void @out(i64 0)
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 0)
+; CHECK-NEXT:    br label %[[JOIN:.*]]
+; CHECK:       [[B]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[C]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @in(i64 2)
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    [[TMP2:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW3:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP2]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG4:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW3]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG4]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  switch i32 %sel, label %c [
+  i32 0, label %a
+  i32 1, label %b
+  ]
+
+a:
+  call void @out(i64 0)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 0)
+  br label %join
+
+b:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br label %join
+
+c:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 2)
+  br label %join
+
+join:
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; The region is opened on one path only. The code after the join can be
+; reached without passing a begin, so it is not in a region, and neither is
+; anything after it, including what follows the conditional end.
+define void @begin_on_one_path(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @begin_on_one_path(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    br i1 [[C]], label %[[THEN:.*]], label %[[JOIN:.*]]
+; CHECK:       [[THEN]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 1)
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    br i1 [[C]], label %[[THEN2:.*]], label %[[EXIT:.*]]
+; CHECK:       [[THEN2]]:
+; CHECK-NEXT:    call void @out(i64 2)
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br label %[[EXIT]]
+; CHECK:       [[EXIT]]:
+; CHECK-NEXT:    call void @out(i64 3)
+; CHECK-NEXT:    ret void
+;
+entry:
+  br i1 %c, label %then, label %join
+
+then:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 1)
+  br label %join
+
+join:
+  call void @out(i64 %x)
+  br i1 %c, label %then2, label %exit
+
+then2:
+  call void @out(i64 2)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  br label %exit
+
+exit:
+  call void @out(i64 3)
+  ret void
+}
+
+; The region is closed on one path only. The code after the join is not in a
+; region.
+define void @end_on_one_path(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @end_on_one_path(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br i1 [[C]], label %[[THEN:.*]], label %[[JOIN:.*]]
+; CHECK:       [[THEN]]:
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 1)
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br i1 %c, label %then, label %join
+
+then:
+  call void @in(i64 1)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  br label %join
+
+join:
+  call void @out(i64 %x)
+  ret void
+}
+
+; Two nested regions, one of which is closed on one path only: the depth after
+; the join is the minimum, 1, so the code is still in the outer region.
+define void @inner_end_on_one_path(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @inner_end_on_one_path(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br i1 [[C]], label %[[THEN:.*]], label %[[JOIN:.*]]
+; CHECK:       [[THEN]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br i1 %c, label %then, label %join
+
+then:
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  br label %join
+
+join:
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; An early return from the middle of a region, as in ct_nmi_exit().
+define void @early_return(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @early_return(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br i1 [[C]], label %[[EARLY:.*]], label %[[CONT:.*]]
+; CHECK:       [[EARLY]]:
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 1)
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+; CHECK:       [[CONT]]:
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @in(i64 2)
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 2)
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br i1 %c, label %early, label %cont
+
+early:
+  call void @in(i64 1)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+
+cont:
+  call void @in(i64 2)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 2)
+  ret void
+}
+
+; A loop that is entirely inside a region, as in mce_start().
+define void @loop_inside(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @loop_inside(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br label %[[LOOP:.*]]
+; CHECK:       [[LOOP]]:
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    [[C:%.*]] = call i1 @cond()
+; CHECK-NEXT:    br i1 [[C]], label %[[LOOP]], label %[[EXIT:.*]]
+; CHECK:       [[EXIT]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br label %loop
+
+loop:
+  call void @in(i64 %x)
+  %c = call i1 @cond()
+  br i1 %c, label %loop, label %exit
+
+exit:
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; A region inside a loop body: balanced in every iteration.
+define void @region_in_loop(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @region_in_loop(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    br label %[[LOOP:.*]]
+; CHECK:       [[LOOP]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[C:%.*]] = call i1 @cond()
+; CHECK-NEXT:    br i1 [[C]], label %[[LOOP]], label %[[EXIT:.*]]
+; CHECK:       [[EXIT]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  br label %loop
+
+loop:
+  call void @out(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  %c = call i1 @cond()
+  br i1 %c, label %loop, label %exit
+
+exit:
+  call void @out(i64 %x)
+  ret void
+}
+
+; The loop body closes a region that was opened before the loop: the loop
+; header is reached with depth 1 from the preheader and with depth 0 over the
+; backedge, so the body is not in a region.
+define void @loop_closes_region(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @loop_closes_region(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 0)
+; CHECK-NEXT:    br label %[[LOOP:.*]]
+; CHECK:       [[LOOP]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[C:%.*]] = call i1 @cond()
+; CHECK-NEXT:    br i1 [[C]], label %[[LOOP]], label %[[EXIT:.*]]
+; CHECK:       [[EXIT]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 0)
+  br label %loop
+
+loop:
+  call void @out(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  %c = call i1 @cond()
+  br i1 %c, label %loop, label %exit
+
+exit:
+  call void @out(i64 %x)
+  ret void
+}
+
+; The loop body opens a region that is never closed: the depth only grows
+; along the backedge, so the minimum at the header is the one from the
+; preheader and the analysis terminates.
+define void @loop_opens_region(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @loop_opens_region(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    br label %[[LOOP:.*]]
+; CHECK:       [[LOOP]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    [[C:%.*]] = call i1 @cond()
+; CHECK-NEXT:    br i1 [[C]], label %[[LOOP]], label %[[EXIT:.*]]
+; CHECK:       [[EXIT]]:
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW2:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG3:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW2]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG3]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  br label %loop
+
+loop:
+  call void @out(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @in(i64 %x)
+  %c = call i1 @cond()
+  br i1 %c, label %loop, label %exit
+
+exit:
+  call void @in(i64 %x)
+  ret void
+}
+
+; A switch inside a region, as in handle_bug().
+define void @switch_inside(i64 %x, i32 %sel) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @switch_inside(
+; CHECK-SAME: i64 [[X:%.*]], i32 [[SEL:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    switch i32 [[SEL]], label %[[DEFAULT:.*]] [
+; CHECK-NEXT:      i32 0, label %[[A:.*]]
+; CHECK-NEXT:      i32 1, label %[[B:.*]]
+; CHECK-NEXT:    ]
+; CHECK:       [[A]]:
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 0)
+; CHECK-NEXT:    br label %[[B]]
+; CHECK:       [[B]]:
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @in(i64 1)
+; CHECK-NEXT:    br label %[[DEFAULT]]
+; CHECK:       [[DEFAULT]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  switch i32 %sel, label %default [
+  i32 0, label %a
+  i32 1, label %b
+  ]
+
+a:
+  call void @in(i64 0)
+  br label %b
+
+b:
+  call void @in(i64 1)
+  br label %default
+
+default:
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
+
+; A block that cannot be reached from the entry is in no region, whatever its
+; markers say, and it does not affect the depth of its successors. The block
+; is not removed, either.
+define void @unreachable_block(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @unreachable_block(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void @out(i64 0)
+; CHECK-NEXT:    br label %[[JOIN:.*]]
+; CHECK:       [[DEAD:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 1)
+; CHECK-NEXT:    br label %[[JOIN]]
+; CHECK:       [[JOIN]]:
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void @out(i64 0)
+  br label %join
+
+dead:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 1)
+  br label %join
+
+join:
+  call void @out(i64 %x)
+  ret void
+}
+
+; A region that ends in a noreturn call, as with panic().
+declare void @panic() noreturn
+define void @noreturn_in_region(i64 %x, i1 %c) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @noreturn_in_region(
+; CHECK-SAME: i64 [[X:%.*]], i1 [[C:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    br i1 [[C]], label %[[DIE:.*]], label %[[CONT:.*]]
+; CHECK:       [[DIE]]:
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @in(i64 [[X]])
+; CHECK-NEXT:    call void @panic()
+; CHECK-NEXT:    unreachable
+; CHECK:       [[CONT]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @out(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  br i1 %c, label %die, label %cont
+
+die:
+  call void @in(i64 %x)
+  call void @panic()
+  unreachable
+
+cont:
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @out(i64 %x)
+  ret void
+}
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-disabled.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-disabled.ll
new file mode 100644
index 00000000000000..af44e928c03338
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-disabled.ll
@@ -0,0 +1,29 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 6
+; RUN: opt < %s -msan-kernel=1 -msan-kernel-instrumentation-regions=0 -S -passes=msan | FileCheck %s
+; RUN: opt < %s -msan-kernel=0 -S -passes=msan | FileCheck %s
+
+; KMSAN instrumentation regions can be switched off with
+; -msan-kernel-instrumentation-regions=0, and userspace MSan does not implement
+; them: in both cases a disable_sanitizer_instrumentation function is left
+; alone, markers or not.
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+declare void @callee(i64)
+
+define void @region(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @region(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0:[0-9]+]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @callee(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-eager.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-eager.ll
new file mode 100644
index 00000000000000..6099ef2d65547e
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions-eager.ll
@@ -0,0 +1,105 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 6
+; RUN: opt < %s -msan-kernel=1 -msan-eager-checks=1 -S -passes=msan | FileCheck %s
+
+; KMSAN instrumentation regions with eager checks (-fsanitize-memory-param-retval,
+; which is what the kernel uses with CONFIG_KMSAN_CHECK_PARAM_RETVAL): a callee
+; does not read the parameter shadow of its noundef arguments, so the calls in
+; a region only clear the shadow of the remaining ones, and do not fetch the
+; context state at all if there are none.
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+%struct.pair = type { i64, i64 }
+
+declare void @callee_noundef(i64 noundef, ptr noundef)
+declare void @callee_mixed(i64 noundef, i64, i64 noundef, %struct.pair)
+declare noundef i64 @callee_ret_noundef(i64 noundef)
+declare i64 @callee_ret(i64 noundef)
+declare void @callee_varargs(ptr noundef, ...)
+
+; All the arguments are noundef: nothing to do, the call is left as it is.
+define void @all_noundef(i64 %x, ptr %p) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @all_noundef(
+; CHECK-SAME: i64 [[X:%.*]], ptr [[P:%.*]]) #[[ATTR0:[0-9]+]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @callee_noundef(i64 noundef [[X]], ptr noundef [[P]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_noundef(i64 noundef %x, ptr noundef %p)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; Only the arguments that are not noundef get their shadow cleared: the ones at
+; the offsets 8 (i64) and 24 (struct).
+define void @mixed(i64 %x, %struct.pair %s) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @mixed(
+; CHECK-SAME: i64 [[X:%.*]], [[STRUCT_PAIR:%.*]] [[S:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 24
+; CHECK-NEXT:    store { i64, i64 } zeroinitializer, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    call void @callee_mixed(i64 noundef [[X]], i64 [[X]], i64 noundef [[X]], [[STRUCT_PAIR]] [[S]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_mixed(i64 noundef %x, i64 %x, i64 noundef %x, %struct.pair %s)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; The shadow of the return value is of no interest, noundef or not.
+define i64 @returns(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define i64 @returns(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[A:%.*]] = call noundef i64 @callee_ret_noundef(i64 noundef [[X]])
+; CHECK-NEXT:    [[B:%.*]] = call i64 @callee_ret(i64 noundef [[A]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret i64 [[B]]
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  %a = call noundef i64 @callee_ret_noundef(i64 noundef %x)
+  %b = call i64 @callee_ret(i64 noundef %a)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret i64 %b
+}
+
+; The shadow of variadic arguments is always passed through the TLS.
+define void @varargs(ptr %fmt, i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @varargs(
+; CHECK-SAME: ptr [[FMT:%.*]], i64 [[X:%.*]]) #[[ATTR0]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[VA_ARG_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 2
+; CHECK-NEXT:    [[VA_ARG_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 3
+; CHECK-NEXT:    [[VA_ARG_OVERFLOW_SIZE:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 4
+; CHECK-NEXT:    [[_MSARG_VA_S1:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 8
+; CHECK-NEXT:    [[_MSARG_VA_O2:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S1]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O2]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[VA_ARG_OVERFLOW_SIZE]], align 8
+; CHECK-NEXT:    call void (ptr, ...) @callee_varargs(ptr noundef [[FMT]], i64 noundef [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void (ptr, ...) @callee_varargs(ptr noundef %fmt, i64 noundef %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
diff --git a/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions.ll b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions.ll
new file mode 100644
index 00000000000000..df279d3adcdf96
--- /dev/null
+++ b/llvm/test/Instrumentation/MemorySanitizer/kmsan-instrumentation-regions.ll
@@ -0,0 +1,557 @@
+; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --version 6
+; RUN: opt < %s -msan-kernel=1 -S -passes=msan | FileCheck %s
+
+; KMSAN instrumentation regions: in a disable_sanitizer_instrumentation
+; function (the kernel's noinstr), the calls between the
+; "kmsan.instrumentation.begin" and "kmsan.instrumentation.end" inline asm
+; markers get clean parameter shadow, the way calls in a
+; no_sanitize("kernel-memory") function do. The context state is fetched right
+; before each such call. Nothing else in the function is instrumented.
+
+target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-i128:128-f80:128-n8:16:32:64-S128"
+target triple = "x86_64-unknown-linux-gnu"
+
+%struct.pair = type { i64, i64 }
+
+declare void @callee()
+declare void @callee_i64(i64)
+declare i64 @callee_ret(i64)
+declare void @callee_mixed(i32, ptr, i64, %struct.pair)
+declare void @callee_byval(ptr byval(%struct.pair), i64)
+declare void @callee_varargs(ptr, ...)
+declare void @llvm.memset.p0.i64(ptr, i8, i64, i1)
+declare i32 @__gxx_personality_v0(...)
+
+; A function without the markers is not touched.
+define void @no_markers(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @no_markers(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1:[0-9]+]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void @callee_i64(i64 %x)
+  ret void
+}
+
+; Only the call between the markers gets the context fetch and the clean
+; shadow for %x. The markers stay.
+define void @basic(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @basic(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_i64(i64 %x)
+  ret void
+}
+
+; Each call in a region fetches the context state on its own, as the context
+; may change between two calls (e.g. on irq_enter_rcu()).
+define void @two_calls(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @two_calls(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    [[TMP1:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW1:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP1]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW1]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_i64(i64 %x)
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; The shadow of every argument is cleared at its offset in the parameter TLS:
+; 0 (i32), 8 (ptr), 16 (i64), 24 (16-byte struct).
+define void @mixed_args(i32 %a, ptr %p, i64 %b, %struct.pair %s) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @mixed_args(
+; CHECK-SAME: i32 [[A:%.*]], ptr [[P:%.*]], i64 [[B:%.*]], [[STRUCT_PAIR:%.*]] [[S:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 16
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    [[_MSARG3:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 24
+; CHECK-NEXT:    store { i64, i64 } zeroinitializer, ptr [[_MSARG3]], align 8
+; CHECK-NEXT:    call void @callee_mixed(i32 [[A]], ptr [[P]], i64 [[B]], [[STRUCT_PAIR]] [[S]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_mixed(i32 %a, ptr %p, i64 %b, %struct.pair %s)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; Constants and values computed outside of the region are all the same: clean.
+define void @constant_and_local_args(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @constant_and_local_args(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    [[LOCAL:%.*]] = alloca i64, align 8
+; CHECK-NEXT:    [[Y:%.*]] = add i64 [[X]], 1
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 16
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    [[_MSARG3:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 24
+; CHECK-NEXT:    store { i64, i64 } zeroinitializer, ptr [[_MSARG3]], align 8
+; CHECK-NEXT:    call void @callee_mixed(i32 42, ptr [[LOCAL]], i64 [[Y]], [[STRUCT_PAIR:%.*]] zeroinitializer)
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  %local = alloca i64
+  %y = add i64 %x, 1
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_mixed(i32 42, ptr %local, i64 %y, %struct.pair zeroinitializer)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; A byval argument: its parameter shadow is zeroed.
+define void @byval_arg(ptr %p, i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @byval_arg(
+; CHECK-SAME: ptr [[P:%.*]], i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr [[_MSARG]], i8 0, i64 16, i1 false)
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 16
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    call void @callee_byval(ptr byval([[STRUCT_PAIR:%.*]]) [[P]], i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_byval(ptr byval(%struct.pair) %p, i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; A byval argument of the function itself that is passed on must not make the
+; pass emit anything in the entry block, which is outside of the region.
+define void @byval_param_passed_on(ptr byval(%struct.pair) %p, i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @byval_param_passed_on(
+; CHECK-SAME: ptr byval([[STRUCT_PAIR:%.*]]) [[P:%.*]], i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr [[_MSARG]], i8 0, i64 16, i1 false)
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 16
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    call void @callee_byval(ptr byval([[STRUCT_PAIR]]) [[P]], i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_byval(ptr byval(%struct.pair) %p, i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; A call to a variadic function publishes clean shadow for the variadic
+; arguments and their overflow size, too.
+define void @varargs_call(ptr %fmt, i64 %x, i32 %y) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @varargs_call(
+; CHECK-SAME: ptr [[FMT:%.*]], i64 [[X:%.*]], i32 [[Y:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[VA_ARG_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 2
+; CHECK-NEXT:    [[VA_ARG_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 3
+; CHECK-NEXT:    [[VA_ARG_OVERFLOW_SIZE:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 4
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 16
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S3:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 8
+; CHECK-NEXT:    [[_MSARG_VA_O4:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S3]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O4]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S5:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 16
+; CHECK-NEXT:    [[_MSARG_VA_O6:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 16
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG_VA_S5]], align 8
+; CHECK-NEXT:    store i32 0, ptr [[_MSARG_VA_O6]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[VA_ARG_OVERFLOW_SIZE]], align 8
+; CHECK-NEXT:    call void (ptr, ...) @callee_varargs(ptr [[FMT]], i64 [[X]], i32 [[Y]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void (ptr, ...) @callee_varargs(ptr %fmt, i64 %x, i32 %y)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; More variadic arguments than fit into the registers: the rest goes to the
+; overflow area.
+define void @varargs_overflow(ptr %fmt, i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @varargs_overflow(
+; CHECK-SAME: ptr [[FMT:%.*]], i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[VA_ARG_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 2
+; CHECK-NEXT:    [[VA_ARG_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 3
+; CHECK-NEXT:    [[VA_ARG_OVERFLOW_SIZE:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 4
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    [[_MSARG2:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 16
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG2]], align 8
+; CHECK-NEXT:    [[_MSARG3:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 24
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG3]], align 8
+; CHECK-NEXT:    [[_MSARG4:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 32
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG4]], align 8
+; CHECK-NEXT:    [[_MSARG5:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 40
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG5]], align 8
+; CHECK-NEXT:    [[_MSARG6:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 48
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG6]], align 8
+; CHECK-NEXT:    [[_MSARG7:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 56
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG7]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S8:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 8
+; CHECK-NEXT:    [[_MSARG_VA_O9:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S8]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O9]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S10:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 16
+; CHECK-NEXT:    [[_MSARG_VA_O11:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 16
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S10]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O11]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S12:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 24
+; CHECK-NEXT:    [[_MSARG_VA_O13:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 24
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S12]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O13]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S14:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 32
+; CHECK-NEXT:    [[_MSARG_VA_O15:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 32
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S14]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O15]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S16:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 40
+; CHECK-NEXT:    [[_MSARG_VA_O17:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 40
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S16]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O17]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S18:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 176
+; CHECK-NEXT:    [[_MSARG_VA_O19:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 176
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S18]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O19]], align 8
+; CHECK-NEXT:    [[_MSARG_VA_S20:%.*]] = getelementptr i8, ptr [[VA_ARG_SHADOW]], i64 184
+; CHECK-NEXT:    [[_MSARG_VA_O21:%.*]] = getelementptr i8, ptr [[VA_ARG_ORIGIN]], i64 184
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_S20]], align 8
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG_VA_O21]], align 8
+; CHECK-NEXT:    store i64 16, ptr [[VA_ARG_OVERFLOW_SIZE]], align 8
+; CHECK-NEXT:    call void (ptr, ...) @callee_varargs(ptr [[FMT]], i64 [[X]], i64 [[X]], i64 [[X]], i64 [[X]], i64 [[X]], i64 [[X]], i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void (ptr, ...) @callee_varargs(ptr %fmt, i64 %x, i64 %x, i64 %x, i64 %x, i64 %x, i64 %x, i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; The function does not read the shadow of the values returned by the calls:
+; nothing is emitted after a call, and a call without arguments needs no
+; context at all.
+define i64 @return_value(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define i64 @return_value(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @callee()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[R:%.*]] = call i64 @callee_ret(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret i64 [[R]]
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee()
+  %r = call i64 @callee_ret(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret i64 %r
+}
+
+; Indirect calls are calls.
+define void @indirect_call(ptr %fn, i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @indirect_call(
+; CHECK-SAME: ptr [[FN:%.*]], i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void [[FN]](i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void %fn(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; Loads, stores, allocas, memory intrinsics, other inline asm and calls marked
+; !nosanitize are left alone inside a region.
+define i64 @not_calls(ptr %p, i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define i64 @not_calls(
+; CHECK-SAME: ptr [[P:%.*]], i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    [[LOCAL:%.*]] = alloca i64, align 8
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    store i64 [[X]], ptr [[LOCAL]], align 8
+; CHECK-NEXT:    [[V:%.*]] = load i64, ptr [[P]], align 8
+; CHECK-NEXT:    store i64 [[V]], ptr [[P]], align 8
+; CHECK-NEXT:    call void @llvm.memset.p0.i64(ptr [[P]], i8 0, i64 8, i1 false)
+; CHECK-NEXT:    call void asm sideeffect "nop", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[W:%.*]] = call i64 asm "movq $1, $0", "=r,r,~{dirflag},~{fpsr},~{flags}"(i64 [[X]])
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]]), !nosanitize [[META1:![0-9]+]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret i64 [[W]]
+;
+entry:
+  %local = alloca i64
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  store i64 %x, ptr %local
+  %v = load i64, ptr %p
+  store i64 %v, ptr %p
+  call void @llvm.memset.p0.i64(ptr %p, i8 0, i64 8, i1 false)
+  call void asm sideeffect "nop", "~{dirflag},~{fpsr},~{flags}"()
+  %w = call i64 asm "movq $1, $0", "=r,r,~{dirflag},~{fpsr},~{flags}"(i64 %x)
+  call void @callee_i64(i64 %x), !nosanitize !0
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret i64 %w
+}
+
+; An argument that is too big for the parameter TLS (800 bytes) is skipped,
+; like in a fully instrumented function.
+declare void @callee_huge([101 x i64], i64)
+define void @huge_arg([101 x i64] %a, i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @huge_arg(
+; CHECK-SAME: [101 x i64] [[A:%.*]], i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @callee_huge([101 x i64] [[A]], i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_huge([101 x i64] %a, i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; Invokes are handled like calls.
+define void @invoke_in_region(i64 %x) disable_sanitizer_instrumentation personality ptr @__gxx_personality_v0 {
+; CHECK-LABEL: define void @invoke_in_region(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1]] personality ptr @__gxx_personality_v0 {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    invoke void @callee_i64(i64 [[X]])
+; CHECK-NEXT:            to label %[[CONT:.*]] unwind label %[[LPAD:.*]]
+; CHECK:       [[CONT]]:
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+; CHECK:       [[LPAD]]:
+; CHECK-NEXT:    [[LP:%.*]] = landingpad { ptr, i32 }
+; CHECK-NEXT:            cleanup
+; CHECK-NEXT:    resume { ptr, i32 } [[LP]]
+;
+entry:
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  invoke void @callee_i64(i64 %x)
+  to label %cont unwind label %lpad
+
+cont:
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+
+lpad:
+  %lp = landingpad { ptr, i32 }
+  cleanup
+  resume { ptr, i32 } %lp
+}
+
+; The markers are recognized by the token being a part of the asm template, so
+; that they can be combined with other annotations, e.g. the nop and the
+; .discard.annotate_insn entry that objtool looks for.
+define void @marker_substring(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @marker_substring(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "771: nop\0A\09# kmsan.instrumentation.begin\0A\09.pushsection .discard.annotate_insn,\22M\22, @progbits, 8\0A\09.long 771b - ., ${0:c}\0A\09.popsection\0A\09", "i,~{dirflag},~{fpsr},~{flags}"(i32 3)
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "772: nop\0A\09# kmsan.instrumentation.end\0A\09.pushsection .discard.annotate_insn,\22M\22, @progbits, 8\0A\09.long 772b - ., ${0:c}\0A\09.popsection\0A\09", "i,~{dirflag},~{fpsr},~{flags}"(i32 4)
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "771: nop\0A\09# kmsan.instrumentation.begin\0A\09.pushsection .discard.annotate_insn,\22M\22, @progbits, 8\0A\09.long 771b - ., ${0:c}\0A\09.popsection\0A\09", "i,~{dirflag},~{fpsr},~{flags}"(i32 3)
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "772: nop\0A\09# kmsan.instrumentation.end\0A\09.pushsection .discard.annotate_insn,\22M\22, @progbits, 8\0A\09.long 772b - ., ${0:c}\0A\09.popsection\0A\09", "i,~{dirflag},~{fpsr},~{flags}"(i32 4)
+  call void @callee_i64(i64 %x)
+  ret void
+}
+
+; Asm that only looks similar does not open a region.
+define void @not_a_marker(i64 %x) disable_sanitizer_instrumentation {
+; CHECK-LABEL: define void @not_a_marker(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR1]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    call void asm sideeffect "# instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void asm sideeffect "# kmsan instrumentation begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void asm sideeffect "# instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void asm sideeffect "# kmsan instrumentation begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; In a function that is instrumented as a whole the markers mean nothing: the
+; context is fetched once in the prologue and every call is instrumented.
+define void @sanitized(i64 %x) sanitize_memory {
+; CHECK-LABEL: define void @sanitized(
+; CHECK-SAME: i64 [[X:%.*]]) #[[ATTR2:[0-9]+]] {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[RETVAL_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 1
+; CHECK-NEXT:    [[VA_ARG_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 2
+; CHECK-NEXT:    [[VA_ARG_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 3
+; CHECK-NEXT:    [[VA_ARG_OVERFLOW_SIZE:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 4
+; CHECK-NEXT:    [[PARAM_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 5
+; CHECK-NEXT:    [[RETVAL_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 6
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    [[TMP1:%.*]] = load i64, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    [[_MSARG_O:%.*]] = getelementptr i8, ptr [[PARAM_ORIGIN]], i64 0
+; CHECK-NEXT:    [[TMP2:%.*]] = load i32, ptr [[_MSARG_O]], align 4
+; CHECK-NEXT:    call void @llvm.donothing()
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 [[TMP1]], ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    [[_MSARG_O2:%.*]] = getelementptr i8, ptr [[PARAM_ORIGIN]], i64 0
+; CHECK-NEXT:    store i32 [[TMP2]], ptr [[_MSARG_O2]], align 4
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[_MSARG3:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 [[TMP1]], ptr [[_MSARG3]], align 8
+; CHECK-NEXT:    [[_MSARG_O4:%.*]] = getelementptr i8, ptr [[PARAM_ORIGIN]], i64 0
+; CHECK-NEXT:    store i32 [[TMP2]], ptr [[_MSARG_O4]], align 4
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+; Likewise in a no_sanitize("kernel-memory") function, which already passes
+; clean shadow to all its callees.
+define void @no_sanitize(i64 %x) {
+; CHECK-LABEL: define void @no_sanitize(
+; CHECK-SAME: i64 [[X:%.*]]) {
+; CHECK-NEXT:  [[ENTRY:.*:]]
+; CHECK-NEXT:    [[TMP0:%.*]] = call ptr @__msan_get_context_state()
+; CHECK-NEXT:    [[PARAM_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 0
+; CHECK-NEXT:    [[RETVAL_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 1
+; CHECK-NEXT:    [[VA_ARG_SHADOW:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 2
+; CHECK-NEXT:    [[VA_ARG_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 3
+; CHECK-NEXT:    [[VA_ARG_OVERFLOW_SIZE:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 4
+; CHECK-NEXT:    [[PARAM_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 5
+; CHECK-NEXT:    [[RETVAL_ORIGIN:%.*]] = getelementptr { [100 x i64], [100 x i64], [100 x i64], [100 x i64], i64, [200 x i32], i32, i32 }, ptr [[TMP0]], i32 0, i32 6
+; CHECK-NEXT:    call void @llvm.donothing()
+; CHECK-NEXT:    [[_MSARG:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG]], align 8
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    [[_MSARG1:%.*]] = getelementptr i8, ptr [[PARAM_SHADOW]], i64 0
+; CHECK-NEXT:    store i64 0, ptr [[_MSARG1]], align 8
+; CHECK-NEXT:    call void @callee_i64(i64 [[X]])
+; CHECK-NEXT:    call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+; CHECK-NEXT:    ret void
+;
+entry:
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.begin", "~{dirflag},~{fpsr},~{flags}"()
+  call void @callee_i64(i64 %x)
+  call void asm sideeffect "# kmsan.instrumentation.end", "~{dirflag},~{fpsr},~{flags}"()
+  ret void
+}
+
+!0 = !{}
+;.
+; CHECK: [[META1]] = !{}
+;.



More information about the cfe-commits mailing list