[llvm] [IR] Introduce intrinsics to model allocator provenance semantics (PR #228048)
via llvm-commits
llvm-commits at lists.llvm.org
Thu Oct 1 06:48:37 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-tablegen
Author: Antonio Frighetto (antoniofrighetto)
<details>
<summary>Changes</summary>
The provenance semantics of memory allocators is reasonably well-defined when these allocators are opaque to LLVM, however, it is left to be specified once their implementation become visible to the optimizer, and the allocator and its deallocator may get inlined asymmetrically. Add the `llvm.provenance.alloc` and `llvm.provenance.dealloc` intrinsics in an attempt to describe how memory allocators create and destroy provenance.
In the context of GPU targets, such intrinsics may be also used standalone to split one larger allocation into multiple separated sub-allocations.
Besides, the semantics of `noalias` in return position is further clarified to be a statement about provenance only: optimizer assumptions about the address of an allocation are now expressed separately via the new `address_unpredictable` and `alloc_disjoint` allockind properties.
Co-authored-by: Nikita Popov <npopov@<!-- -->redhat.com>
Previous discussion: https://discourse.llvm.org/t/rfc-allocator-provenance-model/91106.
---
Patch is 45.45 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/228048.diff
22 Files Affected:
- (modified) llvm/docs/LangRef.md (+176-4)
- (modified) llvm/docs/ReleaseNotes.md (+6)
- (modified) llvm/include/llvm/Analysis/MemoryBuiltins.h (+18)
- (modified) llvm/include/llvm/IR/Attributes.h (+4-1)
- (modified) llvm/include/llvm/IR/Intrinsics.td (+47)
- (modified) llvm/lib/Analysis/BasicAliasAnalysis.cpp (+7)
- (modified) llvm/lib/Analysis/MemoryBuiltins.cpp (+22)
- (modified) llvm/lib/Analysis/MemoryLocation.cpp (+9)
- (modified) llvm/lib/AsmParser/LLParser.cpp (+6)
- (modified) llvm/lib/CodeGen/PreISelIntrinsicLowering.cpp (+9)
- (modified) llvm/lib/IR/Attributes.cpp (+6)
- (modified) llvm/lib/IR/Verifier.cpp (+8-3)
- (modified) llvm/lib/Transforms/Scalar/DeadStoreElimination.cpp (+6-2)
- (modified) llvm/lib/Transforms/Utils/BuildLibCalls.cpp (+10)
- (added) llvm/test/Analysis/BasicAA/provenance-dealloc.ll (+106)
- (added) llvm/test/Assembler/allockind-provenance-intrinsics.ll (+24)
- (modified) llvm/test/Bitcode/compatibility.ll (+14-2)
- (added) llvm/test/Transforms/DeadStoreElimination/provenance-dealloc.ll (+75)
- (modified) llvm/test/Verifier/allockind.ll (+7-1)
- (modified) llvm/utils/TableGen/Basic/CodeGenIntrinsics.cpp (+26)
- (modified) llvm/utils/TableGen/Basic/CodeGenIntrinsics.h (+14)
- (modified) llvm/utils/TableGen/Basic/IntrinsicEmitter.cpp (+26)
``````````diff
diff --git a/llvm/docs/LangRef.md b/llvm/docs/LangRef.md
index ee7068b2125cc..6d011ce62b003 100644
--- a/llvm/docs/LangRef.md
+++ b/llvm/docs/LangRef.md
@@ -1538,7 +1538,13 @@ Currently, only the following parameter attributes are defined:
when used on function arguments. On function return values, the `noalias`
attribute indicates that the function acts like a system memory allocation
function, returning a pointer to allocated storage disjoint from the
- storage for any other object accessible to the caller.
+ storage for any other object accessible to the caller. More specifically,
+ this latter statement is about provenance only: it does not imply anything
+ about the address of the returned pointer, which may coincide with that of,
+ e.g., an alloca, a global or another allocation (more at
+ {ref}`llvm.provenance.alloc <int_provenance_alloc>`). Address assumptions
+ may be expressed via "address_unpredictable" and "alloc_disjoint" options
+ of `allockind`.
(captures_attr)=
@@ -2139,7 +2145,8 @@ define void @f() "no-sse" { ... }
will match that of the `allocptr` argument and the `allocptr`
argument is invalidated, even if the function returns the same address.
* "free": the function frees the block of memory specified by `allocptr`.
- Functions marked as "free" `allockind` must return void.
+ Functions marked as "free" `allockind` that also carry "poisons_memory"
+ must return void.
* "uninitialized": Any newly-allocated memory (either a new block from
a "alloc" function or the enlarged capacity from a "realloc" function)
will be uninitialized.
@@ -2148,10 +2155,29 @@ define void @f() "no-sse" { ... }
zeroed.
* "aligned": the function returns memory aligned according to the
`allocalign` parameter.
+ * "address_unpredictable": the address of the returned allocation cannot
+ be predicted by the caller. Comparisons of the address with pointers not
+ derived from the allocation may assume the two are unequal, provided this
+ is done consistently for all observations of the address. This does not
+ hold for allocators that return storage at an address that is derived from
+ their inputs (more at {ref}`llvm.provenance.alloc <int_provenance_alloc>`).
+ * "alloc_disjoint": the returned allocation does not overlap with storage
+ of allocas, global variables, `byval` arguments, or other allocations.
+ This only holds for top-level allocators which are not nested within
+ another allocator.
+ * "poisons_memory": the contents of the freed region are not observable
+ after the free call site, meaning that stores to such a region prior to
+ the call may be eliminated. This is only valid for "free". Deallocators
+ without this property do preserve the contents, which remain observable
+ through the allocation it originated from, and they may therefore return
+ a pointer to the freed memory (more at {ref}`llvm.provenance.dealloc
+ <int_provenance_dealloc>`).
The first three options are mutually exclusive, and the remaining options
- describe more details of how the function behaves. The remaining options
- are invalid for "free"-type functions.
+ describe more details of how the function behaves. Except for "poisons_memory",
+ the remaining options are invalid for "free"-type functions. Top-level
+ allocators such as `malloc` and `free` are treated as having respectively
+ "address_unpredictable" and "alloc_disjoint", and "poisons_memory".
Calls to functions annotated with `allockind` are subject to allocation
elision: Calls to allocator functions can be removed, and the allocation
@@ -24477,6 +24503,152 @@ for the purposes of `load`/`store` `invariant.group` metadata.
It does not read any accessible memory and the execution can be speculated.
+(int_provenance_alloc_intrinsics)=
+
+### Allocator Provenance Intrinsics
+
+The `llvm.provenance.alloc` and `llvm.provenance.dealloc` intrinsics describe
+the provenance semantics of allocators whose implementation may be visible to
+the optimizer. Informally, memory allocation *creates* pointer provenance, and
+deallocation *destroys* it. When only the allocator declaration is visible to
+LLVM, its attributes imply this. Conversely, when the implementation of an
+allocator and its deallocator is visible too (and they may thus be inlined
+asymmetrically), these intrinsics mark such provenance changes explicitly.
+
+Allocator provenance forms a tree: each allocation adds a new leaf, which
+remembers the provenance of the parent allocation it was derived from, and each
+deallocation removes a leaf, returning a pointer with the parent provenance
+through which the memory is accessible again. Only a leaf has full access to the
+memory range it covers.
+
+Allocators are expected to call `llvm.provenance.alloc` on the pointer they return,
+and deallocators to call `llvm.provenance.dealloc` on the pointer they receive:
+
+```llvm
+define noalias ptr @my_alloc(i64 %size) allockind("alloc") "alloc-family"="my_alloc" {
+ %p = call ptr @get_memory_chunk(i64 %size) ; Obtain memory.
+ %p.alloc = call ptr @llvm.provenance.alloc.p0(ptr %p, i64 %size)
+ ret ptr %p.alloc
+}
+
+define void @my_free(ptr allocptr %p.alloc) allockind("free") "alloc-family"="my_alloc" {
+ %p = call ptr @llvm.provenance.dealloc.p0(ptr %p.alloc)
+ ; Do something with the freed memory, e.g., put it in a free list.
+ call void @push_to_freelist(ptr %p)
+ ret void
+}
+```
+
+The parent allocation may be any allocated object, including a global variable or
+an alloca. The intrinsics may thus also be used standalone to split one larger
+allocation into separate allocated objects, as shown in the following example:
+
+```llvm
+ at pool = internal global [16 x i8] zeroinitializer
+
+; Two separate allocated objects obtained from @pool. Accesses through %a and
+; %b do not alias each other, nor do they alias accesses through @pool.
+%a = call ptr @llvm.provenance.alloc.p0(ptr @pool, i64 8)
+%b = call ptr @llvm.provenance.alloc.p0(ptr getelementptr (i8, ptr @pool, i64 8), i64 8)
+```
+
+It is undefined behavior to deallocate a parent allocation while a child
+allocation created from it is still live.
+
+(int_provenance_alloc)=
+
+#### '`llvm.provenance.alloc`' Intrinsic
+
+##### Syntax:
+
+This is an overloaded intrinsic. The pointer argument can belong to any address
+space. The returned pointer belongs to the same address space as the argument.
+
+```
+declare noalias ptr @llvm.provenance.alloc.p0(ptr <ptr_orig>, i64 <size>)
+ nofree nosync nocallback nounwind willreturn
+ memory(argmem: readwrite, inaccessiblemem: readwrite)
+ allockind("alloc") allocsize(1) "alloc-family"="provenance-alloc"
+```
+
+##### Overview:
+
+The '`llvm.provenance.alloc`' intrinsic creates a new allocated object of `size`
+bytes at the address of `ptr_orig`, sub-allocated from the parent allocation
+`ptr_orig` points into. Allocators should call it on the pointer they return.
+
+##### Arguments:
+
+The first argument is a pointer into the parent allocation. The second argument
+is the size in bytes of the new allocated object.
+
+##### Semantics:
+
+* If `ptr_orig` is a null pointer, a null pointer is returned.
+* The provenance of `ptr_orig` (the provenance of the *parent* allocator) must
+ have access permissions for `size` bytes starting at `ptr_orig`, otherwise
+ the behavior is undefined.
+* The access permissions for `size` bytes starting at `ptr_orig` are *masked*
+ in the parent allocator provenance, including in any pointer based on it.
+ Masked bytes behave similarly to a stack object outside its {ref}`lifetime
+ <objectlifetime>`: through the parent provenance, stores to them are undefined
+ behavior, and loads return poison (rather than being undefined behavior). As a
+ result, the parent allocation remains dereferenceable (loads from it may still
+ be speculated), and the intrinsic does not free memory.
+* The result is a pointer to a new allocated object of `size` bytes at the same
+ address as `ptr_orig`, with fresh provenance that has access permissions for
+ exactly those `size` bytes.
+
+Hence, the address of the returned pointer may be known, and the new object may
+overlap allocas, or global variables. Particularly, unlike other `noalias`
+allocations, it is neither "address_unpredictable" nor "alloc_disjoint"
+({ref}`allockind <fnattrs>`). The parent pointer argument `ptr_orig` is captured by
+the call: its provenance is retained and later returned by `llvm.provenance.dealloc`.
+
+(int_provenance_dealloc)=
+
+#### '`llvm.provenance.dealloc`' Intrinsic
+
+##### Syntax:
+
+This is an overloaded intrinsic. The pointer argument can belong to any address
+space. The returned pointer belongs to the same address space as the argument.
+
+```
+declare ptr @llvm.provenance.dealloc.p0(ptr allocptr captures(address) <ptr>)
+ nosync nocallback nounwind willreturn
+ memory(argmem: readwrite, inaccessiblemem: readwrite)
+ allockind("free") "alloc-family"="provenance-alloc"
+```
+
+##### Overview:
+
+The '`llvm.provenance.dealloc`' intrinsic destroys an allocated object created by
+`llvm.provenance.alloc` and returns a pointer with the provenance of its parent
+allocation.
+
+##### Arguments:
+
+The argument is a pointer with allocator provenance, i.e., a pointer returned by
+`llvm.provenance.alloc`, or by an allocation function whose implementation is
+assumed to perform `llvm.provenance.alloc` on its result. Only the address of the
+argument pointer is captured, as its provenance ends with the call.
+
+##### Semantics:
+
+* If `ptr` is a null pointer, a null pointer is returned.
+* `ptr` must have allocator provenance, otherwise the behavior is undefined.
+* The previously allocated object `ptr` is destroyed: its provenance is *destroyed*,
+ i.e., all its access permissions are disabled, so any access through it (or any
+ pointer derived from it) result in undefined behavior. The masked access
+ permissions for the bytes covered by the allocation are restored in the parent
+ allocator provenance.
+* The result is a pointer with the provenance of the parent allocator.
+
+Because the returned pointer carries the parent provenance, the freed memory may
+still be used through it, e.g., to place the memory object into a free list. This
+likewise allows reallocation to be represented as `llvm.provenance.dealloc` on
+entry to the reallocator followed by `llvm.provenance.alloc` on return.
(constrainedfp)=
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index b262a2a4453d7..5cdb9be3cb1a0 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -172,6 +172,12 @@ Makes programs 10x faster by doing Special New Thing.
* Added the `bitinsert` and `bitextract` instructions for bit-range
manipulation on byte type values.
+* Introduced `llvm.provenance.alloc` and `llvm.provenance.dealloc` intrinsics
+ to model allocator semantics, as well as new `address_unpredictable`,
+ `alloc_disjoint` and `poisons_memory` `allockind` options. `allockind("free")`
+ no longer implies that the freed memory is discarded, you may want to use
+ `allockind("free,poisons_memory")` for that instead.
+
### Changes to LLVM infrastructure
* Removed `TargetOptions::FloatABIType`. The soft float ABI should be
diff --git a/llvm/include/llvm/Analysis/MemoryBuiltins.h b/llvm/include/llvm/Analysis/MemoryBuiltins.h
index ebe644cee53e1..656bd5eb42786 100644
--- a/llvm/include/llvm/Analysis/MemoryBuiltins.h
+++ b/llvm/include/llvm/Analysis/MemoryBuiltins.h
@@ -80,6 +80,11 @@ LLVM_ABI bool isLibFreeFunction(const Function *F, const LibFunc TLIFn);
LLVM_ABI Value *getFreedOperand(const CallBase *CB,
const TargetLibraryInfo *TLI);
+/// Return true if \p V is a call to a free function that discards the contents
+/// of the freed memory, so that stores prior to the call are dead (e.g., known
+/// free functions and those annotated as allockind("free,poisons_memory")).
+LLVM_ABI bool isPoisonMemoryFree(const Value *V, const TargetLibraryInfo *TLI);
+
//===----------------------------------------------------------------------===//
// Properties of allocation functions
//
@@ -95,6 +100,19 @@ LLVM_ABI Value *getFreedOperand(const CallBase *CB,
/// insertion or speculative execution of allocation routines.
LLVM_ABI bool isRemovableAlloc(const CallBase *V, const TargetLibraryInfo *TLI);
+/// Return true if \p V is a call to an allocation function whose returned
+/// address is unpredictable to the caller, so that comparisons of the address
+/// with pointers not based on the allocation may be folded (e.g., known
+/// allocators and routines annotated as allockind("address_unpredictable")).
+LLVM_ABI bool isAddressUnpredictableAlloc(const Value *V,
+ const TargetLibraryInfo *TLI);
+
+/// Return true if \p V is a call to an allocation function whose returned
+/// allocation is known not to overlap allocas, globals, byval arguments or
+/// other allocations (e.g., known allocators and routines annotated as
+/// allockind("alloc_disjoint")).
+LLVM_ABI bool isDisjointAlloc(const Value *V, const TargetLibraryInfo *TLI);
+
/// Gets the alignment argument for an aligned_alloc-like function, using either
/// built-in knowledge based on fuction names/signatures or allocalign
/// attributes. Note: the Value returned may not indicate a valid alignment, per
diff --git a/llvm/include/llvm/IR/Attributes.h b/llvm/include/llvm/IR/Attributes.h
index 7ff7def7525fa..2dae4d3e153ac 100644
--- a/llvm/include/llvm/IR/Attributes.h
+++ b/llvm/include/llvm/IR/Attributes.h
@@ -60,7 +60,10 @@ enum class AllocFnKind : uint64_t {
Zeroed = 1 << 4, // Allocator function returns zeroed memory
Aligned = 1 << 5, // Allocator function aligns allocations per the
// `allocalign` argument
- LLVM_MARK_AS_BITMASK_ENUM(/* LargestValue = */ Aligned)
+ AddressUnpredictable = 1 << 6, // Address of the allocation is unobservable
+ AllocDisjoint = 1 << 7, // Storage is disjoint from allocas, globals, et alia
+ PoisonsMemory = 1 << 8, // Free functions do poison the freed memory
+ LLVM_MARK_AS_BITMASK_ENUM(/* LargestValue = */ PoisonsMemory)
};
class DeadOnReturnInfo {
diff --git a/llvm/include/llvm/IR/Intrinsics.td b/llvm/include/llvm/IR/Intrinsics.td
index 52be391132b7e..197f970c7bde1 100644
--- a/llvm/include/llvm/IR/Intrinsics.td
+++ b/llvm/include/llvm/IR/Intrinsics.td
@@ -101,6 +101,18 @@ class NoCapture<ArgIndex idx> : IntrinsicProperty {
int ArgNo = idx.Value;
}
+// CapturesAddress - The address of the specified pointer argument may be
+// captured.
+class CapturesAddress<ArgIndex idx> : IntrinsicProperty {
+ int ArgNo = idx.Value;
+}
+
+// AllocatedPointer - The specified argument is the pointer that will be
+// manipulated by the allocator.
+class AllocatedPointer<ArgIndex idx> : IntrinsicProperty {
+ int ArgNo = idx.Value;
+}
+
// NoAlias - The return value or the specified argument pointer is not aliasing
// other "noalias" pointer arguments of the intrinsic wrt. the intrinsic scope.
class NoAlias<AttrIndex idx> : IntrinsicProperty {
@@ -275,6 +287,23 @@ def IntrNoCreateUndefOrPoison : IntrinsicProperty;
// This property indicates that the intrinsic is trivially scalarizable.
def IntrTriviallyScalarizable : IntrinsicProperty;
+// IntrAllocKind - The intrinsic is an allocator function with the given
+// allockind property string.
+class IntrAllocKind<string kind> : IntrinsicProperty {
+ string Kind = kind;
+}
+
+// IntrAllocSize - The specified argument is the size in bytes of the allocation
+// returned by the intrinsic.
+class IntrAllocSize<ArgIndex idx> : IntrinsicProperty {
+ int ArgNo = idx.Value;
+}
+
+// IntrAllocFamily - The alloc-family the intrinsic belongs to.
+class IntrAllocFamily<string name> : IntrinsicProperty {
+ string Name = name;
+}
+
//===----------------------------------------------------------------------===//
// IIT constants and utils
//===----------------------------------------------------------------------===//
@@ -1918,6 +1947,24 @@ def int_launder_invariant_group : DefaultAttrsIntrinsic<[llvm_anyptr_ty],
[IntrInaccessibleMemOnly,
IntrSpeculatable]>;
+//===--------------------- Allocator Provenance Intrinsics ----------------===//
+//
+def int_provenance_alloc
+ : DefaultAttrsIntrinsic<[llvm_anyptr_ty], [LLVMMatchType<0>, llvm_i64_ty],
+ [IntrInaccessibleMemOrArgMemOnly,
+ NoAlias<RetIndex>,
+ IntrAllocKind<"alloc">,
+ IntrAllocSize<ArgIndex<1>>,
+ IntrAllocFamily<"provenance-alloc">]>;
+
+def int_provenance_dealloc
+ : Intrinsic<[llvm_anyptr_ty], [LLVMMatchType<0>],
+ [IntrInaccessibleMemOrArgMemOnly, IntrNoSync, IntrNoCallback,
+ IntrWillReturn,
+ CapturesAddress<ArgIndex<0>>, AllocatedPointer<ArgIndex<0>>,
+ IntrAllocKind<"free">,
+ IntrAllocFamily<"provenance-alloc">]>;
+
//===------------------------ Stackmap Intrinsics -------------------------===//
//
def int_experimental_stackmap : DefaultAttrsIntrinsic<[],
diff --git a/llvm/lib/Analysis/BasicAliasAnalysis.cpp b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
index 3aece3ca7d769..98a46f146ac12 100644
--- a/llvm/lib/Analysis/BasicAliasAnalysis.cpp
+++ b/llvm/lib/Analysis/BasicAliasAnalysis.cpp
@@ -892,6 +892,13 @@ MemoryEffects BasicAAResult::getMemoryEffects(const Function *F) {
// inaccessible memory to model control dependence.
return MemoryEffects::readOnly() |
MemoryEffects::inaccessibleMemOnly(ModRefInfo::ModRef);
+ case Intrinsic::provenance_dealloc:
+ // The effects of dealloc also extend to the parent allocation, whose
+ // access permissions it restores. The parent is not memory accessible
+ // through the argument (as the latter carries the child provenance), and
+ // thus it is encompassed in other memory.
+ return F->getMemoryEffects() |
+ MemoryEffects::otherMemOnly(ModRefInfo::ModRef);
}
return F->getMemoryEffects();
diff --git a/llvm/lib/Analysis/MemoryBuiltins.cpp b/llvm/lib/Analysis/MemoryBuiltins.cpp
index 05316cc805bd1..13c27d342a7e9 100644
--- a/llvm/lib/Analysis/MemoryBuiltins.cpp
+++ b/llvm/lib/Analysis/MemoryBuiltins.cpp
@@ -329,6 +329,17 @@ bool llvm::isRemovableAlloc(const CallBase *CB, const TargetLibraryInfo *TLI) {
return isAllocLikeFn(CB, TLI);
}
+bool llvm::isAddressUnpredictableAlloc(const Value *V,
+ const TargetLibraryInfo *TLI) {
+ return getAllocationData(V, AnyAlloc, TLI).has_value() ||
+ checkFnAllocKind(V, AllocFnKind::AddressUnpredictable);
+}
+
+bool llvm::isDisjointAlloc(const Value *V, const TargetLibraryInfo *TLI) {
+ return getAllocationData(V, AnyAlloc, TLI).has_value() ||
+ checkFnAllocKind(V, AllocFnKind::AllocDisjoint);
+}
+
Value *llvm::getAllocAlignment(const CallBase *V,
const TargetLibraryInfo *TLI) {
const std::optional<AllocFnsTy> FnData = getAllocationData(V, AnyAlloc, TLI);
@@ -551,6 +562,17 @@ Value *llvm::getFreedOperand(const CallBase *CB, const TargetLibraryInfo *TLI) {
return nullptr;
}
+bool llvm::isPoisonMemoryFree(const Value *V, const TargetLibraryInfo *TLI) {
+ if (const Function *Callee = getCalledFunction(V)) {
+ LibFunc TLIFn = TLI ? TLI->getLibFunc(*Callee) : NotLibFunc;
+ if (TLIFn != NotLibFunc && TLI->has(TLIFn) &&
+ getFreeFunctionDataForFunction(Callee, TLIFn))
+ return true;
+ }
+
+ return checkFnAllocKind(V, AllocFnKind::PoisonsMemory);
+}
+
//===----------------------------------------------------------------------===//
// Utility functions to compute size of objects.
//
diff --git a/llvm/lib/Analysis/MemoryLocation.cpp b/llvm/lib/Analysis/MemoryLocation.cpp
index 715ae20bbaa3d..a2125cc122d77 100644
--- a/llvm/lib/Analysis/MemoryLocation.cpp
+++ b/llvm/lib/Analysis/MemoryLocation.cpp
@@ -316,6 +316,15 @@ MemoryLocation MemoryLocation::getForArgument(const CallBase *Call,
// otherwise we mer...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/228048
More information about the llvm-commits
mailing list