[flang-commits] [flang] [llvm] [flang][docs] Update runtime trampoline documentation (PR #215233)
Sairudra More via flang-commits
flang-commits at lists.llvm.org
Tue Aug 11 02:44:38 PDT 2026
https://github.com/Saieiei updated https://github.com/llvm/llvm-project/pull/215233
>From 93c30e2a19f1149227ab7978f151ab8ac19a530d Mon Sep 17 00:00:00 2001
From: Sairudra More <sairudra60 at gmail.com>
Date: Mon, 10 Aug 2026 04:19:21 -0500
Subject: [PATCH] [flang][docs] Update runtime trampoline documentation
Align trampoline documentation and runtime comments with the current implementation, including the stack and runtime-pool paths and API wording.
---
flang-rt/lib/runtime/trampoline.cpp | 19 +--
flang/docs/Calls.md | 9 +-
flang/docs/FAQ.md | 9 +-
flang/docs/FortranStandardsSupport.md | 2 +-
flang/docs/InternalProcedureTrampolines.md | 185 +++++++--------------
flang/docs/ProcedurePointer.md | 9 +-
flang/include/flang/Runtime/trampoline.h | 17 +-
7 files changed, 92 insertions(+), 158 deletions(-)
diff --git a/flang-rt/lib/runtime/trampoline.cpp b/flang-rt/lib/runtime/trampoline.cpp
index 871c6d22c5c98..23e0cc3547b2e 100644
--- a/flang-rt/lib/runtime/trampoline.cpp
+++ b/flang-rt/lib/runtime/trampoline.cpp
@@ -18,9 +18,10 @@
// a time on any given thread.
//
// Architecture:
-// - Code region (RX): Contains pre-assembled trampoline stubs that load
-// callee address and static chain from a paired TDATA entry, then jump
-// to the callee with the static chain in the appropriate register.
+// - Code region (RX): Contains trampoline stubs generated during pool
+// initialization that load the callee address and static chain from a
+// paired TDATA entry, then jump to the callee with the static chain in the
+// appropriate register.
// - Data region (RW): Contains TrampolineData entries with {callee_address,
// static_chain_address} pairs, one per trampoline slot.
// - Free list: Tracks available trampoline slots for O(1) alloc/free.
@@ -28,9 +29,6 @@
// Thread safety: Uses Fortran::runtime::Lock (pthreads on POSIX,
// CRITICAL_SECTION on Windows) — not std::mutex — to avoid C++ runtime
// library dependence. A single global lock serializes pool operations.
-// This is a deliberate V1 design choice to keep the initial W^X
-// architectural change minimal. Per-thread lock-free pools are deferred
-// to a future optimization patch.
//
// AddressSanitizer note: The trampoline code region is allocated via
// mmap (not malloc/new), so ASan does not track it. The data region
@@ -138,10 +136,7 @@ class TrampolinePool {
ensureInitialized();
if (freeHead_ == kInvalidIndex) {
- // Pool exhausted — fixed size by design for V1.
- // The pool capacity is controlled by FLANG_TRAMPOLINE_POOL_SIZE
- // (default 1024). Dynamic slab growth can be added in a follow-up
- // patch if real workloads demonstrate a need for it.
+ // Pool capacity is fixed after initialization.
Terminator terminator{__FILE__, __LINE__};
terminator.Crash("Trampoline pool exhausted (max %zu slots). "
"Set FLANG_TRAMPOLINE_POOL_SIZE to increase.",
@@ -199,10 +194,6 @@ class TrampolinePool {
initialized_ = true;
// Check environment variable for pool size override.
- // Fixed-size pool by design (V1): avoids complexity of dynamic growth
- // and re-protection of code pages. The default (1024 slots) is
- // sufficient for typical Fortran programs. Users can override via:
- // export FLANG_TRAMPOLINE_POOL_SIZE=4096
if (const char *envSize = std::getenv("FLANG_TRAMPOLINE_POOL_SIZE")) {
long val{std::strtol(envSize, nullptr, 10)};
if (val > 0) {
diff --git a/flang/docs/Calls.md b/flang/docs/Calls.md
index f27af1a461bac..83515aa5cc410 100644
--- a/flang/docs/Calls.md
+++ b/flang/docs/Calls.md
@@ -529,10 +529,11 @@ PGI passes host instance links in descriptors in additional arguments
that are not always successfully forwarded across implicit interfaces,
sometimes leading to crashes when they turn out to be needed.
-Flang will manage a pool of trampolines in its runtime support library
-that can be used to pass internal procedures as effective arguments
-to F77ish procedures, so that
-a bare code address can serve to represent the effective argument.
+By default, Flang uses a stack-allocated trampoline to pass an internal
+procedure as an effective argument to an F77ish procedure. When
+`-fsafe-trampoline` is enabled on a supported target, Flang instead uses a
+fixed-capacity pool of trampolines in its runtime support library. In either
+case, a bare code address can serve to represent the effective argument.
But targets that can only be called with an explicit interface
have the option of using a "fat pointer" (or additional argument)
to represent a dummy procedure closure so as
diff --git a/flang/docs/FAQ.md b/flang/docs/FAQ.md
index c220d469380ce..05e88a71bcc76 100644
--- a/flang/docs/FAQ.md
+++ b/flang/docs/FAQ.md
@@ -18,14 +18,14 @@ local:
### Why do I get a warning or an error about an executable stack?
-This occurs because Flang's implementation of pointers to internal procedures requires an executable stack.
+This occurs because Flang's default implementation of pointers to internal procedures requires an executable stack.
An internal procedure has a "host scope", which is the scope in which it is contained.
It can access variables defined in that host scope.
When an internal procedure is referenced from outside its host scope (for example, via a procedure pointer), the implementation must ensure that it can still access variables from the host scope.
-To achieve this, the current implementation of Flang generates a small piece of code, called a "trampoline", on the stack.
+By default, Flang achieves this by generating a small piece of code, called a "trampoline", on the stack.
When the procedure is called, this trampoline is executed.
-The trampoline is on the stack, so the stack itself must be executable.
+Since the trampoline is on the stack, the stack itself must be executable.
For a more detailed explanation of trampolines, please refer to the [design document](InternalProcedureTrampolines.md).
An executable stack increases the risk and impact of certain classes of security vulnerabilities, such as [stack buffer overflows](https://llsoftsec.github.io/llsoftsecbook/#stack-buffer-overflows).
@@ -52,5 +52,6 @@ $ flang src.f90 -fuse-ld=lld -Wl,-z,execstack
```
Since LLVM 23, Flang has a new compiler flag `-fsafe-trampoline`, which is disabled by default and is currently supported on x86-64 and AArch64 targets.
-If this is enabled, the trampoline is generated on a dedicated `mmap`'d region instead of the stack.
+If this is enabled, Flang uses a runtime-managed trampoline pool, with executable code stored separately from writeable per-trampoline data. This does not require an executable stack.
Where supported, we recommend using this flag to reduce the security risk of an executable stack.
+The pool capacity is described in the [runtime environment documentation](RuntimeEnvironment.md).
diff --git a/flang/docs/FortranStandardsSupport.md b/flang/docs/FortranStandardsSupport.md
index cff9f274acabb..32e4a5830f3f7 100644
--- a/flang/docs/FortranStandardsSupport.md
+++ b/flang/docs/FortranStandardsSupport.md
@@ -88,7 +88,7 @@ All features except those listed in the following table are supported.
| Feature | Status | Comments |
|------------------------------------------------------------|--------|---------------------------------------------------------|
| do concurrent | Y | See [Do Concurrent Conversion To OpenMP](DoConcurrentConversionToOpenMP.md) for implementation details.|
-| Internal procedure as an actual argument or pointer target | Y | Current implementation requires stack to be executable. See [FAQ](FAQ.md#why-do-i-get-a-warning-or-an-error-about-an-executable-stack) and [Proposal](InternalProcedureTrampolines.md) |
+| Internal procedure as an actual argument or pointer target | Y | The default implementation requires an executable stack; `-fsafe-trampoline` uses a runtime-managed pool on supported targets. See [FAQ](FAQ.md#why-do-i-get-a-warning-or-an-error-about-an-executable-stack) and [design documentation](InternalProcedureTrampolines.md) |
| Intrinsic functions this_image, num_images | P | Experimental support via [PRIF](ParallelMultiImageFortranRuntime.md) |
| Sync all, sync images, sync memory statements | P | Experimental support via [PRIF](ParallelMultiImageFortranRuntime.md) |
| Non-allocatable save coarrays of intrinsic type | P | Experimental support via [PRIF](ParallelMultiImageFortranRuntime.md) |
diff --git a/flang/docs/InternalProcedureTrampolines.md b/flang/docs/InternalProcedureTrampolines.md
index 41f6155332a47..f83b7131b74f9 100644
--- a/flang/docs/InternalProcedureTrampolines.md
+++ b/flang/docs/InternalProcedureTrampolines.md
@@ -182,7 +182,8 @@ Where:
object created inside `host()`.
- `R#` is a target specific register.
-In MLIR LLVM dialect the replacement looks like this:
+With the default stack-trampoline implementation, the replacement in the MLIR
+LLVM dialect looks like this:
```
llvm.call @llvm.init.trampoline(%8, %9, %7) : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> ()
@@ -192,6 +193,9 @@ In MLIR LLVM dialect the replacement looks like this:
```
+When `-fsafe-trampoline` is enabled on a supported target, the pass instead
+uses the runtime API described below.
+
So any call of `fptr` inside `foo()` will result in invocation of the trampoline.
The trampoline will setup `R#` register and jump to `callee()` directly.
@@ -203,10 +207,11 @@ to `callee()` in `R#`:
llvm.func @_QFhostPcallee(%arg0: !llvm.ptr<struct<(ptr<i32>)>> {fir.host_assoc, llvm.nest}) -> i32 attributes {fir.internal_proc} {
```
-#### Trampoline handling
+#### Default stack-trampoline handling
-Currently used [llvm.init.trampoline intrinsic](https://llvm.org/docs/LangRef.html#trampoline-intrinsics)
-expects that the memory for the trampoline content is passed to it as the first argument.
+The default path uses
+the [llvm.init.trampoline intrinsic](https://llvm.org/docs/LangRef.html#trampoline-intrinsics),
+which expects that the memory for the trampoline content is passed to it as the first argument.
The memory has to be writeable at the point of the intrinsic call, and it has to be executable
at any point where `callee()` might be ivoked via the trampoline.
@@ -233,7 +238,7 @@ the second intrinsic was introduced:
> ```
> By the way an example of adjust_trampoline is ARM, which or's a 1 into the address of the trampoline. When the pointer is called the processor sees the 1 and puts itself into thumb mode.
-Currently, the trampolines are allocated on the stack of `host()` subroutine,
+By default, the trampolines are allocated on the stack of `host()` subroutine,
so that they are available throughout the life span of `host()` and are
automatically deallocated at the end of `host()` invocation.
Unfortunately, this requires the program stack to be writeable and executable
@@ -241,14 +246,14 @@ at the same time, which might be a security concern.
> NOTE: LLVM's AArch64 backend supports `nest` attribute, but it requires the compiler-rt runtime selected via the `-rtlib=compiler-rt` flag.
-## Alternative implementation(s)
+## Opt-in runtime trampoline pool
-To address the security risk we may consider managing the trampoline memory
-in a way that it is not writeable and executable at the same time.
-One of the options is to use separate allocations for the trampoline code
-and the trampoline "data".
+On x86-64 and AArch64 targets, `-fsafe-trampoline` selects a Flang runtime
+implementation instead of the LLVM stack-trampoline intrinsics. In this
+implementation, the runtime manages a global, fixed-capacity pool. Trampoline
+code is generated in a separate region that is executable but not writeable
+after initialization, while per-slot data remains writeable but not executable:
-The trampolines may be located in non-writeable executable memory:
```asm
trampoline0:
MOV (TDATA[0].static_chain_address), R#
@@ -259,153 +264,89 @@ trampoline1:
...
```
-The `TDATA` memory is writeable and contains *<static chain address, function address>*
-for each of the trampolines.
-
-A runtime support library may provide APIs for initializing/accessing/deallocating
-the trampolines that can be used by `BoxedProcedure` pass.
-
-### Implementation considerations
-
-* The static chain address still has to be passed in fixed target-specific register,
- and the implementations that rely on LLVM back-ends can use `nest` attribute for this.
-
-* The trampoline area must be able to grow, because there can be a trampoline
- for each internal procedure per host invocation, and an internal procedure can call
- the host recursively. This means that the amount of trampolines in one thread
- may grow pretty quickly.
-
- ```fortran
- recursive subroutine host(local)
- use other
- integer :: local
- call foo(callee)
- return
-
- contains
-
- function callee()
- integer :: callee
- if (local .le. CONST_N) then
- call host(local + 1)
- endif
- end function callee
- end subroutine host
- ```
-
-* On the other hand, putting a hard limit on the number of trampolines live at the same time
- allows putting the trampolines into the static code segment.
+Each `TDATA` entry stores the callee and static-chain addresses for one
+trampoline. The generated stub loads both values, places the static-chain
+address in the target-specific register, and jumps to the callee.
-* Each thread may have its own dynamic trampoline area to reduce the number
- of required locks.
+### Implementation characteristics
-* Some support is required for the offload devices.
+* The pool contains 1024 slots by default. `FLANG_TRAMPOLINE_POOL_SIZE` can set
+ a different capacity, as described in the
+ [runtime environment documentation](RuntimeEnvironment.md).
+* Allocation from a full pool terminates the program with a diagnostic. The
+ pool does not grow dynamically.
+* `TrampolineFree` returns a slot to the synchronized global pool for reuse.
+ The implementation does not use a dynamic trampoline area per thread.
+* Each trampoline invocation loads the static chain and callee addresses from
+ its paired data entry.
-* Each trampoline invocation implies two indirect accesses with this approach.
+### Fortran runtime API
-### Fortran runtime support
-
-The following APIs are suggested:
+The `BoxedProcedure` pass uses these runtime APIs:
```c++
/**
- * \brief Initializes new trampoline and returns its internal handle.
+ * \brief Initializes a new trampoline and returns its internal handle.
*
- * Initializes new trampoline with the given \p callee_address
- * and \p static_chain_address, and returns the new trampoline's
+ * Initializes a new trampoline with the given \p callee_address
+ * and \p static_chain_address, and returns the trampoline's
* internal handle. The compiler calls this method once per host
* invocation for each internal procedure that will need its address
* passed around.
*
- * The initialization is reserving a new entry in TDATA and
- * initializes the entry with the given \p callee_address and
- * \p static_chain_address; it is also reserving a new entry
- * in the trampoline area that is using the corresponding TDATA entry.
- *
- * Optional:
- * \p scratch may be used to switch between the trampoline pool
- * and llvm.init.trampoline implementation, e.g. if compiler passes
- * non-null \p scratch it will be used as a writeable/executable
- * memory for the new trampoline.
+ * \p scratch is reserved and currently ignored. The lowering passes
+ * a null pointer; this argument does not select the default
+ * stack-trampoline implementation.
*/
-const void *InitTrampoline([[maybe_unused]] void *scratch,
- const void *callee_address,
- const void *static_chain_address);
+void *TrampolineInit(void *scratch, const void *callee_address,
+ const void *static_chain_address);
/**
* \brief Returns the trampoline's address for the given handle.
*
- * \p handle is a value returned by InitTrampoline().
- * The result of AdjustTrampoline() is the actual callable
+ * \p handle is a value returned by TrampolineInit().
+ * The result of TrampolineAdjust() is the actual callable
* trampoline's address.
- *
- * Optional: may be implemented via llvm.adjust.trampoline.
*/
-const void *AdjustTrampoline(const void *handle);
+void *TrampolineAdjust(void *handle);
/**
* \brief Frees internal resources occupied for the given trampoline.
*
* The compiler must call this API at every exit from the host function.
- *
- * Optional: may be no-op, if LLVM trampolines are used underneath.
*/
-void FreeTrampoline(void *handle);
+void TrampolineFree(void *handle);
```
-`InitTrampoline` will do the initial allocation of the TDATA memory
-and the trampoline area followed by the initialization of the trampoline
-area with the binary code to "link" the trampolines with the corresponding
-TDATA entries. After the initial allocation the trampoline area is made
-executable and not writeable.
-
-If there is an available entry in the TDATA/trampoline area, then the function
-will initialized the TDATA entry with the given arguments and return
-a handle to the trampoline entry.
-
-`FreeTrampoline` will free the reserved entry.
-
-> NOTE: `FreeTrampoline` may reset the `callee_address` in the trampoline
-being freed to a runtime library function that complains about a dead
-internal procedure being called. This provides some runtime diagnostics
-of dangling procedure pointer usage. Such freed trampolines may still
-have to be reclaimed, if new trampoline is requested and the trampoline
-area is all used.
+`TrampolineInit` initializes the pool on first use, reserves an available slot,
+stores the callee and static chain addresses in its data entry, and returns an
+opaque handle. `TrampolineAdjust` returns the executable code address for that
+slot. `TrampolineFree` invalidates the data entry and returns the slot to the
+pool.
#### Sample IR
```
// Init the trampoline once per host procedure invocation
// (i.e. when the procedure address is emboxed).
- %handle = llvm.call @_FortranAInitTrampoline(%nullptr, %9, %7) : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> !llvm.ptr<i8>
+ %handle = llvm.call @_FortranATrampolineInit(%nullptr, %9, %7) : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> !llvm.ptr<i8>
// Get the actual internal procedure address once per host procedure invocation.
- %10 = llvm.call @_FortranAAdjustTrampoline(%handle) : (!llvm.ptr<i8>) -> !llvm.ptr<i8>
+ %10 = llvm.call @_FortranATrampolineAdjust(%handle) : (!llvm.ptr<i8>) -> !llvm.ptr<i8>
%11 = llvm.bitcast %10 : !llvm.ptr<i8> to !llvm.ptr<func<void ()>>
llvm.call @_QMotherPfoo(%11) {fastmathFlags = #llvm.fastmath<fast>} : (!llvm.ptr<func<void ()>>) -> ()
// The trampoline deallocation must be done only at the exits from the host procedure.
- llvm.call @_FortranAFreeTrampoline(%handle) : (!llvm.ptr<i8>) -> ()
+ llvm.call @_FortranATrampolineFree(%handle) : (!llvm.ptr<i8>) -> ()
```
-### Implementation options
-
-We may try to reuse [libffi](https://github.com/libffi/libffi) implementation for __static trampolines__:
-* Initial implementation added support for x64, i386, aarch64 and arm on Linux: https://github.com/libffi/libffi/pull/624
-* Follow-up patches:
- * Added support for Cygwin: https://github.com/libffi/libffi/commit/a1130f37712c03957c9b0adf316cd006fa92a60b
- * Added support for LoongArch: https://github.com/libffi/libffi/pull/723
- * Page protection for iOS devices: https://github.com/libffi/libffi/pull/718
- * Fix for trampoline code for x32: https://github.com/libffi/libffi/pull/657
-* The author (@madvenka786) initially [proposed](https://sourceware.org/pipermail/libffi-discuss/2021/002587.html) to make the trampoline APIs public,
- but this was not a requirement at the time and the APIs were made private.
- If we want to rely on `libffi`, the APIs have to be made public.
-* We may also try to extract the static trampolines implementation from `libffi`
- into separate library (e.g. `libstatictramp` as mentioned [here](https://sourceware.org/pipermail/libffi-discuss/2021/002592.html)).
-
-Flang's own implementation for trampolines have an advantage that,
-having to support the only Fortran/C interoperable calling convention,
-the implementation may reduce the trampoline overhead. For example,
-it may avoid saving/restoring the scratch registers used by the trampoline code,
-and just clobber some of them according to the particular ABI.
-
-At this point, the recommended approach is to implement the trampoline
-support in Flang runtime.
+The current implementation is self-contained in the Flang runtime.
+Because it needs to support only the Fortran/C interoperable calling convention,
+the implementation may reduce trampoline overhead by clobbering ABI-permitted
+scratch registers rather than saving and restoring them.
+
+### Alternative implementations considered
+
+Alternative implementations were considered but not pursued. Reusing the
+[libffi](https://github.com/libffi/libffi) implementation for __static
+trampolines__ was one option. Another was extracting the static-trampoline
+implementation from `libffi` into a separate library (e.g. `libstatictramp`, as
+mentioned [here](https://sourceware.org/pipermail/libffi-discuss/2021/002592.html)).
diff --git a/flang/docs/ProcedurePointer.md b/flang/docs/ProcedurePointer.md
index ed4381bd23331..c8f6f488088d9 100644
--- a/flang/docs/ProcedurePointer.md
+++ b/flang/docs/ProcedurePointer.md
@@ -277,10 +277,11 @@ due to C721 and C723.
### Procedure pointer to internal procedure
-Initially the current plan is to implement pointers to internal procedures
-using the LLVM Trampoline intrinsics. This has the drawback of requiring the
-stack to be executable, which is a security hole. To avoid this, we will need
-[improve the implementation](InternalProcedureTrampolines.md) to use heap-resident thunks.
+By default, Flang implements pointers to internal procedures using the LLVM
+trampoline intrinsics. This requires the stack to be executable. On x86-64 and
+AArch64 targets, the opt-in `-fsafe-trampoline` mode instead uses a
+runtime-managed trampoline pool that does not require an executable stack. See
+the [trampoline design documentation](InternalProcedureTrampolines.md) for details.
### Procedure pointer assignment `p => proc`
diff --git a/flang/include/flang/Runtime/trampoline.h b/flang/include/flang/Runtime/trampoline.h
index 3322df8b1b340..ae3cadb2d93ee 100644
--- a/flang/include/flang/Runtime/trampoline.h
+++ b/flang/include/flang/Runtime/trampoline.h
@@ -11,9 +11,9 @@
// This provides an alternative to stack-based trampolines for internal
// procedures with host association. Instead of requiring the stack to be
// both writable and executable (violating W^X security policies), this
-// implementation uses a pool of pre-assembled trampolines in a separate
-// executable (but not writable) memory region, paired with writable (but
-// not executable) data entries.
+// implementation uses a pool of trampoline stubs generated during pool
+// initialization in a separate executable (but not writable) memory region,
+// paired with writable (but not executable) data entries.
//
// See flang/docs/InternalProcedureTrampolines.md for design details.
//
@@ -33,15 +33,14 @@ extern "C" {
/// \p calleeAddress with the static chain pointer \p staticChainAddress
/// set in the appropriate register (per target ABI).
///
-/// \p scratch is reserved for future use (e.g., fallback to stack
-/// trampolines). Pass nullptr for pool-based allocation.
+/// \p scratch is currently ignored, and Flang lowering passes nullptr.
///
-/// The returned handle must be passed to FreeTrampoline() when the
+/// The returned handle must be passed to TrampolineFree() when the
/// host procedure exits.
///
-/// Pool capacity: The pool is fixed-size (default 1024 slots, configurable
-/// via FLANG_TRAMPOLINE_POOL_SIZE env var). If all slots are in use, the
-/// runtime issues a fatal error. Dynamic slab growth may be added later.
+/// Pool capacity is fixed after initialization: 1024 slots by default,
+/// configurable via FLANG_TRAMPOLINE_POOL_SIZE. Allocating from a full pool
+/// issues a fatal error.
///
/// Architecture support: Currently x86-64 and AArch64. On unsupported
/// architectures, calling this function issues a fatal diagnostic.
More information about the flang-commits
mailing list