[flang-commits] [flang] [RFC][flang] Trampolines for internal procedures. (PR #66157)
Slava Zakharin via flang-commits
flang-commits at lists.llvm.org
Tue Sep 26 14:45:35 PDT 2023
================
@@ -0,0 +1,373 @@
+<!--===- docs/InternalProcedureTrampolines.md
+
+ Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+ See https://llvm.org/LICENSE.txt for license information.
+ SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+
+-->
+
+# Trampolines for pointers to internal procedures.
+
+## Overview
+
+```fortran
+subroutine host()
+ integer :: local = 10
+ call internal()
+ return
+
+ contains
+ subroutine internal()
+ print *, local
+ end subroutine internal
+end subroutine host
+```
+
+Procedure code generated for subprogram `inernal()` must have access to the scope of
+its host procedure, e.g. to access `local` variable. Flang achieves this by passing
+an extra argument to `internal()` that is a tuple of references to all variables
+used via host association inside `internal()`. We will call this extra argument
+a static chain link.
+
+Fortran standard 2008 allowed using internal procedures as actual arguments or
+procedure pointer targets:
+
+> Fortran 2008 contains several extensions to Fortran 2003; some of these are listed below.
+>
+> * An internal procedure can be used as an actual argument or procedure pointer target.
+>
+> NOTE 12.18
+>
+> An internal procedure cannot be invoked using a procedure pointer from either Fortran or C after the host instance completes execution, because the pointer is then undefined. While the host instance is active, however, the internal procedure may be invoked from outside of the host procedure scoping unit if that internal procedure was passed as an actual argument or is the target of a procedure pointer.
+
+Special handling is required for the internal procedures that might be invoked
+via an argument association or via pointer.
+This document describes Flang implementation to support it.
+
+## Flang current implementation
+
+### Examples
+
+Internal procedure as procedure pointer target:
+
+```fortran
+module other
+ abstract interface
+ function callback()
+ integer :: callback
+ end function callback
+ end interface
+ contains
+ subroutine foo(fptr)
+ procedure(callback), pointer :: fptr
+ ! `fptr` is pointing to `callee`, which needs the static chain link.
+ print *, fptr()
+ end subroutine foo
+end module other
+
+subroutine host(local)
+ use other
+ integer :: local
+ procedure(callback), pointer :: fptr
+ fptr => callee
+ call foo(fptr)
+ return
+
+ contains
+
+ function callee()
+ integer :: callee
+ callee = local
+ end function callee
+end subroutine host
+
+program main
+ call host(10)
+end program main
+```
+
+Internal procedure as actual argument (F90 style):
+
+```fortran
+module other
+ contains
+ subroutine foo(fptr)
+ interface
+ integer function fptr()
+ end function
+ end interface
+ ! `fptr` is pointing to `callee`, which needs the static chain link.
+ print *, fptr()
+ end subroutine foo
+end module other
+
+subroutine host(local)
+ use other
+ integer :: local
+ call foo(callee)
+ return
+
+ contains
+
+ function callee()
+ integer :: callee
+ callee = local
+ end function callee
+end subroutine host
+
+program main
+ call host(10)
+end program main
+```
+
+Internal procedure as actual argument (F77 style):
+
+```fortran
+module other
+ contains
+ subroutine foo(fptr)
+ integer :: fptr
+ ! `fptr` is pointing to `callee`, which needs the static chain link.
+ print *, fptr()
+ end subroutine foo
+end module other
+
+subroutine host(local)
+ use other
+ integer :: local
+ call foo(callee)
+ return
+
+ contains
+
+ function callee()
+ integer :: callee
+ callee = local
+ end function callee
+end subroutine host
+
+program main
+ call host(10)
+end program main
+```
+
+In all cases, the call sequence implementing `fptr()` call site inside `foo()`
+must pass the stack chain link to the actual function `callee()`.
+
+### Usage of trampolines in Flang
+
+`BoxedProcedure` pass recognizes `fir.emboxproc` operations that
+embox a subroutine address together with the static chain link,
+and transforms them into a sequence of operations that replace
+the result of `fir.emboxproc` with an address of a trampoline.
+Eventually, it is the address of the trampoline that is passed
+as an actual argument to `foo()`.
+
+The trampoline has the following structure:
+
+```assembly
+callee_trampoline:
+ MOV <static-chain-address>, R#
+ JMP <callee-address>
+```
+
+Where:
+- `<callee-address>` is the address of function `callee()`.
+- `<static-chain-address>` - the address of the static chain
+ object created inside `host()`.
+- `R#` is a target specific register.
+
+In MLIR LLVM dialect the replacement looks like this:
+
+```
+ llvm.call @llvm.init.trampoline(%8, %9, %7) : (!llvm.ptr<i8>, !llvm.ptr<i8>, !llvm.ptr<i8>) -> ()
+ %10 = llvm.call @llvm.adjust.trampoline(%8) : (!llvm.ptr<i8>) -> !llvm.ptr<i8>
+ %11 = llvm.bitcast %10 : !llvm.ptr<i8> to !llvm.ptr<func<void ()>>
+ llvm.call @_QMotherPfoo(%11) {fastmathFlags = #llvm.fastmath<fast>} : (!llvm.ptr<func<void ()>>) -> ()
+
+```
+
+So any call of `fptr` inside `foo()` will result in invocation of the trampoline.
+The trampoline will setup `R#` register and jump to `callee()` directly.
+
+The ABI of `callee()` is adjusted using `llvm.nest` call argument attribute,
+so that the target code generator assumes the static chain argument is passed
+to `callee()` in `R#`:
+
+```
+ llvm.func @_QFhostPcallee(%arg0: !llvm.ptr<struct<(ptr<i32>)>> {fir.host_assoc, llvm.nest}) -> i32 attributes {fir.internal_proc} {
+```
+
+#### Trampoline handling
+
+Currently used [llvm.init.trampoline intrinsic](https://llvm.org/docs/LangRef.html#trampoline-intrinsics)
+expects that the memory for the trampoline content is passed to it as the first argument.
+The memory has to be writeable at the point of the intrinsic call, and it has to be executable
+at any point where `callee()` might be ivoked via the trampoline.
+
+`@llvm.init.trampoline` intrinsic initializes the trampoline area in a target-specific manner
+so that being executed: the trampoline sets a target-specific register to be equal to the third argument
+(which is a static chain address), and then calls the function defined by the second argument.
+
+Some targets may perform additional actions to guarantee the readiness of the trampoline for execution,
+e.g. [call](https://github.com/llvm/llvm-project/blob/main/compiler-rt/lib/builtins/trampoline_setup.c)
+`__clear_cache` or do something else.
+
+For each internal procedure a trampoline may be initialized once per the host invocation.
+
+The target-specific address of the new trampoline function must be taken via another intrinsic call:
+
+```
+%p = call i8* @llvm.adjust.trampoline(i8* %trampoline_address)
+```
+
+Note that value of `%p` is equal to `%tramp1` in most cases, but this is not
+a requirement - this is partly [why](https://lists.llvm.org/pipermail/llvm-dev/2011-August/042845.html)
+the second intrinsic was introduced:
+
+> ```
+> By the way an example of adjust_trampoline is ARM, which or's a 1 into the address of the trampoline. When the pointer is called the processor sees the 1 and puts itself into thumb mode.
+
+Currently, the trampolines are allocated on the stack of `host()` subroutine,
+so that they are available throughout the life span of `host()` and are
+automatically deallocated at the end of `host()` invocation.
+Unfortunately, this requires the program stack to be writeable and executable
+at the same time, which might be a security concern.
+
+> NOTE: LLVM's AArch64 backend supports `nest` attribute, but it does not seem to support trampoline intrinsics.
+
+## Alternative implementation(s)
+
+To address the security risk we may consider managing the trampoline memory
+in a way that it is not writeable and executable at the same time.
+One of the options is to use separate allocations for the trampoline code
+and the trampoline "data".
+
+The trampolines may be located in non-writeable executable memory:
+```assembly
+trampoline0:
+ MOV (TDATA[0].static_chain_address), R#
+ JMP (TDATA[0].callee_address)
+trampoline1:
+ MOV (TDATA[1].static_chain_address), R#
+ JMP (TDATA[1].callee_address)
+...
+```
+
+The `TDATA` memory is writeable and contains *<static chain address, function address>*
+for each of the trampolines.
+
+A runtime support library may provide APIs for initializing/accessing/deallocating
+the trampolines that can be used by `BoxedProcedure` pass.
+
+### Implementation considerations
+
+* The static chain address still has to be passed in fixed target-specific register,
+ and the implementations that rely on LLVM back-ends can use `nest` attribute for this.
+
+* The trampoline area must be able to grow, because there can be a trampoline
+ for each internal procedure per host invocation, and an internal procedure can call
+ the host recursively. This means that the amount of trampolines in one thread
+ may grow pretty quickly.
+
+ ```fortran
+ recursive subroutine host(local)
+ use other
+ integer :: local
+ call foo(callee)
+ return
+
+ contains
+
+ function callee()
+ integer :: callee
+ if (local .le. CONST_N) then
+ call host(local + 1)
+ endif
+ end function callee
+ end subroutine host
+ ```
+
+* On the other hand, putting a hard limit on the number of trampolines live at the same time
+ allows putting the trampolines into the static code segment.
+
+* Each thread may have its own dynamic trampoline area to reduce the number
+ of required locks.
+
+* Some support is required for the offload devices.
+
+* Each trampoline invocation implies two indirect accesses with this approach.
+
+### Option #1: Fortran runtime support
----------------
vzakhari wrote:
Will do.
https://github.com/llvm/llvm-project/pull/66157
More information about the flang-commits
mailing list