[all-commits] [llvm/llvm-project] 570ecd: [ORC] Introduce LazyReexportsManager, JITLinkTramp...

[Lang Hames via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 570ecdcf8b44aec853ce381a5f6b77222b041afa
      https://github.com/llvm/llvm-project/commit/570ecdcf8b44aec853ce381a5f6b77222b041afa
  Author: Lang Hames <lhames at gmail.com>
  Date:   2024-12-07 (Sat, 07 Dec 2024)

  Changed paths:
    M compiler-rt/lib/orc/CMakeLists.txt
    A compiler-rt/lib/orc/sysv_reentry.arm64.S
    A compiler-rt/lib/orc/sysv_resolve.cpp
    A compiler-rt/test/orc/TestCases/Generic/Inputs/foo-ret-42.ll
    A compiler-rt/test/orc/TestCases/Generic/Inputs/var-x-42.ll
    A compiler-rt/test/orc/TestCases/Generic/lazy-link.ll
    R compiler-rt/test/orc/TestCases/Generic/orc-rt-executor-usage.test
    M compiler-rt/test/orc/lit.cfg.py
    M llvm/include/llvm/ExecutionEngine/JITLink/aarch64.h
    M llvm/include/llvm/ExecutionEngine/Orc/Core.h
    A llvm/include/llvm/ExecutionEngine/Orc/JITLinkLazyCallThroughManager.h
    M llvm/include/llvm/ExecutionEngine/Orc/JITLinkRedirectableSymbolManager.h
    A llvm/include/llvm/ExecutionEngine/Orc/JITLinkReentryTrampolines.h
    M llvm/include/llvm/ExecutionEngine/Orc/LazyObjectLinkingLayer.h
    M llvm/include/llvm/ExecutionEngine/Orc/LazyReexports.h
    M llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
    M llvm/include/llvm/ExecutionEngine/Orc/RedirectionManager.h
    M llvm/lib/ExecutionEngine/JITLink/aarch64.cpp
    M llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
    M llvm/lib/ExecutionEngine/Orc/Core.cpp
    A llvm/lib/ExecutionEngine/Orc/JITLinkReentryTrampolines.cpp
    M llvm/lib/ExecutionEngine/Orc/LazyObjectLinkingLayer.cpp
    M llvm/lib/ExecutionEngine/Orc/LazyReexports.cpp
    M llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp
    R llvm/test/ExecutionEngine/JITLink/Generic/Inputs/foo-ret-42.ll
    R llvm/test/ExecutionEngine/JITLink/Generic/Inputs/var-x-42.ll
    R llvm/test/ExecutionEngine/JITLink/Generic/lazy-link.ll
    M llvm/tools/llvm-jitlink/llvm-jitlink.cpp
    M llvm/tools/llvm-jitlink/llvm-jitlink.h

  Log Message:
  -----------
  [ORC] Introduce LazyReexportsManager, JITLinkTrampolines, ORC-RT base… (#118923)

…d reentry.

These utilities provide new, more generic and easier to use support for
lazy compilation in ORC.

LazyReexportsManager is an alternative to LazyCallThroughManager. It
takes requests for lazy re-entry points in the form of an alias map:
lazy-reexports = {
  ( <entry point symbol #1>, <implementation symbol #1> ),
  ( <entry point symbol #2>, <implementation symbol #2> ),
  ...
  ( <entry point symbol #n>, <implementation symbol #n> )
}

LazyReexportsManager then:
1. binds the entry points to the implementation names in an internal
table.
2. creates a JIT re-entry trampoline for each entry point.
3. creates a redirectable symbol for each of the entry point name and
binds redirectable symbol to the corresponding reentry trampoline.

When an entry point symbol is first called at runtime (which may be on
any thread of the JIT'd program) it will re-enter the JIT via the
trampoline and trigger a lookup for the implementation symbol stored in
LazyReexportsManager's internal table. When the lookup completes the
entry point symbol will be updated (via the RedirectableSymbolManager)
to point at the implementation symbol, and execution will proceed to the
implementation symbol.

Actual construction of the re-entry trampolines and redirectable symbols
is delegated to an EmitTrampolines functor and the
RedirectableSymbolsManager respectively.

JITLinkReentryTrampolines.h provides a JITLink-based implementation of
the EmitTrampolines functor. (AArch64 only in this patch, but other
architectures will be added in the near future).

Register state save and reentry functionality is added to the ORC
runtime in the __orc_rt_sysv_resolve and __orc_rt_resolve_implementation
functions (the latter is generic, the former will need custom
implementations for each ABI and architecture to be supported, however
this should be much less effort than the existing OrcABISupport
approach, since the ORC runtime allows this code to be written as native
assembly).

The resulting system:
1. Works equally well for in-process and out-of-process JIT'd code.
2. Requires less boilerplate to set up.

Given an ObjectLinkingLayer and PlatformJD (JITDylib containing the ORC
runtime), setup is just:

```c++
auto RSMgr = JITLinkRedirectableSymbolManager::Create(OLL);
if (!RSMgr)
  return RSMgr.takeError();

auto LRMgr = createJITLinkLazyReexportsManager(OLL, **RSMgr, PlatformJD);
if (!LRMgr)
  return LRMgr.takeError();
```

after which lazy reexports can be introduced with:

```c++
JD.define(lazyReexports(LRMgr, <alias map>));
```

LazyObectLinkingLayer is updated to use this new method, but the LLVM-IR
level CompileOnDemandLayer will continue to use LazyCallThroughManager
and OrcABISupport until the new system supports a wider range of
architectures and ABIs.

The llvm-jitlink utility's -lazy option now uses the new scheme. Since
it depends on the ORC runtime, the lazy-link.ll testcase and associated
helpers are moved to the ORC runtime.



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications