[all-commits] [llvm/llvm-project] 371cb1: [ORC] LLJIT updates: ExecutorNativePlatform, defau...

[lhames via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 371cb1af61d668234a242b34c773c146cb9077e7
      https://github.com/llvm/llvm-project/commit/371cb1af61d668234a242b34c773c146cb9077e7
  Author: Lang Hames <lhames at gmail.com>
  Date:   2023-04-07 (Fri, 07 Apr 2023)

  Changed paths:
    M llvm/include/llvm/ExecutionEngine/Orc/COFFPlatform.h
    M llvm/include/llvm/ExecutionEngine/Orc/LLJIT.h
    M llvm/lib/ExecutionEngine/Orc/COFFPlatform.cpp
    M llvm/lib/ExecutionEngine/Orc/LLJIT.cpp
    M llvm/test/ExecutionEngine/OrcLazy/emulated-tls.ll
    M llvm/tools/lli/lli.cpp

  Log Message:
  -----------
  [ORC] LLJIT updates: ExecutorNativePlatform, default link order, Process JD.

This commit includes several related ergonomic improvements to LLJIT.

(1) Adds a default JITDylibSearchOrder to be appended to the initial link order
of JITDylibs created via LLJIT::createJITDylib (dropping any duplicate entries).

This was introduced to support automatic reflection of process symbols (see
(2) below), but has been made visible to clients as it's generically useful,
e.g. if clients have some extra set of libraries that they want to be visible
to JIT'd code by default.

The default JITDylibSearchOrder is only appended to the link order of JITDylibs
created via LLJIT::createJITDylib, and will not be apply to JITDylibs created by
directly calling the underlying ExecutionSession -- in that case clients can set
up the link order manually.

(2) Makes process symbols visible to JIT'd code by default via the new "Process"
JITDylib, which is added to the default link order.

LLJIT clients usually want symbols in the executor process to be accessible to
JIT'd code. Until now clients have been left to set this up themselves by adding
a DynamicLibrarySearchGenerator to the Main JITDylib. This patch adds a new
process symbols JITDylib that will be created by default (with an
EPCDynamicLibrarySearchGenerator attached) and added to the default link order,
making process symbols available to JIT'd code.

Clients who do not want process symbols to be visible to JIT'd code by default
can call setLinkProcessSymbolsByDefault(false) on their LLJITBuilder to disable
this:

LLJITBuilder()
  ...
  .setLinkProcessSymbolsByDefault(false)
  ...
  .create();

Clients can also call setProcessSymbolsJITDylibSetup to take over responsibility
for configuring the process symbols JITDylib (the callback that the client
supplies will be called on the bare process symbols JITDylib immediately after
it is created).

If setLinkProcessSymbolsByDefault(false) is called and no JITDylib setup
callback has been set then the process symbols JITDylib will not be created and
LLJIT::getProcessSymbolsJITDylib will return null.

(3) Adds an ExecutorNativePlatform utility that makes it easier to enable
native platform features.

Some object format features (e.g. native static initializers and thread locals)
require runtime support in the executing process. Support for these features in
ORC is implemented cooperatively between the ORC runtime and the LLVM Platform
subclasses (COFFPlatform, ELFNixPlatform, and MachOPlatform).
ExecutorNativePlatfrom simplifies the process of loading the ORC runtime and
creating the appropriate platform class for the executor process.

ExecutorNativePlatform takes a path to the ORC runtime (or a MemoryBuffer
containing the runtime) and other required runtimes for the executor platform
(e.g. MSVC on Windows) and then configures LLJIT with an appropriate platform
class based on the executor's target triple:

LLJITBuilder()
  .setPlatformSetUp(ExecutorNativePlatform("/path/to/orc-runtime.a"));

(The ORC runtime is built as part of compiler-rt, and the exact name of the
archive is platform dependent).

The ORC runtime and platform symbols will be added to a new "Platform" JITDylib,
which will be added to the *front* of the default link order (so JIT'd code will
prefer symbol definitions in the platform/runtime to definitions in the executor
process).

ExecutorNativePlatform assumes that the Process JITDylib is available, as
the ORC runtime may depend on symbols provided by the executor process.

Differential Revision: https://reviews.llvm.org/D144276