[all-commits] [llvm/llvm-project] f9aef4: [ORC][ORC-RT] Rewrite the MachO platform to use al...

[lhames via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: f9aef477ebc6cf7c5e453f39d8ea18863bcbcfd2
      https://github.com/llvm/llvm-project/commit/f9aef477ebc6cf7c5e453f39d8ea18863bcbcfd2
  Author: Lang Hames <lhames at gmail.com>
  Date:   2022-02-08 (Tue, 08 Feb 2022)

  Changed paths:
    M compiler-rt/lib/orc/macho_platform.cpp
    M compiler-rt/lib/orc/macho_platform.h
    A compiler-rt/test/orc/TestCases/Darwin/x86-64/Inputs/standalone-ctor-and-cxa-atexit-dtor.S
    A compiler-rt/test/orc/TestCases/Darwin/x86-64/trivial-jit-dlopen.S
    M compiler-rt/test/orc/lit.cfg.py
    M llvm/include/llvm/ExecutionEngine/Orc/MachOPlatform.h
    M llvm/include/llvm/ExecutionEngine/Orc/Shared/SimplePackedSerialization.h
    M llvm/lib/ExecutionEngine/Orc/Core.cpp
    M llvm/lib/ExecutionEngine/Orc/MachOPlatform.cpp

  Log Message:
  -----------
  [ORC][ORC-RT] Rewrite the MachO platform to use allocation actions.

This patch updates the MachO platform (both the ORC MachOPlatform class and the
ORC-Runtime macho_platform.* files) to use allocation actions, rather than EPC
calls, to transfer the initializer information scraped from each linked object.
Interactions between the ORC and ORC-Runtime sides of the platform are
substantially redesigned to accomodate the change.

The high-level changes in this patch are:

1. The MachOPlatform::setupJITDylib method now calls into the runtime to set up
   a dylib name <-> header mapping, and a dylib state object (JITDylibState).

2. The MachOPlatformPlugin builds an allocation action that calls the
   __orc_rt_macho_register_object_platform_sections and
   __orc_rt_macho_deregister_object_platform_sections functions in the runtime
   to register the address ranges for all "interesting" sections in the object
   being allocated (TLS data sections, initializers, language runtime metadata
   sections, etc.).

3. The MachOPlatform::rt_getInitializers method (the entry point in the
   controller for requests from the runtime for initializer information) is
   replaced by MachOPlatform::rt_pushInitializers. The former returned a data
   structure containing the "interesting" section address ranges, but these are
   now handled by __orc_rt_macho_register_object_platform_sections. The new
   rt_pushInitializers method first issues a lookup to trigger materialization
   of the "interesting" sections, then returns the dylib dependence tree rooted
   at the requested dylib for dlopen to consume. (The dylib dependence tree is
   returned by rt_pushInitializers, rather than being handled by some dedicated
   call, because rt_pushInitializers can alter the dependence tree).

The advantage of these changes (beyond the performance advantages of using
allocation actions) is that it moves more information about the materialized
portions of the JITDylib into the executor. This tends to make the runtime
easier to reason about, e.g. the implementation of dlopen in the runtime is now
recursive, rather than relying on recursive calls in the controller to build a
linear data structure for consumption by the runtime. This change can also make
some operations more efficient, e.g. JITDylibs can be dlclosed and then
re-dlopened without having to pull all initializers over from the controller
again.

In addition to the high-level changes, there are some low-level changes to ORC
and the runtime:

* In ORC, at ExecutionSession teardown time JITDylibs are now destroyed in
reverse creation order. This is on the assumption that the ORC runtime will be
loaded into an earlier dylib that will be used by later JITDylibs. This is a
short-term solution to crashes that arose during testing when the runtime was
torn down before its users. Longer term we will likely destroy dylibs in
dependence order.

* toSPSSerializable(Expected<T> E) is updated to explicitly initialize the T
value, allowing it to be used by Ts that have explicit constructors.

* The ORC runtime now (1) attempts to track ref-counts, and (2) distinguishes
not-yet-processed "interesting" sections from previously processed ones. (1)
is necessary for standard dlopen/dlclose emulation. (2) is intended as a step
towards better REPL support -- it should enable future runtime calls that
run only newly registered initializers ("dlopen_more", "dlopen_additions",
...?).