[all-commits] [llvm/llvm-project] 23973e: [lli] Add --jit-linker command line argument

Tue Mar 2 06:08:23 PST 2021

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 23973e0aac1e13eecebc62547585815e8352f65c
      https://github.com/llvm/llvm-project/commit/23973e0aac1e13eecebc62547585815e8352f65c
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/tools/lli/lli.cpp

  Log Message:
  -----------
  [lli] Add --jit-linker command line argument

The argument value determines the dynamic linker to use (`default`, `rtdyld` or `jitlink`). The JITLink implementation only supports in-process JITing for now. This is the first commit for the reviewed patch.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D97339

  Commit: 0c5e0cf6a206ad68906cab72520c11d9cf71eb2f
      https://github.com/llvm/llvm-project/commit/0c5e0cf6a206ad68906cab72520c11d9cf71eb2f
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/tools/llvm-jitlink/llvm-jitlink.h

  Log Message:
  -----------
  [llvm-jitlink] Remove duplicate type defintion (NFC)

  Commit: 949d1d9937364a18c99bf549a6503285d7e61a5a
      https://github.com/llvm/llvm-project/commit/949d1d9937364a18c99bf549a6503285d7e61a5a
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/lib/ExecutionEngine/JITLink/ELF.cpp
    M llvm/lib/ExecutionEngine/JITLink/JITLink.cpp
    M llvm/lib/ExecutionEngine/JITLink/MachO.cpp
    M llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp

  Log Message:
  -----------
  [JITLink] Remove some std::move(MemoryBufferRef) below createLinkGraphFromObject() (NFC)

  Commit: 48c2acff0c56aab178f2e62090d3f1a7649bd638
      https://github.com/llvm/llvm-project/commit/48c2acff0c56aab178f2e62090d3f1a7649bd638
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/include/llvm/ExecutionEngine/JITLink/JITLink.h

  Log Message:
  -----------
  [JITLink] LinkGraph::getName() can be const

  Commit: b66b73be295c00a11472566f52b80044384f2fb2
      https://github.com/llvm/llvm-project/commit/b66b73be295c00a11472566f52b80044384f2fb2
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/unittests/ExecutionEngine/Orc/ResourceTrackerTest.cpp

  Log Message:
  -----------
  [Orc] Fix a file header (NFC)

  Commit: 171849c2881bc11c82173588596cca8f539ef81d
      https://github.com/llvm/llvm-project/commit/171849c2881bc11c82173588596cca8f539ef81d
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/lib/ExecutionEngine/Orc/LLJIT.cpp

  Log Message:
  -----------
  [Orc] Rename local variable to avoid confusion with equally-named class member (NFC)

  Commit: ef2389235c5dec03be93f8c9585cd9416767ef4c
      https://github.com/llvm/llvm-project/commit/ef2389235c5dec03be93f8c9585cd9416767ef4c
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    A llvm/include/llvm/ExecutionEngine/Orc/DebugObjectManagerPlugin.h
    M llvm/include/llvm/ExecutionEngine/Orc/ObjectLinkingLayer.h
    A llvm/include/llvm/ExecutionEngine/Orc/TPCDebugObjectRegistrar.h
    A llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h
    M llvm/lib/ExecutionEngine/Orc/CMakeLists.txt
    A llvm/lib/ExecutionEngine/Orc/DebugObjectManagerPlugin.cpp
    M llvm/lib/ExecutionEngine/Orc/ObjectLinkingLayer.cpp
    A llvm/lib/ExecutionEngine/Orc/TPCDebugObjectRegistrar.cpp
    M llvm/lib/ExecutionEngine/Orc/TargetProcess/CMakeLists.txt
    A llvm/lib/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.cpp
    M llvm/tools/llvm-jitlink/llvm-jitlink-executor/llvm-jitlink-executor.cpp
    M llvm/tools/llvm-jitlink/llvm-jitlink.cpp

  Log Message:
  -----------
  [Orc] Add JITLink debug support plugin for ELF x86-64

Add a new ObjectLinkingLayer plugin `DebugObjectManagerPlugin` and infrastructure to handle creation of `DebugObject`s as well as their registration in OrcTargetProcess. The current implementation only covers ELF on x86-64, but the infrastructure is not limited to that.

The journey starts with a new `LinkGraph` / `JITLinkContext` pair being created for a `MaterializationResponsibility` in ORC's `ObjectLinkingLayer`. It sends a `notifyMaterializing()` notification, which is forwarded to all registered plugins. The `DebugObjectManagerPlugin` aims to create a  `DebugObject` form the provided target triple and object buffer. (Future implementations might create `DebugObject`s from a `LinkGraph` in other ways.) On success it will track it as the pending `DebugObject` for the `MaterializationResponsibility`.

This patch only implements the `ELFDebugObject` for `x86-64` targets. It follows the RuntimeDyld approach for debug object setup: it captures a copy of the input object, parses all section headers and prepares to patch their load-address fields with their final addresses in target memory. It instructs the plugin to report the section load-addresses once they are available. The plugin overrides `modifyPassConfig()` and installs a JITLink post-allocation pass to capture them.

Once JITLink emitted the finalized executable, the plugin emits and registers the `DebugObject`. For emission it requests a new `JITLinkMemoryManager::Allocation` with a single read-only segment, copies the object with patched section load-addresses over to working memory and triggers finalization to target memory. For registration, it notifies the `DebugObjectRegistrar` provided in the constructor and stores the previously pending`DebugObject` as registered for the corresponding MaterializationResponsibility.

The `DebugObjectRegistrar` registers the `DebugObject` with the target process. `llvm-jitlink` uses the `TPCDebugObjectRegistrar`, which calls `llvm_orc_registerJITLoaderGDBWrapper()` in the target process via `TargetProcessControl` to emit a `jit_code_entry` compatible with the GDB JIT interface [1]. So far the implementation only supports registration and no removal. It appears to me that it wouldn't raise any new design questions, so I left this as an addition for the near future.

[1] https://sourceware.org/gdb/current/onlinedocs/gdb/JIT-Interface.html

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D97335

  Commit: 99a6d003edbe97fcb94854547276ffad3382ec1d
      https://github.com/llvm/llvm-project/commit/99a6d003edbe97fcb94854547276ffad3382ec1d
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/lib/ExecutionEngine/CMakeLists.txt
    M llvm/lib/ExecutionEngine/GDBRegistrationListener.cpp
    M llvm/tools/lli/lli.cpp

  Log Message:
  -----------
  [lli] Add JITLink in-process debug support

lli aims to provide both, RuntimeDyld and JITLink, as the dynamic linkers/loaders for it's JIT implementations. And they both offer debugging via the GDB JIT interface, which builds on the two well-known symbol names `__jit_debug_descriptor` and `__jit_debug_register_code`. As these symbols must be unique accross the linked executable, we can only define them in one of the libraries and make the other depend on it. OrcTargetProcess is a minimal stub for embedding a JIT client in remote executors. For the moment it seems reasonable to have the definition there and let ExecutionEngine depend on it, until we find a better solution.

This is the second commit for the reviewed patch.

Reviewed By: lhames

Differential Revision: https://reviews.llvm.org/D97339

  Commit: ae51fde369dffa34e948c44f002457d27a2b22dd
      https://github.com/llvm/llvm-project/commit/ae51fde369dffa34e948c44f002457d27a2b22dd
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/test/ExecutionEngine/OrcLazy/debug-descriptor-elf-minimal.ll
    M llvm/test/ExecutionEngine/OrcLazy/debug-objects-elf-minimal.ll

  Log Message:
  -----------
  [Orc] Extend lli debug support tests to JITLink

  Commit: 403bdd50069a246a9d3cf914164f1b80444b1ee0
      https://github.com/llvm/llvm-project/commit/403bdd50069a246a9d3cf914164f1b80444b1ee0
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/docs/JITLink.rst

  Log Message:
  -----------
  [docs][JITLink] Fix a typo (NFC)

  Commit: 818772a594031a2c41cd69d946fa353c2670636d
      https://github.com/llvm/llvm-project/commit/818772a594031a2c41cd69d946fa353c2670636d
  Author: Stefan Gränitz <stefan.graenitz at gmail.com>
  Date:   2021-03-02 (Tue, 02 Mar 2021)

  Changed paths:
    M llvm/lib/ExecutionEngine/JITLink/JITLinkMemoryManager.cpp

  Log Message:
  -----------
  [Orc] Fix remaining memory size of slab allocator

Compare: https://github.com/llvm/llvm-project/compare/7f086d74c347...818772a59403