[llvm-dev] LLVM Orc Weekly #28 -- ORC Runtime Prototype update

Sat Jan 23 16:55:09 PST 2021

Hi Stefan,

I am always in favor of getting JIT improvements into mainline as soon as
> possible. Also, simplifying APIs is an important goal.
>

On the other hand this raises a few general questions for me:
>

IIUC the current patch introduces a dependency from LLVM to
> compiler-rt/clang build artifacts, because the llvm-jitlink-executor
> executable is fully functional only if it can find the clang_rt.orc static
> library in the build tree. Do we have dependencies like that in mainline so
> far?

These are excellent questions.

As far as I know we do not have any dependencies like this (LLVM tool
depends on compiler-rt for full functionality) in the mainline yet. Even
superficially similar situations (e.g. building clang without building
compiler-rt or libcxx) are quite different as there must already be
compatible system libraries available to bootstrap LLVM in the first place.

On the other hand, while it's true that llvm-jitlink will no longer have
full functionality without loading the runtime, it's not the case that
there will be any regression from the current functionality: we're only
enabling new functionality here (at least in llvm-jitlink, which never used
to run initializers). This means that we can use the new
'-use-orc-runtime=false' option to disable runtime loading and still run
all llvm-jitlink tests that are in-tree today.

And how does it affect testing? So far, it seems we have no testing for
> out-of-process execution, because the only tool that exercises this
> functionality is llvm-jitlink, which itself is mostly used as a testing
> helper. If the functionality now moves into the TargetProcess library, it
> might be worth thinking through the test strategy first.

I think it is reasonable to maintain the existing tests (adding
'-use-orc-runtime=false'), then add new end-to-end tests of the runtime via
llvm-jitlink that will be dependent on having built compiler-rt.

At the moment, the TargetProcess library is only used by JITLink. Will it
> stay like that? If so, RuntimeDyld-based JITs will not be affected by the
> patch. Will it be possible to build and run e.g. a LLJIT instance with a
> RuntimeDyldLinkingLayer if only LLVM gets built (omitting
> LLVM_ENABLE_PROJECTS="clang;compiler-rt")?

I think that the LLI tool should be migrated to TargetProcess too. We
should distinguish between TargetProcess and the ORC runtime though:
TargetProcess can be linked into the executor without requiring the runtime.

LLI does run static initializers / deinitializers today, but only
in-process using the existing initializer infrastructure (which I think is
a hack). I think two reasonable options going forward would be:
(1) Make running initializers via lli dependent on building compiler-rt (as
in llvm-jitlink). This is my preferred solution.
(2) Move the existing hacks out of ORC and into LLI (or a specific
LLI_LLJIT class in ORC).

For MCJIT-like use cases you will definitely still be able to build and run
an LLJIT instance with either RTDyldObjectLinkingLayer (RuntimeDyld) or
ObjectLinkingLayer (JITLink) with LLVM only. I think this will remain
true indefinitely. What I think may change over time is how advanced
features (e.g. laziness, initializers) are implemented: It is so much
easier and lower maintenance to implement these within the runtime.
Eventually I could see us dropping support for them in LLVM-only builds, at
which point it will be a runtime error to attempt to use those features
(e.g. attempting to add a lazyReexport will yield a "cannot resolve
__orc_rt_jit_reentry" error).

Do these solutions seem reasonable to you?

Thank you very much again for thinking about this -- I think testing and
dependencies are the trickiest aspects of introducing this runtime, and
it's very helpful to get other developers' perspectives.

Regards,
Lang.

On Fri, Jan 22, 2021 at 9:53 PM Stefan Gränitz <stefan.graenitz at gmail.com>
wrote:

> I am always in favor of getting JIT improvements into mainline as soon as
> possible. Also, simplifying APIs is an important goal.
>
> On the other hand this raises a few general questions for me:
>
> IIUC the current patch introduces a dependency from LLVM to
> compiler-rt/clang build artifacts, because the llvm-jitlink-executor
> executable is fully functional only if it can find the clang_rt.orc static
> library in the build tree. Do we have dependencies like that in mainline so
> far?
>
> And how does it affect testing? So far, it seems we have no testing for
> out-of-process execution, because the only tool that exercises this
> functionality is llvm-jitlink, which itself is mostly used as a testing
> helper. If the functionality now moves into the TargetProcess library, it
> might be worth thinking through the test strategy first.
>
> At the moment, the TargetProcess library is only used by JITLink. Will it
> stay like that? If so, RuntimeDyld-based JITs will not be affected by the
> patch. Will it be possible to build and run e.g. a LLJIT instance with a
> RuntimeDyldLinkingLayer if only LLVM gets built (omitting
> LLVM_ENABLE_PROJECTS="clang;compiler-rt")?
>
> Last but not least, there are examples that use JITLink. Could we still
> build and run them if only LLVM gets built?
>
> Thanks,
> Stefan
>
> On 19/01/2021 23:08, Lang Hames wrote:
>
> Big question for JIT clients: Does anyone have any objection to APIs in
> ORC *relying* on the runtime being loaded in the target? If so, now is the
> time to let me know. :)
>
> I think possible objections are JIT'd program startup time (unlikely to be
> very high, and likely fixable via careful runtime design and pre-linking of
> parts of the runtime), and difficulties building compiler-rt (which sounds
> like something we should fix in compiler-rt).
>
> If we can assume that the runtime is loadable then we can significantly
> simplify the TargetProcess library, and TargetProcessControl API, and
> further accelerate feature development in LLVM 13.
>
> -- Lang.
>
> On Tue, Jan 19, 2021 at 8:45 AM Lang Hames <lhames at gmail.com> wrote:
>
>> Hi Stefan,
>>
>> % ./bin/llvm-jitlink -oop-executor inits.o
>>> JIT session error: Symbols not found: [
>>> __ZTIN4llvm6detail14format_adapterE ]
>>
>>
>> I've been testing with a debug build:
>>
>> % xcrun cmake -GNinja -DCMAKE_BUILD_TYPE=Debug
>> -DLLVM_ENABLE_PROJECTS="llvm;clang;compiler-rt" ../llvm
>>
>> Matching this build might fix the issue, though building with my config
>> (if it works) is only a short-term fix. The error that you're seeing
>> implies that the runtime is dependending on a symbol from libSupport that
>> is not being linked in to the target (llvm-jitlink-executor). I'll aim to
>> break these dependencies on libSupport in the future. Mostly that means
>> either removing the dependence on llvm::Error / llvm::Expected (e.g. by
>> creating stripped down versions for the orc runtime), or making those types
>> header-only.
>>
>> -- Lang.
>>
>>
>>
>>
>>
>> On Tue, Jan 19, 2021 at 12:50 AM Stefan Gränitz <
>> stefan.graenitz at gmail.com> wrote:
>>
>>> Wow, thanks for the update. One more ORC milestone in a short period of
>>> time!
>>>
>>> On macOS I built the C++ example like this:
>>>
>>> % cmake -GNinja -DLLVM_TARGETS_TO_BUILD=host
>>> -DLLVM_ENABLE_PROJECTS="clang;compiler-rt" ../llvm
>>> % ninja llvm-jitlink llvm-jitlink-executor
>>> lib/clang/12.0.0/lib/darwin/libclang_rt.orc_osx.a
>>> % clang++ -c -o inits.o inits.cpp
>>>
>>> The in-process version works perfectly, but with the out-of-process flag
>>> the examples fails:
>>>
>>> % ./bin/llvm-jitlink inits.o
>>> Foo::Foo()
>>> Foo::foo()
>>> Foo::~Foo()
>>> % ./bin/llvm-jitlink -oop-executor inits.o
>>> JIT session error: Symbols not found: [
>>> __ZTIN4llvm6detail14format_adapterE ]
>>>
>>> Any idea what could go wrong here? Otherwise I can try to debug it later
>>> this week. (Full error below.)
>>>
>>> Best
>>> Stefan
>>>
>>> --
>>>
>>> JIT session error: Symbols not found: [
>>> __ZTIN4llvm6detail14format_adapterE ]
>>> /Users/staefsn/Develop/LLVM/monorepo/llvm-orc-runtime/build/bin/llvm-jitlink:
>>> Failed to materialize symbols: { (Main, {
>>> ___orc_rt_macho_symbol_lookup_remote,
>>> __ZN4llvm3orc6shared21WrapperFunctionResult22destroyWithArrayDeleteE39LLVMOrcSharedCWrapperFunctionResultDatay,
>>> __ZNSt3__113__vector_baseIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEED2Ev,
>>> __ZNK4llvm8ExpectedINS_3orc6shared21WrapperFunctionResultEE22fatalUncheckedExpectedEv,
>>> __ZN4llvm3orc6shared21toWrapperFunctionBlobIJNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEEEENS1_21WrapperFunctionResultEDpRKT_,
>>> __ZN4llvm3orc6shared20VectorRawByteChannelD1Ev,
>>> __ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJNSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_SA_EE11deserializeISA_JSA_SA_EEENS_5ErrorERS3_RT_DpRT0_,
>>> __ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializers13SectionExtentENS_9allocatorIS5_EEE8__appendEm,
>>> __ZN4llvm3orc6shared21toWrapperFunctionBlobIJyNSt3__112basic_stringIcNS3_11char_traitsIcEENS3_9allocatorIcEEEEEEENS1_21WrapperFunctionResultEDpRKT_,
>>> __ZN4llvm29VerifyEnableABIBreakingChecksE,
>>> __ZN4llvm15format_providerImvE6formatERKmRNS_11raw_ostreamENS_9StringRefE,
>>> __ZNSt3__114__split_bufferIN4llvm3orc6shared25MachOJITDylibInitializersERNS_9allocatorIS4_EEED2Ev,
>>> __ZTVN4llvm6detail23provider_format_adapterIRmEE,
>>> __ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyNSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEEEE9serializeIRKyJRKSA_EEENS_5ErrorERS3_OT_DpOT0_,
>>> __ZTVN4llvm6detail23provider_format_adapterImEE,
>>> __ZN4llvm6detail23provider_format_adapterImE6formatERNS_11raw_ostreamENS_9StringRefE,
>>> __ZN4llvm6detail23provider_format_adapterIRmED1Ev,
>>> ___orc_rt_macho_get_deinitializers_tag,
>>> __ZN4llvm6detail23provider_format_adapterIRmED0Ev,
>>> __ZTVN4llvm3orc6shared14RawByteChannelE,
>>> __ZN6orc_rt12jit_dispatchEPKvN4llvm8ArrayRefIhEE,
>>> __ZN4llvm3orc6shared20VectorRawByteChannelD0Ev,
>>> __ZTVN4llvm3orc6shared20VectorRawByteChannelE,
>>> __ZN4llvm8cantFailENS_5ErrorEPKc,
>>> __ZN4llvm6detail23provider_format_adapterImED1Ev,
>>> __ZTVN4llvm6detail23provider_format_adapterIRjEE,
>>> __ZN4llvm6detail23provider_format_adapterImED0Ev,
>>> __ZN4llvm15format_providerIjvE6formatERKjRNS_11raw_ostreamENS_9StringRefE,
>>> __ZTSN4llvm3orc6shared14RawByteChannelE,
>>> __ZN4llvm6detail23provider_format_adapterIRjE6formatERNS_11raw_ostreamENS_9StringRefE,
>>> __ZN4llvm6detail23provider_format_adapterIRjED0Ev,
>>> __ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__16vectorINS1_25MachOJITDylibInitializersENS4_9allocatorIS6_EEEES9_vE11deserializeERS3_RS9_,
>>> __ZTIN4llvm6detail23provider_format_adapterImEE,
>>> __ZN4llvm6detail15HelperFunctions15consumeHexStyleERNS_9StringRefERNS_13HexPrintStyleE,
>>> __ZTIN4llvm6detail23provider_format_adapterIRmEE,
>>> ___orc_rt_macho_get_initializers_tag,
>>> __ZTSN4llvm6detail23provider_format_adapterImEE,
>>> __ZN4llvm3orc6shared20VectorRawByteChannel4sendEv,
>>> __ZN4llvm6detail23provider_format_adapterIRmE6formatERNS_11raw_ostreamENS_9StringRefE,
>>> ___orc_rt_macho_symbol_lookup_tag,
>>> __ZTSN4llvm6detail23provider_format_adapterIRmEE,
>>> __ZN4llvm3orc6shared20VectorRawByteChannel11appendBytesEPKcj,
>>> __ZTSN4llvm6detail23provider_format_adapterIRjEE,
>>> __ZN4llvm3orc6shared20VectorRawByteChannel9readBytesEPcj,
>>> __ZTSN4llvm3orc6shared20VectorRawByteChannelE,
>>> __ZN4llvm6detail23provider_format_adapterIRjED1Ev,
>>> __ZN4llvm3orc6shared14RawByteChannelD1Ev,
>>> __ZN4llvm3orc6shared23fromWrapperFunctionBlobIJNSt3__16vectorINS1_25MachOJITDylibInitializersENS3_9allocatorIS5_EEEEEEENS_5ErrorENS_8ArrayRefIhEEDpRT_,
>>> __ZN4llvm3orc6shared14RawByteChannelD0Ev,
>>> __ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyyNSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_SA_EE11deserializeIyJySA_SA_SA_EEENS_5ErrorERS3_RT_DpRT0_,
>>> __ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEE8__appendEm,
>>> __ZN4llvm3orc6shared21SequenceSerializationINS1_20VectorRawByteChannelEJyyEE11deserializeIyJyEEENS_5ErrorERS3_RT_DpRT0_,
>>> __ZNSt3__16vectorIN4llvm3orc6shared25MachOJITDylibInitializersENS_9allocatorIS4_EEE6resizeEm,
>>> __ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__112basic_stringIcNS4_11char_traitsIcEENS4_9allocatorIcEEEESA_vE11deserializeERNS1_14RawByteChannelERSA_,
>>> __ZN4llvm3orc6shared23fromWrapperFunctionBlobIJyEEENS_5ErrorENS_8ArrayRefIhEEDpRT_,
>>> __ZTIN4llvm6detail23provider_format_adapterIRjEE,
>>> __ZN4llvm3orc6shared21WrapperFunctionResult20getAnyOutOfBandErrorEv,
>>> __ZTIN4llvm3orc6shared20VectorRawByteChannelE,
>>> ___orc_rt_macho_get_initializers_remote,
>>> __ZTIN4llvm3orc6shared14RawByteChannelE,
>>> __ZNSt3__16vectorIhNS_9allocatorIhEEE26__swap_out_circular_bufferERNS_14__split_bufferIhRS2_EE,
>>> __ZN4llvm3orc6shared19SerializationTraitsINS1_20VectorRawByteChannelENSt3__16vectorINS1_25MachOJITDylibInitializers13SectionExtentENS4_9allocatorIS7_EEEESA_vE11deserializeERS3_RSA_
>>> }) }
>>> /Users/staefsn/Develop/LLVM/monorepo/llvm-orc-runtime/build/bin/llvm-jitlink-executor:Response
>>> has unknown sequence number 527162
>>>
>>> On 18/01/2021 08:55, Lang Hames wrote:
>>>
>>> Hi All,
>>>
>>> Happy 2021!
>>>
>>> I've just posted a new Orc Runtime Preview patch:
>>> https://github.com/lhames/llvm-project/commit/8833a7f24693f1c7a3616438718e7927c6624894
>>>
>>> Quick background:
>>>
>>> To date, neither ORC nor MCJIT have had their own runtime libraries.
>>> This has limited and complicated the implementation of many features (e.g.
>>> jit re-entry functions, exception handling, JID'd initializers and
>>> de-initializers), and more-or-less prevented the implementation of others
>>> (e.g. native thread local storage).
>>>
>>> Late last year I started work on a prototype ORC runtime library to
>>> address this, and with the above commit I've finally got something worth
>>> sharing.
>>>
>>> The prototype above is simultaneously limited and complex. Limited, in
>>> that it only tackles a small subset of the desired functionality. Complex
>>> in that it's one of the most involved pieces of functionality that I
>>> anticipate supporting, as it requires two-way communication between the
>>> executor and JIT processes. My aim in choosing to tackle the hard part
>>> first was to get a sense of our ultimate requirements for the project,
>>> particularly in regards to *where it should live within the LLVM
>>> Project*. It's not a perfect fit for LLVM proper: there will be lots of
>>> target specific code, including assembly, and it should be easily buildable
>>> for multiple targets (that sounds more like compiler-rt). On the other hand
>>> it's not a perfect fit for compiler-rt: it shares data structures with
>>> LLVM, and it would be very useful to be able to re-use llvm::Error  /
>>> llvm::Expected (that sounds like LLVM). At the moment I think the best way
>>> to square things would be to keep it in compiler-rt, allow inclusion of
>>> header-only code from LLVM in compiler-rt, and then make Error / Expected
>>> header-only (or copy / adapt them for this library). This will be a
>>> discussion for llvm-dev at some point in the near future.
>>>
>>> On to the actual functionality though: The prototype makes significant
>>> changes to the MachOPlatform class and introduces an ORC runtime library in
>>> compiler-rt/lib/orc. Together, these changes allow us to emulate the dlopen
>>> / dlsym / dlclose in the JIT executor process. We can use this to define
>>> what it means to run a *JIT program*, rather than just running a JIT
>>> function (the way TargetProcessControl::runAsMain does):
>>>
>>> ORC_RT_INTERFACE int64_t __orc_rt_macho_run_program(int argc, char *argv[])
>>> {
>>>   using MainTy = int (*)(int, char *[]);
>>>
>>>   void *H = __orc_rt_macho_jit_dlopen("Main", ORC_RT_RTLD_LAZY);
>>>   if (!H) {
>>>     __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>>>     return -1;
>>>   }
>>>
>>>   auto *Main = reinterpret_cast<MainTy>(__orc_rt_macho_jit_dlsym(H,
>>> "main"));
>>>   if (!Main) {
>>>     __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>>>     return -1;
>>>   }
>>>
>>>   int Result = Main(argc, argv);
>>>
>>>   if (__orc_rt_macho_jit_dlclose(H) == -1)
>>>     __orc_rt_log_error(__orc_rt_macho_jit_dlerror());
>>>
>>>   return Result;
>>> }
>>>
>>> The functions __orc_rt_macho_jit_dlopen, __orc_rt_macho_jit_dlsym,
>>> and __orc_rt_macho_jit_dlclose behave the same as their dlfcn.h
>>> counterparts (dlopen, dlsym, dlclose), but operate on JITDylibs rather than
>>> regular dylibs. This includes running static initializers and registering
>>> with language runtimes (e.g. ObjC).
>>>
>>> While we could run static initializers before (e.g. via
>>> LLJIT::runConstructors), we had to initiate this from the JIT process side,
>>> which has two significant drawbacks: (1) Extra RPC round trips, and (2) in
>>> the out-of-process case: initializers not running on the executor thread
>>> that requested them, since that thread will be blocked waiting for its call
>>> to return. Issue (1) only affects performance, but (2) can affect
>>> correctness if the initializers modify thread local values, or interact
>>> with locks or threads. Interacting with threads from initializers is
>>> generally best avoided, but nonetheless is done by real-world code, so we
>>> want to support it. By using the runtime we can improve both performance
>>> and correctness (or at least consistency with current behavior).
>>>
>>> The effect of this is that we can now load C++, Objective-C and Swift
>>> programs in the JIT and expect them to run correctly, at least for simple
>>> cases. This works regardless of whether the JIT'd code runs in-process or
>>> out-of-process. To test all this I have integrated support for the
>>> prototype runtime into llvm-jitlink. You can demo output from this tool
>>> below for two simple input programs: One swift, one C++. All of this is
>>> MachO specific at the moment, but provides a template that could be easily
>>> re-used to support this on ELF platforms, and likely on COFF platforms too.
>>>
>>> While the discussion on where the runtime should live plays out I will
>>> continue adding / moving functionality to the prototype runtime. Next up
>>> will be eh-frame registration and resolver functions (both currently in
>>> OrcTargetProcess). After that I'll try to tackle support for native MachO
>>> thread local storage.
>>>
>>> As always: Questions and comments are very welcome.
>>>
>>> -- Lang.
>>>
>>> lhames at Langs-MacBook-Pro scratch % cat foo.swift
>>> class MyClass {
>>>   func foo() {
>>>     print("foo")
>>>   }
>>> }
>>>
>>> let m = MyClass()
>>> m.foo();
>>>
>>> lhames at Langs-MacBook-Pro scratch % xcrun swiftc -emit-object -o foo.o
>>> foo.swift
>>> lhames at Langs-MacBook-Pro scratch % llvm-jitlink -dlopen
>>> /usr/lib/swift/libswiftCore.dylib foo.o
>>> foo
>>> lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor -dlopen
>>> /usr/lib/swift/libswiftCore.dylib foo.o
>>> foo
>>> lhames at Langs-MacBook-Pro scratch % cat inits.cpp
>>> #include <iostream>
>>>
>>> class Foo {
>>> public:
>>>   Foo() { std::cout << "Foo::Foo()\n"; }
>>>   ~Foo() { std::cout << "Foo::~Foo()\n"; }
>>>   void foo() { std::cout << "Foo::foo()\n"; }
>>> };
>>>
>>> Foo F;
>>>
>>> int main(int argc, char *argv[]) {
>>>   F.foo();
>>>   return 0;
>>> }
>>> lhames at Langs-MacBook-Pro scratch % xcrun clang++ -c -o inits.o inits.cpp
>>> lhames at Langs-MacBook-Pro scratch % llvm-jitlink inits.o
>>>
>>> Foo::Foo()
>>> Foo::foo()
>>> Foo::~Foo()
>>> lhames at Langs-MacBook-Pro scratch % llvm-jitlink -oop-executor inits.o
>>> Foo::Foo()
>>> Foo::foo()
>>> Foo::~Foo()
>>>
>>> -- https://flowcrypt.com/pub/stefan.graenitz@gmail.com
>>>
>>> -- https://flowcrypt.com/pub/stefan.graenitz@gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210124/61d0e104/attachment-0001.html>