<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
Thanks for sharing the preview!<br>
<br>
I am referring to this code state:
<a class="moz-txt-link-freetext" href="https://github.com/apple/llvm-project/commit/c305c6389b997e73">https://github.com/apple/llvm-project/commit/c305c6389b997e73</a><br>
<br>
It looks like WrapperFunctionUtils in OrcShared define the common
data structures that can be passed back and forth between runtime
and JIT. It contains a non-trivial amount of code, but all in all it
appears to be self-contained and header-only. I guess that's because
the runtime is supposed to not depend on any LLVM libs?<br>
<br>
The duplication of the Error, Expected<T> and ExtensibleRTTI
classes is a little unfortunate. I assume we won't need arbitrary
data structures in the runtime and for communication with it, but
what we need must be duplicated? Isn't there any way to avoid it? A
separate Error lib above Support? It's probably fine as is for the
moment, but maybe a mid-term perspective could be discussed when
integrating into mainline.<br>
<br>
There are no platform-specific flavors of the clang_rt.orc static
library. The memory footprint will likely be small, but won't it
impact JITLink performance since it will have to process symbols for
all platforms (<span class="blob-code-inner blob-code-marker"
data-code-marker="+"><span class="pl-en">__orc_rt_macho_tlv_get_addr</span></span>,
__orc_rt_elf_tlv_get_addr, etc.)? If not, then why? Can they be
dead-stripped early on?<br>
<br>
Best,<br>
Stefan<br>
<br>
<div class="moz-cite-prefix">On 12/04/2021 04:53, Lang Hames wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CALLttgpYkKs+rKfTUJBTG5D6N91ooOq7PdLJ-TgBAz54ZnEv9w@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">Hi All,
<div><br>
</div>
<div>Apologies for the lack of updates -- I've been flat
out prototyping the ORC runtime, but the details of
that process weren't particularly newsworthy. Good
news though: The result that work is available in the
new preview branch at <a
href="https://github.com/lhames/llvm-project/pull/new/orc-runtime-preview"
moz-do-not-send="true">https://github.com/lhames/llvm-project/pull/new/orc-runtime-preview</a>,
and it's starting to look pretty good.</div>
<div><br>
</div>
<div>A quick recap of this project, since it's been a
while since my last update: Some features of
object files (e.g. thread local variables, exceptions,
static initializers, and language metadata
registration) require support code in the executor
process. We also want support code in the executing
process for other JIT features (e.g. laziness). The
ORC runtime is meant to provide a container for that
support code. The runtime can be loaded into the
executor process via the JIT, and the executor and JIT
processes can communicate via a builtin serialization
format (either serializing/deserializing directly
in-process, or communicating serialized data via
IPC/RPC) to coordinate on complex operations, for
example discovering (on the JIT side) and running (on
the executor side) all initializers in a JITDylib.</div>
<div><br>
</div>
<div>After a bit of hacking, the setup code for all this
is looking very neat and tidy. For example, to turn on
support for advanced MachO features:</div>
<div><br>
</div>
<div>
<div><font face="monospace">if (auto P =
MachOPlatform::Create(ES, ObjLinkingLayer, TPC,
MainJD,<br>
</font></div>
<div><font face="monospace">
OrcRuntimePath))</font></div>
<div><font face="monospace">
ES.setPlatform(std::move(*P));</font></div>
<div><font face="monospace">else</font></div>
<div><font face="monospace"> return P.takeError();</font></div>
<div><br>
</div>
</div>
<div>That's it. The MachOPlatform loads the runtime
archive into the ObjectLinkingLayer, then installs an
ObjectLinkingLayer::Plugin to scan all loaded objects
for features that it needs to react to. When it
encounters such a feature it looks up the
corresponding runtime functionality (loading the
runtime support into the executor as required) and
calls over to the runtime to react. For example, if an
object contains an __eh_frame section then the plugin
will discover its address range during linking and
call over to the runtime to register that range with
libunwind.</div>
<div><br>
</div>
<div>Having set up the platform, you can add objects
compiled from C++, Objective-C, Swift, etc., using
static initializers, thread locals, etc. and
everything should Just Work.</div>
<div><br>
</div>
<div>Of course it doesn't all Just Work yet: The
plumbing work is mostly complete, but I haven't
written handlers for all the special sections yet. A
surprising number of things do work (e.g. C++ code
with static initializers/destructors, TLVs and
exceptions, and simple Objective-C and Swift
programs). An equally surprising number of simple
things don't work (zero-initialized thread locals fail
because I haven't gotten around to handling .tbss
sections yet).</div>
<div><br>
</div>
<div>If you would like to play around with the runtime
(and have access to an x86-64 Mac) you can build it by
checking out the preview branch above and configuring
LLVM like this:</div>
<div><br>
</div>
<div><font face="monospace">xcrun cmake -GNinja \</font></div>
<div><font face="monospace"> -DCMAKE_BUILD_TYPE=Debug \</font></div>
<div><font face="monospace">
-DLLVM_ENABLE_PROJECTS="llvm;clang" \</font></div>
<div><font face="monospace">
-DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi"
\</font></div>
<div><font face="monospace"> /path/to/llvm</font></div>
<div><br>
</div>
<div>Then you can try running arbitrary MachO objects
under llvm-jitlink, which has been updated to load the
built runtime by default. You should be able to run
the objects both in-process (the default), and
out-of-process (using -oop-executor or
-oop-executor-connect) and have them behave exactly
the same way.</div>
<div><br>
</div>
<div>What we have so far is a pretty good proof of
concept, so I'll start a new thread on llvm-dev
tomorrow to discuss how we can land this in the LLVM
mainline.</div>
<div><br>
</div>
<div>-- Lang.<br>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="https://flowcrypt.com/pub/stefan.graenitz@gmail.com">https://flowcrypt.com/pub/stefan.graenitz@gmail.com</a></pre>
</body>
</html>