[llvm-dev] Inclusion of the ORC runtime in compiler-rt.

[Lang Hames via llvm-dev]

Hi Petr,

I'd like to better understand the structure of the ORC runtime and its
> dependencies (both build and runtime). Does it use the C or C++ standard
> library? Does it depend on other parts of LLVM? Do you plan on reusing some
> of the existing compiler-rt libraries like sanitizer_common?


The ORC runtime currently uses the C++ standard library.
Since the ORC runtime needs to communicate with the LLVM ORC library it
also currently uses some header-only includes from LLVM. It does not depend
on any LLVM libraries. We could duplicate this code, but I'd prefer to
share it if possible.
I have not used sanitizer_common, but some parts of it look like they may
be useful.

I gravitated towards implementing the ORC runtime in compiler-rt because I
need to be able to write parts of it in platform-specific assembly (which
compiler-rt supports), and because the runtime should be build for all
targets, not just the host (which seems to be the standard way that
compiler-rt is configured).

To give a bit more background on why I'm interested, compiler-rt has grown
> fairly organically this has been making the maintenance more and more
> difficult, at least from the build perspective. There are some runtimes
> that only use C, some that use C++, some that use C++ standard library.
> When building compiler-rt together with other runtimes like libc or libc++,
> it's difficult to pick up the right order which is why we have several
> entry points into compiler-rt's build system to build different subsets and
> that's been a maintenance headache.


> I've been thinking about this quite a bit recently and I repeatedly came
> to the conclusion that compiler-rt would ideally be broken up into several
> subprojects, but that should probably be discussed as a separate topic.
> However, understanding the build and runtimes dependencies of the ORC
> runtime could help us decide whether it should be a part of compiler-rt or
> be a separate subproject.


That makes sense to me. I think of the ORC runtime as a compiler-rt-style
runtime with a libc++ dependency. In that sense I think it's similar to
libFuzzer, and whatever solution we come up with for libFuzzer would
probably also be applicable to the ORC runtime too.

-- Lang.

On Mon, Apr 12, 2021 at 2:36 PM Petr Hosek <phosek at google.com> wrote:

> I'd like to better understand the structure of the ORC runtime and its
> dependencies (both build and runtime). Does it use the C or C++ standard
> library? Does it depend on other parts of LLVM? Do you plan on reusing some
> of the existing compiler-rt libraries like sanitizer_common?
>
> To give a bit more background on why I'm interested, compiler-rt has grown
> fairly organically this has been making the maintenance more and more
> difficult, at least from the build perspective. There are some runtimes
> that only use C, some that use C++, some that use C++ standard library.
> When building compiler-rt together with other runtimes like libc or libc++,
> it's difficult to pick up the right order which is why we have several
> entry points into compiler-rt's build system to build different subsets and
> that's been a maintenance headache.
>
> I've been thinking about this quite a bit recently and I repeatedly came
> to the conclusion that compiler-rt would ideally be broken up into several
> subprojects, but that should probably be discussed as a separate topic.
> However, understanding the build and runtimes dependencies of the ORC
> runtime could help us decide whether it should be a part of compiler-rt or
> be a separate subproject.
>
> On Mon, Apr 12, 2021 at 12:26 PM Lang Hames <lhames at gmail.com> wrote:
>
>> Hi All,
>>
>> I'd like to add the ORC runtime library (preview available at
>> https://github.com/lhames/llvm-project/tree/orc-runtime-preview) to
>> compiler-rt.
>>
>> Background:
>>
>> ORC, like MCJIT, can link JIT'd code either into the current process
>> ("in-process" mode) or into a remote executor process ("cross-process"
>> mode). Some JIT features require support code in the executor process, but
>> the existing ORC libraries are only linked into the JIT process. This has
>> made cross-process mode support for those features (which include static
>> initializers, thread local variables, exception handling, and others)
>> awkward or impractical to implement. The ORC runtime library aims to
>> provide the necessary support code in a form that is loadable by the JIT
>> itself, which should allow these features to work uniformly in both modes.
>>
>> My prototype branch of the ORC runtime (available at
>> https://github.com/lhames/llvm-project/tree/orc-runtime-preview) has
>> advanced to the point where it can provide uniform support for static
>> initializers, destructors, exceptions, thread locals, and language
>> registration for Objective C and Swift code. This support is all
>> MachO/Darwin only so far, but should be easily adaptable for ELF/Linux/BSD
>> support.
>>
>> Proposal:
>>
>> The proof of concept implementation has been very successful, so I would
>> like to move future development to the LLVM main branch so that others can
>> benefit from this.
>>
>> Before I start posting patches, though:
>>
>> Does anyone see any problems with including this in compiler-rt?
>>
>> Does anyone think that there is a more reasonable home for the ORC
>> runtime within the llvm-project? I considered LLVM itself, or a new
>> top-level project, but neither seemed as natural a fit as compiler-rt.
>>
>> Finally, if everyone is happy for it to be included in principle, are
>> there any volunteers to review ORC runtime patches?
>>
>> Regards,
>> Lang.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210412/ae954bb5/attachment.html>