[llvm-dev] Inclusion of the ORC runtime in compiler-rt.

[Eric Christopher via llvm-dev]

On Mon, Apr 12, 2021 at 6:36 PM Lang Hames <lhames at gmail.com> wrote:

> Hi Petr,
>
> I'd like to better understand the structure of the ORC runtime and its
>> dependencies (both build and runtime). Does it use the C or C++ standard
>> library? Does it depend on other parts of LLVM? Do you plan on reusing some
>> of the existing compiler-rt libraries like sanitizer_common?
>
>
> The ORC runtime currently uses the C++ standard library.
> Since the ORC runtime needs to communicate with the LLVM ORC library it
> also currently uses some header-only includes from LLVM. It does not depend
> on any LLVM libraries. We could duplicate this code, but I'd prefer to
> share it if possible.
> I have not used sanitizer_common, but some parts of it look like they may
> be useful.
>
> I gravitated towards implementing the ORC runtime in compiler-rt because I
> need to be able to write parts of it in platform-specific assembly (which
> compiler-rt supports), and because the runtime should be build for all
> targets, not just the host (which seems to be the standard way that
> compiler-rt is configured).
>
>
>From this it sounds like "convenient reusing of the build system" rather
than "should be included in compiler-rt as a library"? If that's the case
maybe making it clear or lifting the common build system support out might
be maintainable without the "this is a runtime library for the system" sort
of thing?

-eric


> To give a bit more background on why I'm interested, compiler-rt has grown
>> fairly organically this has been making the maintenance more and more
>> difficult, at least from the build perspective. There are some runtimes
>> that only use C, some that use C++, some that use C++ standard library.
>> When building compiler-rt together with other runtimes like libc or libc++,
>> it's difficult to pick up the right order which is why we have several
>> entry points into compiler-rt's build system to build different subsets and
>> that's been a maintenance headache.
>
>
>> I've been thinking about this quite a bit recently and I repeatedly came
>> to the conclusion that compiler-rt would ideally be broken up into several
>> subprojects, but that should probably be discussed as a separate topic.
>> However, understanding the build and runtimes dependencies of the ORC
>> runtime could help us decide whether it should be a part of compiler-rt or
>> be a separate subproject.
>
>
> That makes sense to me. I think of the ORC runtime as a compiler-rt-style
> runtime with a libc++ dependency. In that sense I think it's similar to
> libFuzzer, and whatever solution we come up with for libFuzzer would
> probably also be applicable to the ORC runtime too.
>
> -- Lang.
>
> On Mon, Apr 12, 2021 at 2:36 PM Petr Hosek <phosek at google.com> wrote:
>
>> I'd like to better understand the structure of the ORC runtime and its
>> dependencies (both build and runtime). Does it use the C or C++ standard
>> library? Does it depend on other parts of LLVM? Do you plan on reusing some
>> of the existing compiler-rt libraries like sanitizer_common?
>>
>> To give a bit more background on why I'm interested, compiler-rt has
>> grown fairly organically this has been making the maintenance more and more
>> difficult, at least from the build perspective. There are some runtimes
>> that only use C, some that use C++, some that use C++ standard library.
>> When building compiler-rt together with other runtimes like libc or libc++,
>> it's difficult to pick up the right order which is why we have several
>> entry points into compiler-rt's build system to build different subsets and
>> that's been a maintenance headache.
>>
>> I've been thinking about this quite a bit recently and I repeatedly came
>> to the conclusion that compiler-rt would ideally be broken up into several
>> subprojects, but that should probably be discussed as a separate topic.
>> However, understanding the build and runtimes dependencies of the ORC
>> runtime could help us decide whether it should be a part of compiler-rt or
>> be a separate subproject.
>>
>> On Mon, Apr 12, 2021 at 12:26 PM Lang Hames <lhames at gmail.com> wrote:
>>
>>> Hi All,
>>>
>>> I'd like to add the ORC runtime library (preview available at
>>> https://github.com/lhames/llvm-project/tree/orc-runtime-preview) to
>>> compiler-rt.
>>>
>>> Background:
>>>
>>> ORC, like MCJIT, can link JIT'd code either into the current process
>>> ("in-process" mode) or into a remote executor process ("cross-process"
>>> mode). Some JIT features require support code in the executor process, but
>>> the existing ORC libraries are only linked into the JIT process. This has
>>> made cross-process mode support for those features (which include static
>>> initializers, thread local variables, exception handling, and others)
>>> awkward or impractical to implement. The ORC runtime library aims to
>>> provide the necessary support code in a form that is loadable by the JIT
>>> itself, which should allow these features to work uniformly in both modes.
>>>
>>> My prototype branch of the ORC runtime (available at
>>> https://github.com/lhames/llvm-project/tree/orc-runtime-preview) has
>>> advanced to the point where it can provide uniform support for static
>>> initializers, destructors, exceptions, thread locals, and language
>>> registration for Objective C and Swift code. This support is all
>>> MachO/Darwin only so far, but should be easily adaptable for ELF/Linux/BSD
>>> support.
>>>
>>> Proposal:
>>>
>>> The proof of concept implementation has been very successful, so I would
>>> like to move future development to the LLVM main branch so that others can
>>> benefit from this.
>>>
>>> Before I start posting patches, though:
>>>
>>> Does anyone see any problems with including this in compiler-rt?
>>>
>>> Does anyone think that there is a more reasonable home for the ORC
>>> runtime within the llvm-project? I considered LLVM itself, or a new
>>> top-level project, but neither seemed as natural a fit as compiler-rt.
>>>
>>> Finally, if everyone is happy for it to be included in principle, are
>>> there any volunteers to review ORC runtime patches?
>>>
>>> Regards,
>>> Lang.
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210412/f3bc19e7/attachment.html>