[llvm-dev] [RFC] Support embedding bitcodes in LLD with LTO

[Mehdi AMINI via llvm-dev]

On Thu, Jan 31, 2019 at 3:56 PM Rui Ueyama <ruiu at google.com> wrote:

> We need to set some threshold to avoid feature creeping.
>

I am not sure to understand what you mean by this?
Is this is some sort of view of the tool as a "product" where you want a
close control on the exposed surface or something like this?

I tend to see LLVM as an "open platform" more than a closed "product": it
means that, with the goal of enabling future innovations, I personally tend
to look at proposed new features/patches in light of the cost/burden they
add on the community maintaining the project.

-- 
Mehdi




> As I wrote, lld already has a command line flag to emit a bitcode file
> instead of a compiled one, and perhaps you could do a lot of things you
> think interesting with that option. If you are already using the feature
> and find it less powerful for your purposes, then we can discuss whether
> adding a new feature is the way to go. But as a maintainer of the tool, I
> don't think asking whether or not an existing feature would work as an
> initial response is not unreasonable.
>
> On Thu, Jan 31, 2019 at 3:37 PM Mehdi AMINI <joker.eph at gmail.com> wrote:
>
>>
>>
>> On Thu, Jan 31, 2019 at 11:05 AM Rui Ueyama via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> That feature is probably too specific to your project. Most projects
>>> that use LTO are using LTO just because it generates better code. Your
>>> project is special as your program itself can also interpret LLVM bitcode,
>>> but that's not the case for most other programs.
>>>
>>
>> I agree this is specific "compared to the usual expected output of as
>> linker", but on the other hand it also has potential for opening cool
>> project that can be built on top of this!
>> If this could be supported in lld without too much trouble (maintenance,
>> code complexity, etc.), why not accepting the patches?
>>
>> Best,
>>
>> --
>> Mehdi
>>
>>
>>
>>
>>
>>>
>>> I think the option that's closest to the one you are looking for is
>>> `--plugin-opt=emit-llvm`. That option makes lld to make an output file in
>>> the bitcode file format (so lld doesn't do LTO if the option is given and
>>> instead writes a raw bitcode as an output). With that option, I don't think
>>> it's too hard to embed bitcode file to your executable. Run the linker
>>> twice, with and without `--plugin-opt=emit-llvm`, and embed the generated
>>> bitcode file using objcopy.
>>>
>>> Does that work for you?
>>>
>>> On Thu, Jan 31, 2019 at 4:18 AM Josef Eisl <josef.eisl at oracle.com>
>>> wrote:
>>>
>>>> Thanks for your response!
>>>>
>>>> On 30/01/2019 20:18, Rui Ueyama wrote:
>>>> > Hi Josef,
>>>> >
>>>> > Let me clarify my understanding. Do you want to keep original bitcode
>>>> > files in the output executable when doing LTO, so that the resulting
>>>> > executable contains both compiled bitcode (which is in native machine
>>>> > instructions) and original bitcode files?
>>>>
>>>> Exactly! Kind of analogous to what `clang -fembed-bitcode -c` does, but
>>>> for executables.
>>>>
>>>> >
>>>> > Did you try embedding bitcode files into existing ELF files using
>>>> > objcopy or linker option `--format=binary`?
>>>>
>>>> Yes, that is the alternative. However, having support in the linker for
>>>> that would require less tweaking of exiting build systems. Adding an
>>>> option to CFLAGS/LDFLAGS would then be sufficient.
>>>>
>>>> >
>>>> > On Mon, Jan 28, 2019 at 6:41 AM Josef Eisl via llvm-dev
>>>> > <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>> >
>>>> >     Hi everybody!
>>>> >
>>>> >     I'm Josef and I'm working at Oracle Labs on Sulong [1,2], the
>>>> LLVM IR
>>>> >     execution engine in GraalVM [3]. In addition to executing bare
>>>> bitcode
>>>> >     files, Sulong also accepts ELF files with embedded bitcode
>>>> sections.
>>>> >     Therefore, it would be great if LLD in (Full)LTO mode would
>>>> support
>>>> >     embedding bitcode sections to the resulting object file. Is that
>>>> >     something that would be considered useful and worth contributing?
>>>> >
>>>> >     Thanks,
>>>> >     Josef
>>>> >
>>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190131/856845c9/attachment.html>