[llvm-dev] lld: ELF/COFF main() interface

[Rui Ueyama via llvm-dev]

On Thu, Jan 7, 2016 at 2:52 PM, Eric Christopher <echristo at gmail.com> wrote:

> There are lots of good reasons to have it as a library ranging from
> embedding it in processes as an in-process linker (as Dave mentions later
> in thread) to making it more implicitly testable, to being able to do
> complete build to executable/library/etc with a single command (this goes
> back to one, but is a more specific example). There are many good reasons -
> some of which are being worked on with a library based lld as a
> prerequisite to have lld be a library based linker.
>
> Rewriting every function as ErrorOr sounds terrible and we should avoid
> that as much as possible, but keeping the general llvm style of "library
> first" seems to be an important use case.
>

Do you have any idea to avoid ErrorOr? We do not have a pass to ensure that
inputs are correct (because such pass would be slow). Most data is read
on-demand, so almost everything can fail.


> -eric
>
>
> On Thu, Jan 7, 2016 at 9:06 AM Rui Ueyama via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> Designing it a command makes things simpler because you can safely assume
>> that most functions always success, or otherwise the entire process
>> terminates. You can defer the operating system to clean up all resources
>> that was used by the process however it fails (with a few exceptions such
>> as temporary files.) I actually like to use the linker as an external
>> command since the operating system provides good isolation between my
>> process and the linker which could fail by an unknown bug or something.
>> Rewriting all of them as functions that return ErrorOr<Something> is
>> technically doable, but it needs strong justification, so I guess that
>> unlikely to change.
>>
>> Why do you want to use that as a library? I don't think "because LLVM and
>> Clang allow that" is not a compelling argument since the linker and the
>> compiler are pretty different programs. I clearly see many reasons to use
>> LLVM and part of Clang as libraries, but they are not directly applicable
>> to LLD. The new linker is more like a command which is built on top of the
>> libraries that LLVM provides.
>>
>> On Thu, Jan 7, 2016 at 8:28 AM, Arseny Kapoulkine <
>> arseny.kapoulkine at gmail.com> wrote:
>>
>>> This is really unfortunate.
>>>
>>> I've read the discussion threads for the atom/chunk controversy and I
>>> feel like I understand the reasons for rewriting the linker, but this does
>>> not seem to have anything to do with whether the linker is usable as a
>>> library or not.
>>>
>>> As it stands, not only does lld have two completely different linkers
>>> (I'm treating COFF/ELF2 as one since they are really two different
>>> implementations of the same concept, AFAIU), but one is usable as a library
>>> (and even does not require round-tripping generated code through an object
>>> file! I was really happy to use that) and the other one isn't. Not sure
>>> what the future plans are for Mach-O linker (at this point it seems logical
>>> to rewrite that using the new designs but I'm not sure if it ever happens),
>>> so maybe at some point we'll just have one linker application instead of a
>>> library and an application.
>>>
>>> Anyway, since linker is the only missing piece for full compilation
>>> stack (source language to runnable executable), it's sad to see this
>>> specific part of LLVM not working as a library when everything else does.
>>>
>>> Are there specific concerns in terms of implementation that prevent new
>>> lld from being a library? I understand that using global variables and
>>> error() function is simpler, but the rest of LLVM does not do that and the
>>> codebase there is significantly larger. Am I missing any other issues
>>> except for the ones I mentioned in my original e-mail that will come up in
>>> a library-like usage scenario?
>>>
>>> Arseny
>>>
>>
>>> On Thu, Jan 7, 2016 at 7:17 AM, Rui Ueyama <ruiu at google.com> wrote:
>>>
>>>> On Thu, Jan 7, 2016 at 7:03 AM, Arseny Kapoulkine via llvm-dev <
>>>> llvm-dev at lists.llvm.org> wrote:
>>>>
>>>>> In the process of migrating from old lld ELF linker to new (previously
>>>>> ELF2) I noticed the interface lost several important features (ordered by
>>>>> importance for my use case):
>>>>>
>>>>> 1. Detecting errors in the first place. New linker seems to call
>>>>> exit(1) for any error.
>>>>>
>>>>> 2. Reporting messages to non-stderr outputs. Previously all link
>>>>> functions had a raw_ostream argument so it was possible to delay the error
>>>>> output, aggregate it for multiple linked files, output via a different
>>>>> format, etc.
>>>>>
>>>>> 3. Linking multiple outputs in parallel (useful for test drivers) in a
>>>>> single process. Not really an interface issue but there are at least two
>>>>> global pointers (Config & Driver) that refer to stack variables and are
>>>>> used in various places in the code.
>>>>>
>>>>> All of this seems to indicate a departure from the linker being
>>>>> useable as a library. To maintain the previous behavior you'd have to use a
>>>>> linker binary & popen.
>>>>>
>>>>> Is this a conscious design decision or a temporary limitation?
>>>>>
>>>>
>>>> That the new ELF and COFF linkers are designed as commands instead of
>>>> libraries is very much an intended design change.
>>>>
>>>>
>>>>> If it's a limitation, how would one go about fixing this? I'm not too
>>>>> familiar with the idiomatical error handling in LLVM. Normally in this
>>>>> situation I'd just throw from error() but lld is probably compiled without
>>>>> exceptions. Is ErrorOr the established practice? How does LLVM generally
>>>>> deal with error handling for parsers (like linker script parser) where it's
>>>>> a lot of mutually recursive functions where every single one can fail?
>>>>>
>>>>
>>>> Since it's as designed, please run the linker as an external command
>>>> using fork/exec (or wrappers of them) instead of trying to use that inside
>>>> the same process.
>>>>
>>>>
>>>>> Arseny
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>>
>>>>
>>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160107/23c653a9/attachment.html>