[llvm-dev] lld: ELF/COFF main() interface

[Rui Ueyama via llvm-dev]

I think I have an idea to cover your need and possibly other people's on
this thread. It provides the "main() as a library function" feature,
input/output files wouldn't go through disks nor file systems, and it
doesn't require any major design changes. Sounds too good?

That is, we can provide a function that takes command line parameters, do
fork, and call the linker's main function.

This may sound too simple, but I think that is fairly powerful. Because the
child process has a copy(-on-write) of the parent's memory, it can read
parent's in-memory object files directly with no overhead. The child can
pass the resulting file back to the parent through a shared memory object
(which we can obtain using shm_open or something like that). In addition to
that, your main process gets a protection from linker's bugs thanks to the
operating system's memory protection. But from the user's point of view,
that is just a linker's main function that you can call and that works as
expected.

Even if we want to call exec for whatever reason, we can copy in-memory
objects to shared memory objects and exec the linker, so the basic design
should work in such case too.

The function signature would be something like:

  bool link(ArrayRef<LinkerArg> CommandLineArgs, MemoryBuffer &OutputFile,
std::string &ErrorMsg);

where the return value indicates success/failure. LinkerArg is a union type
of StringRef and MemoryBufferRef. The result is returned as OutputFile
memory buffer. If it prints out any message, ErrorMsg will hold it.

(I want to point out that the function "bool link(ArrayRef<const char*>
args, raw_ostream& diagnostics, ArrayRef<unique_ptr<MemoryBuffer>> inputs)"
doesn't work because the order of command line parameters matters. Some
command line parameters, such as --whole-archive/--no-whole-archive,
affects how files in between will be interpreted, so you can't separate
command line parameters from a list of files.)

I think this is a practical solution that we can do now. I can implement
this for you.

On Thu, Jan 21, 2016 at 9:28 PM, Arseny Kapoulkine <
arseny.kapoulkine at gmail.com> wrote:

> > In any case, I have simply wasted too much time on a thread with
> someone with no patches on the new elf linker. It is really annoying that
> you don't put effort into it and seem entitled to dictate its direction.
>
> Sorry about that. I was initially planning to work on a patch to enhance
> the interface for new lld - hence my questions in the original post. Since
> I learned that people writing the code for lld are hostile to the idea of
> linker-as-a-library, error_code is treated as spaghetti (which would be
> fine if LLVM used exceptions which it does not) and patches, even if
> submitted, will not actually be reviewed in a timely manner, I'll try to
> adapt my code to either not use lld or use lld-as-a-binary.
>
> I'm disappointed by all of this but obviously it's not my project so I
> should not have a say in this.
>
> Thank you for your time.
>
> Arseny
>
> On Thu, Jan 21, 2016 at 8:44 PM, Rafael Espíndola <
> rafael.espindola at gmail.com> wrote:
>
>> > Also, one of the other possible motivations of using LLD directly from
>> Clang would be to avoid process overhead on operating systems where that is
>> a much more significant part of the compile time cost. We could today
>> actually take the fork out of the Clang driver because the Clang frontend
>> *is* designed in this way. But we would also need LLD to work in this way.
>>
>> Then go change clang and send a patch for lld once you are done. It will
>> be interested to see if you can measure a single fork in an entire build.
>>
>> Even better, please finish the new pass manager before working on clang
>> forking cc1.
>>
>> In any case, I have simply wasted too much time on a thread with someone
>> with no patches on the new elf linker. It is really annoying that you don't
>> put effort into it and seem entitled to dictate its direction.
>>
>> If you want to kick us out of the llvm project, please start a thread on
>> llvm-dev.
>>
>> If you want lld to be a library, figure out how to do it without
>> sacrificing lld's productivity,  error reporting and performance (no
>> error_code spaghetti) and write a patch. Just don't expect it to be
>> reviewed while we have actual missing features.
>>
>> I will go back to implementing the linker.
>>
>> Rafael
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160122/1b61c8cd/attachment.html>