[llvm-dev] lld: ELF/COFF main() interface

[Yaron Keren via llvm-dev]

On Windows fork() is not available. If exec() is used instead, process
creation time is several times slower than Linux. This may be or not a
problem dependeing on how lld is used. In general, on Windows the best
solution is multi-threading.


2016-01-22 18:31 GMT+02:00 Rui Ueyama via llvm-dev <llvm-dev at lists.llvm.org>
:

> I think I have an idea to cover your need and possibly other people's on
> this thread. It provides the "main() as a library function" feature,
> input/output files wouldn't go through disks nor file systems, and it
> doesn't require any major design changes. Sounds too good?
>
> That is, we can provide a function that takes command line parameters, do
> fork, and call the linker's main function.
>
> This may sound too simple, but I think that is fairly powerful. Because
> the child process has a copy(-on-write) of the parent's memory, it can read
> parent's in-memory object files directly with no overhead. The child can
> pass the resulting file back to the parent through a shared memory object
> (which we can obtain using shm_open or something like that). In addition to
> that, your main process gets a protection from linker's bugs thanks to the
> operating system's memory protection. But from the user's point of view,
> that is just a linker's main function that you can call and that works as
> expected.
>
> Even if we want to call exec for whatever reason, we can copy in-memory
> objects to shared memory objects and exec the linker, so the basic design
> should work in such case too.
>
> The function signature would be something like:
>
>   bool link(ArrayRef<LinkerArg> CommandLineArgs, MemoryBuffer &OutputFile,
> std::string &ErrorMsg);
>
> where the return value indicates success/failure. LinkerArg is a union
> type of StringRef and MemoryBufferRef. The result is returned as OutputFile
> memory buffer. If it prints out any message, ErrorMsg will hold it.
>
> (I want to point out that the function "bool link(ArrayRef<const char*>
> args, raw_ostream& diagnostics, ArrayRef<unique_ptr<MemoryBuffer>>
> inputs)" doesn't work because the order of command line parameters matters.
> Some command line parameters, such as --whole-archive/--no-whole-archive,
> affects how files in between will be interpreted, so you can't separate
> command line parameters from a list of files.)
>
> I think this is a practical solution that we can do now. I can implement
> this for you.
>
> On Thu, Jan 21, 2016 at 9:28 PM, Arseny Kapoulkine <
> arseny.kapoulkine at gmail.com> wrote:
>
>> > In any case, I have simply wasted too much time on a thread with
>> someone with no patches on the new elf linker. It is really annoying that
>> you don't put effort into it and seem entitled to dictate its direction.
>>
>> Sorry about that. I was initially planning to work on a patch to enhance
>> the interface for new lld - hence my questions in the original post. Since
>> I learned that people writing the code for lld are hostile to the idea of
>> linker-as-a-library, error_code is treated as spaghetti (which would be
>> fine if LLVM used exceptions which it does not) and patches, even if
>> submitted, will not actually be reviewed in a timely manner, I'll try to
>> adapt my code to either not use lld or use lld-as-a-binary.
>>
>> I'm disappointed by all of this but obviously it's not my project so I
>> should not have a say in this.
>>
>> Thank you for your time.
>>
>> Arseny
>>
>> On Thu, Jan 21, 2016 at 8:44 PM, Rafael Espíndola <
>> rafael.espindola at gmail.com> wrote:
>>
>>> > Also, one of the other possible motivations of using LLD directly from
>>> Clang would be to avoid process overhead on operating systems where that is
>>> a much more significant part of the compile time cost. We could today
>>> actually take the fork out of the Clang driver because the Clang frontend
>>> *is* designed in this way. But we would also need LLD to work in this way.
>>>
>>> Then go change clang and send a patch for lld once you are done. It will
>>> be interested to see if you can measure a single fork in an entire build.
>>>
>>> Even better, please finish the new pass manager before working on clang
>>> forking cc1.
>>>
>>> In any case, I have simply wasted too much time on a thread with someone
>>> with no patches on the new elf linker. It is really annoying that you don't
>>> put effort into it and seem entitled to dictate its direction.
>>>
>>> If you want to kick us out of the llvm project, please start a thread on
>>> llvm-dev.
>>>
>>> If you want lld to be a library, figure out how to do it without
>>> sacrificing lld's productivity,  error reporting and performance (no
>>> error_code spaghetti) and write a patch. Just don't expect it to be
>>> reviewed while we have actual missing features.
>>>
>>> I will go back to implementing the linker.
>>>
>>> Rafael
>>>
>>
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160122/7ed13646/attachment.html>