[llvm-dev] lld: ELF/COFF main() interface

[Rui Ueyama via llvm-dev]

If you want to link ELF object files, you are likely to be using a Unix
machine. I'm not trying to address all possible problems but suggesting a
practical solution.

On Fri, Jan 22, 2016 at 9:49 AM, Yaron Keren <yaron.keren at gmail.com> wrote:

> On Windows fork() is not available. If exec() is used instead, process
> creation time is several times slower than Linux. This may be or not a
> problem dependeing on how lld is used. In general, on Windows the best
> solution is multi-threading.
>
>
> 2016-01-22 18:31 GMT+02:00 Rui Ueyama via llvm-dev <
> llvm-dev at lists.llvm.org>:
>
>> I think I have an idea to cover your need and possibly other people's on
>> this thread. It provides the "main() as a library function" feature,
>> input/output files wouldn't go through disks nor file systems, and it
>> doesn't require any major design changes. Sounds too good?
>>
>> That is, we can provide a function that takes command line parameters, do
>> fork, and call the linker's main function.
>>
>> This may sound too simple, but I think that is fairly powerful. Because
>> the child process has a copy(-on-write) of the parent's memory, it can read
>> parent's in-memory object files directly with no overhead. The child can
>> pass the resulting file back to the parent through a shared memory object
>> (which we can obtain using shm_open or something like that). In addition to
>> that, your main process gets a protection from linker's bugs thanks to the
>> operating system's memory protection. But from the user's point of view,
>> that is just a linker's main function that you can call and that works as
>> expected.
>>
>> Even if we want to call exec for whatever reason, we can copy in-memory
>> objects to shared memory objects and exec the linker, so the basic design
>> should work in such case too.
>>
>> The function signature would be something like:
>>
>>   bool link(ArrayRef<LinkerArg> CommandLineArgs, MemoryBuffer
>> &OutputFile, std::string &ErrorMsg);
>>
>> where the return value indicates success/failure. LinkerArg is a union
>> type of StringRef and MemoryBufferRef. The result is returned as OutputFile
>> memory buffer. If it prints out any message, ErrorMsg will hold it.
>>
>> (I want to point out that the function "bool link(ArrayRef<const char*>
>> args, raw_ostream& diagnostics, ArrayRef<unique_ptr<MemoryBuffer>>
>> inputs)" doesn't work because the order of command line parameters matters.
>> Some command line parameters, such as --whole-archive/--no-whole-archive,
>> affects how files in between will be interpreted, so you can't separate
>> command line parameters from a list of files.)
>>
>> I think this is a practical solution that we can do now. I can implement
>> this for you.
>>
>> On Thu, Jan 21, 2016 at 9:28 PM, Arseny Kapoulkine <
>> arseny.kapoulkine at gmail.com> wrote:
>>
>>> > In any case, I have simply wasted too much time on a thread with
>>> someone with no patches on the new elf linker. It is really annoying that
>>> you don't put effort into it and seem entitled to dictate its direction.
>>>
>>> Sorry about that. I was initially planning to work on a patch to enhance
>>> the interface for new lld - hence my questions in the original post. Since
>>> I learned that people writing the code for lld are hostile to the idea of
>>> linker-as-a-library, error_code is treated as spaghetti (which would be
>>> fine if LLVM used exceptions which it does not) and patches, even if
>>> submitted, will not actually be reviewed in a timely manner, I'll try to
>>> adapt my code to either not use lld or use lld-as-a-binary.
>>>
>>> I'm disappointed by all of this but obviously it's not my project so I
>>> should not have a say in this.
>>>
>>> Thank you for your time.
>>>
>>> Arseny
>>>
>>> On Thu, Jan 21, 2016 at 8:44 PM, Rafael Espíndola <
>>> rafael.espindola at gmail.com> wrote:
>>>
>>>> > Also, one of the other possible motivations of using LLD directly
>>>> from Clang would be to avoid process overhead on operating systems where
>>>> that is a much more significant part of the compile time cost. We could
>>>> today actually take the fork out of the Clang driver because the Clang
>>>> frontend *is* designed in this way. But we would also need LLD to work in
>>>> this way.
>>>>
>>>> Then go change clang and send a patch for lld once you are done. It
>>>> will be interested to see if you can measure a single fork in an entire
>>>> build.
>>>>
>>>> Even better, please finish the new pass manager before working on clang
>>>> forking cc1.
>>>>
>>>> In any case, I have simply wasted too much time on a thread with
>>>> someone with no patches on the new elf linker. It is really annoying that
>>>> you don't put effort into it and seem entitled to dictate its direction.
>>>>
>>>> If you want to kick us out of the llvm project, please start a thread
>>>> on llvm-dev.
>>>>
>>>> If you want lld to be a library, figure out how to do it without
>>>> sacrificing lld's productivity,  error reporting and performance (no
>>>> error_code spaghetti) and write a patch. Just don't expect it to be
>>>> reviewed while we have actual missing features.
>>>>
>>>> I will go back to implementing the linker.
>>>>
>>>> Rafael
>>>>
>>>
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160122/53f7edce/attachment.html>