[llvm-dev] RFC: A new llvm-dlltool driver and llvm-lib driver improvements

Mon Feb 13 13:16:41 PST 2017

>
> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process
> wrapper, meaning that you could add a main function for mingw which
> translates all arguments into the MSVC style and then invoke the COFF
> linker's main function.

That should work, if I am understanding this correctly I can create an
argument parser (probably partially based on the ELF one) then I can set
the data in the COFF Config object and call COFF LinkerDriver::run etc
directly from that?

This proposal is for generally moving code from lld into llvm that could be
part of lib.exe / dlltool, what are your thoughts here?

On Mon, Feb 13, 2017 at 8:57 PM, Rui Ueyama <ruiu at google.com> wrote:

> On Mon, Feb 13, 2017 at 12:52 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
>> Also you need to make a change to LLD/COFF to accept GNU command
>>> arguments, right? (Looks like you already have that patch locally.)
>>
>> Yes
>>
>
> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process
> wrapper, meaning that you could add a main function for mingw which
> translates all arguments into the MSVC style and then invoke the COFF
> linker's main function.
>
>
>>
>> My patch to hack lld into accepting some very basic gnu front end
>>> arguments was enough to get all the above working which was enough to
>>> develop further.
>>
>>
>>
>> On Mon, Feb 13, 2017 at 8:41 PM, Rui Ueyama <ruiu at google.com> wrote:
>>
>>> On Mon, Feb 13, 2017 at 12:33 PM, Martell Malone <
>>> martellmalone at gmail.com> wrote:
>>>
>>>> Hey Rui,
>>>>
>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw
>>>>> support. I don't think dlltool is the last missing piece. What do you need
>>>>> to do other than that to fully support mingw using LLVM toolchain?
>>>>
>>>> Other then changing `lib/MC/WinCOFFStreamer.cpp` to not use -aligncomm
>>>> within the EmitCommonSymbol function and a single patch for mingw-w64
>>>> itself to pre-populate it's .ctors and .dtors list, so llvm-dlltool is
>>>> infact the only missing part really.
>>>>
>>>
>>> Also you need to make a change to LLD/COFF to accept GNU command
>>> arguments, right? (Looks like you already have that patch locally.)
>>>
>>>
>>>> This gives us a fully working clang based mingw-w64 C compiler.
>>>>
>>>> C++ and exception handling is a different story.
>>>>
>>>> libc++ is somewhat working with the following test results
>>>>
>>>> Expected Passes    : 2188
>>>> Expected Failures  : 44
>>>> Unsupported Tests  : 588
>>>> Unexpected Failures: 2816
>>>>
>>>> I was able to build it with exceptions disabled and I actually managed
>>>> to bootstrap llvm and clang itself.
>>>> It didn't run very well though as you can imagine based on the above
>>>> tests.
>>>>
>>>> It's not a performance issue but a code maintenance issue. The initial
>>>>> patches to support mingw was trying to add a new linker driver and a
>>>>> different linkin semantics to the COFF linker which seemed too complicated
>>>>> to me. IIRC, I suggested adding a shim which translates GNU command line
>>>>> arguments to MSVC linker arguments. Didn't it work?
>>>>
>>>> There were just so many differences between link and ld I never
>>>> followed down that path. I pushed forward with the short import library
>>>> support based on your later suggestions. My patch to hack lld into
>>>> accepting some very basic gnu front end arguments was enough to get all the
>>>> above working which was enough to develop further.
>>>>
>>>> Best,
>>>> Martell
>>>>
>>>> On Mon, Feb 13, 2017 at 8:08 PM, Rui Ueyama <ruiu at google.com> wrote:
>>>>
>>>>> Hi Martell,
>>>>>
>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw
>>>>> support. I don't think dlltool is the last missing piece. What do you need
>>>>> to do other than that to fully support mingw using LLVM toolchain?
>>>>>
>>>>> On Mon, Feb 13, 2017 at 8:56 AM, Martell Malone via llvm-dev <
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> Hey llvm'ers,
>>>>>>
>>>>>> I have been working on a dlltool replacement for llvm.
>>>>>> Here is my initial differential https://reviews.llvm.org/D29892
>>>>>> It is based on some functionality that already exists in lld.
>>>>>> I added functionality to support, PE COFF Weak Externals and of
>>>>>> course a front end to actually use it.
>>>>>> I believe the work here can also be used for llvm-lib and lessen the
>>>>>> load on lld.
>>>>>> I would like some comments about how this could be be structured to
>>>>>> live in llvm with a shared code base across lib ar and dlltool.
>>>>>> I also have a section below called "Difference from lib" which is
>>>>>> somewhat of a rationale for the tool.
>>>>>>
>>>>>> Many Thanks,
>>>>>> Martell
>>>>>>
>>>>>>
>>>>>> Context
>>>>>> ==========
>>>>>>
>>>>>> Awhile back I talked to various llvm'ers about getting mingw-w64
>>>>>> support for lld.
>>>>>> There were a few issues raised but the main issue was that mingw-w64
>>>>>> should try best to comply with the PECOFF spec, adding support for custom
>>>>>> sections and various binutils/mingw hacks would impact the performance of
>>>>>> the COFF linker and in general is not something that lld should support.
>>>>>>
>>>>>
>>>>> It's not a performance issue but a code maintenance issue. The initial
>>>>> patches to support mingw was trying to add a new linker driver and a
>>>>> different linkin semantics to the COFF linker which seemed too complicated
>>>>> to me. IIRC, I suggested adding a shim which translates GNU command line
>>>>> arguments to MSVC linker arguments. Didn't it work?
>>>>>
>>>>>
>>>>>> Motivation
>>>>>> ==========
>>>>>>
>>>>>> The main motivation was because dlltool and ld did not comply with
>>>>>> PECOFF Spec.
>>>>>> It has some custom formatting and uses the assembler for some reason
>>>>>> to generate import libraries, it did not use the short import library
>>>>>> format at all.
>>>>>>
>>>>>> There has been many work arounds for the problem this creates such as
>>>>>> the creation of the `reimp` tool. Which imports MSVC built libs creates a
>>>>>> def and uses dlltool so that the binutils linker can use it.
>>>>>>
>>>>>> We should just be using the short import format and the linker should
>>>>>> support that.
>>>>>> Thus llvm-dlltool was born.
>>>>>>
>>>>>>
>>>>>> Difference from lib
>>>>>> ===================
>>>>>>
>>>>>> Using PE COFF spec (section 8, Import Library Format) should be self
>>>>>> explanatory.
>>>>>> lib.exe is able to accept def files and create libraries using this
>>>>>> format.
>>>>>>
>>>>>> example
>>>>>>
>>>>>>   `LIBRARY "user32.dll"
>>>>>>    EXPORTS
>>>>>>    MessageBoxA`
>>>>>>
>>>>>> LIB.exe can create a user32.lib with the function MessageBoxA from
>>>>>> the above definition.
>>>>>>
>>>>>> Mingw-w64 is different MSVC in that we need to compile the runtime.
>>>>>> MS provide us with their crt prebuilt so lib.exe doesn't have support
>>>>>> for external function aliasing.
>>>>>> We often use aliases for posix naming reasons as well as avoid using
>>>>>> the MS version of a function.
>>>>>>
>>>>>> example
>>>>>>
>>>>>>   `LIBRARY "user32.dll"
>>>>>>    EXPORTS
>>>>>>    MessageBoxA
>>>>>>    MessageBoxW==MessageBoxA`
>>>>>>
>>>>>> LIB.exe doesn't not have support for this but dlltool does.
>>>>>> The best fit for this within the spec is PE COFF spec (Aux Format 3:
>>>>>> Weak Externals)
>>>>>>
>>>>>> Mingw-w64 also uses lib prefix and .a file extensions for the library
>>>>>> name so the driver should be different. The above example would be
>>>>>> libuser32.a, for when shared and static versions of the same library exist
>>>>>> there is the .dll.a variant.
>>>>>>
>>>>>>
>>>>>> Issues
>>>>>> =======
>>>>>>
>>>>>> Like more of the gnu suite dlltool was not designed in one build
>>>>>> multi target manner.
>>>>>> We would have to introduce custom arguments to specify the target,
>>>>>> "i686", "x86_64", "thumb2pe" etc.
>>>>>> This will most likely break some level of backwards compatibility
>>>>>> with the binutils version.
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> llvm-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170213/841dea1c/attachment.html>