[llvm-dev] RFC: A new llvm-dlltool driver and llvm-lib driver improvements

Martell Malone via llvm-dev llvm-dev at lists.llvm.org
Mon Feb 13 16:46:02 PST 2017 

>
> No, I meant an even thinner wrapper which textually translates arguments.
> For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o
> foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything
> with Config object nor LinkerDriver::run and have absolutely zero knowledge
> on the internals of LLD.

Ohh okay I misunderstood.

(Why I'm asking this is because I want to sure that there's no difference
> between the regular Windows linker and mingw. If all differences are
> superficial, the command line translator should just work. If not, there's
> some semantic difference.)

The libraries can be used with MSVC link.exe directly assuming the correct
arguments are passed to the linker.
I tested this after creating a working llvm-dlltool.
The only change I had to make to support this was alter ming-w64 crt to
change all references from __image_base__ to _ImageBase to support that

That should be fine.

Great

On Mon, Feb 13, 2017 at 9:34 PM, Rui Ueyama <ruiu at google.com> wrote:

> On Mon, Feb 13, 2017 at 1:16 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
>> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process
>>> wrapper, meaning that you could add a main function for mingw which
>>> translates all arguments into the MSVC style and then invoke the COFF
>>> linker's main function.
>>
>> That should work, if I am understanding this correctly I can create an
>> argument parser (probably partially based on the ELF one) then I can set
>> the data in the COFF Config object and call COFF LinkerDriver::run etc
>> directly from that?
>>
>
> No, I meant an even thinner wrapper which textually translates arguments.
> For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o
> foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything
> with Config object nor LinkerDriver::run and have absolutely zero knowledge
> on the internals of LLD.
>
> (Why I'm asking this is because I want to sure that there's no difference
> between the regular Windows linker and mingw. If all differences are
> superficial, the command line translator should just work. If not, there's
> some semantic difference.)
>
> This proposal is for generally moving code from lld into llvm that could
>> be part of lib.exe / dlltool, what are your thoughts here?
>>
>
> That should be fine.
>
>
>> On Mon, Feb 13, 2017 at 8:57 PM, Rui Ueyama <ruiu at google.com> wrote:
>>
>>> On Mon, Feb 13, 2017 at 12:52 PM, Martell Malone <
>>> martellmalone at gmail.com> wrote:
>>>
>>>> Also you need to make a change to LLD/COFF to accept GNU command
>>>>> arguments, right? (Looks like you already have that patch locally.)
>>>>
>>>> Yes
>>>>
>>>
>>> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process
>>> wrapper, meaning that you could add a main function for mingw which
>>> translates all arguments into the MSVC style and then invoke the COFF
>>> linker's main function.
>>>
>>>
>>>>
>>>> My patch to hack lld into accepting some very basic gnu front end
>>>>> arguments was enough to get all the above working which was enough to
>>>>> develop further.
>>>>
>>>>
>>>>
>>>> On Mon, Feb 13, 2017 at 8:41 PM, Rui Ueyama <ruiu at google.com> wrote:
>>>>
>>>>> On Mon, Feb 13, 2017 at 12:33 PM, Martell Malone <
>>>>> martellmalone at gmail.com> wrote:
>>>>>
>>>>>> Hey Rui,
>>>>>>
>>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw
>>>>>>> support. I don't think dlltool is the last missing piece. What do you need
>>>>>>> to do other than that to fully support mingw using LLVM toolchain?
>>>>>>
>>>>>> Other then changing `lib/MC/WinCOFFStreamer.cpp` to not use
>>>>>> -aligncomm within the EmitCommonSymbol function and a single patch for
>>>>>> mingw-w64 itself to pre-populate it's .ctors and .dtors list, so
>>>>>> llvm-dlltool is infact the only missing part really.
>>>>>>
>>>>>
>>>>> Also you need to make a change to LLD/COFF to accept GNU command
>>>>> arguments, right? (Looks like you already have that patch locally.)
>>>>>
>>>>>
>>>>>> This gives us a fully working clang based mingw-w64 C compiler.
>>>>>>
>>>>>> C++ and exception handling is a different story.
>>>>>>
>>>>>> libc++ is somewhat working with the following test results
>>>>>>
>>>>>> Expected Passes    : 2188
>>>>>> Expected Failures  : 44
>>>>>> Unsupported Tests  : 588
>>>>>> Unexpected Failures: 2816
>>>>>>
>>>>>> I was able to build it with exceptions disabled and I actually
>>>>>> managed to bootstrap llvm and clang itself.
>>>>>> It didn't run very well though as you can imagine based on the above
>>>>>> tests.
>>>>>>
>>>>>> It's not a performance issue but a code maintenance issue. The
>>>>>>> initial patches to support mingw was trying to add a new linker driver and
>>>>>>> a different linkin semantics to the COFF linker which seemed too
>>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU
>>>>>>> command line arguments to MSVC linker arguments. Didn't it work?
>>>>>>
>>>>>> There were just so many differences between link and ld I never
>>>>>> followed down that path. I pushed forward with the short import library
>>>>>> support based on your later suggestions. My patch to hack lld into
>>>>>> accepting some very basic gnu front end arguments was enough to get all the
>>>>>> above working which was enough to develop further.
>>>>>>
>>>>>> Best,
>>>>>> Martell
>>>>>>
>>>>>> On Mon, Feb 13, 2017 at 8:08 PM, Rui Ueyama <ruiu at google.com> wrote:
>>>>>>
>>>>>>> Hi Martell,
>>>>>>>
>>>>>>> I wonder how llvm-dlltool would fit in the entire picture of mingw
>>>>>>> support. I don't think dlltool is the last missing piece. What do you need
>>>>>>> to do other than that to fully support mingw using LLVM toolchain?
>>>>>>>
>>>>>>> On Mon, Feb 13, 2017 at 8:56 AM, Martell Malone via llvm-dev <
>>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> Hey llvm'ers,
>>>>>>>>
>>>>>>>> I have been working on a dlltool replacement for llvm.
>>>>>>>> Here is my initial differential https://reviews.llvm.org/D29892
>>>>>>>> It is based on some functionality that already exists in lld.
>>>>>>>> I added functionality to support, PE COFF Weak Externals and of
>>>>>>>> course a front end to actually use it.
>>>>>>>> I believe the work here can also be used for llvm-lib and lessen
>>>>>>>> the load on lld.
>>>>>>>> I would like some comments about how this could be be structured to
>>>>>>>> live in llvm with a shared code base across lib ar and dlltool.
>>>>>>>> I also have a section below called "Difference from lib" which is
>>>>>>>> somewhat of a rationale for the tool.
>>>>>>>>
>>>>>>>> Many Thanks,
>>>>>>>> Martell
>>>>>>>>
>>>>>>>>
>>>>>>>> Context
>>>>>>>> ==========
>>>>>>>>
>>>>>>>> Awhile back I talked to various llvm'ers about getting mingw-w64
>>>>>>>> support for lld.
>>>>>>>> There were a few issues raised but the main issue was that
>>>>>>>> mingw-w64 should try best to comply with the PECOFF spec, adding support
>>>>>>>> for custom sections and various binutils/mingw hacks would impact the
>>>>>>>> performance of the COFF linker and in general is not something that lld
>>>>>>>> should support.
>>>>>>>>
>>>>>>>
>>>>>>> It's not a performance issue but a code maintenance issue. The
>>>>>>> initial patches to support mingw was trying to add a new linker driver and
>>>>>>> a different linkin semantics to the COFF linker which seemed too
>>>>>>> complicated to me. IIRC, I suggested adding a shim which translates GNU
>>>>>>> command line arguments to MSVC linker arguments. Didn't it work?
>>>>>>>
>>>>>>>
>>>>>>>> Motivation
>>>>>>>> ==========
>>>>>>>>
>>>>>>>> The main motivation was because dlltool and ld did not comply with
>>>>>>>> PECOFF Spec.
>>>>>>>> It has some custom formatting and uses the assembler for some
>>>>>>>> reason to generate import libraries, it did not use the short import
>>>>>>>> library format at all.
>>>>>>>>
>>>>>>>> There has been many work arounds for the problem this creates such
>>>>>>>> as the creation of the `reimp` tool. Which imports MSVC built libs creates
>>>>>>>> a def and uses dlltool so that the binutils linker can use it.
>>>>>>>>
>>>>>>>> We should just be using the short import format and the linker
>>>>>>>> should support that.
>>>>>>>> Thus llvm-dlltool was born.
>>>>>>>>
>>>>>>>>
>>>>>>>> Difference from lib
>>>>>>>> ===================
>>>>>>>>
>>>>>>>> Using PE COFF spec (section 8, Import Library Format) should be
>>>>>>>> self explanatory.
>>>>>>>> lib.exe is able to accept def files and create libraries using this
>>>>>>>> format.
>>>>>>>>
>>>>>>>> example
>>>>>>>>
>>>>>>>>   `LIBRARY "user32.dll"
>>>>>>>>    EXPORTS
>>>>>>>>    MessageBoxA`
>>>>>>>>
>>>>>>>> LIB.exe can create a user32.lib with the function MessageBoxA from
>>>>>>>> the above definition.
>>>>>>>>
>>>>>>>> Mingw-w64 is different MSVC in that we need to compile the runtime.
>>>>>>>> MS provide us with their crt prebuilt so lib.exe doesn't have
>>>>>>>> support for external function aliasing.
>>>>>>>> We often use aliases for posix naming reasons as well as avoid
>>>>>>>> using the MS version of a function.
>>>>>>>>
>>>>>>>> example
>>>>>>>>
>>>>>>>>   `LIBRARY "user32.dll"
>>>>>>>>    EXPORTS
>>>>>>>>    MessageBoxA
>>>>>>>>    MessageBoxW==MessageBoxA`
>>>>>>>>
>>>>>>>> LIB.exe doesn't not have support for this but dlltool does.
>>>>>>>> The best fit for this within the spec is PE COFF spec (Aux Format
>>>>>>>> 3: Weak Externals)
>>>>>>>>
>>>>>>>> Mingw-w64 also uses lib prefix and .a file extensions for the
>>>>>>>> library name so the driver should be different. The above example would be
>>>>>>>> libuser32.a, for when shared and static versions of the same library exist
>>>>>>>> there is the .dll.a variant.
>>>>>>>>
>>>>>>>>
>>>>>>>> Issues
>>>>>>>> =======
>>>>>>>>
>>>>>>>> Like more of the gnu suite dlltool was not designed in one build
>>>>>>>> multi target manner.
>>>>>>>> We would have to introduce custom arguments to specify the target,
>>>>>>>> "i686", "x86_64", "thumb2pe" etc.
>>>>>>>> This will most likely break some level of backwards compatibility
>>>>>>>> with the binutils version.
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> LLVM Developers mailing list
>>>>>>>> llvm-dev at lists.llvm.org
>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170214/0aa058ba/attachment.html>

More information about the llvm-dev mailing list