[llvm-dev] RFC: A new llvm-dlltool driver and llvm-lib driver improvements

[Martell Malone via llvm-dev]

Well that means it would just be the only plan then.
I assume the first step would be to move the code from lld into this new
library and lld can use that? I can then re add the extra functionality for
weak externals?
We can then have lib and dlltool use this then.

On Tue 14 Feb 2017 at 01:58, Rui Ueyama <ruiu at google.com> wrote:

> On Mon, Feb 13, 2017 at 5:56 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
> Yes, that would be the long term plan.
> llvm-link ideally would share the same library.
> I'm not sure where it should live though.
>
>
> What is the short-term plan? I don't think you can submit this
> llvm-dlltool as-is because it has to much duplicate code with LLD.
>
>
>
> On Tue 14 Feb 2017 at 01:49, Rui Ueyama <ruiu at google.com> wrote:
>
> I took a look at the code. Looks like you need a library to create import
> library files in LLVM and use that from llvm-dlltool and LLD. Is that what
> you are planning?
>
> On Mon, Feb 13, 2017 at 5:37 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
> Ohh nice.
> With that method I can support it without upsetting ld users by
> introducing an api breakage.
>
> On Tue 14 Feb 2017 at 01:32, Rui Ueyama <ruiu at google.com> wrote:
>
> On Mon, Feb 13, 2017 at 5:26 PM, Peter Collingbourne <peter at pcc.me.uk>
> wrote:
>
> On Mon, Feb 13, 2017 at 5:20 PM, Rui Ueyama via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> On Mon, Feb 13, 2017 at 4:46 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
> No, I meant an even thinner wrapper which textually translates arguments.
> For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o
> foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything
> with Config object nor LinkerDriver::run and have absolutely zero knowledge
> on the internals of LLD.
>
> Ohh okay I misunderstood.
>
> (Why I'm asking this is because I want to sure that there's no difference
> between the regular Windows linker and mingw. If all differences are
> superficial, the command line translator should just work. If not, there's
> some semantic difference.)
>
> The libraries can be used with MSVC link.exe directly assuming the correct
> arguments are passed to the linker.
> I tested this after creating a working llvm-dlltool.
> The only change I had to make to support this was alter ming-w64 crt to
> change all references from __image_base__ to _ImageBase to support that
>
>
> You may be able to define __ImageBase as a weak external symbol to
> __image_base__. Then, if __ImageBase is not defined, all references against
> __ImageBase will be resolved using __image_base__.
>
>
> Or you could have your wrapper driver pass
> "/alternatename:__image_base__=__ImageBase".
>
>
> Ah, that's much better indeed.
>
>
> Peter
>
>
>
>
> That should be fine.
>
> Great
>
> On Mon, Feb 13, 2017 at 9:34 PM, Rui Ueyama <ruiu at google.com> wrote:
>
> On Mon, Feb 13, 2017 at 1:16 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process
> wrapper, meaning that you could add a main function for mingw which
> translates all arguments into the MSVC style and then invoke the COFF
> linker's main function.
>
> That should work, if I am understanding this correctly I can create an
> argument parser (probably partially based on the ELF one) then I can set
> the data in the COFF Config object and call COFF LinkerDriver::run etc
> directly from that?
>
>
> No, I meant an even thinner wrapper which textually translates arguments.
> For example, the wrapper would translates "/out:foo.exe foo.obj" to "-o
> foo.exe foo.obj" and then call lld::COFFF:link(). It doesn't do anything
> with Config object nor LinkerDriver::run and have absolutely zero knowledge
> on the internals of LLD.
>
> (Why I'm asking this is because I want to sure that there's no difference
> between the regular Windows linker and mingw. If all differences are
> superficial, the command line translator should just work. If not, there's
> some semantic difference.)
>
> This proposal is for generally moving code from lld into llvm that could
> be part of lib.exe / dlltool, what are your thoughts here?
>
>
> That should be fine.
>
>
> On Mon, Feb 13, 2017 at 8:57 PM, Rui Ueyama <ruiu at google.com> wrote:
>
> On Mon, Feb 13, 2017 at 12:52 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
> Also you need to make a change to LLD/COFF to accept GNU command
> arguments, right? (Looks like you already have that patch locally.)
>
> Yes
>
>
> I wonder if it can be a wrapper for LLD/COFF. It can be an in-process
> wrapper, meaning that you could add a main function for mingw which
> translates all arguments into the MSVC style and then invoke the COFF
> linker's main function.
>
>
>
> My patch to hack lld into accepting some very basic gnu front end
> arguments was enough to get all the above working which was enough to
> develop further.
>
>
>
> On Mon, Feb 13, 2017 at 8:41 PM, Rui Ueyama <ruiu at google.com> wrote:
>
> On Mon, Feb 13, 2017 at 12:33 PM, Martell Malone <martellmalone at gmail.com>
> wrote:
>
> Hey Rui,
>
> I wonder how llvm-dlltool would fit in the entire picture of mingw
> support. I don't think dlltool is the last missing piece. What do you need
> to do other than that to fully support mingw using LLVM toolchain?
>
> Other then changing `lib/MC/WinCOFFStreamer.cpp` to not use -aligncomm
> within the EmitCommonSymbol function and a single patch for mingw-w64
> itself to pre-populate it's .ctors and .dtors list, so llvm-dlltool is
> infact the only missing part really.
>
>
> Also you need to make a change to LLD/COFF to accept GNU command
> arguments, right? (Looks like you already have that patch locally.)
>
>
> This gives us a fully working clang based mingw-w64 C compiler.
>
> C++ and exception handling is a different story.
>
> libc++ is somewhat working with the following test results
>
> Expected Passes    : 2188
> Expected Failures  : 44
> Unsupported Tests  : 588
> Unexpected Failures: 2816
>
> I was able to build it with exceptions disabled and I actually managed to
> bootstrap llvm and clang itself.
> It didn't run very well though as you can imagine based on the above tests.
>
> It's not a performance issue but a code maintenance issue. The initial
> patches to support mingw was trying to add a new linker driver and a
> different linkin semantics to the COFF linker which seemed too complicated
> to me. IIRC, I suggested adding a shim which translates GNU command line
> arguments to MSVC linker arguments. Didn't it work?
>
> There were just so many differences between link and ld I never followed
> down that path. I pushed forward with the short import library support
> based on your later suggestions. My patch to hack lld into accepting some
> very basic gnu front end arguments was enough to get all the above working
> which was enough to develop further.
>
> Best,
> Martell
>
> On Mon, Feb 13, 2017 at 8:08 PM, Rui Ueyama <ruiu at google.com> wrote:
>
> Hi Martell,
>
> I wonder how llvm-dlltool would fit in the entire picture of mingw
> support. I don't think dlltool is the last missing piece. What do you need
> to do other than that to fully support mingw using LLVM toolchain?
>
> On Mon, Feb 13, 2017 at 8:56 AM, Martell Malone via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hey llvm'ers,
>
> I have been working on a dlltool replacement for llvm.
> Here is my initial differential https://reviews.llvm.org/D29892
> It is based on some functionality that already exists in lld.
> I added functionality to support, PE COFF Weak Externals and of course a
> front end to actually use it.
> I believe the work here can also be used for llvm-lib and lessen the load
> on lld.
> I would like some comments about how this could be be structured to live
> in llvm with a shared code base across lib ar and dlltool.
> I also have a section below called "Difference from lib" which is somewhat
> of a rationale for the tool.
>
> Many Thanks,
> Martell
>
>
> Context
> ==========
>
> Awhile back I talked to various llvm'ers about getting mingw-w64 support
> for lld.
> There were a few issues raised but the main issue was that mingw-w64
> should try best to comply with the PECOFF spec, adding support for custom
> sections and various binutils/mingw hacks would impact the performance of
> the COFF linker and in general is not something that lld should support.
>
>
> It's not a performance issue but a code maintenance issue. The initial
> patches to support mingw was trying to add a new linker driver and a
> different linkin semantics to the COFF linker which seemed too complicated
> to me. IIRC, I suggested adding a shim which translates GNU command line
> arguments to MSVC linker arguments. Didn't it work?
>
>
> Motivation
> ==========
>
> The main motivation was because dlltool and ld did not comply with PECOFF
> Spec.
> It has some custom formatting and uses the assembler for some reason to
> generate import libraries, it did not use the short import library format
> at all.
>
> There has been many work arounds for the problem this creates such as the
> creation of the `reimp` tool. Which imports MSVC built libs creates a def
> and uses dlltool so that the binutils linker can use it.
>
> We should just be using the short import format and the linker should
> support that.
> Thus llvm-dlltool was born.
>
>
> Difference from lib
> ===================
>
> Using PE COFF spec (section 8, Import Library Format) should be self
> explanatory.
> lib.exe is able to accept def files and create libraries using this format.
>
> example
>
>   `LIBRARY "user32.dll"
>    EXPORTS
>    MessageBoxA`
>
> LIB.exe can create a user32.lib with the function MessageBoxA from the
> above definition.
>
> Mingw-w64 is different MSVC in that we need to compile the runtime.
> MS provide us with their crt prebuilt so lib.exe doesn't have support for
> external function aliasing.
> We often use aliases for posix naming reasons as well as avoid using the
> MS version of a function.
>
> example
>
>   `LIBRARY "user32.dll"
>    EXPORTS
>    MessageBoxA
>    MessageBoxW==MessageBoxA`
>
> LIB.exe doesn't not have support for this but dlltool does.
> The best fit for this within the spec is PE COFF spec (Aux Format 3: Weak
> Externals)
>
> Mingw-w64 also uses lib prefix and .a file extensions for the library name
> so the driver should be different. The above example would be libuser32.a,
> for when shared and static versions of the same library exist there is the
> .dll.a variant.
>
>
> Issues
> =======
>
> Like more of the gnu suite dlltool was not designed in one build multi
> target manner.
> We would have to introduce custom arguments to specify the target, "i686",
> "x86_64", "thumb2pe" etc.
> This will most likely break some level of backwards compatibility with the
> binutils version.
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
> --
> --
> Peter
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170214/ee105b64/attachment-0001.html>