[llvm-dev] llvm-link has no --allow-multiple-definition

[Eli Friedman via llvm-dev]

Probably you could patch the gold plugin to support something similar? I haven't tried it, but I don't see any reason it couldn't work.  Should look similar to the lld patch I posted.  gold doesn't support "-mllvm", but I think you can do the same thing with "-plugin-opt".

ld.lld is generally compatible with commonly used GNU linker flags.

As far as I know, loading pass plugins into lld or the gold plugin has never been implemented before, and my patch is the first time anyone has looked at it.  I don't think that many people are using pass plugins in the first place; if you have to build your own LLVM for whatever reason, it's almost as easy to just hack the pass manager.

-Eli

> -----Original Message-----
> From: Marc <mh at mh-sec.de>
> Sent: Tuesday, March 10, 2020 12:00 AM
> To: Eli Friedman <efriedma at quicinc.com>
> Cc: llvm-dev <llvm-dev at lists.llvm.org>
> Subject: [EXT] Re: [llvm-dev] llvm-link has no --allow-multiple-definition
>
> HI Eli,
>
> that sounds like the -mllvm option would be a good a mention in the
> WritingAnLLVMPass document ;-)
>
> Two further questions for this and I am keeping this on the list so
> other people have pointers to this if they struggle with the same thing:
>
> 1. Does this require to use lld? (I am sure it is, just to clarify, for
> the very low chance that it would also work with gnu ld and the llvm
> gold plugin performce this magic)
>
> 1a. Follow up question if this required lld - does lld has feature
> parity (and same parameter names) with e.g. gcc-9 ld?
>
> 2. When you say it does not work with trunk at the moment - did it work
> previously? e.g. with the llvm 9 release? Or did it never work so far
> and is in the progress of being fixed?
>
> Thanks a lot!
>
> Regards,
> Marc
>
>
> On 09.03.20 23:41, Eli Friedman wrote:
> > If you want to load a pass plugin into lld, you need to write something like " -
> Wl,-mllvm=-load=plugin.so".  Granted, that doesn't actually work on trunk at the
> moment; posted https://reviews.llvm.org/D75879 .
> >
> > Once you actually have the plugin loaded into the linker,
> EP_FullLinkTimeOptimizationLast should work.
> >
> > -Eli
> >
> >> -----Original Message-----
> >> From: Marc <mh at mh-sec.de>
> >> Sent: Monday, March 9, 2020 2:26 PM
> >> To: Eli Friedman <efriedma at quicinc.com>
> >> Cc: llvm-dev <llvm-dev at lists.llvm.org>
> >> Subject: [EXT] Re: [llvm-dev] llvm-link has no --allow-multiple-definition
> >>
> >> Hi Eli,
> >>
> >> I know the page, and the only mentioning about LTO or link time
> >> optimization I found is the option
> >> PassManagerBuilder::EP_FullLinkTimeOptimizationLast - and nobody knows
> >> how this works (and asked about it on this list too before with no
> >> reply), and the author who wrote that part in the llvm code at Redhat
> >> did not reply to a question how to actually use it. And there is no
> >> example code or any code using this option anywhere on the Internet I
> >> could find.
> >>
> >> And loading such a plugin via -Xclang -load -Xclang plugin.so with
> >> EP_FullLinkTimeOptimizationLast doesnt work.
> >>
> >> So if you can shed any light about how to write a plugin that allows
> >> running a pass within ld - I would be very greatful! (and a large
> >> community of people fuzzing code :)
> >>
> >> Regards,
> >> Marc
> >>
> >>
> >> On 09.03.20 20:24, Eli Friedman wrote:
> >>> Despite the name, llvm-link isn't really a proper linker.  If you need symbols
> to
> >> get resolved exactly the same way an ELF linker would resolve them, I would
> >> recommend using lld.
> >>>
> >>> It's possible to write a pass plugin that hooks into the LTO pipeline.
> >> http://llvm.org/docs/WritingAnLLVMPass.html describes this.
> >>>
> >>> -Eli
> >>>
> >>>> -----Original Message-----
> >>>> From: llvm-dev <llvm-dev-bounces at lists.llvm.org> On Behalf Of Marc via
> >> llvm-
> >>>> dev
> >>>> Sent: Sunday, March 8, 2020 10:01 AM
> >>>> To: llvm-dev <llvm-dev at lists.llvm.org>
> >>>> Subject: [EXT] [llvm-dev] llvm-link has no --allow-multiple-definition
> >>>>
> >>>> Hello,
> >>>>
> >>>> I am developing an LTO instrumentation of programs for fuzzing (for
> >>>> afl++) and I run into the following problem:
> >>>>
> >>>> a lot of open source program have multiple definitons of the same
> >>>> functions. In normal compile mode this is not a problem, however with
> >>>> LTO this only works with the linker option:
> >>>>  -Wl,--allow-multiple-definition
> >>>>
> >>>> Now to be able to instrument (it is an "opt" pass") at link time I need
> >>>> to merge all bitcode files together, and this works great - unless I run
> >>>> into this error when I link them to llvm-link:
> >>>>
> >>>> error: Linking globals named 'process_arg': symbol multiply defined!
> >>>>
> >>>> (this is for the bogofilter tools, but happens with other open source
> >>>> program too).
> >>>>
> >>>> My issue is that neither is --allow-multiple-definition an llvm-link
> >>>> command line option, nor can I find an equivalent from the --help output.
> >>>>
> >>>> is there a plan to add this (IMHO) important feature? Thank you!
> >>>>
> >>>> Regards,
> >>>> Marc
> >>>>
> >>>> --
> >>>> Marc Heuse
> >>>> www.mh-sec.de
> >>>>
> >>>> PGP: AF3D 1D4C D810 F0BB 977D  3807 C7EE D0A0 6BE9 F573
> >>>> _______________________________________________
> >>>> LLVM Developers mailing list
> >>>> llvm-dev at lists.llvm.org
> >>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>
> >> --
> >> Marc Heuse
> >> Mobil: +49 177 9611560
> >> Fax: +49 30 37309726
> >> www.mh-sec.de
> >>
> >> Marc Heuse - IT-Security Consulting
> >> Winsstr. 68
> >> 10405 Berlin
> >>
> >> Ust.-Ident.-Nr.: DE244222388
> >> PGP: AF3D 1D4C D810 F0BB 977D  3807 C7EE D0A0 6BE9 F573
>
> --
> Marc Heuse
> Mobil: +49 177 9611560
> Fax: +49 30 37309726
> www.mh-sec.de
>
> Marc Heuse - IT-Security Consulting
> Winsstr. 68
> 10405 Berlin
>
> Ust.-Ident.-Nr.: DE244222388
> PGP: AF3D 1D4C D810 F0BB 977D  3807 C7EE D0A0 6BE9 F573