[LLVMdev] LTO and Optimized libraries don't mix

[Daniel Stewart]

Actually I used it as

add_input_library("opt"), without the .a, as the documents say use it as you would with a -l. But the result is still the same. I also get a crash when I try to use the add_input_file() as well. 

-----Original Message-----
From: Daniel Stewart [mailto:stewartd at codeaurora.org] 
Sent: Tuesday, June 10, 2014 8:13 AM
To: 'Rafael Espíndola'
Cc: 'LLVM Developers Mailing List'
Subject: RE: [LLVMdev] LTO and Optimized libraries don't mix

I had tried to use the add_input _library as

(*add_input_library)("opt.a");

but got a seg fault every time. I'm not sure exactly how to use it, apparently. 

I also noticed that in gold_plugin.cpp:onload(), that add_input_library is defined as

case LDPT_ADD_INPUT_LIBRARY:
 add_input_library = tv->tv_u.tv_add_input_file; break;

Seems odd that add_input_library doesn't actually use the tv_add_input_library. I changed mine locally to point to add_input_library, but still got a crash. 

Am I using the callback correctly?

-----Original Message-----
From: Rafael Espíndola [mailto:rafael.espindola at gmail.com]
Sent: Monday, June 09, 2014 5:51 PM
To: Daniel Stewart
Cc: LLVM Developers Mailing List
Subject: Re: [LLVMdev] LTO and Optimized libraries don't mix

On 9 June 2014 16:26, Daniel Stewart <stewartd at codeaurora.org> wrote:
> When using the ARM cross compiler we’ve run into an issue with LTO and 
> optimized libraries.
>
>
>
> Consider you have an optimized library opt.a, which contains a version 
> of memcpy.
>
>
>
> Compiling with LTO (something like),
>
> clang myTest.c opt.a –flto –o myTest
>
>
>
> causes myTest.c to get compiled to bitcode.
>
>
>
> Then the bitcode gets passed to the linker. The linker looks through 
> the bitcode (via the gold plugin) searching for symbols it needs to 
> resolve. But any memcpy() calls in myTest.c got converted to
> llvm.memcpy() calls in the bitcode. Any symbols that start with ‘llvm.’ get ignored in LTOModule.cpp.
> So the linker doesn’t know that it needs memcpy. Now once the linker 
> processes opt.a it doesn’t know that it needs memcpy, so the linker 
> doesn’t keep the definition of memcpy from opt.a.
>
>
>
> After the (full) compilation of myTest is complete, the linker winds 
> up trying to pull memcpy from libc. But once it hits opt.a to process 
> again, it sees a multiple definition issue.
>
>
>
> It appears that optimized libraries and LTO don’t currently mix well. 
> Does anyone have any insight about how this should be accomplished?
>

I don't think gold has an option for doing a full second pass.  What is has is the ADD_INPUT_LIBRARY callback which should cause it to look at the provided library again.

It is probably a good idea to try to find a convenient way to use it from llvm. If the performance impact is not too horrible, it might make sense to change gold-plugin.cpp to use that callback for every .a that it sees that contains at least one native object file.

Cheers,
Rafael