[PATCH] D22677: [ThinLTO/gold] Support for getting list of included objects from gold

Xinliang David Li via llvm-commits llvm-commits at lists.llvm.org
Fri Jul 22 10:45:11 PDT 2016


On Fri, Jul 22, 2016 at 10:41 AM, Teresa Johnson <tejohnson at google.com>
wrote:

>
>
> On Fri, Jul 22, 2016 at 10:35 AM, Xinliang David Li <davidxl at google.com>
> wrote:
>
>>
>>
>> On Fri, Jul 22, 2016 at 10:06 AM, Teresa Johnson <tejohnson at google.com>
>> wrote:
>>
>>> tejohnson added a comment.
>>>
>>> > I thought one of the issue is that the first link does not drop any,
>>> but the second link drops it if the same command line is used.  If the
>>> first link does not drop any, is v1.12 still needed?
>>>
>>>
>>> Ah, good point, I had forgotten that the issue there was that we don't
>>> want to drop one file (and always include all the rest). (In some cases you
>>> can get into trouble if you include all objects from the archives, but not
>>> there).
>>>
>>> That being said, since the patch no longer addresses the specific
>>> instance in that test case, I think it is a bit too heavy-weight - we know
>>> that it is fixed by including that second file in the link, and the test
>>> case I added here ensures that the file does include objects from libraries
>>> that are needed by the link. So my preference is not to include it here,
>>> but if you feel strongly I will add it too.
>>>
>>
>>
>> Probably as a follow up patch.
>>
>>>
>>>
>>> ================
>>> Comment at: test/tools/gold/X86/thinlto_emit_linked_objects.ll:18
>>> @@ +17,3 @@
>>> +; RUN:    %t.o \
>>> +; RUN:    --start-lib %t2.o --end-lib
>>> +
>>> ----------------
>>> davidxl wrote:
>>> > tejohnson wrote:
>>> > > davidxl wrote:
>>> > > > tejohnson wrote:
>>> > > > > davidxl wrote:
>>> > > > > > Perhaps add test case about real archive case as well.
>>> > > > > For a distributed build the build system needs to extract the
>>> constituent objects (otherwise the combined index paths will not point to
>>> an object file)
>>> > > > I think this feature is more about two step link (not about a
>>> particular distributed build system). It is possible that some other
>>> distributed build system uses archive, right?
>>> > > Using archives directly with a distributed build won't work - the
>>> combined index module paths need to point to an object file for the
>>> distributed backends to import from. The build system needs to extract the
>>> objects (e.g. into a temp dir) and pass those.
>>> > >
>>> > > Note that archives passed to a non-distributed build work fine
>>> because we load the modules from the offset within the archive passed to
>>> the plugin from gold and serve those to the importer.
>>> > how about thin archive?
>>> Gold does not seem to like thin archives (regardless of LTO vs ThinLTO).
>>> I get an error (from gold, not the plugin) like:
>>>
>>> error: foo.a: no archive symbol table (run ranlib)
>>>
>>> (even when I run ranlib)
>>>
>>
>>
>>
>> ok -- a path not yet supported then.
>>
>>
>>
>>>
>>> ================
>>> Comment at: test/tools/gold/X86/thinlto_emit_linked_objects.ll:21
>>> @@ +20,3 @@
>>> +; RUN: cat %t3 | FileCheck %s
>>> +; CHECK: thinlto_emit_linked_objects.ll.tmp.o
>>> +; CHECK: thinlto_emit_linked_objects.ll.tmp2.o
>>> ----------------
>>> davidxl wrote:
>>> > tejohnson wrote:
>>> > > davidxl wrote:
>>> > > > tejohnson wrote:
>>> > > > > davidxl wrote:
>>> > > > > > Can the name between bitcode file and final object files be
>>> made more related ?
>>> > > > > The ThinLink gold invocation doesn't know the name of the final
>>> object files. But note that if a prefix replace path is specified
>>> (thinlto-prefix-replace=oldprefix:newprefix plugin option), then the names
>>> emitted will have the matching prefix path replaced with the new one. This
>>> can be used to get the paths to the final object files into the file, as
>>> long as the final objects use the same names but a different path (true in
>>> bazel). Otherwise the build system would need to do post-processing of the
>>> list to map to the final object files it is going to create - known by the
>>> build system but not by gold here.
>>> > > > This request is more about test case readability. Suppose
>>> --start-lib and --end-lib pair includes two bit code files, but one of them
>>> is dropped in the first link. In the second step, we need to be able to
>>> tell which one maps to the kept one..
>>> > > Is the concern that it isn't obvious how %t.o and %t2.o map to lines
>>> in the CHECK? Note though that both are included in the link due to the
>>> gold version issue mentioned in the comments, so it should be pretty clear
>>> in this test case I think?
>>> > yes for this case it is clear more or less.  A more general concern is
>>> that if we have a very large link that has a bug in the command line, how
>>> do we debug the problem? Whatever debugging facility we can use to trace
>>> the native .o back to the bitcode file can be used in tests like this.
>>> I'm not following - that mechanism has to be in the build system. The
>>> separate processes in the distributed backend case mean that the two
>>> invocations of gold don't have a way to identify what the intervening clang
>>> invocations use as the output native object file name (the backend clang
>>> processes take the bitcode file and the index produced by the thin link as
>>> input, and an output file name, and invokes the importing/optimization
>>> pipelines).
>>>
>>
>> How does the clang invocation determine the name of the native output?
>> There is no fundamental reason we can not establish the name connection. If
>> it is limited by the current implementation, we can revisit the issue later.
>>
>
> Clang determines it based on the "-o filename" option passed to it. So it
> is entirely up to the build system. I don't think we want to hardcode an
> expectation in the compiler/plugin in the distributed backend case. It will
> limit flexibility on the build system side.
>

Implementation wise, it is possible for the plugin to control and dump the
name mapping in some file and let build system to honor it.  However as I
said we can follow up on that later.

David



>
> Teresa
>
>>
>> David
>>
>>>
>>>
>>> https://reviews.llvm.org/D22677
>>>
>>>
>>>
>>>
>>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
> 408-460-2413
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160722/6ba40724/attachment-0001.html>


More information about the llvm-commits mailing list