[PATCH] D22677: [ThinLTO/gold] Support for getting list of included objects from gold

Fri Jul 22 10:41:05 PDT 2016

On Fri, Jul 22, 2016 at 10:35 AM, Xinliang David Li <davidxl at google.com>
wrote:

>
>
> On Fri, Jul 22, 2016 at 10:06 AM, Teresa Johnson <tejohnson at google.com>
> wrote:
>
>> tejohnson added a comment.
>>
>> > I thought one of the issue is that the first link does not drop any,
>> but the second link drops it if the same command line is used.  If the
>> first link does not drop any, is v1.12 still needed?
>>
>>
>> Ah, good point, I had forgotten that the issue there was that we don't
>> want to drop one file (and always include all the rest). (In some cases you
>> can get into trouble if you include all objects from the archives, but not
>> there).
>>
>> That being said, since the patch no longer addresses the specific
>> instance in that test case, I think it is a bit too heavy-weight - we know
>> that it is fixed by including that second file in the link, and the test
>> case I added here ensures that the file does include objects from libraries
>> that are needed by the link. So my preference is not to include it here,
>> but if you feel strongly I will add it too.
>>
>
>
> Probably as a follow up patch.
>
>>
>>
>> ================
>> Comment at: test/tools/gold/X86/thinlto_emit_linked_objects.ll:18
>> @@ +17,3 @@
>> +; RUN:    %t.o \
>> +; RUN:    --start-lib %t2.o --end-lib
>> +
>> ----------------
>> davidxl wrote:
>> > tejohnson wrote:
>> > > davidxl wrote:
>> > > > tejohnson wrote:
>> > > > > davidxl wrote:
>> > > > > > Perhaps add test case about real archive case as well.
>> > > > > For a distributed build the build system needs to extract the
>> constituent objects (otherwise the combined index paths will not point to
>> an object file)
>> > > > I think this feature is more about two step link (not about a
>> particular distributed build system). It is possible that some other
>> distributed build system uses archive, right?
>> > > Using archives directly with a distributed build won't work - the
>> combined index module paths need to point to an object file for the
>> distributed backends to import from. The build system needs to extract the
>> objects (e.g. into a temp dir) and pass those.
>> > >
>> > > Note that archives passed to a non-distributed build work fine
>> because we load the modules from the offset within the archive passed to
>> the plugin from gold and serve those to the importer.
>> > how about thin archive?
>> Gold does not seem to like thin archives (regardless of LTO vs ThinLTO).
>> I get an error (from gold, not the plugin) like:
>>
>> error: foo.a: no archive symbol table (run ranlib)
>>
>> (even when I run ranlib)
>>
>
>
>
> ok -- a path not yet supported then.
>
>
>
>>
>> ================
>> Comment at: test/tools/gold/X86/thinlto_emit_linked_objects.ll:21
>> @@ +20,3 @@
>> +; RUN: cat %t3 | FileCheck %s
>> +; CHECK: thinlto_emit_linked_objects.ll.tmp.o
>> +; CHECK: thinlto_emit_linked_objects.ll.tmp2.o
>> ----------------
>> davidxl wrote:
>> > tejohnson wrote:
>> > > davidxl wrote:
>> > > > tejohnson wrote:
>> > > > > davidxl wrote:
>> > > > > > Can the name between bitcode file and final object files be
>> made more related ?
>> > > > > The ThinLink gold invocation doesn't know the name of the final
>> object files. But note that if a prefix replace path is specified
>> (thinlto-prefix-replace=oldprefix:newprefix plugin option), then the names
>> emitted will have the matching prefix path replaced with the new one. This
>> can be used to get the paths to the final object files into the file, as
>> long as the final objects use the same names but a different path (true in
>> bazel). Otherwise the build system would need to do post-processing of the
>> list to map to the final object files it is going to create - known by the
>> build system but not by gold here.
>> > > > This request is more about test case readability. Suppose
>> --start-lib and --end-lib pair includes two bit code files, but one of them
>> is dropped in the first link. In the second step, we need to be able to
>> tell which one maps to the kept one..
>> > > Is the concern that it isn't obvious how %t.o and %t2.o map to lines
>> in the CHECK? Note though that both are included in the link due to the
>> gold version issue mentioned in the comments, so it should be pretty clear
>> in this test case I think?
>> > yes for this case it is clear more or less.  A more general concern is
>> that if we have a very large link that has a bug in the command line, how
>> do we debug the problem? Whatever debugging facility we can use to trace
>> the native .o back to the bitcode file can be used in tests like this.
>> I'm not following - that mechanism has to be in the build system. The
>> separate processes in the distributed backend case mean that the two
>> invocations of gold don't have a way to identify what the intervening clang
>> invocations use as the output native object file name (the backend clang
>> processes take the bitcode file and the index produced by the thin link as
>> input, and an output file name, and invokes the importing/optimization
>> pipelines).
>>
>
> How does the clang invocation determine the name of the native output?
> There is no fundamental reason we can not establish the name connection. If
> it is limited by the current implementation, we can revisit the issue later.
>

Clang determines it based on the "-o filename" option passed to it. So it
is entirely up to the build system. I don't think we want to hardcode an
expectation in the compiler/plugin in the distributed backend case. It will
limit flexibility on the build system side.

Teresa

>
> David
>
>>
>>
>> https://reviews.llvm.org/D22677
>>
>>
>>
>>
>

-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |  408-460-2413
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160722/1af43aa4/attachment.html>