[llvm-dev] getting nowhere with thinLTO

[Davis, Alan via llvm-dev]

Teresa, thanks, that’s helpful.

I discovered the resolution-based API and am now converting to that. I can also see that the summaries are in the bitcode files, via llvm-bcanalyzer.

I hope the llvm-dis support will happen. Also it would be nice if the documentation were more up-to-date. But this will definitely help get me moving again.

For whatever reason in this simple example t3 is not being inlined, but I can set that aside for now.

// t2.c:
int t3();

int t2()
{
   if (t3()) return 1;
   else return 0;
}

// t3.c:
int t3()
{
   return 10;
}

% clang -c -O2 -flto=thin t2.c t3.c
% llvm-lto t2.o t3.o --exported-symbol=t2 --thinlto-action=run -debug-only=function-import -thinlto-save-temps=.
Live root: 11079814748834529807
2 symbols Live, and 1 symbols Dead
Computing import for Module 't3.o'
Ignores Dead GUID: 8059958135287337232
Initialize import for 14314392591860533259
Computing import for Module 't2.o'
Initialize import for 11079814748834529807
Import/Export lists for 2 modules:
* Module t3.o exports 0 functions. Imports from 0 modules.
* Module t2.o exports 0 functions. Imports from 0 modules.
Starting import for Module t2.o
Imported 0 functions for Module t2.o
Starting import for Module t3.o
Imported 0 functions for Module t3.o

-Alan

From: Teresa Johnson [mailto:tejohnson at google.com]
Sent: Thursday, November 09, 2017 10:33 AM
To: Davis, Alan
Cc: llvm-dev at lists.llvm.org; Peter Collingbourne
Subject: [EXTERNAL] Re: [llvm-dev] getting nowhere with thinLTO



On Wed, Nov 8, 2017 at 2:31 PM, Davis, Alan via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
I’m trying to incorporate thinLTO into our proprietary linker. I can’t seem to make it work at all (although I do have the ‘thick’ form working).

Sorry to hear that, hopefully we can help get that sorted out.

When I try to invoke the ThinGenerator API it crashes somewhere way down in the bowels of IPO. So stepping away from the linker, I am just trying to explore the processing steps using standalone off-the shelf llvm built from tip-of-tree.

For a new implementation I would highly recommend using the new LTO API, which utilizes linker resolution info and is better supported at this point. See http://llvm-cs.pcc.me.uk/include/llvm/LTO/LTO.h#234.
See also gold-plugin.cpp and the lld sources - both utilize this API. To test it use the llvm-lto2 tool instead of llvm-lto. Because this interface uses linker resolution info, invoking via llvm-lto2 is more tedious since resolutions must be specified for all symbols in the input files. See invocations in the llvm tests for examples.


1. When I do ‘clang -c -flto=thin t1.c’ on a simple input file, the resulting .o file is a bitcode file as expected, but when I llvm-dis it there is no evidence of the module summary information. Is there a way to view the summary index for a given module?

Unfortunately not. This has come up a few times and the long-term plan is to serialize it through llvm assembly, but no one has had the bandwidth to completely flesh this out yet, and there was disagreement about what to do in the short term, so nothing went in. For now, you have to use llvm-bcanalyzer -dump, but that is a raw output format, although it will at least allow you to confirm the summary sections are there.


2. When I take two or three .o files generated using -flto=thin and pass them to the standalone llvm-lto tool, as in:
   llvm-lto --thinlto t2.o t3.o --exported-symbol=t2 -o index
the combined index is (apparently) empty.

Use llvm-bcanalyzer -dump to see the contents of the output index. It shouldn't be empty.

3. When I take a program and pass all the .o files to llvm-lto --thinlto-action=run, I get recompiled .o files for each but there seems to be no IPO (no inlining, etc).

How do you know? BTW you can invoke llvm-lto with the -thinlto-save-temps=<pathprefix> option to get bitcode files emitted from the backends after the major phases of ThinLTO backend compilation. I.e. to see what was imported.
You can also pass internal llvm options to llvm-lto, i.e. -debug-only=function-import will give you debug info from the function importing done both during the thin link and during the thinlto backends.

Let me know what you find.

Teresa


What am I missing?

-Alan

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev



--
Teresa Johnson |

 Software Engineer |

 tejohnson at google.com<mailto:tejohnson at google.com> |

 408-460-2413<tel:(408)%20460-2413>


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171110/ea1c562d/attachment.html>