[llvm-dev] Question about thinLTO

Thu Jul 13 09:38:35 PDT 2017

I understand the potential risk of having thinLTO in our model now. No, we
don't support LTO with our custom linker. We are moving to gold soon, it is
under progress. I hope this feature can be enabled at that point of time.

Thanks,
Christu

On Thu, Jul 13, 2017 at 9:15 PM, Teresa Johnson <tejohnson at google.com>
wrote:

>
>
> On Thu, Jul 13, 2017 at 8:37 AM, Christudasan D <xander.cd at gmail.com>
> wrote:
>
>>
>> Hi Teresa,
>>
>> Yes, we plan to have our code at CG directly.
>> We use our own linker. That's the pain. We might only get a partial
>> benefit of thinLTO which occurs at compile time.
>>
>
> There is no compile-time only benefit of ThinLTO. You'll need the linker
> to interface with the LTO API for either ThinLTO or LTO to work. Unless you
> use internal tools to get native objects from ThinLTO, and feed those to
> your linker. But that is not a supported model the internal tools are just
> developer tools. Or you could use gold or lld but have them save the temps
> files which will give you the native .o files after ThinLTO, and feed those
> through your linker. But in either case (llvm-lto or gold/lld), you may not
> get the same symbol resolution as with your own linker, which would be a
> problem.
>
> Do you support regular LTO right now? If so, how do you do get that to
> work with your linker?
>
> Teresa
>
>
> I will have a close look at the LTO-tests for any useful command-line
>> option.
>>
>> Thanks for the quick response.
>>
>> Regards,
>> Christu
>>
>> On Thu, Jul 13, 2017 at 7:49 PM, Teresa Johnson <tejohnson at google.com>
>> wrote:
>>
>>>
>>>
>>> On Thu, Jul 13, 2017 at 2:54 AM, Christudasan D <xander.cd at gmail.com>
>>> wrote:
>>>
>>>> Thank you Teresa.
>>>>
>>>> Yes, I would like to save the IR (*.bc and/or *.ll) after all
>>>> optimizations (especially thinLTO) are done and call *llc* separately.
>>>> Is there any specific document available online to see more about this
>>>> feature and various command-line switches that a compiler developer can
>>>> take advantage of?
>>>> It would help us to enable this feature for a custom architecture.
>>>>
>>>
>>> Unfortunately I don't see any documentation online of llvm-lto. I guess
>>> the best bet for now is to look at llvm-lto.cpp, or some of the ThinLTO
>>> tests in the tree that use it with those options. I assume you plan to use
>>> the internal tools just for testing, and eventually hook up your own code
>>> generator to the compiler directly? Which linker do you use? Both gold and
>>> lld have support for ThinLTO and a newer LTO interface that uses linker
>>> resolution information.
>>>
>>> Teresa
>>>
>>>
>>>>
>>>> Regards,
>>>> Christu
>>>>
>>>> On Wed, Jul 12, 2017 at 10:55 PM, Teresa Johnson <tejohnson at google.com>
>>>> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Jul 12, 2017 at 10:19 AM, Teresa Johnson <tejohnson at google.com
>>>>> > wrote:
>>>>>
>>>>>> Hi Christu,
>>>>>>
>>>>>> Thanks for the note!
>>>>>>
>>>>>> On Wed, Jul 12, 2017 at 9:56 AM, Christudasan D via llvm-dev <
>>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> My impression on *thinLTO* when I first heard of it, (EuroLLVM2015)
>>>>>>> was about achieving Cross Module Optimization (CMO) at the IR level.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Having parallel front-end compilation & initial optimization first,
>>>>>>> a thin-link of individual input units, more optimization by calling opt
>>>>>>> again on the combined IR, and finally the target codegen using llc.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> A transformation similar to the following:
>>>>>>>
>>>>>>> Input File 1: Clang+opt (with thinLTO)
>>>>>>>
>>>>>>> Input File 2: Clang+opt (with thinLTO)           ---
>>>>>>> llvm-link       ----       opt (for CMO)      ---             llc for
>>>>>>> target codegen.
>>>>>>>
>>>>>>>                        …..
>>>>>>>
>>>>>>> Input File n: Clang+opt (with thinLTO)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> But from the presentation on LLVM Developers’ Meeting 2016, I
>>>>>>> believe thinLTO is more than that. The full advantage of this optimization
>>>>>>> should require a significant changes in the backend as well (I suppose).
>>>>>>>
>>>>>>
>>>>>> Right, the model in the EuroLLVM talk was just an initial prototype
>>>>>> that used llvm-link/opt, and yes now we do whole program optimizations
>>>>>> during the thin link, beyond just linking in additional IR for inlining etc.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Before I post my question, kudos to the entire team behind thinLTO
>>>>>>> optimization - Teresa Johnson, Mehdi Amni, Xinliang David Li, other
>>>>>>> developers and test engineers across the globe.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I am working on the compiler for a target wherein the code size
>>>>>>> improvement is a critical factor. We are still using LLVM 3.5 code base. We
>>>>>>> assume, by moving to LLVM4.0.0 (with thinLTO and other recent target
>>>>>>> independent optimizations) we would be able to improve the codegen to a
>>>>>>> considerable number.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> With thinLTO in LLVM4.0.0 compiler, when we build an application
>>>>>>> with multiple compilation units, is it possible to achieve any benefit
>>>>>>> purely with LLVM IR passes (without really involving the compiler backend)?
>>>>>>>
>>>>>>> If yes, can anyone provide me the information about the command-line
>>>>>>> options and the sequence to call the llvm components (clang, opt, etc.) to
>>>>>>> achieve it.
>>>>>>>
>>>>>>
>>>>>> Do you just want the bitcode out after all optimization passes and
>>>>>> before codegen? It is doable with llvm-lto -thinlto-action=run -save-temps
>>>>>>
>>>>>
>>>>> Sorry, that last option should be -thinlto-save-temps=foo, where "foo"
>>>>> will be the prefix of the generate temp files (can include a path). You
>>>>> want the foo*.opt.bc files for the output of the opt pipeline in the
>>>>> backends.
>>>>>
>>>>> I believe (although you will get more bitcode output files than you
>>>>>> want, but one of the output bitcode files per input file should be the
>>>>>> post-opt pre-codegen bitcode file. The input files to llvm-lto would be
>>>>>> bitcode files created by "clang -flto=thin -c -O2 ..."
>>>>>>
>>>>>> Teresa
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> I truly value any input in this regard.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>>
>>>>>>> Christu
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> LLVM Developers mailing list
>>>>>>> llvm-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>>>>> 408-460-2413 <(408)%20460-2413>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>>>> 408-460-2413 <(408)%20460-2413>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>>> 408-460-2413 <(408)%20460-2413>
>>>
>>
>>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>  408-460-2413
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170713/4162ce5f/attachment.html>