[llvm-dev] Supporting Regular and Thin LTO with a Single LTO Bitcode Format

[Petr Hosek via llvm-dev]

Yes, I don't think that using a different pipeline for test binaries would
be a problem.

On Tue, Apr 13, 2021 at 3:04 PM Teresa Johnson <tejohnson at google.com> wrote:

> Since we do get a module summary even for regular LTO (with metadata added
> to indicate whether the LTO link should do regular or ThinLTO), I think the
> issue is just the divergent pass managers, which is also discussed in the
> thread Steven pointed to. Matthew - can you remind me of the different
> summary issue?
>
> Petr, in your case where you don't care about the performance (so much)
> when running in a ThinLTO mode since it is just tests, presumably the
> different pipeline issue isn't a big deal.
>
> Teresa
>
> On Tue, Apr 13, 2021 at 2:46 PM <Matthew.Voss at sony.com> wrote:
>
>> Hi Petr,
>>
>>
>>
>> This does sound like a good use case for our pipeline. We’ve seen good
>> runtime performance overall, as we stated in the talk. I’ve been working on
>> upstreaming our patches off and on for a couple months now. Our pipeline
>> needs to be ported to the NPM, but once that work is done, the patch is
>> ready for review. I should be able to finish that work within the next
>> month or two and would love to get some feedback on our approach.
>>
>>
>>
>> Thanks,
>>
>> Matthew
>>
>>
>>
>>
>>
>> *From:* Steven Wu <stevenwu at apple.com>
>> *Sent:* Tuesday, April 13, 2021 2:39 PM
>> *To:* David Blaikie <dblaikie at gmail.com>
>> *Cc:* Petr Hosek <phosek at google.com>; Voss, Matthew <
>> Matthew.Voss at sony.com>; Teresa Johnson <tejohnson at google.com>; llvm-dev <
>> llvm-dev at lists.llvm.org>
>> *Subject:* Re: [llvm-dev] Supporting Regular and Thin LTO with a Single
>> LTO Bitcode Format
>>
>>
>>
>> This is a really good thread to read:
>> https://lists.llvm.org/pipermail/llvm-dev/2018-April/122469.html
>>
>>
>>
>> There is no fundamental technical reasons why this cannot happen but it
>> requires lots of work to fine tuning the pipeline (yes, fullLTO and thinLTO
>> uses different pipeline) so that it reaches a good balance of
>> performance/build overhead for general users.
>>
>>
>>
>> Steven
>>
>>
>>
>> On Apr 13, 2021, at 2:23 PM, David Blaikie via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>
>>
>> +Matthew and Teresa for any context they might have
>>
>> High level sounds like a reasonable thing to me, for what it's worth.
>>
>> On Tue, Apr 13, 2021 at 2:19 PM Petr Hosek via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>>
>>
>> We're using regular LTO for our Clang toolchain build because we don't
>> mind spending more resources to squeeze out as much performance as
>> possible. However, when looking into our build performance, I've noticed
>> that we only spent about 1/3 of the total build time in building
>> distribution components, the rest is spent on building unit tests and tools
>> that are only used by lit tests. For the latter, we don't care about the
>> performance, so it'd be nice to avoid doing regular LTO to speed up the
>> build.
>>
>> The idea I had would be to use a single LTO bitcode format for all
>> translation units, and then decide only at link time whether to use regular
>> LTO for distribution components or ThinLTO for everything else.
>>
>> After doing some research, I found the "Supporting Regular and Thin LTO
>> with a Single LTO Bitcode Format" talk presented by Matthew Voss at LLVM
>> Developers’ Meeting 2019 which does exactly what I described, but it seems
>> like this was only implemented downstream.
>>
>> Has there been any progress on upstreaming the implementation? Is there
>> any way to do what I described using the in-tree LTO implementation?
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>>
>
>
> --
> Teresa Johnson |  Software Engineer |  tejohnson at google.com |
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210413/318ecc2a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3996 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210413/318ecc2a/attachment.bin>