[llvm-dev] Supporting Regular and Thin LTO with a Single LTO Bitcode Format

[Teresa Johnson via llvm-dev]

Since we do get a module summary even for regular LTO (with metadata added
to indicate whether the LTO link should do regular or ThinLTO), I think the
issue is just the divergent pass managers, which is also discussed in the
thread Steven pointed to. Matthew - can you remind me of the different
summary issue?

Petr, in your case where you don't care about the performance (so much)
when running in a ThinLTO mode since it is just tests, presumably the
different pipeline issue isn't a big deal.

Teresa

On Tue, Apr 13, 2021 at 2:46 PM <Matthew.Voss at sony.com> wrote:

> Hi Petr,
>
>
>
> This does sound like a good use case for our pipeline. We’ve seen good
> runtime performance overall, as we stated in the talk. I’ve been working on
> upstreaming our patches off and on for a couple months now. Our pipeline
> needs to be ported to the NPM, but once that work is done, the patch is
> ready for review. I should be able to finish that work within the next
> month or two and would love to get some feedback on our approach.
>
>
>
> Thanks,
>
> Matthew
>
>
>
>
>
> *From:* Steven Wu <stevenwu at apple.com>
> *Sent:* Tuesday, April 13, 2021 2:39 PM
> *To:* David Blaikie <dblaikie at gmail.com>
> *Cc:* Petr Hosek <phosek at google.com>; Voss, Matthew <Matthew.Voss at sony.com>;
> Teresa Johnson <tejohnson at google.com>; llvm-dev <llvm-dev at lists.llvm.org>
> *Subject:* Re: [llvm-dev] Supporting Regular and Thin LTO with a Single
> LTO Bitcode Format
>
>
>
> This is a really good thread to read:
> https://lists.llvm.org/pipermail/llvm-dev/2018-April/122469.html
>
>
>
> There is no fundamental technical reasons why this cannot happen but it
> requires lots of work to fine tuning the pipeline (yes, fullLTO and thinLTO
> uses different pipeline) so that it reaches a good balance of
> performance/build overhead for general users.
>
>
>
> Steven
>
>
>
> On Apr 13, 2021, at 2:23 PM, David Blaikie via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>
>
> +Matthew and Teresa for any context they might have
>
> High level sounds like a reasonable thing to me, for what it's worth.
>
> On Tue, Apr 13, 2021 at 2:19 PM Petr Hosek via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
>
> We're using regular LTO for our Clang toolchain build because we don't
> mind spending more resources to squeeze out as much performance as
> possible. However, when looking into our build performance, I've noticed
> that we only spent about 1/3 of the total build time in building
> distribution components, the rest is spent on building unit tests and tools
> that are only used by lit tests. For the latter, we don't care about the
> performance, so it'd be nice to avoid doing regular LTO to speed up the
> build.
>
> The idea I had would be to use a single LTO bitcode format for all
> translation units, and then decide only at link time whether to use regular
> LTO for distribution components or ThinLTO for everything else.
>
> After doing some research, I found the "Supporting Regular and Thin LTO
> with a Single LTO Bitcode Format" talk presented by Matthew Voss at LLVM
> Developers’ Meeting 2019 which does exactly what I described, but it seems
> like this was only implemented downstream.
>
> Has there been any progress on upstreaming the implementation? Is there
> any way to do what I described using the in-tree LTO implementation?
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>


-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210413/43deba11/attachment.html>