[cfe-dev] JumboSupport: making unity builds easier in Clang

[Bruce Dawson via cfe-dev]

I think it was Churchill who said that "Jumbo builds are the worst form of
build optimization, except for all the others."

The simple reality is that Chromium has a lot of translation units and
there is a minimum (often a large minimum) cost to compiling each of these
translation units. In order to make Chromium build times closer to
reasonable we either need to dramatically reduce the number of translation
units (jumbo) or dramatically reduce the cost of compiling each of these
translation units.

Jumbo builds have some practical and philosophical problems but ultimately
they work, and they work now, and they will get even better. Speculative
efforts to reduce the per-translation-unit are, well, speculative. If and
when modules improve our build times we can assess whether they should
replace or coexist with jumbo builds, but until they are proven we need to
proceed with what works today.

Making jumbo builds more maintainable through compiler changes appears to
have an excellent cost/benefit ratio, and does nothing to stop us from
pursuing other options in parallel. We will always continue to do non-jumbo
builds, so our code will not be compromised by this effort.


On Tue, Apr 10, 2018 at 10:19 AM Reid Kleckner <rnk at google.com> wrote:

> On Tue, Apr 10, 2018 at 7:34 AM Nico Weber via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> On Tue, Apr 10, 2018 at 10:27 AM, David Blaikie <dblaikie at gmail.com>
>> wrote:
>>
>>> I haven't looked at the patches in detail - but generally a jumbo build
>>> feels like a bit of a workaround & maybe there are better long-term
>>> solutions that might fit into the compiler.
>>>
>>
>> People will use this, if we want them to or not (I have some influence in
>> chrome land and wasn't able to talk them out of it, since it does provide
>> huge benefits), and the workarounds needed without compiler support are
>> gnarly.
>>
>> So I think we might want to revisit our "you don't really want this"
>> stance on this topic we've had historically and instead try to make this
>> work well.
>>
>
> I'm also revisiting my position on this. We've discussed unity build
> support in the past (I think Ubisoft proposed it), and at the time I felt
> that it was very backwards-facing. It's not a long term solution to
> reducing the overall cost of C++ compilation, and it can lead to creeping
> transitive dependencies between C++ files.
>
> However, more than a year later, we have not produced a solution that is
> as easy to deploy and as compelling as unity builds are today. I think we
> need to seriously weigh cost of adding features to support unity/jumbo
> builds. The initial patches necessary to get things off the ground look
> small and relatively low-maintenance. They may be just the tip of the
> iceberg, so we need to gather more input, but I think it's worth a try.
>
> FYI, my availability this week is low, so I don't expect to be able to
> participate more in this thread.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180410/6c12c077/attachment.html>