[cfe-dev] JumboSupport: making unity builds easier in Clang

[David Blaikie via cfe-dev]

On Tue, Apr 10, 2018 at 10:42 AM Bruce Dawson <brucedawson at chromium.org>
wrote:

> I think it was Churchill who said that "Jumbo builds are the worst form of
> build optimization, except for all the others."
>
> The simple reality is that Chromium has a lot of translation units and
> there is a minimum (often a large minimum) cost to compiling each of these
> translation units. In order to make Chromium build times closer to
> reasonable we either need to dramatically reduce the number of translation
> units (jumbo) or dramatically reduce the cost of compiling each of these
> translation units.
>
> Jumbo builds have some practical and philosophical problems but ultimately
> they work, and they work now, and they will get even better. Speculative
> efforts to reduce the per-translation-unit are, well, speculative.
>

Not quite sure what you mean here - Clang header modules have been deployed
across a variety of users (Apple initially shipped them to reduce compile
time for Apple developers, especially around Cocoa.h, as I understand it -
then Google's used them internally for protobufs & the like for a few years
now). So it's not exactly speculative that these features exist, are
implemented fairly robustly, and do offer improvements.


> If and when modules improve our build times we can assess whether they
> should replace or coexist with jumbo builds, but until they are proven we
> need to proceed with what works today.
>
> Making jumbo builds more maintainable through compiler changes appears to
> have an excellent cost/benefit ratio, and does nothing to stop us from
> pursuing other options in parallel. We will always continue to do non-jumbo
> builds, so our code will not be compromised by this effort.
>
>
> On Tue, Apr 10, 2018 at 10:19 AM Reid Kleckner <rnk at google.com> wrote:
>
>> On Tue, Apr 10, 2018 at 7:34 AM Nico Weber via cfe-dev <
>> cfe-dev at lists.llvm.org> wrote:
>>
>>> On Tue, Apr 10, 2018 at 10:27 AM, David Blaikie <dblaikie at gmail.com>
>>> wrote:
>>>
>>>> I haven't looked at the patches in detail - but generally a jumbo build
>>>> feels like a bit of a workaround & maybe there are better long-term
>>>> solutions that might fit into the compiler.
>>>>
>>>
>>> People will use this, if we want them to or not (I have some influence
>>> in chrome land and wasn't able to talk them out of it, since it does
>>> provide huge benefits), and the workarounds needed without compiler support
>>> are gnarly.
>>>
>>> So I think we might want to revisit our "you don't really want this"
>>> stance on this topic we've had historically and instead try to make this
>>> work well.
>>>
>>
>> I'm also revisiting my position on this. We've discussed unity build
>> support in the past (I think Ubisoft proposed it), and at the time I felt
>> that it was very backwards-facing. It's not a long term solution to
>> reducing the overall cost of C++ compilation, and it can lead to creeping
>> transitive dependencies between C++ files.
>>
>> However, more than a year later, we have not produced a solution that is
>> as easy to deploy and as compelling as unity builds are today. I think we
>> need to seriously weigh cost of adding features to support unity/jumbo
>> builds. The initial patches necessary to get things off the ground look
>> small and relatively low-maintenance. They may be just the tip of the
>> iceberg, so we need to gather more input, but I think it's worth a try.
>>
>> FYI, my availability this week is low, so I don't expect to be able to
>> participate more in this thread.
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180410/1ad1ec8c/attachment.html>