[cfe-dev] JumboSupport: making unity builds easier in Clang

[Bruce Dawson via cfe-dev]

It sounds like I was exaggerating how speculative modules are. On the other
hand, until they are in use and demonstrating compile-time reductions in
Chromium they are still speculative - we don't know how big a reduction
they will give, how much work will be required to use them, how they will
interact with goma builds, etc.

On Tue, Apr 10, 2018 at 10:57 AM David Blaikie <dblaikie at gmail.com> wrote:

>
>
> On Tue, Apr 10, 2018 at 10:42 AM Bruce Dawson <brucedawson at chromium.org>
> wrote:
>
>> I think it was Churchill who said that "Jumbo builds are the worst form
>> of build optimization, except for all the others."
>>
>> The simple reality is that Chromium has a lot of translation units and
>> there is a minimum (often a large minimum) cost to compiling each of these
>> translation units. In order to make Chromium build times closer to
>> reasonable we either need to dramatically reduce the number of translation
>> units (jumbo) or dramatically reduce the cost of compiling each of these
>> translation units.
>>
>> Jumbo builds have some practical and philosophical problems but
>> ultimately they work, and they work now, and they will get even better.
>> Speculative efforts to reduce the per-translation-unit are, well,
>> speculative.
>>
>
> Not quite sure what you mean here - Clang header modules have been
> deployed across a variety of users (Apple initially shipped them to reduce
> compile time for Apple developers, especially around Cocoa.h, as I
> understand it - then Google's used them internally for protobufs & the like
> for a few years now). So it's not exactly speculative that these features
> exist, are implemented fairly robustly, and do offer improvements.
>
>
>> If and when modules improve our build times we can assess whether they
>> should replace or coexist with jumbo builds, but until they are proven we
>> need to proceed with what works today.
>>
>> Making jumbo builds more maintainable through compiler changes appears to
>> have an excellent cost/benefit ratio, and does nothing to stop us from
>> pursuing other options in parallel. We will always continue to do non-jumbo
>> builds, so our code will not be compromised by this effort.
>>
>>
>> On Tue, Apr 10, 2018 at 10:19 AM Reid Kleckner <rnk at google.com> wrote:
>>
>>> On Tue, Apr 10, 2018 at 7:34 AM Nico Weber via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> On Tue, Apr 10, 2018 at 10:27 AM, David Blaikie <dblaikie at gmail.com>
>>>> wrote:
>>>>
>>>>> I haven't looked at the patches in detail - but generally a jumbo
>>>>> build feels like a bit of a workaround & maybe there are better long-term
>>>>> solutions that might fit into the compiler.
>>>>>
>>>>
>>>> People will use this, if we want them to or not (I have some influence
>>>> in chrome land and wasn't able to talk them out of it, since it does
>>>> provide huge benefits), and the workarounds needed without compiler support
>>>> are gnarly.
>>>>
>>>> So I think we might want to revisit our "you don't really want this"
>>>> stance on this topic we've had historically and instead try to make this
>>>> work well.
>>>>
>>>
>>> I'm also revisiting my position on this. We've discussed unity build
>>> support in the past (I think Ubisoft proposed it), and at the time I felt
>>> that it was very backwards-facing. It's not a long term solution to
>>> reducing the overall cost of C++ compilation, and it can lead to creeping
>>> transitive dependencies between C++ files.
>>>
>>> However, more than a year later, we have not produced a solution that is
>>> as easy to deploy and as compelling as unity builds are today. I think we
>>> need to seriously weigh cost of adding features to support unity/jumbo
>>> builds. The initial patches necessary to get things off the ground look
>>> small and relatively low-maintenance. They may be just the tip of the
>>> iceberg, so we need to gather more input, but I think it's worth a try.
>>>
>>> FYI, my availability this week is low, so I don't expect to be able to
>>> participate more in this thread.
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180410/69bb90f6/attachment.html>