[cfe-dev] JumboSupport: making unity builds easier in Clang

[Jens Widell via cfe-dev]

On Tue, May 22, 2018 at 2:04 PM, Hans Wennborg <hans at chromium.org> wrote:
> Hi Jens,
>
> On Fri, May 18, 2018 at 9:17 AM, Jens Widell via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
>> Hi all,
>>
>> I'd like to summarize this thread (so far) and try to agree on a way forward.
>>
>> First, we had an interesting discussion on unity building in general,
>> or perhaps rather other, more "modern", ways to achieve the same or
>> similar effects. In the short term, this discussion is mostly
>> academic; the fact is that the Chromium project already has a unity
>> build configuration as described at the start of this thread, and it's
>> in "production" use. We're proposing these changes to Clang to reduce
>> an existing maintenance cost.
>>
>> Second, we've had two principal proposed ways to support unity builds in Clang:
>>
>> 1) The "custom" but ultimately pretty trivial, in terms of changes to
>> the core Clang code, way implemented by my proof-of-concept patch.
>>
>> 2) An thus-far nowhere implemented way built on top of Clang's support
>> for C++ modules.
>>
>> I'm not qualified to tell how much work (2) would be, how big the
>> changes to Clang would be, or how much of a maintenance burden this
>> would be on the Clang project, but it sounds more complicated to me,
>> so I'm inclined to think that (2) > (1) in all of those metrics.
>>
>> I will also not be able to implement (2), because of the inevitably
>> limited nature of time.
>>
>> So, if I don't receive push-back here (e.g. someone committing to work
>> on (2)), I'll proceed to submit patches for (1).
>
> My concern is whether (1) would carry its own weight. Even though it's
> fairly simple, would it be useful enough for projects besides Chromium
> that it warrants the complexity of having it in the tree? Are there
> other problems it would need to handle, like explicit template
> instantiation declarations and definitions that weren't intended to
> end up in the same translation unit.. I feel there could be lots of
> problems.

So, we/I never really intended for the Clang extension to completely
fix all issues that are or can be caused by combining source files
into a single compilation unit. Sure, that would be awesome to have,
but the intention was something that addressed some major annoyances,
and thus makes the unity build configuration less burdensome to
support.

The complexity of a solution, any solution, that addresses all
possible issues may well be unfeasible to have in the Clang source
tree. I don't have such a solution to look at, and never intended to
produce one, so I can't really say.

I don't really know about other projects. Chromium certainly isn't the
first project to use unity builds, but I don't know whether other
projects would find a Clang extension useful, or if they've worked
around the issues in other ways. The Chromium code base is in certain
ways rather painful to apply unity builds to, which would not
necessarily apply to all code bases, but also surely isn't
particularly unique.


> But if it's possible to implement this in a very simple and clean way
> (that also handles static functions and variables the same as
> anonymous namespaces) that works well for many projects? If so it
> seems hard to argue against it.

I think my proof-of-concept is fairly simple and clean, in particular
the parts that's in Clang and not the plugin.

It does not handle static functions or variables. Would that be a
specific requirement for this extension to be acceptable? Would
anything else?

> For Chromium, part of me hopes we could do something better though.
> Maybe using proper modules, or maybe just by restructuring parts of
> Blink.

And I'm of course in no way against that.

But we've had a huge compilation time issue for a long while. And
we've been using a very effective solution to address that issue for
quite some time. At this point we're dealing with the maintenance cost
of the existing solution. Any discussion about alternative solutions
ought to happen in parallel with discussions about improving the
existing solution, I think.

--
Jens