[cfe-dev] JumboSupport: making unity builds easier in Clang

[Hans Wennborg via cfe-dev]

On Tue, May 22, 2018 at 4:19 PM, Jens Widell <jl at opera.com> wrote:
> On Tue, May 22, 2018 at 2:04 PM, Hans Wennborg <hans at chromium.org> wrote:
>> Hi Jens,
>>
>> On Fri, May 18, 2018 at 9:17 AM, Jens Widell via cfe-dev
>> <cfe-dev at lists.llvm.org> wrote:
>>> Hi all,
>>>
>>> I'd like to summarize this thread (so far) and try to agree on a way forward.
>>>
>>> First, we had an interesting discussion on unity building in general,
>>> or perhaps rather other, more "modern", ways to achieve the same or
>>> similar effects. In the short term, this discussion is mostly
>>> academic; the fact is that the Chromium project already has a unity
>>> build configuration as described at the start of this thread, and it's
>>> in "production" use. We're proposing these changes to Clang to reduce
>>> an existing maintenance cost.
>>>
>>> Second, we've had two principal proposed ways to support unity builds in Clang:
>>>
>>> 1) The "custom" but ultimately pretty trivial, in terms of changes to
>>> the core Clang code, way implemented by my proof-of-concept patch.
>>>
>>> 2) An thus-far nowhere implemented way built on top of Clang's support
>>> for C++ modules.
>>>
>>> I'm not qualified to tell how much work (2) would be, how big the
>>> changes to Clang would be, or how much of a maintenance burden this
>>> would be on the Clang project, but it sounds more complicated to me,
>>> so I'm inclined to think that (2) > (1) in all of those metrics.
>>>
>>> I will also not be able to implement (2), because of the inevitably
>>> limited nature of time.
>>>
>>> So, if I don't receive push-back here (e.g. someone committing to work
>>> on (2)), I'll proceed to submit patches for (1).
>>
>> My concern is whether (1) would carry its own weight. Even though it's
>> fairly simple, would it be useful enough for projects besides Chromium
>> that it warrants the complexity of having it in the tree? Are there
>> other problems it would need to handle, like explicit template
>> instantiation declarations and definitions that weren't intended to
>> end up in the same translation unit.. I feel there could be lots of
>> problems.
>
> So, we/I never really intended for the Clang extension to completely
> fix all issues that are or can be caused by combining source files
> into a single compilation unit. Sure, that would be awesome to have,
> but the intention was something that addressed some major annoyances,
> and thus makes the unity build configuration less burdensome to
> support.

Right, and that's fine for a plugin, but then it becomes hard to
motivate changes to Clang itself.

I'm starting to think that maybe one way forward is trying to figure
out ways to do what you need to do from the plugin without changing
Clang itself. After all, it has very good access to the AST and other
Clang internals.

For example, instead of introducing NamespaceDecl::IsDisabled to hide
the names, maybe the plugin could set Decl::IdentifierNamespace to
zero on the decls that should be hidden. (That's not a public member,
but perhaps it's possible to get around that obstacle.)

The mangling problem is harder to solve from the plugin obviously, but
perhaps one way could be that after each "main" file is finished, to
walk the LLVM Module and rename internal symbols to something
file-specific.

These are hacks of course, but if they're effective maybe that's OK
until some more sophisticated solution is ready.

 - Hans