[cfe-dev] JumboSupport: making unity builds easier in Clang

[David Blaikie via cfe-dev]

This would have issues with distributed builds, though, right? Unless clang
then took on the burden of doing the distribution too, which might be a bit
much.

On Wed, Apr 11, 2018 at 12:43 AM David Chisnall via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> On 10 Apr 2018, at 21:28, Daniel Bratell via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
> >
> > I've heard (hearsay, I admit) from profiling that it seems the single
> largest time consumer in clang is template instantiation, something I
> assume can't easily be prepared in advance.
> >
> > One example is chromium's chrome/browser/browser target which is 732
> files that normally need 6220 CPU seconds to compile, average 8,5 seconds
> per file. All combined together gives a single translation unit that takes
> 400 seconds to compile, a mere 0.54 seconds on average per file. That
> indicates that about 8 seconds per compiled file is related to the
> processing of headers.
>
> It sounds as if there are two things here:
>
> 1. The time taken to parse the headers
> 2. The time taken to repeatedly instantiate templates that the linker will
> then discard
>
> Assuming a command line where all of the relevant source files are
> provided to the compiler invocation:
>
> Solving the first one is relatively easy if the files have a common prefix
> (which can be determined by simple string comparison).  Find the common
> prefix in the source files, build the clang AST, and then do a clone for
> each compilation unit.  Hopefully, the clone is a lot cheaper than
> re-parsing (and can ideally share source locations).
>
> The second is slightly more difficult, because it relies on sharing parts
> of the AST across notional compilation units.
>
> To make this work well with incremental builds, ideally you’d spit out all
> of the common template instantiations into a separate IR file, which could
> then be used with ThinLTO.
>
> Personally, I would prefer to have an interface where a build system can
> invoke clang with all of the files that need building and the degree of
> parallelism to use and let it share as much state as it wants across
> builds.  In an ideal world, clang would record which templates have been
> instantiated in a prior build (or a previous build step in the current
> build) and avoid any IRGen for them, at the very least.
>
> Old C++ compilers, predating linker support for COMDATs, emitted templates
> lazily, simply emitting references to them, then parsing the linker errors
> and generating missing implementations until the linker errors went away.
> Modern C++ compilers generate many instantiations of the same templates and
> then discard most of them.  It would be nice to find an intermediate point,
> which worked well with ThinLTO, where templates could be emitted once and
> be available for inlining everywhere.
>
> David
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20180411/e35ea881/attachment.html>