<div dir="ltr"><div dir="ltr">This seems like a nice idea, and has a lot in common with our existing "modular codegen" mode, which does largely the same thing but for PCMs rather than PCHs. I'd hope we could share a lot of the implementation between the two features.<div><br></div><div>+David Blaikie, who implemented modular codegen and might be able to advise as to the best way to integrate similar functionality into our PCH support.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 25 May 2019 at 12:32, Lubos Lunak via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Hello,<br>
<br>
I'm working on a Clang patch that can make C++ builds noticeably faster in <br>
some setups by allowing control over how templates are instantiated, but I <br>
have some problems finishing it and need advice.<br>
<br>
Background: I am a LibreOffice developer. When enabling precompiled headers, <br>
e.g. for LO Calc precompiled headers save ~2/3 of build time when MSVC is <br>
used, but with Clang they save only ~10%. Moreover the larger the PCH the <br>
more time is saved with MSVC, but this is not so with Clang, in fact larger <br>
PCHs often make things slower.<br>
<br>
The recent -ftime-trace feature allowed me to investigate this and it turns <br>
out that the time saved by having to parse less is outweighted by having to <br>
instantiate (many) more templates. You can see -ftime-trace graphs at <br>
<a href="http://llunak.blogspot.com/2019/05/why-precompiled-headers-do-not-improve.html" rel="noreferrer" target="_blank">http://llunak.blogspot.com/2019/05/why-precompiled-headers-do-not-improve.html</a> <br>
(1nd row - no PCH, 2nd row - small PCH, 3rd row - large PCH), the .json files <br>
are at <a href="http://ge.tt/7RHeLHw2" rel="noreferrer" target="_blank">http://ge.tt/7RHeLHw2</a> if somebody wants to see them.<br>
<br>
Specifically, the time is spent in Sema::PerformPendingInstantiations() and <br>
Sema::InstantiateFunctionDefinition(). The vast majority of the <br>
instantiations comes from the PCH itself. This means that this is performed <br>
for every TU using the PCH, and it also means that it's useless work, as the <br>
linker will discard all but one copy of that.<br>
<br>
My WIP patch implements a new option to avoid that. The idea is that all <br>
sources using the PCH will be built with -fpch-template-instantiation=skip, <br>
which will prevent Sema::InstantiateFunctionDefinition() from actually <br>
instantiating templates coming from the PCH if they would be uneeded <br>
duplicates (note that means almost all PCH template instantiations in the <br>
case of a developer build with -O0 -g, which is my primary use case). Then <br>
one extra source file is built with -fpch-template-instantiation=force, which <br>
will provide one copy of instantiations. I assume that this is similar to how <br>
MSVC manages to have much better gains with PCH, the .obj created during PCH <br>
creation presumably contains single instantiations.<br>
<br>
In the -ftime-trace graphs linked above, the 4th row is large PCH with my <br>
patch. The compilation time saved by this is 50% and 60% for the two examples <br>
(and I think moving some templates into the PCH might get it to 70-75% for <br>
the second file).<br>
<br>
As I said, I have some problems that prevent the patch from being fully <br>
usable, so in order to finish it, could somebody help me with the following:<br>
<br>
- I don't understand how it is controlled which kind of ctor/dtor is emitted <br>
(complete ctor vs base ctor, i.e. C1 vs C2 type in the Itanium ABI). I get <br>
many undefined references because the TU built with instances does not have <br>
both types, yet other TUs refer to them. How can I force both of them be <br>
emitted?<br>
<br>
- I have an undefined reference to one template function that should be <br>
included in the TU with instances, but it isn't. The Sema part instantiates <br>
it and I could track it as far as getting generated in Codegen, but then I'm <br>
lost. I assume that it gets discarded because something in codegen or llvm <br>
considers it unused. Is there a place like that and where is it? Are there <br>
other places in codegen/llvm where I could check to see why this function <br>
doesn't get generated in the object file?<br>
<br>
- In Sema::InstantiateFunctionDefinition() the code for extern templates still <br>
instantiates a function if it has getContainedAutoType(), so my code should <br>
probably also check that. But I'm not sure what that is (is that 'auto foo() <br>
{ return 1; }' ?) or why that would need an instance in every TU.<br>
<br>
- I used BENIGN_ENUM_LANGOPT because Clang otherwise complains that the PCH is <br>
used with a different option than what it was generated with, which is <br>
necessary in this case, but I'm not sure if this is the correct handling of <br>
the option.<br>
<br>
- Is there a simple rule for what decides that a template needs to be <br>
instantiated? As far as I can tell, even using a template as a class member <br>
or having an inline member function manipulating it doesn't. When I mentioned <br>
moving some templates into the PCH in order to get possible 70% savings, I <br>
actually don't know how to cause an instantiation from the PCH, the templates <br>
and their uses are included there.<br>
<br>
Thank you.<br>
<br>
-- <br>
Lubos Lunak<br>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div></div>