<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, May 26, 2019 at 2:36 PM Richard Smith <<a href="mailto:richard@metafoo.co.uk">richard@metafoo.co.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">On Sun, 26 May 2019 at 13:26, David Blaikie via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks Richard - yeah sounds pretty similar though I'm a bit confused about what's happening in this case, in part because I know next to nothing about how Clang's PCH works (& especially how it differs from PCM/modules).<br><br>Richard: why would any module or PCH cause a subsequent compilation to perform more pending instantiations? (I would've thought/my understanding was that nothing in the module would be used if it wasn't referenced from the source file, so why would a pch cause more pending instantiations?)<br></div></blockquote><div><br></div><div>Our design philosophy for modules and preamble precompilation is for a compilation using a precompiled header / preamble to behave identically to a compilation that parsed the header rather than using a precompiled form. So we don't perform end-of-translation-unit template instantiation at the end of a precompiled header, and instead perform the instantiation (and emit all the instantiated definitions and likewise all definitions of all used inline functions in the PCH) in all consumers of the PCH.</div></div></div></blockquote><div><br>What would happen if we didn't perform pending instantiations that came from (were already pending) from the module? We'd miss some error messages in the code that uses the module. But if we instantiated the templates during the module building - would that be OK? If we did them only in the modular code generation?<br><br>- Dave<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div class="gmail_quote"><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Lubos: Could you provide a small standalone example of this increase in pending instantiations so it's a bit easier for me to understand the kind of code & what's happening?<br>You mentioned in the blog post that the use of a PCH causes more functions to be emitted into the final object file (than if a PCH had not been used, and the source remained the same). Especially the possibility of functions being emitted into the object file that are totally unused by the object file. (again, I'm especially interested in comparing the non-PCH with the PCH case here, rather than the Clang PCH with the VS PCH situation) - those are situations that would be very surprising to me.</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, May 25, 2019 at 6:38 PM Richard Smith <<a href="mailto:richard@metafoo.co.uk" target="_blank">richard@metafoo.co.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">This seems like a nice idea, and has a lot in common with our existing "modular codegen" mode, which does largely the same thing but for PCMs rather than PCHs. I'd hope we could share a lot of the implementation between the two features.<div><br></div><div>+David Blaikie, who implemented modular codegen and might be able to advise as to the best way to integrate similar functionality into our PCH support.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 25 May 2019 at 12:32, Lubos Lunak via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><br>
Hello,<br>
<br>
I'm working on a Clang patch that can make C++ builds noticeably faster in <br>
some setups by allowing control over how templates are instantiated, but I <br>
have some problems finishing it and need advice.<br>
<br>
Background: I am a LibreOffice developer. When enabling precompiled headers, <br>
e.g. for LO Calc precompiled headers save ~2/3 of build time when MSVC is <br>
used, but with Clang they save only ~10%. Moreover the larger the PCH the <br>
more time is saved with MSVC, but this is not so with Clang, in fact larger <br>
PCHs often make things slower.<br>
<br>
The recent -ftime-trace feature allowed me to investigate this and it turns <br>
out that the time saved by having to parse less is outweighted by having to <br>
instantiate (many) more templates. You can see -ftime-trace graphs at <br>
<a href="http://llunak.blogspot.com/2019/05/why-precompiled-headers-do-not-improve.html" rel="noreferrer" target="_blank">http://llunak.blogspot.com/2019/05/why-precompiled-headers-do-not-improve.html</a> <br>
(1nd row - no PCH, 2nd row - small PCH, 3rd row - large PCH), the .json files <br>
are at <a href="http://ge.tt/7RHeLHw2" rel="noreferrer" target="_blank">http://ge.tt/7RHeLHw2</a> if somebody wants to see them.<br>
<br>
Specifically, the time is spent in Sema::PerformPendingInstantiations() and <br>
Sema::InstantiateFunctionDefinition(). The vast majority of the <br>
instantiations comes from the PCH itself. This means that this is performed <br>
for every TU using the PCH, and it also means that it's useless work, as the <br>
linker will discard all but one copy of that.<br>
<br>
My WIP patch implements a new option to avoid that. The idea is that all <br>
sources using the PCH will be built with -fpch-template-instantiation=skip, <br>
which will prevent Sema::InstantiateFunctionDefinition() from actually <br>
instantiating templates coming from the PCH if they would be uneeded <br>
duplicates (note that means almost all PCH template instantiations in the <br>
case of a developer build with -O0 -g, which is my primary use case). Then <br>
one extra source file is built with -fpch-template-instantiation=force, which <br>
will provide one copy of instantiations. I assume that this is similar to how <br>
MSVC manages to have much better gains with PCH, the .obj created during PCH <br>
creation presumably contains single instantiations.<br>
<br>
In the -ftime-trace graphs linked above, the 4th row is large PCH with my <br>
patch. The compilation time saved by this is 50% and 60% for the two examples <br>
(and I think moving some templates into the PCH might get it to 70-75% for <br>
the second file).<br>
<br>
As I said, I have some problems that prevent the patch from being fully <br>
usable, so in order to finish it, could somebody help me with the following:<br>
<br>
- I don't understand how it is controlled which kind of ctor/dtor is emitted <br>
(complete ctor vs base ctor, i.e. C1 vs C2 type in the Itanium ABI). I get <br>
many undefined references because the TU built with instances does not have <br>
both types, yet other TUs refer to them. How can I force both of them be <br>
emitted?<br>
<br>
- I have an undefined reference to one template function that should be <br>
included in the TU with instances, but it isn't. The Sema part instantiates <br>
it and I could track it as far as getting generated in Codegen, but then I'm <br>
lost. I assume that it gets discarded because something in codegen or llvm <br>
considers it unused. Is there a place like that and where is it? Are there <br>
other places in codegen/llvm where I could check to see why this function <br>
doesn't get generated in the object file?<br>
<br>
- In Sema::InstantiateFunctionDefinition() the code for extern templates still <br>
instantiates a function if it has getContainedAutoType(), so my code should <br>
probably also check that. But I'm not sure what that is (is that 'auto foo() <br>
{ return 1; }' ?) or why that would need an instance in every TU.<br>
<br>
- I used BENIGN_ENUM_LANGOPT because Clang otherwise complains that the PCH is <br>
used with a different option than what it was generated with, which is <br>
necessary in this case, but I'm not sure if this is the correct handling of <br>
the option.<br>
<br>
- Is there a simple rule for what decides that a template needs to be <br>
instantiated? As far as I can tell, even using a template as a class member <br>
or having an inline member function manipulating it doesn't. When I mentioned <br>
moving some templates into the PCH in order to get possible 70% savings, I <br>
actually don't know how to cause an instantiation from the PCH, the templates <br>
and their uses are included there.<br>
<br>
Thank you.<br>
<br>
-- <br>
Lubos Lunak<br>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div></div>
</blockquote></div>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div></div>
</blockquote></div></div>