[cfe-dev] Controlling instantiation of templates from PCH

Tue Jun 18 01:37:06 PDT 2019

Hi everyone,

Just wanted to chime in and mention another mode of operations for PCHs
that has not been discussed yet.

> So, step 1) perhaps we can just remove the PCH mode ^ here, and do the
header module thing (perform pending instantiations at the end of building
the PCH) for PCH as well.

Preamble optimization in code completion (used by libclang and clangd) is
where this mode is actually useful, so we should probably avoid removing
this completely without prior investigation of how this affects the time
until initial completion (I would expect the optimization to be significant
and useful).
That being said, changing the clang binary behavior seems ok, but we
probably want to keep this option in the C++ API.

On Tue, Jun 18, 2019 at 12:18 AM David Blaikie via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> Sorry for the delayed response.
>
> I'm getting a bit confused by the layers of this thread, so I'll see if I
> can sum up my understanding & see if that's any use.
>
> Currently there's a difference between PCH and Clang Header Modules
> semantics with regard to pending instantiations
> In PCH, pending instantiations are all performed at any use of the PCH.
> In modules pending instantiations are done at the end of building the
> header module (so they're performed once).
> This difference is the cause of the original compile time cost you were
> investigating (somewhat independent of the duplicate code in object files)
>
> So, step 1) perhaps we can just remove the PCH mode ^ here, and do the
> header module thing (perform pending instantiations at the end of building
> the PCH) for PCH as well.
> If you want to do this ^ work, I think a patch would be welcome (I don't
> know much about where the distinction is between PCH and modules that
> causes this difference in how pending instantiations are handled - I might
> be able to find the code a little easier than you, but perhaps you can have
> a look around & see if you can find it - or Richard might be able to tell
> you/us)
>
>
> Separate from that, there's some desire to be able to build an object file
> from a PCH to remove duplicate template instantiations in object files.
> (reducing compile time, reducing object size, maybe reducing link time).
> To do this - it sounds like there's already a flag
> (-building-pch-with-obj) for building PCHs that will have a separate object
> file. I've no idea what this flag does today - does it already do what you
> want in terms of moving template instantiations and inline functions into
> that object? Or does it do something else (what else?)? If it doesn't do
> what you want already, and you want to extend it to do what you want - yes,
> I'd recommend piggy-backing on the exsiting Clang Header Modules modular
> code generation features for this.
>
> & yes, potentially the pending instantiations could be moved over from the
> PCH-or-header-module generation step, to the PCH-or-header-module->Object
> step. (to benefit both PCH and header modules)
>
> - Dave
>
>
> On Fri, May 31, 2019 at 2:36 PM Lubos Lunak <l.lunak at centrum.cz> wrote:
>
>> On Friday 31 of May 2019, David Blaikie wrote:
>> > On Thu, May 30, 2019 at 6:04 AM Lubos Lunak <l.lunak at centrum.cz> wrote:
>> > > >  Two questions come to mind here:
>> > > > - is it reasonably ready for use?
>> > > > - how much work would it be to use it?
>> > > >
>> > > >  I tried Clang modules after they were mentioned in the first reply,
>> > > > and I got the impression that they need preparation for every header
>> > > > file used by the project, even external ones. Unless that can be
>> > > > automated, I don't
>> > > > quite see that happening for something the size and complexity of
>> > > > LibreOffice (we don't manually create our headers-to-become-PCHs
>> > > > either). And an unofficial tongue-in-cheek motton of LibreOffice is
>> > > > "proudly breaking your toolchain since 1985", so unless modules are
>> > > > reasonably usable, we'll run into all the bugs there and nobody will
>> > > > want to use it.
>> > > >
>> > > >  If doing this is good technically, long-term, fine, do it. But I'd
>> > > > like to
>> > > > have something that works this summer, and my rather simple patch
>> can
>> > > > deliver that.
>> > >
>> > >  And so this part would be irrelevant in that case I hope?
>> >
>> > Right. (I mean, a separate question is whether you'd want to use
>> modules -
>> > but yes, at the very least it does involve standardizing your headers on
>> > "well behaved" sort of restraints (basically "you can include the header
>> > anywhere, any time, and it always behaves the same way" - so not having
>> > headers that depend on macros locally defined to different values in
>> > different translation units, etc) - but yeah, it's a lot more work than
>> the
>> > PCH situation)
>>
>>
>>  We can try modules eventually, but as said above, I expect the switch to
>> those would be possibly quite some work, and there's also the question
>> how
>> build tools like ccache and icecream would cope with modules, so unlikely
>> now.
>>
>> >
>> > > > > * Moving the pending instantiation processing to the end of the
>> PCH
>> > > > > would make PCH generation a little slower, but given a project
>> would
>> > > > > only have one PCH that might not be a huge problem.
>> > > >
>> > > >  I think that would be very well worth it.
>> > >
>> > >  This should be the same either way.
>>
>>  Just to make it clear, this means "this should be worth it for either
>> case of
>> whether I understood correctly or not that we should use modules".
>> However
>> I'm later basically contradicting this by saying that if the build mode
>> is
>> switched to generate an object file to aacompany the PCH then this
>> shouldn't
>> be done.
>>
>> > Yeah, I think I'd misunderstood your proposal - I had assumed there was
>> a
>> > separation between PCH generation and the PCH->Object step (& in that
>> > latter step, the pending instantiations would be done). Sounds like your
>> > .h->PCH step also generates the object?
>>
>>
>>  My current plan is still to have a separate PCH->object step, but the
>> idea
>> now is to use -building-pch-with-obj. So the build steps now should be:
>>
>> # generate PCH and mark it as having accompanying .o
>> clang++ -c precompiled.hxx -o precompiled.pch -Xclang
>> -building-pch-with-obj
>> # generate that .o
>> clang++ -c empty.cxx -include-pch
>> precompiled.pch -Xclang -building-pch-with-obj
>> # compile the rest
>> clang++ -c whatever.cxx -include-pch precompiled.pch
>>
>>  So the first step will mark the PCH, the second step will generate all
>> the
>> shared code, and the remaining steps will see the PCH as marked and will
>> skip
>> generating things again. I like this separation better than /Yu merging
>> the
>> first two steps into one, here each step generates just one output and
>> compilations depending on the PCH can already run alongside generating
>> the
>> PCH's object. And keeping instantiating PCH templates the way it is now
>> instead of moving it to the end of PCH creation would mean that work is
>> done
>> only in the second step, so the PCH generation wouldn't get slower.
>>
>>  And this could be easily later extended to e.g. non-template inline
>> functions
>> that get out-of-line copies in debug mode:
>> - Are we using a marked PCH and -building-pch-with-obj is set? =>
>> generate
>> shared copies
>> - Are we using a marked PCH but -building-pch-with-obj is not set? =>
>> skip
>> generation
>> - The PCH is not marked as having an object? => work normally
>>
>>  I don't know how this relates to modules, but for PCHs I expect this
>> should
>> work fine.
>>
>> > But I still suspect whatever that implements isn't quite
>> modules-codegen,
>> > but I could be wrong.
>>
>>
>>  I still don't quite get how modules relate to my patch, so I can't
>> really
>> comment on this.
>>
>> --
>>  Lubos Lunak
>>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>

-- 
Regards,
Ilya Biryukov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190618/bd4bd2dd/attachment.html>