[cfe-dev] Controlling instantiation of templates from PCH

[David Blaikie via cfe-dev]

Sorry for the delayed response.

I'm getting a bit confused by the layers of this thread, so I'll see if I
can sum up my understanding & see if that's any use.

Currently there's a difference between PCH and Clang Header Modules
semantics with regard to pending instantiations
In PCH, pending instantiations are all performed at any use of the PCH.
In modules pending instantiations are done at the end of building the
header module (so they're performed once).
This difference is the cause of the original compile time cost you were
investigating (somewhat independent of the duplicate code in object files)

So, step 1) perhaps we can just remove the PCH mode ^ here, and do the
header module thing (perform pending instantiations at the end of building
the PCH) for PCH as well.
If you want to do this ^ work, I think a patch would be welcome (I don't
know much about where the distinction is between PCH and modules that
causes this difference in how pending instantiations are handled - I might
be able to find the code a little easier than you, but perhaps you can have
a look around & see if you can find it - or Richard might be able to tell
you/us)


Separate from that, there's some desire to be able to build an object file
from a PCH to remove duplicate template instantiations in object files.
(reducing compile time, reducing object size, maybe reducing link time).
To do this - it sounds like there's already a flag (-building-pch-with-obj)
for building PCHs that will have a separate object file. I've no idea what
this flag does today - does it already do what you want in terms of moving
template instantiations and inline functions into that object? Or does it
do something else (what else?)? If it doesn't do what you want already, and
you want to extend it to do what you want - yes, I'd recommend
piggy-backing on the exsiting Clang Header Modules modular code generation
features for this.

& yes, potentially the pending instantiations could be moved over from the
PCH-or-header-module generation step, to the PCH-or-header-module->Object
step. (to benefit both PCH and header modules)

- Dave


On Fri, May 31, 2019 at 2:36 PM Lubos Lunak <l.lunak at centrum.cz> wrote:

> On Friday 31 of May 2019, David Blaikie wrote:
> > On Thu, May 30, 2019 at 6:04 AM Lubos Lunak <l.lunak at centrum.cz> wrote:
> > > >  Two questions come to mind here:
> > > > - is it reasonably ready for use?
> > > > - how much work would it be to use it?
> > > >
> > > >  I tried Clang modules after they were mentioned in the first reply,
> > > > and I got the impression that they need preparation for every header
> > > > file used by the project, even external ones. Unless that can be
> > > > automated, I don't
> > > > quite see that happening for something the size and complexity of
> > > > LibreOffice (we don't manually create our headers-to-become-PCHs
> > > > either). And an unofficial tongue-in-cheek motton of LibreOffice is
> > > > "proudly breaking your toolchain since 1985", so unless modules are
> > > > reasonably usable, we'll run into all the bugs there and nobody will
> > > > want to use it.
> > > >
> > > >  If doing this is good technically, long-term, fine, do it. But I'd
> > > > like to
> > > > have something that works this summer, and my rather simple patch can
> > > > deliver that.
> > >
> > >  And so this part would be irrelevant in that case I hope?
> >
> > Right. (I mean, a separate question is whether you'd want to use modules
> -
> > but yes, at the very least it does involve standardizing your headers on
> > "well behaved" sort of restraints (basically "you can include the header
> > anywhere, any time, and it always behaves the same way" - so not having
> > headers that depend on macros locally defined to different values in
> > different translation units, etc) - but yeah, it's a lot more work than
> the
> > PCH situation)
>
>
>  We can try modules eventually, but as said above, I expect the switch to
> those would be possibly quite some work, and there's also the question how
> build tools like ccache and icecream would cope with modules, so unlikely
> now.
>
> >
> > > > > * Moving the pending instantiation processing to the end of the PCH
> > > > > would make PCH generation a little slower, but given a project
> would
> > > > > only have one PCH that might not be a huge problem.
> > > >
> > > >  I think that would be very well worth it.
> > >
> > >  This should be the same either way.
>
>  Just to make it clear, this means "this should be worth it for either
> case of
> whether I understood correctly or not that we should use modules". However
> I'm later basically contradicting this by saying that if the build mode is
> switched to generate an object file to aacompany the PCH then this
> shouldn't
> be done.
>
> > Yeah, I think I'd misunderstood your proposal - I had assumed there was a
> > separation between PCH generation and the PCH->Object step (& in that
> > latter step, the pending instantiations would be done). Sounds like your
> > .h->PCH step also generates the object?
>
>
>  My current plan is still to have a separate PCH->object step, but the
> idea
> now is to use -building-pch-with-obj. So the build steps now should be:
>
> # generate PCH and mark it as having accompanying .o
> clang++ -c precompiled.hxx -o precompiled.pch -Xclang
> -building-pch-with-obj
> # generate that .o
> clang++ -c empty.cxx -include-pch
> precompiled.pch -Xclang -building-pch-with-obj
> # compile the rest
> clang++ -c whatever.cxx -include-pch precompiled.pch
>
>  So the first step will mark the PCH, the second step will generate all
> the
> shared code, and the remaining steps will see the PCH as marked and will
> skip
> generating things again. I like this separation better than /Yu merging
> the
> first two steps into one, here each step generates just one output and
> compilations depending on the PCH can already run alongside generating the
> PCH's object. And keeping instantiating PCH templates the way it is now
> instead of moving it to the end of PCH creation would mean that work is
> done
> only in the second step, so the PCH generation wouldn't get slower.
>
>  And this could be easily later extended to e.g. non-template inline
> functions
> that get out-of-line copies in debug mode:
> - Are we using a marked PCH and -building-pch-with-obj is set? => generate
> shared copies
> - Are we using a marked PCH but -building-pch-with-obj is not set? => skip
> generation
> - The PCH is not marked as having an object? => work normally
>
>  I don't know how this relates to modules, but for PCHs I expect this
> should
> work fine.
>
> > But I still suspect whatever that implements isn't quite modules-codegen,
> > but I could be wrong.
>
>
>  I still don't quite get how modules relate to my patch, so I can't really
> comment on this.
>
> --
>  Lubos Lunak
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190617/6ce70e23/attachment.html>