[PATCH] D17212: [ThinLTO] Support for call graph in per-module and combined summary.

[Teresa Johnson via llvm-commits]

On Fri, Feb 26, 2016 at 10:53 AM, Mehdi Amini <mehdi.amini at apple.com> wrote:

>
> > On Feb 26, 2016, at 10:38 AM, Teresa Johnson <tejohnson at google.com>
> wrote:
> >
> > tejohnson added a comment.
> >
> > In http://reviews.llvm.org/D17212#362864, @joker.eph wrote:
> >
> >> How could we integrate accesses to global variable as part of this?
> >> It turns out that to be able to benefit from the linker information on
> what symbol is exported during the import, this is a must.
> >
> >
> > Well without it you still can see which function symbols will be
> exported, just not the variables, so you are running with less info and I
> guess need to assume that all static variables will be exposed and promote
> them.
>
> The scheme I am currently setting up is:
>
> 1) The linker gives us the list of symbols that need to be "preserved"
> (can't internalize)
> 2) Link the combined index
> 3) Compute the import list for every module *by just looking at the
> profile*
> 4) Do the promotion
>
> There is absolutely no assumption for the promotion (last step): you
> exactly know what will be imported by *every module*, and you can promote
> the optimal minimal amount of symbols.
>
> All of that is good and should work with your call-graph patch "as is".
>
> I'm looking to go two steps further during stage 3:
>
> 1) I want to drive the importing heuristic cost to actually take into
> account the need for promotion.
> I'll start some test on the extreme case by *forbiding* any promotion,
> i.e. if a function references an internal function or global, then it can't
> be imported in any other module. On the long term it may be interesting to
> include this in the importing threshold.
> This can be implemented with a flag or an int in the summary
> "numberOfInternalGlobalsReferenced", but will not be enough for step 2
> (below).
>
> 2) I want to benefit from the linker information from stage 1 to
> internalize symbols.
> It means that the information about the fact that a function is
> referencing an internal global *can't be in the summary* because the
> front-end does not know that the global will be internalized.
>

For ThinLTO I assume these internalized globals only includes those that
were declared externally-visible, but are not actually used cross-module as
per the linker?


> This can be implemented by not having a "call graph" but a "reference
> graph" (not sure on the terminology): i.e. edges would be there for any
> uses of a symbol to another one and not only calls.
>
>
>
> > To refine that behavior for variables, yes, we'd need additional info in
> the summary.
> >
> > (For davidxl or anyone else who didn't see the IRC conversation, Mehdi
> is looking at doing pure summary-based importing decisions in the linker
> step, then giving this info to the ThinLTO backends to avoid promotion of
> local values that aren't exported. For a distributed build if we wanted to
> do it this way the importing decisions would all be made in the plugin
> step, then would need to be serialized out for the distributed backends to
> check.)
> >
> > Two possibilities depending on the fidelity of the info you think you
> need:
> >
> > 1. One possibility is to just put a flag in the function summary if it
> accesses *any* local variables, and adjust the importing threshold
> accordingly. Then in the ThinLTO backend for the exporting module you need
> to check which of your own functions are on the import list, and which
> local variables they access, and promote accordingly.
> >
> > 2. If it will be really beneficial to note exactly which local variables
> are accessed by which function, we'll need to broaden the edges list to
> include accesses to variables (I assume you only care about local variables
> here). E.g. the per-module summary edge list for a function would need to
> include value ids for any local variables referenced by that function (not
> sure that the other parts of the triple, the static and profile counts, are
> needed for that). Then in the combined VST we need to include entries for
> GUIDs of things that don't have a function summary, but are referenced by
> these edges. When accessing a function summary edge list for a candidate
> function to import, you could then see the GUID of any local variables
> accessed. You wouldn't know them by name, but if for example you wanted a
> heuristic like "if >=N hot import candidate functions from module M access
> a local variable with GUID G, go ahead and import those and let G be
> promoted by the backend (which like in approach #1 needs to check which
> local variables are accessed by any functions on an import list)".
> >
> > Obviously 1) is easier and cheaper space-wise. What are your thoughts?
>
>
> So 1) is cheaper, but 2) a lot more powerful as explained above :)
>
>
> --
> Mehdi
>
>


-- 
Teresa Johnson |  Software Engineer |  tejohnson at google.com |  408-460-2413
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160226/ac53e674/attachment.html>