[llvm-commits] [llvm] r98099 - in /llvm/trunk: include/llvm/Analysis/InlineCost.h include/llvm/Transforms/IPO/InlinerPass.h lib/Analysis/InlineCost.cpp lib/Transforms/IPO/InlineAlways.cpp lib/Transforms/IPO/InlineSimple.cpp lib/Transforms/IPO/Inl
Daniel Dunbar
daniel at zuster.org
Tue Mar 9 15:52:31 PST 2010
Hi Jakob,
On Tue, Mar 9, 2010 at 3:02 PM, Jakob Stoklund Olesen <stoklund at 2pi.dk> wrote:
> Author: stoklund
> Date: Tue Mar 9 17:02:17 2010
> New Revision: 98099
>
> URL: http://llvm.org/viewvc/llvm-project?rev=98099&view=rev
> Log:
> Try to keep the cached inliner costs around for a bit longer for big functions.
>
> The Caller cost info would be reset everytime a callee was inlined. If the
> caller has lots of calls and there is some mutual recursion going on, the
> caller cost info could be calculated many times.
>
> This patch reduces inliner runtime from 240s to 0.5s for a function with 20000
> small function calls.
>
> This is a more conservative version of r98089 that doesn't break the clang
> test CodeGenCXX/temp-order.cpp. That test relies on rather extreme inlining
> for constant folding.
That test doesn't *have* to pass, and we can change it if need be.
However, it sounds like this should just be a performance improvement
not a functionality change, so I wouldn't expect it to break the test.
- Daniel
> Modified:
> llvm/trunk/include/llvm/Analysis/InlineCost.h
> llvm/trunk/include/llvm/Transforms/IPO/InlinerPass.h
> llvm/trunk/lib/Analysis/InlineCost.cpp
> llvm/trunk/lib/Transforms/IPO/InlineAlways.cpp
> llvm/trunk/lib/Transforms/IPO/InlineSimple.cpp
> llvm/trunk/lib/Transforms/IPO/Inliner.cpp
>
> Modified: llvm/trunk/include/llvm/Analysis/InlineCost.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Analysis/InlineCost.h?rev=98099&r1=98098&r2=98099&view=diff
> ==============================================================================
> --- llvm/trunk/include/llvm/Analysis/InlineCost.h (original)
> +++ llvm/trunk/include/llvm/Analysis/InlineCost.h Tue Mar 9 17:02:17 2010
> @@ -179,6 +179,11 @@
> void resetCachedCostInfo(Function* Caller) {
> CachedFunctionInfo[Caller] = FunctionInfo();
> }
> +
> + /// growCachedCostInfo - update the cached cost info for Caller after Callee
> + /// has been inlined. If Callee is NULL it means a dead call has been
> + /// eliminated.
> + void growCachedCostInfo(Function* Caller, Function* Callee);
> };
> }
>
>
> Modified: llvm/trunk/include/llvm/Transforms/IPO/InlinerPass.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/IPO/InlinerPass.h?rev=98099&r1=98098&r2=98099&view=diff
> ==============================================================================
> --- llvm/trunk/include/llvm/Transforms/IPO/InlinerPass.h (original)
> +++ llvm/trunk/include/llvm/Transforms/IPO/InlinerPass.h Tue Mar 9 17:02:17 2010
> @@ -75,6 +75,10 @@
> ///
> virtual void resetCachedCostInfo(Function* Caller) = 0;
>
> + /// growCachedCostInfo - update the cached cost info for Caller after Callee
> + /// has been inlined.
> + virtual void growCachedCostInfo(Function* Caller, Function* Callee) = 0;
> +
> /// removeDeadFunctions - Remove dead functions that are not included in
> /// DNR (Do Not Remove) list.
> bool removeDeadFunctions(CallGraph &CG,
>
> Modified: llvm/trunk/lib/Analysis/InlineCost.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Analysis/InlineCost.cpp?rev=98099&r1=98098&r2=98099&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Analysis/InlineCost.cpp (original)
> +++ llvm/trunk/lib/Analysis/InlineCost.cpp Tue Mar 9 17:02:17 2010
> @@ -383,3 +383,45 @@
> Factor += 1.5f;
> return Factor;
> }
> +
> +/// growCachedCostInfo - update the cached cost info for Caller after Callee has
> +/// been inlined.
> +void
> +InlineCostAnalyzer::growCachedCostInfo(Function* Caller, Function* Callee) {
> + FunctionInfo &CallerFI = CachedFunctionInfo[Caller];
> +
> + // For small functions we prefer to recalculate the cost for better accuracy.
> + if (CallerFI.Metrics.NumBlocks < 10 || CallerFI.Metrics.NumInsts < 1000) {
> + resetCachedCostInfo(Caller);
> + return;
> + }
> +
> + // For large functions, we can save a lot of computation time by skipping
> + // recalculations.
> + if (CallerFI.Metrics.NumCalls > 0)
> + --CallerFI.Metrics.NumCalls;
> +
> + if (Callee) {
> + FunctionInfo &CalleeFI = CachedFunctionInfo[Callee];
> + if (!CalleeFI.Metrics.NumBlocks) {
> + resetCachedCostInfo(Caller);
> + return;
> + }
> + CallerFI.Metrics.NeverInline |= CalleeFI.Metrics.NeverInline;
> + CallerFI.Metrics.usesDynamicAlloca |= CalleeFI.Metrics.usesDynamicAlloca;
> +
> + CallerFI.Metrics.NumInsts += CalleeFI.Metrics.NumInsts;
> + CallerFI.Metrics.NumBlocks += CalleeFI.Metrics.NumBlocks;
> + CallerFI.Metrics.NumCalls += CalleeFI.Metrics.NumCalls;
> + CallerFI.Metrics.NumVectorInsts += CalleeFI.Metrics.NumVectorInsts;
> + CallerFI.Metrics.NumRets += CalleeFI.Metrics.NumRets;
> +
> + // analyzeBasicBlock counts each function argument as an inst.
> + if (CallerFI.Metrics.NumInsts >= Callee->arg_size())
> + CallerFI.Metrics.NumInsts -= Callee->arg_size();
> + else
> + CallerFI.Metrics.NumInsts = 0;
> + }
> + // We are not updating the argumentweights. We have already determined that
> + // Caller is a fairly large function, so we accept the loss of precision.
> +}
>
> Modified: llvm/trunk/lib/Transforms/IPO/InlineAlways.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/InlineAlways.cpp?rev=98099&r1=98098&r2=98099&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Transforms/IPO/InlineAlways.cpp (original)
> +++ llvm/trunk/lib/Transforms/IPO/InlineAlways.cpp Tue Mar 9 17:02:17 2010
> @@ -45,7 +45,10 @@
> return CA.getInlineFudgeFactor(CS);
> }
> void resetCachedCostInfo(Function *Caller) {
> - return CA.resetCachedCostInfo(Caller);
> + CA.resetCachedCostInfo(Caller);
> + }
> + void growCachedCostInfo(Function* Caller, Function* Callee) {
> + CA.growCachedCostInfo(Caller, Callee);
> }
> virtual bool doFinalization(CallGraph &CG) {
> return removeDeadFunctions(CG, &NeverInline);
>
> Modified: llvm/trunk/lib/Transforms/IPO/InlineSimple.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/InlineSimple.cpp?rev=98099&r1=98098&r2=98099&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Transforms/IPO/InlineSimple.cpp (original)
> +++ llvm/trunk/lib/Transforms/IPO/InlineSimple.cpp Tue Mar 9 17:02:17 2010
> @@ -45,6 +45,9 @@
> void resetCachedCostInfo(Function *Caller) {
> CA.resetCachedCostInfo(Caller);
> }
> + void growCachedCostInfo(Function* Caller, Function* Callee) {
> + CA.growCachedCostInfo(Caller, Callee);
> + }
> virtual bool doInitialization(CallGraph &CG);
> };
> }
>
> Modified: llvm/trunk/lib/Transforms/IPO/Inliner.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/Inliner.cpp?rev=98099&r1=98098&r2=98099&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Transforms/IPO/Inliner.cpp (original)
> +++ llvm/trunk/lib/Transforms/IPO/Inliner.cpp Tue Mar 9 17:02:17 2010
> @@ -369,6 +369,8 @@
> CG[Caller]->removeCallEdgeFor(CS);
> CS.getInstruction()->eraseFromParent();
> ++NumCallsDeleted;
> + // Update the cached cost info with the missing call
> + growCachedCostInfo(Caller, NULL);
> } else {
> // We can only inline direct calls to non-declarations.
> if (Callee == 0 || Callee->isDeclaration()) continue;
> @@ -382,6 +384,9 @@
> if (!InlineCallIfPossible(CS, CG, TD, InlinedArrayAllocas))
> continue;
> ++NumInlined;
> +
> + // Update the cached cost info with the inlined call.
> + growCachedCostInfo(Caller, Callee);
> }
>
> // If we inlined or deleted the last possible call site to the function,
> @@ -407,11 +412,6 @@
> delete CG.removeFunctionFromModule(CalleeNode);
> ++NumDeleted;
> }
> -
> - // Remove any cached cost info for this caller, as inlining the
> - // callee has increased the size of the caller (which may be the
> - // same as the callee).
> - resetCachedCostInfo(Caller);
>
> // Remove this call site from the list. If possible, use
> // swap/pop_back for efficiency, but do not use it if doing so would
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
More information about the llvm-commits
mailing list