[llvm] r275073 - Implement callsite-hotness based inline cost for Sample-based PGO
Jingyue Wu via llvm-commits
llvm-commits at lists.llvm.org
Mon Jul 11 10:35:56 PDT 2016
This commit (r275072) and r275073 seem to have broken many tests AFAICT
from http://lab.llvm.org:8011/console.
On Mon, Jul 11, 2016 at 9:48 AM, Dehao Chen via llvm-commits <
llvm-commits at lists.llvm.org> wrote:
> Author: dehao
> Date: Mon Jul 11 11:48:54 2016
> New Revision: 275073
>
> URL: http://llvm.org/viewvc/llvm-project?rev=275073&view=rev
> Log:
> Implement callsite-hotness based inline cost for Sample-based PGO
>
> Summary:
> For sample-based PGO, using BFI to calculate callsite count is sometime
> not accurate. This is because with sampling based approach, if a callsite
> resides in a hot loop deeply nested in a bunch of cold branches, the
> callsite's BFI frequency would be inaccurately calculated due to lack of
> samples in the cold branch.
>
> E.g.
>
> if (A1 && A2 && A3 && ..... && A10) {
> for (i=0; i < 100000000; i++) {
> callsite();
> }
> }
>
> Assume that A1 to A100 are all 100% taken, and callsite has 1000 samples
> and thus is considerred hot. Because the loop's trip count is huge, it's
> normal that all branches outside the loop has no sample at all. As a
> result, we can only use static branch probability to derive the the
> frequency of the loop header. Assuming that static heuristic thinks each
> branch is 50% taken, then the count calculated from BFI will be 1/(2^10) of
> the actual value.
>
> In order to get more accurate callsite count, we directly annotate the
> weight on the call instruction, and directly use it when checking callsite
> hotness.
>
> Note that this mechanism can also be shared by instrumentation based
> callsite hotness analysis. The side benefit is that it breaks the
> dependency from Inliner to BFI as call count is embedded in the IR.
>
> Reviewers: davidxl, eraman, dnovillo
>
> Subscribers: llvm-commits
>
> Differential Revision: http://reviews.llvm.org/D22118
>
> Added:
> llvm/trunk/test/Transforms/Inline/inline-hot-callsite.ll
> Modified:
> llvm/trunk/include/llvm/IR/Instruction.h
> llvm/trunk/lib/Analysis/InlineCost.cpp
> llvm/trunk/lib/IR/Metadata.cpp
> llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp
>
> Modified: llvm/trunk/include/llvm/IR/Instruction.h
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Instruction.h?rev=275073&r1=275072&r2=275073&view=diff
>
> ==============================================================================
> --- llvm/trunk/include/llvm/IR/Instruction.h (original)
> +++ llvm/trunk/include/llvm/IR/Instruction.h Mon Jul 11 11:48:54 2016
> @@ -222,6 +222,11 @@ public:
> /// Returns false if no metadata or invalid metadata was found.
> bool extractProfMetadata(uint64_t &TrueVal, uint64_t &FalseVal);
>
> + /// Retrieve total raw weight values of a branch.
> + /// Returns true on success with profile total weights filled in.
> + /// Returns false if no metadata was found.
> + bool extractProfTotalWeight(uint64_t &TotalVal);
> +
> /// Set the debug location information for this instruction.
> void setDebugLoc(DebugLoc Loc) { DbgLoc = std::move(Loc); }
>
>
> Modified: llvm/trunk/lib/Analysis/InlineCost.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Analysis/InlineCost.cpp?rev=275073&r1=275072&r2=275073&view=diff
>
> ==============================================================================
> --- llvm/trunk/lib/Analysis/InlineCost.cpp (original)
> +++ llvm/trunk/lib/Analysis/InlineCost.cpp Mon Jul 11 11:48:54 2016
> @@ -633,11 +633,18 @@ void CallAnalyzer::updateThreshold(CallS
> Threshold = OptSizeThreshold;
> }
>
> + bool HotCallsite = false;
> + uint64_t TotalWeight;
> + if (CS.getInstruction()->extractProfTotalWeight(TotalWeight) &&
> + PSI->isHotCount(TotalWeight))
> + HotCallsite = true;
> +
> // Listen to the inlinehint attribute or profile based hotness
> information
> // when it would increase the threshold and the caller does not need to
> // minimize its size.
> bool InlineHint = Callee.hasFnAttribute(Attribute::InlineHint) ||
> - PSI->isHotFunction(&Callee);
> + PSI->isHotFunction(&Callee) ||
> + HotCallsite;
> if (InlineHint && HintThreshold > Threshold && !Caller->optForMinSize())
> Threshold = HintThreshold;
>
>
> Modified: llvm/trunk/lib/IR/Metadata.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Metadata.cpp?rev=275073&r1=275072&r2=275073&view=diff
>
> ==============================================================================
> --- llvm/trunk/lib/IR/Metadata.cpp (original)
> +++ llvm/trunk/lib/IR/Metadata.cpp Mon Jul 11 11:48:54 2016
> @@ -1312,6 +1312,31 @@ bool Instruction::extractProfMetadata(ui
> return true;
> }
>
> +bool Instruction::extractProfTotalWeight(uint64_t &TotalVal) {
> + assert((getOpcode() == Instruction::Br ||
> + getOpcode() == Instruction::Select ||
> + getOpcode() == Instruction::Call) &&
> + "Looking for branch weights on something besides branch");
> +
> + TotalVal = 0;
> + auto *ProfileData = getMetadata(LLVMContext::MD_prof);
> + if (!ProfileData)
> + return false;
> +
> + auto *ProfDataName = dyn_cast<MDString>(ProfileData->getOperand(0));
> + if (!ProfDataName ||
> !ProfDataName->getString().equals("branch_weights"))
> + return false;
> +
> + TotalVal = 0;
> + for (int i = 1; i < ProfileData->getNumOperands(); i++) {
> + auto *V =
> mdconst::dyn_extract<ConstantInt>(ProfileData->getOperand(i));
> + if (!V)
> + return false;
> + TotalVal += V->getValue().getZExtValue();
> + }
> + return true;
> +}
> +
> void Instruction::clearMetadataHashEntries() {
> assert(hasMetadataHashEntry() && "Caller should check");
> getContext().pImpl->InstructionMetadata.erase(this);
>
> Modified: llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp?rev=275073&r1=275072&r2=275073&view=diff
>
> ==============================================================================
> --- llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp (original)
> +++ llvm/trunk/lib/Transforms/IPO/SampleProfile.cpp Mon Jul 11 11:48:54
> 2016
> @@ -987,6 +987,19 @@ void SampleProfileLoader::propagateWeigh
> MDBuilder MDB(Ctx);
> for (auto &BI : F) {
> BasicBlock *BB = &BI;
> +
> + if (BlockWeights[BB]) {
> + for (auto &I : BB->getInstList()) {
> + if (CallInst *CI = dyn_cast<CallInst>(&I)) {
> + if (!dyn_cast<IntrinsicInst>(&I)) {
> + SmallVector<uint32_t, 1> Weights;
> + Weights.push_back(BlockWeights[BB]);
> + CI->setMetadata(LLVMContext::MD_prof,
> + MDB.createBranchWeights(Weights));
> + }
> + }
> + }
> + }
> TerminatorInst *TI = BB->getTerminator();
> if (TI->getNumSuccessors() == 1)
> continue;
>
> Added: llvm/trunk/test/Transforms/Inline/inline-hot-callsite.ll
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/Inline/inline-hot-callsite.ll?rev=275073&view=auto
>
> ==============================================================================
> --- llvm/trunk/test/Transforms/Inline/inline-hot-callsite.ll (added)
> +++ llvm/trunk/test/Transforms/Inline/inline-hot-callsite.ll Mon Jul 11
> 11:48:54 2016
> @@ -0,0 +1,52 @@
> +; RUN: opt < %s -inline -inline-threshold=0 -inlinehint-threshold=100 -S
> | FileCheck %s
> +
> +; This tests that a hot callsite gets the (higher) inlinehint-threshold
> even without
> +; without inline hints and gets inlined because the cost is less than
> +; inlinehint-threshold. A cold callee with identical body does not get
> inlined because
> +; cost exceeds the inline-threshold
> +
> +define i32 @callee1(i32 %x) {
> + %x1 = add i32 %x, 1
> + %x2 = add i32 %x1, 1
> + %x3 = add i32 %x2, 1
> +
> + ret i32 %x3
> +}
> +
> +define i32 @callee2(i32 %x) {
> +; CHECK-LABEL: @callee2(
> + %x1 = add i32 %x, 1
> + %x2 = add i32 %x1, 1
> + %x3 = add i32 %x2, 1
> +
> + ret i32 %x3
> +}
> +
> +define i32 @caller2(i32 %y1) {
> +; CHECK-LABEL: @caller2(
> +; CHECK: call i32 @callee2
> +; CHECK-NOT: call i32 @callee1
> +; CHECK: ret i32 %x3.i
> + %y2 = call i32 @callee2(i32 %y1), !prof !22
> + %y3 = call i32 @callee1(i32 %y2), !prof !21
> + ret i32 %y3
> +}
> +
> +!llvm.module.flags = !{!1}
> +!21 = !{!"branch_weights", i64 300}
> +!22 = !{!"branch_weights", i64 1}
> +
> +!1 = !{i32 1, !"ProfileSummary", !2}
> +!2 = !{!3, !4, !5, !6, !7, !8, !9, !10}
> +!3 = !{!"ProfileFormat", !"InstrProf"}
> +!4 = !{!"TotalCount", i64 10000}
> +!5 = !{!"MaxCount", i64 1000}
> +!6 = !{!"MaxInternalCount", i64 1}
> +!7 = !{!"MaxFunctionCount", i64 1000}
> +!8 = !{!"NumCounts", i64 3}
> +!9 = !{!"NumFunctions", i64 3}
> +!10 = !{!"DetailedSummary", !11}
> +!11 = !{!12, !13, !14}
> +!12 = !{i32 10000, i64 100, i32 1}
> +!13 = !{i32 999000, i64 100, i32 1}
> +!14 = !{i32 999999, i64 1, i32 2}
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160711/d55b61be/attachment.html>
More information about the llvm-commits
mailing list