[llvm] r315635 - [XRay][tools] Updated stacks tool with flamegraph output.
Keith Wyss via llvm-commits
llvm-commits at lists.llvm.org
Thu Oct 12 16:41:48 PDT 2017
Thanks Evgenii.
I ran the 'check-llvm' target before submitting and it was successful.
I guess I forgot to provide a CMake option to get -Werror turned on.
It's trivial to fix the issue, and I'm running checks before fixing forward.
On Thu, Oct 12, 2017 at 4:23 PM, Evgenii Stepanov <eugeni.stepanov at gmail.com
> wrote:
> Please test your code with -Werror before committing.
>
> /code/llvm-project/llvm/tools/llvm-xray/xray-stacks.cc:801:7: error:
> default label in switch which covers all enumeration values
> [-Werror,-Wcovered-switch-default]
> default:
> ^
> /code/llvm-project/llvm/tools/llvm-xray/xray-stacks.cc:816:7: error:
> default label in switch which covers all enumeration values
> [-Werror,-Wcovered-switch-default]
> default:
> ^
> 2 errors generated.
>
> On Thu, Oct 12, 2017 at 3:47 PM, Keith Wyss via llvm-commits
> <llvm-commits at lists.llvm.org> wrote:
> > Author: kpw
> > Date: Thu Oct 12 15:47:42 2017
> > New Revision: 315635
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=315635&view=rev
> > Log:
> > [XRay][tools] Updated stacks tool with flamegraph output.
> >
> > Summary:
> > As the first step to allow analysis and visualization of xray collected
> data,
> > allow using the llvm-xray stacks tool to emit a complete listing of
> stacks in
> > the format consumable by a flamegraph tool.
> >
> > Possible follow up formats include chrome trace viewer format and sql
> load
> > files.
> >
> > As a POC, I'm able to generate flamegraphs of an xray instrumented llc
> compiling
> > hello world.
> >
> > Reviewers: dberris, pelikan
> >
> > Subscribers: llvm-commits
> >
> > Differential Revision: https://reviews.llvm.org/D38650
> >
> > Modified:
> > llvm/trunk/tools/llvm-xray/xray-stacks.cc
> >
> > Modified: llvm/trunk/tools/llvm-xray/xray-stacks.cc
> > URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-
> xray/xray-stacks.cc?rev=315635&r1=315634&r2=315635&view=diff
> > ============================================================
> ==================
> > --- llvm/trunk/tools/llvm-xray/xray-stacks.cc (original)
> > +++ llvm/trunk/tools/llvm-xray/xray-stacks.cc Thu Oct 12 15:47:42 2017
> > @@ -66,8 +66,52 @@ static cl::opt<bool>
> > cl::desc("Aggregate stack times across threads"),
> > cl::sub(Stack), cl::init(false));
> >
> > -/// A helper struct to work with formatv and XRayRecords. Makes it
> easier to use
> > -/// instrumentation map names or addresses in formatted output.
> > +static cl::opt<bool>
> > + DumpAllStacks("all-stacks",
> > + cl::desc("Dump sum of timings for all stacks. "
> > + "By default separates stacks per-thread."),
> > + cl::sub(Stack), cl::init(false));
> > +static cl::alias DumpAllStacksShort("all", cl::aliasopt(DumpAllStacks),
> > + cl::desc("Alias for -all-stacks"),
> > + cl::sub(Stack));
> > +
> > +// TODO(kpw): Add other interesting formats. Perhaps chrome trace
> viewer format
> > +// possibly with aggregations or just a linear trace of timings.
> > +enum StackOutputFormat { HUMAN, FLAMETOOL };
> > +
> > +static cl::opt<StackOutputFormat> StacksOutputFormat(
> > + "stack-format",
> > + cl::desc("The format that output stacks should be "
> > + "output in. Only applies with all-stacks."),
> > + cl::values(
> > + clEnumValN(HUMAN, "human",
> > + "Human readable output. Only valid without
> -all-stacks."),
> > + clEnumValN(FLAMETOOL, "flame",
> > + "Format consumable by Brendan Gregg's FlameGraph
> tool. "
> > + "Only valid with -all-stacks.")),
> > + cl::sub(Stack), cl::init(HUMAN));
> > +
> > +// Types of values for each stack in a CallTrie.
> > +enum class AggregationType {
> > + TOTAL_TIME, // The total time spent in a stack and its callees.
> > + INVOCATION_COUNT // The number of times the stack was invoked.
> > +};
> > +
> > +static cl::opt<AggregationType> RequestedAggregation(
> > + "aggregation-type",
> > + cl::desc("The type of aggregation to do on call stacks."),
> > + cl::values(
> > + clEnumValN(
> > + AggregationType::TOTAL_TIME, "time",
> > + "Capture the total time spent in an all invocations of a
> stack."),
> > + clEnumValN(AggregationType::INVOCATION_COUNT, "count",
> > + "Capture the number of times a stack was invoked. "
> > + "In flamegraph mode, this count also includes
> invocations "
> > + "of all callees.")),
> > + cl::sub(Stack), cl::init(AggregationType::TOTAL_TIME));
> > +
> > +/// A helper struct to work with formatv and XRayRecords. Makes it
> easier to
> > +/// use instrumentation map names or addresses in formatted output.
> > struct format_xray_record : public FormatAdapter<XRayRecord> {
> > explicit format_xray_record(XRayRecord record,
> > const FuncIdConversionHelper &conv)
> > @@ -274,10 +318,45 @@ TrieNode *mergeTrieNodes(const TrieNode
> > return Node;
> > }
> >
> > +template <AggregationType AggType>
> > +std::size_t GetValueForStack(const TrieNode *Node);
> > +
> > +// When computing total time spent in a stack, we're adding the timings
> from
> > +// its callees and the timings from when it was a leaf.
> > +template <>
> > +std::size_t
> > +GetValueForStack<AggregationType::TOTAL_TIME>(const TrieNode *Node) {
> > + auto TopSum = std::accumulate(Node->TerminalDurations.begin(),
> > + Node->TerminalDurations.end(), 0uLL);
> > + return std::accumulate(Node->IntermediateDurations.begin(),
> > + Node->IntermediateDurations.end(), TopSum);
> > +}
> > +
> > +// Calculates how many times a function was invoked.
> > +// TODO: Hook up option to produce stacks
> > +template <>
> > +std::size_t
> > +GetValueForStack<AggregationType::INVOCATION_COUNT>(const TrieNode
> *Node) {
> > + return Node->TerminalDurations.size() + Node->IntermediateDurations.
> size();
> > +}
> > +
> > +// Make sure there are implementations for each enum value.
> > +template <AggregationType T> struct DependentFalseType :
> std::false_type {};
> > +
> > +template <AggregationType AggType>
> > +std::size_t GetValueForStack(const TrieNode *Node) {
> > + static_assert(DependentFalseType<AggType>::value,
> > + "No implementation found for aggregation type
> provided.");
> > + return 0;
> > +}
> > +
> > class StackTrie {
> > + // Avoid the magic number of 4 propagated through the code with an
> alias.
> > + // We use this SmallVector to track the root nodes in a call graph.
> > + using RootVector = SmallVector<TrieNode *, 4>;
> >
> > // We maintain pointers to the roots of the tries we see.
> > - DenseMap<uint32_t, SmallVector<TrieNode *, 4>> Roots;
> > + DenseMap<uint32_t, RootVector> Roots;
> >
> > // We make sure all the nodes are accounted for in this list.
> > std::forward_list<TrieNode> NodeStore;
> > @@ -439,11 +518,23 @@ public:
> > }
> > }
> >
> > + /// Prints timing sums for each stack in each threads.
> > + template <AggregationType AggType>
> > + void printAllPerThread(raw_ostream &OS, FuncIdConversionHelper &FN,
> > + StackOutputFormat format) {
> > + for (auto iter : Roots) {
> > + uint32_t threadId = iter.first;
> > + RootVector &perThreadRoots = iter.second;
> > + bool reportThreadId = true;
> > + printAll<AggType>(OS, FN, perThreadRoots, threadId,
> reportThreadId);
> > + }
> > + }
> > +
> > /// Prints top stacks from looking at all the leaves and ignoring
> thread IDs.
> > /// Stacks that consist of the same function IDs but were called in
> different
> > /// thread IDs are not considered unique in this printout.
> > void printIgnoringThreads(raw_ostream &OS, FuncIdConversionHelper
> &FN) {
> > - SmallVector<TrieNode *, 4> RootValues;
> > + RootVector RootValues;
> >
> > // Function to pull the values out of a map iterator.
> > using RootsType = decltype(Roots.begin())::value_type;
> > @@ -459,30 +550,88 @@ public:
> > print(OS, FN, RootValues);
> > }
> >
> > - /// Merges the trie by thread id before printing top stacks.
> > - void printAggregatingThreads(raw_ostream &OS, FuncIdConversionHelper
> &FN) {
> > - std::forward_list<TrieNode> AggregatedNodeStore;
> > - SmallVector<TrieNode *, 4> RootValues;
> > + /// Creates a merged list of Tries for unique stacks that disregards
> their
> > + /// thread IDs.
> > + RootVector mergeAcrossThreads(std::forward_list<TrieNode>
> &NodeStore) {
> > + RootVector MergedByThreadRoots;
> > for (auto MapIter : Roots) {
> > const auto &RootNodeVector = MapIter.second;
> > for (auto *Node : RootNodeVector) {
> > - auto MaybeFoundIter = find_if(RootValues, [Node](TrieNode
> *elem) {
> > - return Node->FuncId == elem->FuncId;
> > - });
> > - if (MaybeFoundIter == RootValues.end()) {
> > - RootValues.push_back(Node);
> > + auto MaybeFoundIter =
> > + find_if(MergedByThreadRoots, [Node](TrieNode *elem) {
> > + return Node->FuncId == elem->FuncId;
> > + });
> > + if (MaybeFoundIter == MergedByThreadRoots.end()) {
> > + MergedByThreadRoots.push_back(Node);
> > } else {
> > - RootValues.push_back(mergeTrieNodes(**MaybeFoundIter, *Node,
> nullptr,
> > - AggregatedNodeStore));
> > - RootValues.erase(MaybeFoundIter);
> > + MergedByThreadRoots.push_back(
> > + mergeTrieNodes(**MaybeFoundIter, *Node, nullptr,
> NodeStore));
> > + MergedByThreadRoots.erase(MaybeFoundIter);
> > }
> > }
> > }
> > - print(OS, FN, RootValues);
> > + return MergedByThreadRoots;
> > + }
> > +
> > + /// Print timing sums for all stacks merged by Thread ID.
> > + template <AggregationType AggType>
> > + void printAllAggregatingThreads(raw_ostream &OS,
> FuncIdConversionHelper &FN,
> > + StackOutputFormat format) {
> > + std::forward_list<TrieNode> AggregatedNodeStore;
> > + RootVector MergedByThreadRoots = mergeAcrossThreads(
> AggregatedNodeStore);
> > + bool reportThreadId = false;
> > + printAll<AggType>(OS, FN, MergedByThreadRoots,
> > + /*threadId*/ 0, reportThreadId);
> > + }
> > +
> > + /// Merges the trie by thread id before printing top stacks.
> > + void printAggregatingThreads(raw_ostream &OS, FuncIdConversionHelper
> &FN) {
> > + std::forward_list<TrieNode> AggregatedNodeStore;
> > + RootVector MergedByThreadRoots = mergeAcrossThreads(
> AggregatedNodeStore);
> > + print(OS, FN, MergedByThreadRoots);
> > + }
> > +
> > + // TODO: Add a format option when more than one are supported.
> > + template <AggregationType AggType>
> > + void printAll(raw_ostream &OS, FuncIdConversionHelper &FN,
> > + RootVector RootValues, uint32_t ThreadId, bool
> ReportThread) {
> > + SmallVector<const TrieNode *, 16> S;
> > + for (const auto *N : RootValues) {
> > + S.clear();
> > + S.push_back(N);
> > + while (!S.empty()) {
> > + auto *Top = S.pop_back_val();
> > + printSingleStack<AggType>(OS, FN, ReportThread, ThreadId, Top);
> > + for (const auto *C : Top->Callees)
> > + S.push_back(C);
> > + }
> > + }
> > + }
> > +
> > + /// Prints values for stacks in a format consumable for the
> flamegraph.pl
> > + /// tool. This is a line based format that lists each level in the
> stack
> > + /// hierarchy in a semicolon delimited form followed by a space and a
> numeric
> > + /// value. If breaking down by thread, the thread ID will be added as
> the
> > + /// root level of the stack.
> > + template <AggregationType AggType>
> > + void printSingleStack(raw_ostream &OS, FuncIdConversionHelper
> &Converter,
> > + bool ReportThread, uint32_t ThreadId,
> > + const TrieNode *Node) {
> > + if (ReportThread)
> > + OS << "thread_" << ThreadId << ";";
> > + SmallVector<const TrieNode *, 5> lineage{};
> > + lineage.push_back(Node);
> > + while (lineage.back()->Parent != nullptr)
> > + lineage.push_back(lineage.back()->Parent);
> > + while (!lineage.empty()) {
> > + OS << Converter.SymbolOrNumber(lineage.back()->FuncId) << ";";
> > + lineage.pop_back();
> > + }
> > + OS << " " << GetValueForStack<AggType>(Node) << "\n";
> > }
> >
> > void print(raw_ostream &OS, FuncIdConversionHelper &FN,
> > - SmallVector<TrieNode *, 4> RootValues) {
> > + RootVector RootValues) {
> > // Go through each of the roots, and traverse the call stack,
> producing the
> > // aggregates as you go along. Remember these aggregates and
> stacks, and
> > // show summary statistics about:
> > @@ -502,7 +651,7 @@ public:
> > S.emplace_back(N);
> >
> > while (!S.empty()) {
> > - auto Top = S.pop_back_val();
> > + auto *Top = S.pop_back_val();
> >
> > // We only start printing the stack (by walking up the parent
> pointers)
> > // when we get to a leaf function.
> > @@ -587,6 +736,17 @@ static CommandRegistration Unused(&Stack
> > "that aggregates threads."),
> > std::make_error_code(std::errc::invalid_argument));
> >
> > + if (!DumpAllStacks && StacksOutputFormat != HUMAN)
> > + return make_error<StringError>(
> > + Twine("Can't specify a non-human format without -all-stacks."),
> > + std::make_error_code(std::errc::invalid_argument));
> > +
> > + if (DumpAllStacks && StacksOutputFormat == HUMAN)
> > + return make_error<StringError>(
> > + Twine("You must specify a non-human format when reporting with "
> > + "-all-stacks."),
> > + std::make_error_code(std::errc::invalid_argument));
> > +
> > symbolize::LLVMSymbolizer::Options Opts(
> > symbolize::FunctionNameKind::LinkageName, true, true, false, "");
> > symbolize::LLVMSymbolizer Symbolizer(Opts);
> > @@ -625,6 +785,44 @@ static CommandRegistration Unused(&Stack
> > "No instrumented calls were accounted in the input file.",
> > make_error_code(errc::result_out_of_range));
> > }
> > +
> > + // Report the stacks in a long form mode for another tool to analyze.
> > + if (DumpAllStacks) {
> > + if (AggregateThreads) {
> > + switch (RequestedAggregation) {
> > + case AggregationType::TOTAL_TIME:
> > + ST.printAllAggregatingThreads<AggregationType::TOTAL_TIME>(
> > + outs(), FuncIdHelper, StacksOutputFormat);
> > + break;
> > + case AggregationType::INVOCATION_COUNT:
> > + ST.printAllAggregatingThreads<AggregationType::INVOCATION_
> COUNT>(
> > + outs(), FuncIdHelper, StacksOutputFormat);
> > + break;
> > + default:
> > + return make_error<StringError>(
> > + "Illegal value for aggregation-type.",
> > + make_error_code(errc::result_out_of_range));
> > + }
> > + } else {
> > + switch (RequestedAggregation) {
> > + case AggregationType::TOTAL_TIME:
> > + ST.printAllPerThread<AggregationType::TOTAL_TIME>(outs(),
> FuncIdHelper,
> > +
> StacksOutputFormat);
> > + break;
> > + case AggregationType::INVOCATION_COUNT:
> > + ST.printAllPerThread<AggregationType::INVOCATION_COUNT>(
> > + outs(), FuncIdHelper, StacksOutputFormat);
> > + break;
> > + default:
> > + return make_error<StringError>(
> > + "Illegal value for aggregation-type.",
> > + make_error_code(errc::result_out_of_range));
> > + }
> > + }
> > + return Error::success();
> > + }
> > +
> > + // We're only outputting top stacks.
> > if (AggregateThreads) {
> > ST.printAggregatingThreads(outs(), FuncIdHelper);
> > } else if (SeparateThreadStacks) {
> >
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20171012/0bf5c3fc/attachment.html>
More information about the llvm-commits
mailing list