[llvm] r338825 - [XRay][llvm] Load XRay Profiles
Galina Kistanova via llvm-commits
llvm-commits at lists.llvm.org
Fri Aug 3 19:58:31 PDT 2018
I have reverted this by r338948.
Thanks
Galina
On Fri, Aug 3, 2018 at 11:21 AM, Galina Kistanova <gkistanova at gmail.com>
wrote:
> Hello Dean,
>
> This commit broke couple of our builders:
>
> http://lab.llvm.org:8011/builders/llvm-clang-x86_64-
> expensive-checks-win/builds/11464
> http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_
> 64-scei-ps4-windows10pro-fast
> . . .
> FAILED: lib/XRay/CMakeFiles/LLVMXRay.dir/Profile.cpp.obj
> C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe /nologo /TP
> -DEXPENSIVE_CHECKS -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE
> -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE
> -D_CRT_SECURE_NO_WARNINGS -D_GLIBCXX_DEBUG -D_HAS_EXCEPTIONS=0
> -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE
> -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS
> -Ilib\XRay -IC:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\lib\XRay
> -Iinclude -IC:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\include
> /DWIN32 /D_WINDOWS /Zc:inline /Zc:strictStrings /Oi /Zc:rvalueCast /W4
> -wd4141 -wd4146 -wd4180 -wd4244 -wd4258 -wd4267 -wd4291 -wd4345 -wd4351
> -wd4355 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503 -wd4624 -wd4722 -wd4800
> -wd4100 -wd4127 -wd4512 -wd4505 -wd4610 -wd4510 -wd4702 -wd4245 -wd4706
> -wd4310 -wd4701 -wd4703 -wd4389 -wd4611 -wd4805 -wd4204 -wd4577 -wd4091
> -wd4592 -wd4319 -wd4324 -w14062 -we4238 /MDd /Zi /Ob0 /Od /RTC1 /EHs-c-
> /GR- /showIncludes /Folib\XRay\CMakeFiles\LLVMXRay.dir\Profile.cpp.obj
> /Fdlib\XRay\CMakeFiles\LLVMXRay.dir\LLVMXRay.pdb /FS -c
> C:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\
> llvm\lib\XRay\Profile.cpp
>
> Please have a look?
>
> Thanks
>
> Galina
>
> On Fri, Aug 3, 2018 at 12:18 AM, Dean Michael Berris via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> Author: dberris
>> Date: Fri Aug 3 00:18:39 2018
>> New Revision: 338825
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=338825&view=rev
>> Log:
>> [XRay][llvm] Load XRay Profiles
>>
>> Summary:
>> This change implements the profile loading functionality in LLVM to
>> support XRay's profiling mode in compiler-rt.
>>
>> We introduce a type named `llvm::xray::Profile` which allows building a
>> profile representation. We can load an XRay profile from a file to build
>> Profile instances, or do it manually through the Profile type's API.
>>
>> The intent is to get the `llvm-xray` tool to generate `Profile`
>> instances and use that as the common abstraction through which all
>> conversion and analysis can be done. In the future we can generate
>> `Profile` instances from `Trace` instances as well, through conversion
>> functions.
>>
>> Some of the key operations supported by the `Profile` API are:
>>
>> - Path interning (`Profile::internPath(...)`) which returns a unique path
>> identifier.
>>
>> - Block appending (`Profile::addBlock(...)`) to add thread-associated
>> profile information.
>>
>> - Path ID to Path lookup (`Profile::expandPath(...)`) to look up a
>> PathID and return the original interned path.
>>
>> - Block iteration.
>>
>> A 'Path' in this context represents the function call stack in
>> leaf-to-root order. This is represented as a path in an internally
>> managed prefix tree in the `Profile` instance. Having a handle (PathID)
>> to identify the unique Paths we encounter for a particular Profile
>> allows us to reduce the amount of memory required to associate profile
>> data to a particular Path.
>>
>> This is the first of a series of patches to migrate the `llvm-stacks`
>> tool towards using a single profile representation.
>>
>> Depends on D48653.
>>
>> Reviewers: kpw, eizan
>>
>> Reviewed By: kpw
>>
>> Subscribers: mgorny, llvm-commits, hiraditya
>>
>> Differential Revision: https://reviews.llvm.org/D48370
>>
>> Added:
>> llvm/trunk/include/llvm/XRay/Profile.h
>> llvm/trunk/lib/XRay/Profile.cpp
>> llvm/trunk/unittests/XRay/ProfileTest.cpp
>> Modified:
>> llvm/trunk/lib/XRay/CMakeLists.txt
>> llvm/trunk/unittests/XRay/CMakeLists.txt
>>
>> Added: llvm/trunk/include/llvm/XRay/Profile.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/
>> XRay/Profile.h?rev=338825&view=auto
>> ============================================================
>> ==================
>> --- llvm/trunk/include/llvm/XRay/Profile.h (added)
>> +++ llvm/trunk/include/llvm/XRay/Profile.h Fri Aug 3 00:18:39 2018
>> @@ -0,0 +1,130 @@
>> +//===- Profile.h - XRay Profile Abstraction
>> -------------------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===------------------------------------------------------
>> ----------------===//
>> +//
>> +// Defines the XRay Profile class representing the latency profile
>> generated by
>> +// XRay's profiling mode.
>> +//
>> +//===------------------------------------------------------
>> ----------------===//
>> +#ifndef LLVM_XRAY_PROFILE_H
>> +#define LLVM_XRAY_PROFILE_H
>> +
>> +#include "llvm/ADT/DenseMap.h"
>> +#include "llvm/ADT/SmallVector.h"
>> +#include "llvm/ADT/StringRef.h"
>> +#include "llvm/Support/Error.h"
>> +#include <list>
>> +#include <utility>
>> +#include <vector>
>> +
>> +namespace llvm {
>> +namespace xray {
>> +
>> +class Profile;
>> +
>> +// We forward declare the Trace type for turning a Trace into a Profile.
>> +class Trace;
>> +
>> +/// This function will attempt to load an XRay Profiling Mode profile
>> from the
>> +/// provided |Filename|.
>> +///
>> +/// For any errors encountered in the loading of the profile data from
>> +/// |Filename|, this function will return an Error condition
>> appropriately.
>> +Expected<Profile> loadProfile(StringRef Filename);
>> +
>> +/// This algorithm will merge two Profile instances into a single Profile
>> +/// instance, aggregating blocks by Thread ID.
>> +Profile mergeProfilesByThread(const Profile &L, const Profile &R);
>> +
>> +/// This algorithm will merge two Profile instances into a single Profile
>> +/// instance, aggregating blocks by function call stack.
>> +Profile mergeProfilesByStack(const Profile &L, const Profile &R);
>> +
>> +/// This function takes a Trace and creates a Profile instance from it.
>> +Expected<Profile> profileFromTrace(const Trace &T);
>> +
>> +/// Profile instances are thread-compatible.
>> +class Profile {
>> +public:
>> + using ThreadID = uint64_t;
>> + using PathID = unsigned;
>> + using FuncID = int32_t;
>> +
>> + struct Data {
>> + uint64_t CallCount;
>> + uint64_t CumulativeLocalTime;
>> + };
>> +
>> + struct Block {
>> + ThreadID Thread;
>> + std::vector<std::pair<PathID, Data>> PathData;
>> + };
>> +
>> + /// Provides a sequence of function IDs from a previously interned
>> PathID.
>> + ///
>> + /// Returns an error if |P| had not been interned before into the
>> Profile.
>> + ///
>> + Expected<std::vector<FuncID>> expandPath(PathID P) const;
>> +
>> + /// The stack represented in |P| must be in stack order (leaf to
>> root). This
>> + /// will always return the same PathID for |P| that has the same
>> sequence.
>> + PathID internPath(ArrayRef<FuncID> P);
>> +
>> + /// Appends a fully-formed Block instance into the Profile.
>> + ///
>> + /// Returns an error condition in the following cases:
>> + ///
>> + /// - The PathData component of the Block is empty
>> + ///
>> + Error addBlock(Block &&B);
>> +
>> + Profile() = default;
>> + ~Profile() = default;
>> + Profile(Profile &&) noexcept = default;
>> + Profile &operator=(Profile &&) noexcept = default;
>> +
>> + // Disable copy construction and assignment.
>> + Profile(const Profile &) = delete;
>> + Profile &operator=(const Profile &) = delete;
>> +
>> +private:
>> + using BlockList = std::list<Block>;
>> +
>> + struct TrieNode {
>> + FuncID Func = 0;
>> + std::vector<TrieNode *> Callees{};
>> + TrieNode *Caller = nullptr;
>> + PathID ID = 0;
>> + };
>> +
>> + // List of blocks associated with a Profile.
>> + BlockList Blocks;
>> +
>> + // List of TrieNode elements we've seen.
>> + std::list<TrieNode> NodeStorage;
>> +
>> + // List of call stack roots.
>> + SmallVector<TrieNode *, 4> Roots;
>> +
>> + // Reverse mapping between a PathID to a TrieNode*.
>> + DenseMap<PathID, TrieNode *> PathIDMap;
>> +
>> + // Used to increment
>> + PathID NextID = 1;
>> +
>> +public:
>> + using const_iterator = BlockList::const_iterator;
>> + const_iterator begin() const { return Blocks.begin(); }
>> + const_iterator end() const { return Blocks.end(); }
>> + bool empty() const { return Blocks.empty(); }
>> +};
>> +
>> +} // namespace xray
>> +} // namespace llvm
>> +
>> +#endif
>>
>> Modified: llvm/trunk/lib/XRay/CMakeLists.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/XRay/CMak
>> eLists.txt?rev=338825&r1=338824&r2=338825&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/lib/XRay/CMakeLists.txt (original)
>> +++ llvm/trunk/lib/XRay/CMakeLists.txt Fri Aug 3 00:18:39 2018
>> @@ -1,5 +1,6 @@
>> add_llvm_library(LLVMXRay
>> InstrumentationMap.cpp
>> + Profile.cpp
>> Trace.cpp
>>
>> ADDITIONAL_HEADER_DIRS
>>
>> Added: llvm/trunk/lib/XRay/Profile.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/XRay/Prof
>> ile.cpp?rev=338825&view=auto
>> ============================================================
>> ==================
>> --- llvm/trunk/lib/XRay/Profile.cpp (added)
>> +++ llvm/trunk/lib/XRay/Profile.cpp Fri Aug 3 00:18:39 2018
>> @@ -0,0 +1,379 @@
>> +//===- Profile.cpp - XRay Profile Abstraction
>> -----------------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===------------------------------------------------------
>> ----------------===//
>> +//
>> +// Defines the XRay Profile class representing the latency profile
>> generated by
>> +// XRay's profiling mode.
>> +//
>> +//===------------------------------------------------------
>> ----------------===//
>> +#include "llvm/XRay/Profile.h"
>> +
>> +#include "llvm/Support/DataExtractor.h"
>> +#include "llvm/Support/Error.h"
>> +#include "llvm/Support/FileSystem.h"
>> +#include "llvm/XRay/Trace.h"
>> +#include <deque>
>> +#include <memory>
>> +
>> +namespace llvm {
>> +namespace xray {
>> +
>> +namespace {
>> +
>> +struct BlockHeader {
>> + uint32_t Size;
>> + uint32_t Number;
>> + uint64_t Thread;
>> +};
>> +
>> +static Expected<BlockHeader> readBlockHeader(DataExtractor &Extractor,
>> + uint32_t &Offset) {
>> + BlockHeader H;
>> + uint32_t CurrentOffset = Offset;
>> + H.Size = Extractor.getU32(&Offset);
>> + if (Offset == CurrentOffset)
>> + return make_error<StringError>(
>> + Twine("Error parsing block header size at offset '") +
>> + Twine(CurrentOffset) + "'",
>> + std::make_error_code(std::errc::invalid_argument));
>> + CurrentOffset = Offset;
>> + H.Number = Extractor.getU32(&Offset);
>> + if (Offset == CurrentOffset)
>> + return make_error<StringError>(
>> + Twine("Error parsing block header number at offset '") +
>> + Twine(CurrentOffset) + "'",
>> + std::make_error_code(std::errc::invalid_argument));
>> + CurrentOffset = Offset;
>> + H.Thread = Extractor.getU64(&Offset);
>> + if (Offset == CurrentOffset)
>> + return make_error<StringError>(
>> + Twine("Error parsing block header thread id at offset '") +
>> + Twine(CurrentOffset) + "'",
>> + std::make_error_code(std::errc::invalid_argument));
>> + return H;
>> +}
>> +
>> +static Expected<std::vector<Profile::FuncID>> readPath(DataExtractor
>> &Extractor,
>> + uint32_t &Offset)
>> {
>> + // We're reading a sequence of int32_t's until we find a 0.
>> + std::vector<Profile::FuncID> Path;
>> + auto CurrentOffset = Offset;
>> + int32_t FuncId;
>> + do {
>> + FuncId = Extractor.getSigned(&Offset, 4);
>> + if (CurrentOffset == Offset)
>> + return make_error<StringError>(
>> + Twine("Error parsing path at offset '") + Twine(CurrentOffset)
>> + "'",
>> + std::make_error_code(std::errc::invalid_argument));
>> + CurrentOffset = Offset;
>> + Path.push_back(FuncId);
>> + } while (FuncId != 0);
>> + return std::move(Path);
>> +}
>> +
>> +static Expected<Profile::Data> readData(DataExtractor &Extractor,
>> + uint32_t &Offset) {
>> + // We expect a certain number of elements for Data:
>> + // - A 64-bit CallCount
>> + // - A 64-bit CumulativeLocalTime counter
>> + Profile::Data D;
>> + auto CurrentOffset = Offset;
>> + D.CallCount = Extractor.getU64(&Offset);
>> + if (CurrentOffset == Offset)
>> + return make_error<StringError>(
>> + Twine("Error parsing call counts at offset '") +
>> Twine(CurrentOffset) +
>> + "'",
>> + std::make_error_code(std::errc::invalid_argument));
>> + CurrentOffset = Offset;
>> + D.CumulativeLocalTime = Extractor.getU64(&Offset);
>> + if (CurrentOffset == Offset)
>> + return make_error<StringError>(
>> + Twine("Error parsing cumulative local time at offset '") +
>> + Twine(CurrentOffset) + "'",
>> + std::make_error_code(std::errc::invalid_argument));
>> + return D;
>> +}
>> +
>> +} // namespace
>> +
>> +Error Profile::addBlock(Block &&B) {
>> + if (B.PathData.empty())
>> + return make_error<StringError>(
>> + "Block may not have empty path data.",
>> + std::make_error_code(std::errc::invalid_argument));
>> +
>> + Blocks.emplace_back(std::move(B));
>> + return Error::success();
>> +}
>> +
>> +Expected<std::vector<Profile::FuncID>> Profile::expandPath(PathID P)
>> const {
>> + auto It = PathIDMap.find(P);
>> + if (It == PathIDMap.end())
>> + return make_error<StringError>(
>> + Twine("PathID not found: ") + Twine(P),
>> + std::make_error_code(std::errc::invalid_argument));
>> + std::vector<Profile::FuncID> Path;
>> + for (auto Node = It->second; Node; Node = Node->Caller)
>> + Path.push_back(Node->Func);
>> + return std::move(Path);
>> +}
>> +
>> +Profile::PathID Profile::internPath(ArrayRef<FuncID> P) {
>> + if (P.empty())
>> + return 0;
>> +
>> + auto RootToLeafPath = reverse(P);
>> +
>> + // Find the root.
>> + auto It = RootToLeafPath.begin();
>> + auto PathRoot = *It++;
>> + auto RootIt =
>> + find_if(Roots, [PathRoot](TrieNode *N) { return N->Func ==
>> PathRoot; });
>> +
>> + // If we've not seen this root before, remember it.
>> + TrieNode *Node = nullptr;
>> + if (RootIt == Roots.end()) {
>> + NodeStorage.emplace_back();
>> + Node = &NodeStorage.back();
>> + Node->Func = PathRoot;
>> + Roots.push_back(Node);
>> + } else {
>> + Node = *RootIt;
>> + }
>> +
>> + // Now traverse the path, re-creating if necessary.
>> + while (It != RootToLeafPath.end()) {
>> + auto NodeFuncID = *It++;
>> + auto CalleeIt = find_if(Node->Callees, [NodeFuncID](TrieNode *N) {
>> + return N->Func == NodeFuncID;
>> + });
>> + if (CalleeIt == Node->Callees.end()) {
>> + NodeStorage.emplace_back();
>> + auto NewNode = &NodeStorage.back();
>> + NewNode->Func = NodeFuncID;
>> + NewNode->Caller = Node;
>> + Node->Callees.push_back(NewNode);
>> + Node = NewNode;
>> + } else {
>> + Node = *CalleeIt;
>> + }
>> + }
>> +
>> + // At this point, Node *must* be pointing at the leaf.
>> + assert(Node->Func == P.front());
>> + if (Node->ID == 0) {
>> + Node->ID = NextID++;
>> + PathIDMap.insert({Node->ID, Node});
>> + }
>> + return Node->ID;
>> +}
>> +
>> +Profile mergeProfilesByThread(const Profile &L, const Profile &R) {
>> + Profile Merged;
>> + using PathDataMap = DenseMap<Profile::PathID, Profile::Data>;
>> + using PathDataMapPtr = std::unique_ptr<PathDataMap>;
>> + using PathDataVector = decltype(Profile::Block::PathData);
>> + using ThreadProfileIndexMap = DenseMap<Profile::ThreadID,
>> PathDataMapPtr>;
>> + ThreadProfileIndexMap ThreadProfileIndex;
>> +
>> + for (const auto &P : {std::ref(L), std::ref(R)})
>> + for (const auto &Block : P.get()) {
>> + ThreadProfileIndexMap::iterator It;
>> + std::tie(It, std::ignore) = ThreadProfileIndex.insert(
>> + {Block.Thread, PathDataMapPtr{new PathDataMap()}});
>> + for (const auto &PathAndData : Block.PathData) {
>> + auto &PathID = PathAndData.first;
>> + auto &Data = PathAndData.second;
>> + auto NewPathID =
>> + Merged.internPath(cantFail(P.get().expandPath(PathID)));
>> + PathDataMap::iterator PathDataIt;
>> + bool Inserted;
>> + std::tie(PathDataIt, Inserted) = It->second->insert({NewPathID,
>> Data});
>> + if (!Inserted) {
>> + auto &ExistingData = PathDataIt->second;
>> + ExistingData.CallCount += Data.CallCount;
>> + ExistingData.CumulativeLocalTime += Data.CumulativeLocalTime;
>> + }
>> + }
>> + }
>> +
>> + for (const auto &IndexedThreadBlock : ThreadProfileIndex) {
>> + PathDataVector PathAndData;
>> + PathAndData.reserve(IndexedThreadBlock.second->size());
>> + copy(*IndexedThreadBlock.second, std::back_inserter(PathAndData));
>> + cantFail(
>> + Merged.addBlock({IndexedThreadBlock.first,
>> std::move(PathAndData)}));
>> + }
>> + return Merged;
>> +}
>> +
>> +Profile mergeProfilesByStack(const Profile &L, const Profile &R) {
>> + Profile Merged;
>> + using PathDataMap = DenseMap<Profile::PathID, Profile::Data>;
>> + PathDataMap PathData;
>> + using PathDataVector = decltype(Profile::Block::PathData);
>> + for (const auto &P : {std::ref(L), std::ref(R)})
>> + for (const auto &Block : P.get())
>> + for (const auto &PathAndData : Block.PathData) {
>> + auto &PathId = PathAndData.first;
>> + auto &Data = PathAndData.second;
>> + auto NewPathID =
>> + Merged.internPath(cantFail(P.get().expandPath(PathId)));
>> + PathDataMap::iterator PathDataIt;
>> + bool Inserted;
>> + std::tie(PathDataIt, Inserted) = PathData.insert({NewPathID,
>> Data});
>> + if (!Inserted) {
>> + auto &ExistingData = PathDataIt->second;
>> + ExistingData.CallCount += Data.CallCount;
>> + ExistingData.CumulativeLocalTime += Data.CumulativeLocalTime;
>> + }
>> + }
>> +
>> + // In the end there's a single Block, for thread 0.
>> + PathDataVector Block;
>> + Block.reserve(PathData.size());
>> + copy(PathData, std::back_inserter(Block));
>> + cantFail(Merged.addBlock({0, std::move(Block)}));
>> + return Merged;
>> +}
>> +
>> +Expected<Profile> loadProfile(StringRef Filename) {
>> + int Fd;
>> + if (auto EC = sys::fs::openFileForRead(Filename, Fd))
>> + return make_error<StringError>(
>> + Twine("Cannot read profile from '") + Filename + "'", EC);
>> +
>> + uint64_t FileSize;
>> + if (auto EC = sys::fs::file_size(Filename, FileSize))
>> + return make_error<StringError>(
>> + Twine("Cannot get filesize of '") + Filename + "'", EC);
>> +
>> + std::error_code EC;
>> + sys::fs::mapped_file_region MappedFile(
>> + Fd, sys::fs::mapped_file_region::mapmode::readonly, FileSize, 0,
>> EC);
>> + if (EC)
>> + return make_error<StringError>(
>> + Twine("Cannot mmap profile '") + Filename + "'", EC);
>> + StringRef Data(MappedFile.data(), MappedFile.size());
>> +
>> + Profile P;
>> + uint32_t Offset = 0;
>> + DataExtractor Extractor(Data, true, 8);
>> +
>> + // For each block we get from the file:
>> + while (Offset != MappedFile.size()) {
>> + auto HeaderOrError = readBlockHeader(Extractor, Offset);
>> + if (!HeaderOrError)
>> + return HeaderOrError.takeError();
>> +
>> + // TODO: Maybe store this header information for each block, even
>> just for
>> + // debugging?
>> + const auto &Header = HeaderOrError.get();
>> +
>> + // Read in the path data.
>> + auto PathOrError = readPath(Extractor, Offset);
>> + if (!PathOrError)
>> + return PathOrError.takeError();
>> + const auto &Path = PathOrError.get();
>> +
>> + // For each path we encounter, we should intern it to get a PathID.
>> + auto DataOrError = readData(Extractor, Offset);
>> + if (!DataOrError)
>> + return DataOrError.takeError();
>> + auto &Data = DataOrError.get();
>> +
>> + if (auto E =
>> + P.addBlock(Profile::Block{Profile::ThreadID{Header.Thread},
>> + {{P.internPath(Path),
>> std::move(Data)}}}))
>> + return std::move(E);
>> + }
>> +
>> + return P;
>> +}
>> +
>> +namespace {
>> +
>> +struct StackEntry {
>> + uint64_t Timestamp;
>> + Profile::FuncID FuncId;
>> +};
>> +
>> +} // namespace
>> +
>> +Expected<Profile> profileFromTrace(const Trace &T) {
>> + Profile P;
>> +
>> + // The implementation of the algorithm re-creates the execution of
>> + // the functions based on the trace data. To do this, we set up a
>> number of
>> + // data structures to track the execution context of every thread in
>> the
>> + // Trace.
>> + DenseMap<Profile::ThreadID, std::vector<StackEntry>> ThreadStacks;
>> + DenseMap<Profile::ThreadID, DenseMap<Profile::PathID, Profile::Data>>
>> + ThreadPathData;
>> +
>> + // We then do a pass through the Trace to account data on a
>> per-thread-basis.
>> + for (const auto &E : T) {
>> + auto &TSD = ThreadStacks[E.TId];
>> + switch (E.Type) {
>> + case RecordTypes::ENTER:
>> + case RecordTypes::ENTER_ARG:
>> +
>> + // Push entries into the function call stack.
>> + TSD.push_back({E.TSC, E.FuncId});
>> + break;
>> +
>> + case RecordTypes::EXIT:
>> + case RecordTypes::TAIL_EXIT:
>> +
>> + // Exits cause some accounting to happen, based on the state of
>> the stack.
>> + // For each function we pop off the stack, we take note of the
>> path and
>> + // record the cumulative state for this path. As we're doing this,
>> we
>> + // intern the path into the Profile.
>> + while (!TSD.empty()) {
>> + auto Top = TSD.back();
>> + auto FunctionLocalTime = AbsoluteDifference(Top.Timestamp,
>> E.TSC);
>> + SmallVector<Profile::FuncID, 16> Path;
>> + transform(reverse(TSD), std::back_inserter(Path),
>> + std::mem_fn(&StackEntry::FuncId));
>> + auto InternedPath = P.internPath(Path);
>> + auto &TPD = ThreadPathData[E.TId][InternedPath];
>> + ++TPD.CallCount;
>> + TPD.CumulativeLocalTime += FunctionLocalTime;
>> + TSD.pop_back();
>> +
>> + // If we've matched the corresponding entry event for this
>> function,
>> + // then we exit the loop.
>> + if (Top.FuncId == E.FuncId)
>> + break;
>> +
>> + // FIXME: Consider the intermediate times and the cumulative
>> tree time
>> + // as well.
>> + }
>> +
>> + break;
>> + }
>> + }
>> +
>> + // Once we've gone through the Trace, we now create one Block per
>> thread in
>> + // the Profile.
>> + for (const auto &ThreadPaths : ThreadPathData) {
>> + const auto &TID = ThreadPaths.first;
>> + const auto &PathsData = ThreadPaths.second;
>> + if (auto E = P.addBlock({
>> + TID,
>> + std::vector<std::pair<Profile::PathID, Profile::Data>>(
>> + PathsData.begin(), PathsData.end()),
>> + }))
>> + return std::move(E);
>> + }
>> +
>> + return P;
>> +}
>> +
>> +} // namespace xray
>> +} // namespace llvm
>>
>> Modified: llvm/trunk/unittests/XRay/CMakeLists.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/XRa
>> y/CMakeLists.txt?rev=338825&r1=338824&r2=338825&view=diff
>> ============================================================
>> ==================
>> --- llvm/trunk/unittests/XRay/CMakeLists.txt (original)
>> +++ llvm/trunk/unittests/XRay/CMakeLists.txt Fri Aug 3 00:18:39 2018
>> @@ -1,9 +1,11 @@
>> set(LLVM_LINK_COMPONENTS
>> Support
>> + XRay
>> )
>>
>> add_llvm_unittest(XRayTests
>> GraphTest.cpp
>> + ProfileTest.cpp
>> )
>>
>> add_dependencies(XRayTests intrinsics_gen)
>>
>> Added: llvm/trunk/unittests/XRay/ProfileTest.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/XRa
>> y/ProfileTest.cpp?rev=338825&view=auto
>> ============================================================
>> ==================
>> --- llvm/trunk/unittests/XRay/ProfileTest.cpp (added)
>> +++ llvm/trunk/unittests/XRay/ProfileTest.cpp Fri Aug 3 00:18:39 2018
>> @@ -0,0 +1,218 @@
>> +//===- ProfileTest.cpp - XRay Profile unit tests ----------------*- C++
>> -*-===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===------------------------------------------------------
>> ----------------===//
>> +#include "llvm/XRay/Profile.h"
>> +#include "gmock/gmock.h"
>> +#include "gtest/gtest.h"
>> +
>> +#include <numeric>
>> +
>> +namespace llvm {
>> +namespace xray {
>> +namespace {
>> +
>> +using ::testing::AllOf;
>> +using ::testing::ElementsAre;
>> +using ::testing::Eq;
>> +using ::testing::Field;
>> +using ::testing::Not;
>> +using ::testing::Pair;
>> +using ::testing::UnorderedElementsAre;
>> +
>> +TEST(ProfileTest, CreateProfile) { Profile P; }
>> +
>> +TEST(ProfileTest, InternPath) {
>> + Profile P;
>> + auto Path0 = P.internPath({3, 2, 1});
>> + auto Path1 = P.internPath({3, 2, 1});
>> + auto Path2 = P.internPath({2, 1});
>> + EXPECT_THAT(Path0, Eq(Path1));
>> + EXPECT_THAT(Path0, Not(Eq(Path2)));
>> +}
>> +
>> +TEST(ProfileTest, ExpandPath) {
>> + Profile P;
>> + auto PathID = P.internPath({3, 2, 1});
>> + auto PathOrError = P.expandPath(PathID);
>> + if (!PathOrError)
>> + FAIL() << "Error: " << PathOrError.takeError();
>> + EXPECT_THAT(PathOrError.get(), ElementsAre(3, 2, 1));
>> +}
>> +
>> +TEST(ProfileTest, AddBlocks) {
>> + Profile P;
>> + // Expect an error on adding empty blocks.
>> + EXPECT_TRUE(errorToBool(P.addBlock({})));
>> +
>> + // Thread blocks may not be empty.
>> + EXPECT_TRUE(errorToBool(P.addBlock({1, {}})));
>> +
>> + // Thread blocks with data must succeed.
>> + EXPECT_FALSE(errorToBool(P.addBlock(
>> + Profile::Block{Profile::ThreadID{1},
>> + {
>> + {P.internPath({2, 1}), Profile::Data{1, 1000}},
>> + {P.internPath({3, 2, 1}), Profile::Data{10,
>> 100}},
>> + }})));
>> +}
>> +
>> +TEST(ProfileTest, MergeProfilesByThread) {
>> + Profile P0, P1;
>> +
>> + // Set up the blocks for two different threads in P0.
>> + EXPECT_FALSE(errorToBool(P0.addBlock(
>> + Profile::Block{Profile::ThreadID{1},
>> + {{P0.internPath({2, 1}), Profile::Data{1, 1000}},
>> + {P0.internPath({4, 1}), Profile::Data{1,
>> 1000}}}})));
>> + EXPECT_FALSE(errorToBool(P0.addBlock(
>> + Profile::Block{Profile::ThreadID{2},
>> + {{P0.internPath({3, 1}), Profile::Data{1,
>> 1000}}}})));
>> +
>> + // Set up the blocks for two different threads in P1.
>> + EXPECT_FALSE(errorToBool(P1.addBlock(
>> + Profile::Block{Profile::ThreadID{1},
>> + {{P1.internPath({2, 1}), Profile::Data{1,
>> 1000}}}})));
>> + EXPECT_FALSE(errorToBool(P1.addBlock(
>> + Profile::Block{Profile::ThreadID{2},
>> + {{P1.internPath({3, 1}), Profile::Data{1, 1000}},
>> + {P1.internPath({4, 1}), Profile::Data{1,
>> 1000}}}})));
>> +
>> + Profile Merged = mergeProfilesByThread(P0, P1);
>> + EXPECT_THAT(
>> + Merged,
>> + UnorderedElementsAre(
>> + // We want to see two threads after the merge.
>> + AllOf(Field(&Profile::Block::Thread,
>> Eq(Profile::ThreadID{1})),
>> + Field(&Profile::Block::PathData,
>> + UnorderedElementsAre(
>> + Pair(Merged.internPath({2, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(2u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(2000u)))),
>> + Pair(Merged.internPath({4, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(1u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(1000u))))))),
>> + AllOf(Field(&Profile::Block::Thread,
>> Eq(Profile::ThreadID{2})),
>> + Field(&Profile::Block::PathData,
>> + UnorderedElementsAre(
>> + Pair(Merged.internPath({3, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(2u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(2000u)))),
>> + Pair(Merged.internPath({4, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(1u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(1000u)))))))));
>> +}
>> +
>> +TEST(ProfileTest, MergeProfilesByStack) {
>> + Profile P0, P1;
>> + EXPECT_FALSE(errorToBool(P0.addBlock(
>> + Profile::Block{Profile::ThreadID{1},
>> + {{P0.internPath({2, 1}), Profile::Data{1,
>> 1000}}}})));
>> + EXPECT_FALSE(errorToBool(P1.addBlock(
>> + Profile::Block{Profile::ThreadID{2},
>> + {{P1.internPath({2, 1}), Profile::Data{1,
>> 1000}}}})));
>> +
>> + Profile Merged = mergeProfilesByStack(P0, P1);
>> + EXPECT_THAT(Merged,
>> + ElementsAre(AllOf(
>> + // We expect that we lose the ThreadID dimension in
>> this
>> + // algorithm.
>> + Field(&Profile::Block::Thread,
>> Eq(Profile::ThreadID{0})),
>> + Field(&Profile::Block::PathData,
>> + ElementsAre(Pair(
>> + Merged.internPath({2, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(2u)),
>> + Field(&Profile::Data::Cumulati
>> veLocalTime,
>> + Eq(2000u)))))))));
>> +}
>> +
>> +TEST(ProfileTest, MergeProfilesByStackAccumulate) {
>> + std::vector<Profile> Profiles(3);
>> + EXPECT_FALSE(errorToBool(Profiles[0].addBlock(Profile::Block{
>> + Profile::ThreadID{1},
>> + {{Profiles[0].internPath({2, 1}), Profile::Data{1, 1000}}}})));
>> + EXPECT_FALSE(errorToBool(Profiles[1].addBlock(Profile::Block{
>> + Profile::ThreadID{2},
>> + {{Profiles[1].internPath({2, 1}), Profile::Data{1, 1000}}}})));
>> + EXPECT_FALSE(errorToBool(Profiles[2].addBlock(Profile::Block{
>> + Profile::ThreadID{3},
>> + {{Profiles[2].internPath({2, 1}), Profile::Data{1, 1000}}}})));
>> + Profile Merged = std::accumulate(Profiles.begin(), Profiles.end(),
>> Profile(),
>> + mergeProfilesByStack);
>> + EXPECT_THAT(Merged,
>> + ElementsAre(AllOf(
>> + // We expect that we lose the ThreadID dimension in
>> this
>> + // algorithm.
>> + Field(&Profile::Block::Thread,
>> Eq(Profile::ThreadID{0})),
>> + Field(&Profile::Block::PathData,
>> + ElementsAre(Pair(
>> + Merged.internPath({2, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(3u)),
>> + Field(&Profile::Data::Cumulati
>> veLocalTime,
>> + Eq(3000u)))))))));
>> +}
>> +
>> +TEST(ProfileTest, MergeProfilesByThreadAccumulate) {
>> + std::vector<Profile> Profiles(2);
>> +
>> + // Set up the blocks for two different threads in Profiles[0].
>> + EXPECT_FALSE(errorToBool(Profiles[0].addBlock(Profile::Block{
>> + Profile::ThreadID{1},
>> + {{Profiles[0].internPath({2, 1}), Profile::Data{1, 1000}},
>> + {Profiles[0].internPath({4, 1}), Profile::Data{1, 1000}}}})));
>> + EXPECT_FALSE(errorToBool(Profiles[0].addBlock(Profile::Block{
>> + Profile::ThreadID{2},
>> + {{Profiles[0].internPath({3, 1}), Profile::Data{1, 1000}}}})));
>> +
>> + // Set up the blocks for two different threads in Profiles[1].
>> + EXPECT_FALSE(errorToBool(Profiles[1].addBlock(Profile::Block{
>> + Profile::ThreadID{1},
>> + {{Profiles[1].internPath({2, 1}), Profile::Data{1, 1000}}}})));
>> + EXPECT_FALSE(errorToBool(Profiles[1].addBlock(Profile::Block{
>> + Profile::ThreadID{2},
>> + {{Profiles[1].internPath({3, 1}), Profile::Data{1, 1000}},
>> + {Profiles[1].internPath({4, 1}), Profile::Data{1, 1000}}}})));
>> +
>> + Profile Merged = std::accumulate(Profiles.begin(), Profiles.end(),
>> Profile(),
>> + mergeProfilesByThread);
>> + EXPECT_THAT(
>> + Merged,
>> + UnorderedElementsAre(
>> + // We want to see two threads after the merge.
>> + AllOf(Field(&Profile::Block::Thread,
>> Eq(Profile::ThreadID{1})),
>> + Field(&Profile::Block::PathData,
>> + UnorderedElementsAre(
>> + Pair(Merged.internPath({2, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(2u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(2000u)))),
>> + Pair(Merged.internPath({4, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(1u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(1000u))))))),
>> + AllOf(Field(&Profile::Block::Thread,
>> Eq(Profile::ThreadID{2})),
>> + Field(&Profile::Block::PathData,
>> + UnorderedElementsAre(
>> + Pair(Merged.internPath({3, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(2u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(2000u)))),
>> + Pair(Merged.internPath({4, 1}),
>> + AllOf(Field(&Profile::Data::CallCount,
>> Eq(1u)),
>> + Field(&Profile::Data::Cumulat
>> iveLocalTime,
>> + Eq(1000u)))))))));
>> +}
>> +// FIXME: Add a test creating a Trace and generating a Profile
>> +// FIXME: Add tests for ranking/sorting profile blocks by dimension
>> +
>> +} // namespace
>> +} // namespace xray
>> +} // namespace llvm
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180803/8c213863/attachment-0001.html>
More information about the llvm-commits
mailing list