<div dir="ltr">Hello Dean,<br><br>This commit broke couple of our builders:<br><br><a href="http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/11464">http://lab.llvm.org:8011/builders/llvm-clang-x86_64-expensive-checks-win/builds/11464</a><br><a href="http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast">http://lab.llvm.org:8011/builders/llvm-clang-lld-x86_64-scei-ps4-windows10pro-fast</a><br>. . . <br>FAILED: lib/XRay/CMakeFiles/LLVMXRay.dir/Profile.cpp.obj <br>C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe /nologo /TP -DEXPENSIVE_CHECKS -DGTEST_HAS_RTTI=0 -DUNICODE -D_CRT_NONSTDC_NO_DEPRECATE -D_CRT_NONSTDC_NO_WARNINGS -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_GLIBCXX_DEBUG -D_HAS_EXCEPTIONS=0 -D_SCL_SECURE_NO_DEPRECATE -D_SCL_SECURE_NO_WARNINGS -D_UNICODE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Ilib\XRay -IC:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\lib\XRay -Iinclude -IC:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\include /DWIN32 /D_WINDOWS /Zc:inline /Zc:strictStrings /Oi /Zc:rvalueCast /W4 -wd4141 -wd4146 -wd4180 -wd4244 -wd4258 -wd4267 -wd4291 -wd4345 -wd4351 -wd4355 -wd4456 -wd4457 -wd4458 -wd4459 -wd4503 -wd4624 -wd4722 -wd4800 -wd4100 -wd4127 -wd4512 -wd4505 -wd4610 -wd4510 -wd4702 -wd4245 -wd4706 -wd4310 -wd4701 -wd4703 -wd4389 -wd4611 -wd4805 -wd4204 -wd4577 -wd4091 -wd4592 -wd4319 -wd4324 -w14062 -we4238 /MDd /Zi /Ob0 /Od /RTC1 /EHs-c- /GR- /showIncludes /Folib\XRay\CMakeFiles\LLVMXRay.dir\Profile.cpp.obj /Fdlib\XRay\CMakeFiles\LLVMXRay.dir\LLVMXRay.pdb /FS -c C:\ps4-buildslave2\llvm-clang-x86_64-expensive-checks-win\llvm\lib\XRay\Profile.cpp<br><br>Please have a look?<br><br>Thanks<br><br>Galina<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Aug 3, 2018 at 12:18 AM, Dean Michael Berris via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Author: dberris<br>
Date: Fri Aug 3 00:18:39 2018<br>
New Revision: 338825<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=338825&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project?rev=338825&view=rev</a><br>
Log:<br>
[XRay][llvm] Load XRay Profiles<br>
<br>
Summary:<br>
This change implements the profile loading functionality in LLVM to<br>
support XRay's profiling mode in compiler-rt.<br>
<br>
We introduce a type named `llvm::xray::Profile` which allows building a<br>
profile representation. We can load an XRay profile from a file to build<br>
Profile instances, or do it manually through the Profile type's API.<br>
<br>
The intent is to get the `llvm-xray` tool to generate `Profile`<br>
instances and use that as the common abstraction through which all<br>
conversion and analysis can be done. In the future we can generate<br>
`Profile` instances from `Trace` instances as well, through conversion<br>
functions.<br>
<br>
Some of the key operations supported by the `Profile` API are:<br>
<br>
- Path interning (`Profile::internPath(...)`) which returns a unique path<br>
identifier.<br>
<br>
- Block appending (`Profile::addBlock(...)`) to add thread-associated<br>
profile information.<br>
<br>
- Path ID to Path lookup (`Profile::expandPath(...)`) to look up a<br>
PathID and return the original interned path.<br>
<br>
- Block iteration.<br>
<br>
A 'Path' in this context represents the function call stack in<br>
leaf-to-root order. This is represented as a path in an internally<br>
managed prefix tree in the `Profile` instance. Having a handle (PathID)<br>
to identify the unique Paths we encounter for a particular Profile<br>
allows us to reduce the amount of memory required to associate profile<br>
data to a particular Path.<br>
<br>
This is the first of a series of patches to migrate the `llvm-stacks`<br>
tool towards using a single profile representation.<br>
<br>
Depends on D48653.<br>
<br>
Reviewers: kpw, eizan<br>
<br>
Reviewed By: kpw<br>
<br>
Subscribers: mgorny, llvm-commits, hiraditya<br>
<br>
Differential Revision: <a href="https://reviews.llvm.org/D48370" rel="noreferrer" target="_blank">https://reviews.llvm.org/<wbr>D48370</a><br>
<br>
Added:<br>
llvm/trunk/include/llvm/XRay/<wbr>Profile.h<br>
llvm/trunk/lib/XRay/Profile.<wbr>cpp<br>
llvm/trunk/unittests/XRay/<wbr>ProfileTest.cpp<br>
Modified:<br>
llvm/trunk/lib/XRay/<wbr>CMakeLists.txt<br>
llvm/trunk/unittests/XRay/<wbr>CMakeLists.txt<br>
<br>
Added: llvm/trunk/include/llvm/XRay/<wbr>Profile.h<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/XRay/Profile.h?rev=338825&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/include/<wbr>llvm/XRay/Profile.h?rev=<wbr>338825&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/include/llvm/XRay/<wbr>Profile.h (added)<br>
+++ llvm/trunk/include/llvm/XRay/<wbr>Profile.h Fri Aug 3 00:18:39 2018<br>
@@ -0,0 +1,130 @@<br>
+//===- Profile.h - XRay Profile Abstraction ------------------------------<wbr>-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+//<br>
+// Defines the XRay Profile class representing the latency profile generated by<br>
+// XRay's profiling mode.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+#ifndef LLVM_XRAY_PROFILE_H<br>
+#define LLVM_XRAY_PROFILE_H<br>
+<br>
+#include "llvm/ADT/DenseMap.h"<br>
+#include "llvm/ADT/SmallVector.h"<br>
+#include "llvm/ADT/StringRef.h"<br>
+#include "llvm/Support/Error.h"<br>
+#include <list><br>
+#include <utility><br>
+#include <vector><br>
+<br>
+namespace llvm {<br>
+namespace xray {<br>
+<br>
+class Profile;<br>
+<br>
+// We forward declare the Trace type for turning a Trace into a Profile.<br>
+class Trace;<br>
+<br>
+/// This function will attempt to load an XRay Profiling Mode profile from the<br>
+/// provided |Filename|.<br>
+///<br>
+/// For any errors encountered in the loading of the profile data from<br>
+/// |Filename|, this function will return an Error condition appropriately.<br>
+Expected<Profile> loadProfile(StringRef Filename);<br>
+<br>
+/// This algorithm will merge two Profile instances into a single Profile<br>
+/// instance, aggregating blocks by Thread ID.<br>
+Profile mergeProfilesByThread(const Profile &L, const Profile &R);<br>
+<br>
+/// This algorithm will merge two Profile instances into a single Profile<br>
+/// instance, aggregating blocks by function call stack.<br>
+Profile mergeProfilesByStack(const Profile &L, const Profile &R);<br>
+<br>
+/// This function takes a Trace and creates a Profile instance from it.<br>
+Expected<Profile> profileFromTrace(const Trace &T);<br>
+<br>
+/// Profile instances are thread-compatible.<br>
+class Profile {<br>
+public:<br>
+ using ThreadID = uint64_t;<br>
+ using PathID = unsigned;<br>
+ using FuncID = int32_t;<br>
+<br>
+ struct Data {<br>
+ uint64_t CallCount;<br>
+ uint64_t CumulativeLocalTime;<br>
+ };<br>
+<br>
+ struct Block {<br>
+ ThreadID Thread;<br>
+ std::vector<std::pair<PathID, Data>> PathData;<br>
+ };<br>
+<br>
+ /// Provides a sequence of function IDs from a previously interned PathID.<br>
+ ///<br>
+ /// Returns an error if |P| had not been interned before into the Profile.<br>
+ ///<br>
+ Expected<std::vector<FuncID>> expandPath(PathID P) const;<br>
+<br>
+ /// The stack represented in |P| must be in stack order (leaf to root). This<br>
+ /// will always return the same PathID for |P| that has the same sequence.<br>
+ PathID internPath(ArrayRef<FuncID> P);<br>
+<br>
+ /// Appends a fully-formed Block instance into the Profile.<br>
+ ///<br>
+ /// Returns an error condition in the following cases:<br>
+ ///<br>
+ /// - The PathData component of the Block is empty<br>
+ ///<br>
+ Error addBlock(Block &&B);<br>
+<br>
+ Profile() = default;<br>
+ ~Profile() = default;<br>
+ Profile(Profile &&) noexcept = default;<br>
+ Profile &operator=(Profile &&) noexcept = default;<br>
+<br>
+ // Disable copy construction and assignment.<br>
+ Profile(const Profile &) = delete;<br>
+ Profile &operator=(const Profile &) = delete;<br>
+<br>
+private:<br>
+ using BlockList = std::list<Block>;<br>
+<br>
+ struct TrieNode {<br>
+ FuncID Func = 0;<br>
+ std::vector<TrieNode *> Callees{};<br>
+ TrieNode *Caller = nullptr;<br>
+ PathID ID = 0;<br>
+ };<br>
+<br>
+ // List of blocks associated with a Profile.<br>
+ BlockList Blocks;<br>
+<br>
+ // List of TrieNode elements we've seen.<br>
+ std::list<TrieNode> NodeStorage;<br>
+<br>
+ // List of call stack roots.<br>
+ SmallVector<TrieNode *, 4> Roots;<br>
+<br>
+ // Reverse mapping between a PathID to a TrieNode*.<br>
+ DenseMap<PathID, TrieNode *> PathIDMap;<br>
+<br>
+ // Used to increment<br>
+ PathID NextID = 1;<br>
+<br>
+public:<br>
+ using const_iterator = BlockList::const_iterator;<br>
+ const_iterator begin() const { return Blocks.begin(); }<br>
+ const_iterator end() const { return Blocks.end(); }<br>
+ bool empty() const { return Blocks.empty(); }<br>
+};<br>
+<br>
+} // namespace xray<br>
+} // namespace llvm<br>
+<br>
+#endif<br>
<br>
Modified: llvm/trunk/lib/XRay/<wbr>CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/XRay/CMakeLists.txt?rev=338825&r1=338824&r2=338825&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/XRay/<wbr>CMakeLists.txt?rev=338825&r1=<wbr>338824&r2=338825&view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/lib/XRay/<wbr>CMakeLists.txt (original)<br>
+++ llvm/trunk/lib/XRay/<wbr>CMakeLists.txt Fri Aug 3 00:18:39 2018<br>
@@ -1,5 +1,6 @@<br>
add_llvm_library(LLVMXRay<br>
InstrumentationMap.cpp<br>
+ Profile.cpp<br>
Trace.cpp<br>
<br>
ADDITIONAL_HEADER_DIRS<br>
<br>
Added: llvm/trunk/lib/XRay/Profile.<wbr>cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/XRay/Profile.cpp?rev=338825&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/lib/XRay/<wbr>Profile.cpp?rev=338825&view=<wbr>auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/lib/XRay/Profile.<wbr>cpp (added)<br>
+++ llvm/trunk/lib/XRay/Profile.<wbr>cpp Fri Aug 3 00:18:39 2018<br>
@@ -0,0 +1,379 @@<br>
+//===- Profile.cpp - XRay Profile Abstraction -----------------------------=<wbr>==//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+//<br>
+// Defines the XRay Profile class representing the latency profile generated by<br>
+// XRay's profiling mode.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+#include "llvm/XRay/Profile.h"<br>
+<br>
+#include "llvm/Support/DataExtractor.h"<br>
+#include "llvm/Support/Error.h"<br>
+#include "llvm/Support/FileSystem.h"<br>
+#include "llvm/XRay/Trace.h"<br>
+#include <deque><br>
+#include <memory><br>
+<br>
+namespace llvm {<br>
+namespace xray {<br>
+<br>
+namespace {<br>
+<br>
+struct BlockHeader {<br>
+ uint32_t Size;<br>
+ uint32_t Number;<br>
+ uint64_t Thread;<br>
+};<br>
+<br>
+static Expected<BlockHeader> readBlockHeader(DataExtractor &Extractor,<br>
+ uint32_t &Offset) {<br>
+ BlockHeader H;<br>
+ uint32_t CurrentOffset = Offset;<br>
+ H.Size = Extractor.getU32(&Offset);<br>
+ if (Offset == CurrentOffset)<br>
+ return make_error<StringError>(<br>
+ Twine("Error parsing block header size at offset '") +<br>
+ Twine(CurrentOffset) + "'",<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+ CurrentOffset = Offset;<br>
+ H.Number = Extractor.getU32(&Offset);<br>
+ if (Offset == CurrentOffset)<br>
+ return make_error<StringError>(<br>
+ Twine("Error parsing block header number at offset '") +<br>
+ Twine(CurrentOffset) + "'",<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+ CurrentOffset = Offset;<br>
+ H.Thread = Extractor.getU64(&Offset);<br>
+ if (Offset == CurrentOffset)<br>
+ return make_error<StringError>(<br>
+ Twine("Error parsing block header thread id at offset '") +<br>
+ Twine(CurrentOffset) + "'",<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+ return H;<br>
+}<br>
+<br>
+static Expected<std::vector<Profile::<wbr>FuncID>> readPath(DataExtractor &Extractor,<br>
+ uint32_t &Offset) {<br>
+ // We're reading a sequence of int32_t's until we find a 0.<br>
+ std::vector<Profile::FuncID> Path;<br>
+ auto CurrentOffset = Offset;<br>
+ int32_t FuncId;<br>
+ do {<br>
+ FuncId = Extractor.getSigned(&Offset, 4);<br>
+ if (CurrentOffset == Offset)<br>
+ return make_error<StringError>(<br>
+ Twine("Error parsing path at offset '") + Twine(CurrentOffset) + "'",<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+ CurrentOffset = Offset;<br>
+ Path.push_back(FuncId);<br>
+ } while (FuncId != 0);<br>
+ return std::move(Path);<br>
+}<br>
+<br>
+static Expected<Profile::Data> readData(DataExtractor &Extractor,<br>
+ uint32_t &Offset) {<br>
+ // We expect a certain number of elements for Data:<br>
+ // - A 64-bit CallCount<br>
+ // - A 64-bit CumulativeLocalTime counter<br>
+ Profile::Data D;<br>
+ auto CurrentOffset = Offset;<br>
+ D.CallCount = Extractor.getU64(&Offset);<br>
+ if (CurrentOffset == Offset)<br>
+ return make_error<StringError>(<br>
+ Twine("Error parsing call counts at offset '") + Twine(CurrentOffset) +<br>
+ "'",<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+ CurrentOffset = Offset;<br>
+ D.CumulativeLocalTime = Extractor.getU64(&Offset);<br>
+ if (CurrentOffset == Offset)<br>
+ return make_error<StringError>(<br>
+ Twine("Error parsing cumulative local time at offset '") +<br>
+ Twine(CurrentOffset) + "'",<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+ return D;<br>
+}<br>
+<br>
+} // namespace<br>
+<br>
+Error Profile::addBlock(Block &&B) {<br>
+ if (B.PathData.empty())<br>
+ return make_error<StringError>(<br>
+ "Block may not have empty path data.",<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+<br>
+ Blocks.emplace_back(std::move(<wbr>B));<br>
+ return Error::success();<br>
+}<br>
+<br>
+Expected<std::vector<Profile:<wbr>:FuncID>> Profile::expandPath(PathID P) const {<br>
+ auto It = PathIDMap.find(P);<br>
+ if (It == PathIDMap.end())<br>
+ return make_error<StringError>(<br>
+ Twine("PathID not found: ") + Twine(P),<br>
+ std::make_error_code(std::<wbr>errc::invalid_argument));<br>
+ std::vector<Profile::FuncID> Path;<br>
+ for (auto Node = It->second; Node; Node = Node->Caller)<br>
+ Path.push_back(Node->Func);<br>
+ return std::move(Path);<br>
+}<br>
+<br>
+Profile::PathID Profile::internPath(ArrayRef<<wbr>FuncID> P) {<br>
+ if (P.empty())<br>
+ return 0;<br>
+<br>
+ auto RootToLeafPath = reverse(P);<br>
+<br>
+ // Find the root.<br>
+ auto It = RootToLeafPath.begin();<br>
+ auto PathRoot = *It++;<br>
+ auto RootIt =<br>
+ find_if(Roots, [PathRoot](TrieNode *N) { return N->Func == PathRoot; });<br>
+<br>
+ // If we've not seen this root before, remember it.<br>
+ TrieNode *Node = nullptr;<br>
+ if (RootIt == Roots.end()) {<br>
+ NodeStorage.emplace_back();<br>
+ Node = &NodeStorage.back();<br>
+ Node->Func = PathRoot;<br>
+ Roots.push_back(Node);<br>
+ } else {<br>
+ Node = *RootIt;<br>
+ }<br>
+<br>
+ // Now traverse the path, re-creating if necessary.<br>
+ while (It != RootToLeafPath.end()) {<br>
+ auto NodeFuncID = *It++;<br>
+ auto CalleeIt = find_if(Node->Callees, [NodeFuncID](TrieNode *N) {<br>
+ return N->Func == NodeFuncID;<br>
+ });<br>
+ if (CalleeIt == Node->Callees.end()) {<br>
+ NodeStorage.emplace_back();<br>
+ auto NewNode = &NodeStorage.back();<br>
+ NewNode->Func = NodeFuncID;<br>
+ NewNode->Caller = Node;<br>
+ Node->Callees.push_back(<wbr>NewNode);<br>
+ Node = NewNode;<br>
+ } else {<br>
+ Node = *CalleeIt;<br>
+ }<br>
+ }<br>
+<br>
+ // At this point, Node *must* be pointing at the leaf.<br>
+ assert(Node->Func == P.front());<br>
+ if (Node->ID == 0) {<br>
+ Node->ID = NextID++;<br>
+ PathIDMap.insert({Node->ID, Node});<br>
+ }<br>
+ return Node->ID;<br>
+}<br>
+<br>
+Profile mergeProfilesByThread(const Profile &L, const Profile &R) {<br>
+ Profile Merged;<br>
+ using PathDataMap = DenseMap<Profile::PathID, Profile::Data>;<br>
+ using PathDataMapPtr = std::unique_ptr<PathDataMap>;<br>
+ using PathDataVector = decltype(Profile::Block::<wbr>PathData);<br>
+ using ThreadProfileIndexMap = DenseMap<Profile::ThreadID, PathDataMapPtr>;<br>
+ ThreadProfileIndexMap ThreadProfileIndex;<br>
+<br>
+ for (const auto &P : {std::ref(L), std::ref(R)})<br>
+ for (const auto &Block : P.get()) {<br>
+ ThreadProfileIndexMap::<wbr>iterator It;<br>
+ std::tie(It, std::ignore) = ThreadProfileIndex.insert(<br>
+ {Block.Thread, PathDataMapPtr{new PathDataMap()}});<br>
+ for (const auto &PathAndData : Block.PathData) {<br>
+ auto &PathID = PathAndData.first;<br>
+ auto &Data = PathAndData.second;<br>
+ auto NewPathID =<br>
+ Merged.internPath(cantFail(P.<wbr>get().expandPath(PathID)));<br>
+ PathDataMap::iterator PathDataIt;<br>
+ bool Inserted;<br>
+ std::tie(PathDataIt, Inserted) = It->second->insert({NewPathID, Data});<br>
+ if (!Inserted) {<br>
+ auto &ExistingData = PathDataIt->second;<br>
+ ExistingData.CallCount += Data.CallCount;<br>
+ ExistingData.<wbr>CumulativeLocalTime += Data.CumulativeLocalTime;<br>
+ }<br>
+ }<br>
+ }<br>
+<br>
+ for (const auto &IndexedThreadBlock : ThreadProfileIndex) {<br>
+ PathDataVector PathAndData;<br>
+ PathAndData.reserve(<wbr>IndexedThreadBlock.second-><wbr>size());<br>
+ copy(*IndexedThreadBlock.<wbr>second, std::back_inserter(<wbr>PathAndData));<br>
+ cantFail(<br>
+ Merged.addBlock({<wbr>IndexedThreadBlock.first, std::move(PathAndData)}));<br>
+ }<br>
+ return Merged;<br>
+}<br>
+<br>
+Profile mergeProfilesByStack(const Profile &L, const Profile &R) {<br>
+ Profile Merged;<br>
+ using PathDataMap = DenseMap<Profile::PathID, Profile::Data>;<br>
+ PathDataMap PathData;<br>
+ using PathDataVector = decltype(Profile::Block::<wbr>PathData);<br>
+ for (const auto &P : {std::ref(L), std::ref(R)})<br>
+ for (const auto &Block : P.get())<br>
+ for (const auto &PathAndData : Block.PathData) {<br>
+ auto &PathId = PathAndData.first;<br>
+ auto &Data = PathAndData.second;<br>
+ auto NewPathID =<br>
+ Merged.internPath(cantFail(P.<wbr>get().expandPath(PathId)));<br>
+ PathDataMap::iterator PathDataIt;<br>
+ bool Inserted;<br>
+ std::tie(PathDataIt, Inserted) = PathData.insert({NewPathID, Data});<br>
+ if (!Inserted) {<br>
+ auto &ExistingData = PathDataIt->second;<br>
+ ExistingData.CallCount += Data.CallCount;<br>
+ ExistingData.<wbr>CumulativeLocalTime += Data.CumulativeLocalTime;<br>
+ }<br>
+ }<br>
+<br>
+ // In the end there's a single Block, for thread 0.<br>
+ PathDataVector Block;<br>
+ Block.reserve(PathData.size())<wbr>;<br>
+ copy(PathData, std::back_inserter(Block));<br>
+ cantFail(Merged.addBlock({0, std::move(Block)}));<br>
+ return Merged;<br>
+}<br>
+<br>
+Expected<Profile> loadProfile(StringRef Filename) {<br>
+ int Fd;<br>
+ if (auto EC = sys::fs::openFileForRead(<wbr>Filename, Fd))<br>
+ return make_error<StringError>(<br>
+ Twine("Cannot read profile from '") + Filename + "'", EC);<br>
+<br>
+ uint64_t FileSize;<br>
+ if (auto EC = sys::fs::file_size(Filename, FileSize))<br>
+ return make_error<StringError>(<br>
+ Twine("Cannot get filesize of '") + Filename + "'", EC);<br>
+<br>
+ std::error_code EC;<br>
+ sys::fs::mapped_file_region MappedFile(<br>
+ Fd, sys::fs::mapped_file_region::<wbr>mapmode::readonly, FileSize, 0, EC);<br>
+ if (EC)<br>
+ return make_error<StringError>(<br>
+ Twine("Cannot mmap profile '") + Filename + "'", EC);<br>
+ StringRef Data(MappedFile.data(), MappedFile.size());<br>
+<br>
+ Profile P;<br>
+ uint32_t Offset = 0;<br>
+ DataExtractor Extractor(Data, true, 8);<br>
+<br>
+ // For each block we get from the file:<br>
+ while (Offset != MappedFile.size()) {<br>
+ auto HeaderOrError = readBlockHeader(Extractor, Offset);<br>
+ if (!HeaderOrError)<br>
+ return HeaderOrError.takeError();<br>
+<br>
+ // TODO: Maybe store this header information for each block, even just for<br>
+ // debugging?<br>
+ const auto &Header = HeaderOrError.get();<br>
+<br>
+ // Read in the path data.<br>
+ auto PathOrError = readPath(Extractor, Offset);<br>
+ if (!PathOrError)<br>
+ return PathOrError.takeError();<br>
+ const auto &Path = PathOrError.get();<br>
+<br>
+ // For each path we encounter, we should intern it to get a PathID.<br>
+ auto DataOrError = readData(Extractor, Offset);<br>
+ if (!DataOrError)<br>
+ return DataOrError.takeError();<br>
+ auto &Data = DataOrError.get();<br>
+<br>
+ if (auto E =<br>
+ P.addBlock(Profile::Block{<wbr>Profile::ThreadID{Header.<wbr>Thread},<br>
+ {{P.internPath(Path), std::move(Data)}}}))<br>
+ return std::move(E);<br>
+ }<br>
+<br>
+ return P;<br>
+}<br>
+<br>
+namespace {<br>
+<br>
+struct StackEntry {<br>
+ uint64_t Timestamp;<br>
+ Profile::FuncID FuncId;<br>
+};<br>
+<br>
+} // namespace<br>
+<br>
+Expected<Profile> profileFromTrace(const Trace &T) {<br>
+ Profile P;<br>
+<br>
+ // The implementation of the algorithm re-creates the execution of<br>
+ // the functions based on the trace data. To do this, we set up a number of<br>
+ // data structures to track the execution context of every thread in the<br>
+ // Trace.<br>
+ DenseMap<Profile::ThreadID, std::vector<StackEntry>> ThreadStacks;<br>
+ DenseMap<Profile::ThreadID, DenseMap<Profile::PathID, Profile::Data>><br>
+ ThreadPathData;<br>
+<br>
+ // We then do a pass through the Trace to account data on a per-thread-basis.<br>
+ for (const auto &E : T) {<br>
+ auto &TSD = ThreadStacks[E.TId];<br>
+ switch (E.Type) {<br>
+ case RecordTypes::ENTER:<br>
+ case RecordTypes::ENTER_ARG:<br>
+<br>
+ // Push entries into the function call stack.<br>
+ TSD.push_back({E.TSC, E.FuncId});<br>
+ break;<br>
+<br>
+ case RecordTypes::EXIT:<br>
+ case RecordTypes::TAIL_EXIT:<br>
+<br>
+ // Exits cause some accounting to happen, based on the state of the stack.<br>
+ // For each function we pop off the stack, we take note of the path and<br>
+ // record the cumulative state for this path. As we're doing this, we<br>
+ // intern the path into the Profile.<br>
+ while (!TSD.empty()) {<br>
+ auto Top = TSD.back();<br>
+ auto FunctionLocalTime = AbsoluteDifference(Top.<wbr>Timestamp, E.TSC);<br>
+ SmallVector<Profile::FuncID, 16> Path;<br>
+ transform(reverse(TSD), std::back_inserter(Path),<br>
+ std::mem_fn(&StackEntry::<wbr>FuncId));<br>
+ auto InternedPath = P.internPath(Path);<br>
+ auto &TPD = ThreadPathData[E.TId][<wbr>InternedPath];<br>
+ ++TPD.CallCount;<br>
+ TPD.CumulativeLocalTime += FunctionLocalTime;<br>
+ TSD.pop_back();<br>
+<br>
+ // If we've matched the corresponding entry event for this function,<br>
+ // then we exit the loop.<br>
+ if (Top.FuncId == E.FuncId)<br>
+ break;<br>
+<br>
+ // FIXME: Consider the intermediate times and the cumulative tree time<br>
+ // as well.<br>
+ }<br>
+<br>
+ break;<br>
+ }<br>
+ }<br>
+<br>
+ // Once we've gone through the Trace, we now create one Block per thread in<br>
+ // the Profile.<br>
+ for (const auto &ThreadPaths : ThreadPathData) {<br>
+ const auto &TID = ThreadPaths.first;<br>
+ const auto &PathsData = ThreadPaths.second;<br>
+ if (auto E = P.addBlock({<br>
+ TID,<br>
+ std::vector<std::pair<Profile:<wbr>:PathID, Profile::Data>>(<br>
+ PathsData.begin(), PathsData.end()),<br>
+ }))<br>
+ return std::move(E);<br>
+ }<br>
+<br>
+ return P;<br>
+}<br>
+<br>
+} // namespace xray<br>
+} // namespace llvm<br>
<br>
Modified: llvm/trunk/unittests/XRay/<wbr>CMakeLists.txt<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/XRay/CMakeLists.txt?rev=338825&r1=338824&r2=338825&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/unittests/<wbr>XRay/CMakeLists.txt?rev=<wbr>338825&r1=338824&r2=338825&<wbr>view=diff</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/unittests/XRay/<wbr>CMakeLists.txt (original)<br>
+++ llvm/trunk/unittests/XRay/<wbr>CMakeLists.txt Fri Aug 3 00:18:39 2018<br>
@@ -1,9 +1,11 @@<br>
set(LLVM_LINK_COMPONENTS<br>
Support<br>
+ XRay<br>
)<br>
<br>
add_llvm_unittest(XRayTests<br>
GraphTest.cpp<br>
+ ProfileTest.cpp<br>
)<br>
<br>
add_dependencies(XRayTests intrinsics_gen)<br>
<br>
Added: llvm/trunk/unittests/XRay/<wbr>ProfileTest.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/unittests/XRay/ProfileTest.cpp?rev=338825&view=auto" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-<wbr>project/llvm/trunk/unittests/<wbr>XRay/ProfileTest.cpp?rev=<wbr>338825&view=auto</a><br>
==============================<wbr>==============================<wbr>==================<br>
--- llvm/trunk/unittests/XRay/<wbr>ProfileTest.cpp (added)<br>
+++ llvm/trunk/unittests/XRay/<wbr>ProfileTest.cpp Fri Aug 3 00:18:39 2018<br>
@@ -0,0 +1,218 @@<br>
+//===- ProfileTest.cpp - XRay Profile unit tests ----------------*- C++ -*-===//<br>
+//<br>
+// The LLVM Compiler Infrastructure<br>
+//<br>
+// This file is distributed under the University of Illinois Open Source<br>
+// License. See LICENSE.TXT for details.<br>
+//<br>
+//===------------------------<wbr>------------------------------<wbr>----------------===//<br>
+#include "llvm/XRay/Profile.h"<br>
+#include "gmock/gmock.h"<br>
+#include "gtest/gtest.h"<br>
+<br>
+#include <numeric><br>
+<br>
+namespace llvm {<br>
+namespace xray {<br>
+namespace {<br>
+<br>
+using ::testing::AllOf;<br>
+using ::testing::ElementsAre;<br>
+using ::testing::Eq;<br>
+using ::testing::Field;<br>
+using ::testing::Not;<br>
+using ::testing::Pair;<br>
+using ::testing::<wbr>UnorderedElementsAre;<br>
+<br>
+TEST(ProfileTest, CreateProfile) { Profile P; }<br>
+<br>
+TEST(ProfileTest, InternPath) {<br>
+ Profile P;<br>
+ auto Path0 = P.internPath({3, 2, 1});<br>
+ auto Path1 = P.internPath({3, 2, 1});<br>
+ auto Path2 = P.internPath({2, 1});<br>
+ EXPECT_THAT(Path0, Eq(Path1));<br>
+ EXPECT_THAT(Path0, Not(Eq(Path2)));<br>
+}<br>
+<br>
+TEST(ProfileTest, ExpandPath) {<br>
+ Profile P;<br>
+ auto PathID = P.internPath({3, 2, 1});<br>
+ auto PathOrError = P.expandPath(PathID);<br>
+ if (!PathOrError)<br>
+ FAIL() << "Error: " << PathOrError.takeError();<br>
+ EXPECT_THAT(PathOrError.get(), ElementsAre(3, 2, 1));<br>
+}<br>
+<br>
+TEST(ProfileTest, AddBlocks) {<br>
+ Profile P;<br>
+ // Expect an error on adding empty blocks.<br>
+ EXPECT_TRUE(errorToBool(P.<wbr>addBlock({})));<br>
+<br>
+ // Thread blocks may not be empty.<br>
+ EXPECT_TRUE(errorToBool(P.<wbr>addBlock({1, {}})));<br>
+<br>
+ // Thread blocks with data must succeed.<br>
+ EXPECT_FALSE(errorToBool(P.<wbr>addBlock(<br>
+ Profile::Block{Profile::<wbr>ThreadID{1},<br>
+ {<br>
+ {P.internPath({2, 1}), Profile::Data{1, 1000}},<br>
+ {P.internPath({3, 2, 1}), Profile::Data{10, 100}},<br>
+ }})));<br>
+}<br>
+<br>
+TEST(ProfileTest, MergeProfilesByThread) {<br>
+ Profile P0, P1;<br>
+<br>
+ // Set up the blocks for two different threads in P0.<br>
+ EXPECT_FALSE(errorToBool(P0.<wbr>addBlock(<br>
+ Profile::Block{Profile::<wbr>ThreadID{1},<br>
+ {{P0.internPath({2, 1}), Profile::Data{1, 1000}},<br>
+ {P0.internPath({4, 1}), Profile::Data{1, 1000}}}})));<br>
+ EXPECT_FALSE(errorToBool(P0.<wbr>addBlock(<br>
+ Profile::Block{Profile::<wbr>ThreadID{2},<br>
+ {{P0.internPath({3, 1}), Profile::Data{1, 1000}}}})));<br>
+<br>
+ // Set up the blocks for two different threads in P1.<br>
+ EXPECT_FALSE(errorToBool(P1.<wbr>addBlock(<br>
+ Profile::Block{Profile::<wbr>ThreadID{1},<br>
+ {{P1.internPath({2, 1}), Profile::Data{1, 1000}}}})));<br>
+ EXPECT_FALSE(errorToBool(P1.<wbr>addBlock(<br>
+ Profile::Block{Profile::<wbr>ThreadID{2},<br>
+ {{P1.internPath({3, 1}), Profile::Data{1, 1000}},<br>
+ {P1.internPath({4, 1}), Profile::Data{1, 1000}}}})));<br>
+<br>
+ Profile Merged = mergeProfilesByThread(P0, P1);<br>
+ EXPECT_THAT(<br>
+ Merged,<br>
+ UnorderedElementsAre(<br>
+ // We want to see two threads after the merge.<br>
+ AllOf(Field(&Profile::Block::<wbr>Thread, Eq(Profile::ThreadID{1})),<br>
+ Field(&Profile::Block::<wbr>PathData,<br>
+ UnorderedElementsAre(<br>
+ Pair(Merged.internPath({2, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(2u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(2000u)))),<br>
+ Pair(Merged.internPath({4, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(1u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(1000u))))))),<br>
+ AllOf(Field(&Profile::Block::<wbr>Thread, Eq(Profile::ThreadID{2})),<br>
+ Field(&Profile::Block::<wbr>PathData,<br>
+ UnorderedElementsAre(<br>
+ Pair(Merged.internPath({3, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(2u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(2000u)))),<br>
+ Pair(Merged.internPath({4, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(1u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(1000u)))))))));<br>
+}<br>
+<br>
+TEST(ProfileTest, MergeProfilesByStack) {<br>
+ Profile P0, P1;<br>
+ EXPECT_FALSE(errorToBool(P0.<wbr>addBlock(<br>
+ Profile::Block{Profile::<wbr>ThreadID{1},<br>
+ {{P0.internPath({2, 1}), Profile::Data{1, 1000}}}})));<br>
+ EXPECT_FALSE(errorToBool(P1.<wbr>addBlock(<br>
+ Profile::Block{Profile::<wbr>ThreadID{2},<br>
+ {{P1.internPath({2, 1}), Profile::Data{1, 1000}}}})));<br>
+<br>
+ Profile Merged = mergeProfilesByStack(P0, P1);<br>
+ EXPECT_THAT(Merged,<br>
+ ElementsAre(AllOf(<br>
+ // We expect that we lose the ThreadID dimension in this<br>
+ // algorithm.<br>
+ Field(&Profile::Block::Thread, Eq(Profile::ThreadID{0})),<br>
+ Field(&Profile::Block::<wbr>PathData,<br>
+ ElementsAre(Pair(<br>
+ Merged.internPath({2, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(2u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(2000u)))))))));<br>
+}<br>
+<br>
+TEST(ProfileTest, MergeProfilesByStackAccumulate<wbr>) {<br>
+ std::vector<Profile> Profiles(3);<br>
+ EXPECT_FALSE(errorToBool(<wbr>Profiles[0].addBlock(Profile::<wbr>Block{<br>
+ Profile::ThreadID{1},<br>
+ {{Profiles[0].internPath({2, 1}), Profile::Data{1, 1000}}}})));<br>
+ EXPECT_FALSE(errorToBool(<wbr>Profiles[1].addBlock(Profile::<wbr>Block{<br>
+ Profile::ThreadID{2},<br>
+ {{Profiles[1].internPath({2, 1}), Profile::Data{1, 1000}}}})));<br>
+ EXPECT_FALSE(errorToBool(<wbr>Profiles[2].addBlock(Profile::<wbr>Block{<br>
+ Profile::ThreadID{3},<br>
+ {{Profiles[2].internPath({2, 1}), Profile::Data{1, 1000}}}})));<br>
+ Profile Merged = std::accumulate(Profiles.<wbr>begin(), Profiles.end(), Profile(),<br>
+ mergeProfilesByStack);<br>
+ EXPECT_THAT(Merged,<br>
+ ElementsAre(AllOf(<br>
+ // We expect that we lose the ThreadID dimension in this<br>
+ // algorithm.<br>
+ Field(&Profile::Block::Thread, Eq(Profile::ThreadID{0})),<br>
+ Field(&Profile::Block::<wbr>PathData,<br>
+ ElementsAre(Pair(<br>
+ Merged.internPath({2, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(3u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(3000u)))))))));<br>
+}<br>
+<br>
+TEST(ProfileTest, MergeProfilesByThreadAccumulat<wbr>e) {<br>
+ std::vector<Profile> Profiles(2);<br>
+<br>
+ // Set up the blocks for two different threads in Profiles[0].<br>
+ EXPECT_FALSE(errorToBool(<wbr>Profiles[0].addBlock(Profile::<wbr>Block{<br>
+ Profile::ThreadID{1},<br>
+ {{Profiles[0].internPath({2, 1}), Profile::Data{1, 1000}},<br>
+ {Profiles[0].internPath({4, 1}), Profile::Data{1, 1000}}}})));<br>
+ EXPECT_FALSE(errorToBool(<wbr>Profiles[0].addBlock(Profile::<wbr>Block{<br>
+ Profile::ThreadID{2},<br>
+ {{Profiles[0].internPath({3, 1}), Profile::Data{1, 1000}}}})));<br>
+<br>
+ // Set up the blocks for two different threads in Profiles[1].<br>
+ EXPECT_FALSE(errorToBool(<wbr>Profiles[1].addBlock(Profile::<wbr>Block{<br>
+ Profile::ThreadID{1},<br>
+ {{Profiles[1].internPath({2, 1}), Profile::Data{1, 1000}}}})));<br>
+ EXPECT_FALSE(errorToBool(<wbr>Profiles[1].addBlock(Profile::<wbr>Block{<br>
+ Profile::ThreadID{2},<br>
+ {{Profiles[1].internPath({3, 1}), Profile::Data{1, 1000}},<br>
+ {Profiles[1].internPath({4, 1}), Profile::Data{1, 1000}}}})));<br>
+<br>
+ Profile Merged = std::accumulate(Profiles.<wbr>begin(), Profiles.end(), Profile(),<br>
+ mergeProfilesByThread);<br>
+ EXPECT_THAT(<br>
+ Merged,<br>
+ UnorderedElementsAre(<br>
+ // We want to see two threads after the merge.<br>
+ AllOf(Field(&Profile::Block::<wbr>Thread, Eq(Profile::ThreadID{1})),<br>
+ Field(&Profile::Block::<wbr>PathData,<br>
+ UnorderedElementsAre(<br>
+ Pair(Merged.internPath({2, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(2u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(2000u)))),<br>
+ Pair(Merged.internPath({4, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(1u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(1000u))))))),<br>
+ AllOf(Field(&Profile::Block::<wbr>Thread, Eq(Profile::ThreadID{2})),<br>
+ Field(&Profile::Block::<wbr>PathData,<br>
+ UnorderedElementsAre(<br>
+ Pair(Merged.internPath({3, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(2u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(2000u)))),<br>
+ Pair(Merged.internPath({4, 1}),<br>
+ AllOf(Field(&Profile::Data::<wbr>CallCount, Eq(1u)),<br>
+ Field(&Profile::Data::<wbr>CumulativeLocalTime,<br>
+ Eq(1000u)))))))));<br>
+}<br>
+// FIXME: Add a test creating a Trace and generating a Profile<br>
+// FIXME: Add tests for ranking/sorting profile blocks by dimension<br>
+<br>
+} // namespace<br>
+} // namespace xray<br>
+} // namespace llvm<br>
<br>
<br>
______________________________<wbr>_________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div>