[llvm] r291529 - [XRay] Implement `llvm-xray convert` -- trace file conversion

Dimitry Andric via llvm-commits llvm-commits at lists.llvm.org
Tue Feb 14 12:33:07 PST 2017


Hi Dean,

I have been trying to figure out one llvm-xray test failure that occurs while testing the release_40 branch (but this same failure also shows up for me on trunk):

FAIL: LLVM :: tools/llvm-xray/X86/convert-with-debug-syms.txt (30283 of 37661)
******************** TEST 'LLVM :: tools/llvm-xray/X86/convert-with-debug-syms.txt' FAILED ********************
Script:
--
/home/dim/obj/llvm-294803-rel40-freebsd12-i386-ninja-rel-1/./bin/llvm-xray convert -m /share/dim/src/llvm/release_40/test
/tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin -y /share/dim/src/llvm/release_40/test/tools/llvm-xray/X86/Inputs/naive-l
og-simple.xray -f=yaml -o - 2>&1 | /home/dim/obj/llvm-294803-rel40-freebsd12-i386-ninja-rel-1/./bin/FileCheck /share/dim/
src/llvm/release_40/test/tools/llvm-xray/X86/convert-with-debug-syms.txt
--
Exit Code: 1

Command Output (stderr):
--
/share/dim/src/llvm/release_40/test/tools/llvm-xray/X86/convert-with-debug-syms.txt:13:15: error: expected string not fou
nd in input
; CHECK-NEXT: - { type: 0, func-id: 2, function: {{.*foo.*}}, cpu: 37, thread: 84697, kind: function-enter,
              ^
<stdin>:11:2: note: scanning from here
 - { type: 0, func-id: 2, function: 'foo(void)', cpu: 37, thread: 84697,
 ^
<stdin>:19:2: note: possible intended match here
 - { type: 0, func-id: 3, function: main, cpu: 37, thread: 84697, kind: function-exit,
 ^

--

********************

What happens here is that the test/tools/llvm-xray/X86/convert-with-debug-syms.txt test case has the following CHECK-NEXT lines:

; CHECK-NEXT:   - { type: 0, func-id: 2, function: {{.*foo.*}}, cpu: 37, thread: 84697, kind: function-enter,
; CHECK-NEXT:       tsc: 3315356841454542 }

However, on my FreeBSD-based systems, the output for the 'foo' and 'bar' lines is formatted slightly differently, wrapping at the 'kind' key instead of the 'tsc' key:

  - { type: 0, func-id: 2, function: 'foo(void)', cpu: 37, thread: 84697,
      kind: function-enter, tsc: 3315356841454542 }

Any idea what causes this?  Maybe the yaml outputter is retrieving the terminal size differently somehow, and thus wrapping it at another margin?

If so, maybe there is some way of ensuring the output either never wraps, or always wraps at the same margin?

-Dimitry

> On 10 Jan 2017, at 03:38, Dean Michael Berris via llvm-commits <llvm-commits at lists.llvm.org> wrote:
> 
> Author: dberris
> Date: Mon Jan  9 20:38:11 2017
> New Revision: 291529
> 
> URL: http://llvm.org/viewvc/llvm-project?rev=291529&view=rev
> Log:
> [XRay] Implement `llvm-xray convert` -- trace file conversion
> 
> This is the second part of a multi-part change to define additional
> subcommands to the `llvm-xray` tool.
> 
> This change defines a conversion subcommand to take XRay log files, and
> turns them from one format to another (binary or YAML). This currently
> only supports the first version of the log file format, defined in the
> compiler-rt runtime.
> 
> Depends on D21987.
> 
> Reviewers: dblaikie, echristo
> 
> Subscribers: mehdi_amini, dberris, beanz, llvm-commits
> 
> Differential Revision: https://reviews.llvm.org/D24376
> 
> Added:
>    llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-objcopied-instrmap.bin   (with props)
>    llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin   (with props)
>    llvm/trunk/test/tools/llvm-xray/X86/Inputs/naive-log-simple.xray
>    llvm/trunk/test/tools/llvm-xray/X86/Inputs/simple-xray-instrmap.yaml
>    llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.txt
>      - copied, changed from r291518, llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin
>    llvm/trunk/test/tools/llvm-xray/X86/convert-roundtrip.yaml
>    llvm/trunk/test/tools/llvm-xray/X86/convert-to-yaml.txt
>    llvm/trunk/test/tools/llvm-xray/X86/convert-with-debug-syms.txt
>    llvm/trunk/test/tools/llvm-xray/X86/convert-with-standalone-instrmap.txt
>    llvm/trunk/test/tools/llvm-xray/X86/convert-with-yaml-instrmap.txt
>    llvm/trunk/tools/llvm-xray/func-id-helper.cc
>    llvm/trunk/tools/llvm-xray/func-id-helper.h
>    llvm/trunk/tools/llvm-xray/xray-converter.cc
>    llvm/trunk/tools/llvm-xray/xray-converter.h
>    llvm/trunk/tools/llvm-xray/xray-log-reader.cc
>    llvm/trunk/tools/llvm-xray/xray-log-reader.h
>    llvm/trunk/tools/llvm-xray/xray-record-yaml.h
>    llvm/trunk/tools/llvm-xray/xray-record.h
> Removed:
>    llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin
> Modified:
>    llvm/trunk/tools/llvm-xray/CMakeLists.txt
>    llvm/trunk/tools/llvm-xray/xray-extract.cc
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-objcopied-instrmap.bin
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-objcopied-instrmap.bin?rev=291529&view=auto
> ==============================================================================
> Binary files llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-objcopied-instrmap.bin (added) and llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-objcopied-instrmap.bin Mon Jan  9 20:38:11 2017 differ
> 
> Propchange: llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-objcopied-instrmap.bin
> ------------------------------------------------------------------------------
>    svn:executable = *
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin?rev=291529&view=auto
> ==============================================================================
> Binary files llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin (added) and llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin Mon Jan  9 20:38:11 2017 differ
> 
> Propchange: llvm/trunk/test/tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin
> ------------------------------------------------------------------------------
>    svn:executable = *
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/Inputs/naive-log-simple.xray
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/Inputs/naive-log-simple.xray?rev=291529&view=auto
> ==============================================================================
> Binary files llvm/trunk/test/tools/llvm-xray/X86/Inputs/naive-log-simple.xray (added) and llvm/trunk/test/tools/llvm-xray/X86/Inputs/naive-log-simple.xray Mon Jan  9 20:38:11 2017 differ
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/Inputs/simple-xray-instrmap.yaml
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/Inputs/simple-xray-instrmap.yaml?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-xray/X86/Inputs/simple-xray-instrmap.yaml (added)
> +++ llvm/trunk/test/tools/llvm-xray/X86/Inputs/simple-xray-instrmap.yaml Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,14 @@
> +---
> +- { id: 1, address: 0x000000000041CA40, function: 0x000000000041CA40, kind: function-enter,
> +    always-instrument: true }
> +- { id: 1, address: 0x000000000041CA50, function: 0x000000000041CA40, kind: tail-exit,
> +    always-instrument: true }
> +- { id: 2, address: 0x000000000041CA70, function: 0x000000000041CA70, kind: function-enter,
> +    always-instrument: true }
> +- { id: 2, address: 0x000000000041CA7C, function: 0x000000000041CA70, kind: tail-exit,
> +    always-instrument: true }
> +- { id: 3, address: 0x000000000041CAA0, function: 0x000000000041CAA0, kind: function-enter,
> +    always-instrument: true }
> +- { id: 3, address: 0x000000000041CAB4, function: 0x000000000041CAA0, kind: function-exit,
> +    always-instrument: true }
> +...
> 
> Removed: llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin?rev=291528&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin (original)
> +++ llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin (removed)
> @@ -1,3 +0,0 @@
> -; RUN: not llvm-xray extract %S/Inputs/elf64-badentrysizes.bin 2>&1 | FileCheck %s
> -; CHECK: llvm-xray: Cannot extract instrumentation map from '{{.*}}elf64-badentrysizes.bin'.
> -; CHECK-NEXT: Instrumentation map entries not evenly divisible by size of an XRay sled entry in ELF64.
> 
> Copied: llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.txt (from r291518, llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin)
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.txt?p2=llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.txt&p1=llvm/trunk/test/tools/llvm-xray/X86/bad-instrmap-sizes.bin&r1=291518&r2=291529&rev=291529&view=diff
> ==============================================================================
>    (empty)
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/convert-roundtrip.yaml
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/convert-roundtrip.yaml?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-xray/X86/convert-roundtrip.yaml (added)
> +++ llvm/trunk/test/tools/llvm-xray/X86/convert-roundtrip.yaml Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,28 @@
> +#RUN: llvm-xray convert %s -i=yaml -f=raw -o %t && llvm-xray convert %t -f=yaml -o - | FileCheck %s
> +---
> +header:
> +  version: 1
> +  type: 0
> +  constant-tsc: true
> +  nonstop-tsc: true
> +  cycle-frequency: 2601000000
> +records:
> +  - { type: 0, func-id: 1, cpu: 1, thread: 111, kind: function-enter,
> +    tsc: 10001 }
> +  - { type: 0, func-id: 1, cpu: 1, thread: 111, kind: function-exit,
> +    tsc: 10100 }
> +...
> +
> +#CHECK:       ---
> +#CHECK-NEXT:  header:
> +#CHECK-NEXT:    version: 1
> +#CHECK-NEXT:    type: 0
> +#CHECK-NEXT:    constant-tsc: true
> +#CHECK-NEXT:    nonstop-tsc: true
> +#CHECK-NEXT:    cycle-frequency: 2601000000
> +#CHECK-NEXT:  records:
> +#CHECK-NEXT:    - { type: 0, func-id: 1, function: '1', cpu: 1, thread: 111, kind: function-enter,
> +#CHECK-NEXT:      tsc: 10001 }
> +#CHECK-NEXT:    - { type: 0, func-id: 1, function: '1', cpu: 1, thread: 111, kind: function-exit,
> +#CHECK-NEXT:      tsc: 10100 }
> +#CHECK-NEXT:  ...
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/convert-to-yaml.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/convert-to-yaml.txt?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-xray/X86/convert-to-yaml.txt (added)
> +++ llvm/trunk/test/tools/llvm-xray/X86/convert-to-yaml.txt Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,23 @@
> +; RUN: llvm-xray convert %S/Inputs/naive-log-simple.xray -f=yaml -o - | FileCheck %s
> +
> +; CHECK:      ---
> +; CHECK-NEXT: header:
> +; CHECK-NEXT:   version:         1
> +; CHECK-NEXT:   type:            0
> +; CHECK-NEXT:   constant-tsc:    true
> +; CHECK-NEXT:   nonstop-tsc:     true
> +; CHECK-NEXT:   cycle-frequency: 2601000000
> +; CHECK-NEXT: records:
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: '3', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841453914 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: '2', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841454542 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: '2', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841454670 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: '1', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841454762 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: '1', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841454802 }
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: '3', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841494828 }
> +; CHECK-NEXT: ...
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/convert-with-debug-syms.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/convert-with-debug-syms.txt?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-xray/X86/convert-with-debug-syms.txt (added)
> +++ llvm/trunk/test/tools/llvm-xray/X86/convert-with-debug-syms.txt Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,23 @@
> +; RUN: llvm-xray convert -m %S/Inputs/elf64-sample-o2.bin -y %S/Inputs/naive-log-simple.xray -f=yaml -o - 2>&1 | FileCheck %s
> +
> +; CHECK:      ---
> +; CHECK-NEXT: header:
> +; CHECK-NEXT:   version:         1
> +; CHECK-NEXT:   type:            0
> +; CHECK-NEXT:   constant-tsc:    true
> +; CHECK-NEXT:   nonstop-tsc:     true
> +; CHECK-NEXT:   cycle-frequency: 2601000000
> +; CHECK-NEXT: records:
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: main, cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841453914 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: 'foo()', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841454542 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: 'foo()', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841454670 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: 'bar()', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841454762 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: 'bar()', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841454802 }
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: main, cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841494828 }
> +; CHECK-NEXT: ...
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/convert-with-standalone-instrmap.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/convert-with-standalone-instrmap.txt?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-xray/X86/convert-with-standalone-instrmap.txt (added)
> +++ llvm/trunk/test/tools/llvm-xray/X86/convert-with-standalone-instrmap.txt Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,23 @@
> +; RUN: llvm-xray convert -m %S/Inputs/elf64-objcopied-instrmap.bin -y %S/Inputs/naive-log-simple.xray -f=yaml -o - 2>&1 | FileCheck %s
> +
> +; CHECK:      ---
> +; CHECK-NEXT: header:
> +; CHECK-NEXT:   version:         1
> +; CHECK-NEXT:   type:            0
> +; CHECK-NEXT:   constant-tsc:    true
> +; CHECK-NEXT:   nonstop-tsc:     true
> +; CHECK-NEXT:   cycle-frequency: 2601000000
> +; CHECK-NEXT: records:
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: '@(41caa0)', cpu: 37, thread: 84697,
> +; CHECK-NEXT:       kind: function-enter, tsc: 3315356841453914 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: '@(41ca70)', cpu: 37, thread: 84697,
> +; CHECK-NEXT:       kind: function-enter, tsc: 3315356841454542 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: '@(41ca70)', cpu: 37, thread: 84697,
> +; CHECK-NEXT:       kind: function-exit, tsc: 3315356841454670 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: '@(41ca40)', cpu: 37, thread: 84697,
> +; CHECK-NEXT:       kind: function-enter, tsc: 3315356841454762 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: '@(41ca40)', cpu: 37, thread: 84697,
> +; CHECK-NEXT:       kind: function-exit, tsc: 3315356841454802 }
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: '@(41caa0)', cpu: 37, thread: 84697,
> +; CHECK-NEXT:       kind: function-exit, tsc: 3315356841494828 }
> +; CHECK-NEXT: ...
> 
> Added: llvm/trunk/test/tools/llvm-xray/X86/convert-with-yaml-instrmap.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-xray/X86/convert-with-yaml-instrmap.txt?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-xray/X86/convert-with-yaml-instrmap.txt (added)
> +++ llvm/trunk/test/tools/llvm-xray/X86/convert-with-yaml-instrmap.txt Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,23 @@
> +; RUN: llvm-xray convert -m %S/Inputs/simple-xray-instrmap.yaml -t yaml %S/Inputs/naive-log-simple.xray -f=yaml -o - | FileCheck %s
> +
> +; CHECK:      ---
> +; CHECK-NEXT: header:
> +; CHECK-NEXT:   version:         1
> +; CHECK-NEXT:   type:            0
> +; CHECK-NEXT:   constant-tsc:    true
> +; CHECK-NEXT:   nonstop-tsc:     true
> +; CHECK-NEXT:   cycle-frequency: 2601000000
> +; CHECK-NEXT: records:
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: '3', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841453914 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: '2', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841454542 }
> +; CHECK-NEXT:   - { type: 0, func-id: 2, function: '2', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841454670 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: '1', cpu: 37, thread: 84697, kind: function-enter,
> +; CHECK-NEXT:       tsc: 3315356841454762 }
> +; CHECK-NEXT:   - { type: 0, func-id: 1, function: '1', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841454802 }
> +; CHECK-NEXT:   - { type: 0, func-id: 3, function: '3', cpu: 37, thread: 84697, kind: function-exit,
> +; CHECK-NEXT:       tsc: 3315356841494828 }
> +; CHECK-NEXT: ...
> 
> Modified: llvm/trunk/tools/llvm-xray/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/CMakeLists.txt?rev=291529&r1=291528&r2=291529&view=diff
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/CMakeLists.txt (original)
> +++ llvm/trunk/tools/llvm-xray/CMakeLists.txt Mon Jan  9 20:38:11 2017
> @@ -1,10 +1,16 @@
> set(LLVM_LINK_COMPONENTS
>   ${LLVM_TARGETS_TO_BUILD}
> +  DebugInfoDWARF
> +  Object
>   Support
> -  Object)
> +  Symbolize)
> 
> set(LLVM_XRAY_TOOLS
> +  func-id-helper.cc
> +  xray-converter.cc
>   xray-extract.cc
> +  xray-extract.cc
> +  xray-log-reader.cc
>   xray-registry.cc)
> 
> add_llvm_tool(llvm-xray llvm-xray.cc ${LLVM_XRAY_TOOLS})
> 
> Added: llvm/trunk/tools/llvm-xray/func-id-helper.cc
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/func-id-helper.cc?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/func-id-helper.cc (added)
> +++ llvm/trunk/tools/llvm-xray/func-id-helper.cc Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,60 @@
> +//===- xray-fc-account.cc - XRay Function Call Accounting Tool ------------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// Implementation of the helper tools dealing with XRay-generated function ids.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "func-id-helper.h"
> +#include "llvm/Support/Path.h"
> +#include <sstream>
> +
> +using namespace llvm;
> +using namespace xray;
> +
> +std::string FuncIdConversionHelper::SymbolOrNumber(int32_t FuncId) const {
> +  std::ostringstream F;
> +  auto It = FunctionAddresses.find(FuncId);
> +  if (It == FunctionAddresses.end()) {
> +    F << "#" << FuncId;
> +    return F.str();
> +  }
> +
> +  if (auto ResOrErr = Symbolizer.symbolizeCode(BinaryInstrMap, It->second)) {
> +    auto &DI = *ResOrErr;
> +    if (DI.FunctionName == "<invalid>")
> +      F << "@(" << std::hex << It->second << ")";
> +    else
> +      F << DI.FunctionName;
> +  } else
> +    handleAllErrors(ResOrErr.takeError(), [&](const ErrorInfoBase &) {
> +      F << "@(" << std::hex << It->second << ")";
> +    });
> +
> +  return F.str();
> +}
> +
> +std::string FuncIdConversionHelper::FileLineAndColumn(int32_t FuncId) const {
> +  auto It = FunctionAddresses.find(FuncId);
> +  if (It == FunctionAddresses.end())
> +    return "(unknown)";
> +
> +  std::ostringstream F;
> +  auto ResOrErr = Symbolizer.symbolizeCode(BinaryInstrMap, It->second);
> +  if (!ResOrErr) {
> +    consumeError(ResOrErr.takeError());
> +    return "(unknown)";
> +  }
> +
> +  auto &DI = *ResOrErr;
> +  F << sys::path::filename(DI.FileName).str() << ":" << DI.Line << ":"
> +    << DI.Column;
> +
> +  return F.str();
> +}
> 
> Added: llvm/trunk/tools/llvm-xray/func-id-helper.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/func-id-helper.h?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/func-id-helper.h (added)
> +++ llvm/trunk/tools/llvm-xray/func-id-helper.h Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,49 @@
> +//===- func-id-helper.h - XRay Function ID Conversion Helpers -------------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// Defines helper tools dealing with XRay-generated function ids.
> +//
> +//===----------------------------------------------------------------------===//
> +#ifndef LLVM_TOOLS_LLVM_XRAY_FUNC_ID_HELPER_H
> +#define LLVM_TOOLS_LLVM_XRAY_FUNC_ID_HELPER_H
> +
> +#include "llvm/DebugInfo/Symbolize/Symbolize.h"
> +#include <unordered_map>
> +
> +namespace llvm {
> +namespace xray {
> +
> +// This class consolidates common operations related to Function IDs.
> +class FuncIdConversionHelper {
> +public:
> +  using FunctionAddressMap = std::unordered_map<int32_t, uint64_t>;
> +
> +private:
> +  std::string BinaryInstrMap;
> +  symbolize::LLVMSymbolizer &Symbolizer;
> +  const FunctionAddressMap &FunctionAddresses;
> +
> +public:
> +  FuncIdConversionHelper(std::string BinaryInstrMap,
> +                         symbolize::LLVMSymbolizer &Symbolizer,
> +                         const FunctionAddressMap &FunctionAddresses)
> +      : BinaryInstrMap(std::move(BinaryInstrMap)), Symbolizer(Symbolizer),
> +        FunctionAddresses(FunctionAddresses) {}
> +
> +  // Returns the symbol or a string representation of the function id.
> +  std::string SymbolOrNumber(int32_t FuncId) const;
> +
> +  // Returns the file and column from debug info for the given function id.
> +  std::string FileLineAndColumn(int32_t FuncId) const;
> +};
> +
> +} // namespace xray
> +} // namespace llvm
> +
> +#endif // LLVM_TOOLS_LLVM_XRAY_FUNC_ID_HELPER_H
> 
> Added: llvm/trunk/tools/llvm-xray/xray-converter.cc
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/xray-converter.cc?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/xray-converter.cc (added)
> +++ llvm/trunk/tools/llvm-xray/xray-converter.cc Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,222 @@
> +//===- xray-converter.cc - XRay Trace Conversion --------------------------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// Implements the trace conversion functions.
> +//
> +//===----------------------------------------------------------------------===//
> +#include "xray-converter.h"
> +
> +#include "xray-extract.h"
> +#include "xray-record-yaml.h"
> +#include "xray-registry.h"
> +#include "llvm/DebugInfo/Symbolize/Symbolize.h"
> +#include "llvm/Support/EndianStream.h"
> +#include "llvm/Support/FileSystem.h"
> +#include "llvm/Support/YAMLTraits.h"
> +#include "llvm/Support/raw_ostream.h"
> +#include <unistd.h>
> +
> +using namespace llvm;
> +using namespace xray;
> +
> +// llvm-xray convert
> +// ----------------------------------------------------------------------------
> +static cl::SubCommand Convert("convert", "Trace Format Conversion");
> +static cl::opt<std::string> ConvertInput(cl::Positional,
> +                                         cl::desc("<xray log file>"),
> +                                         cl::Required, cl::sub(Convert));
> +enum class ConvertFormats { BINARY, YAML };
> +static cl::opt<ConvertFormats> ConvertInputFormat(
> +    "input-format", cl::desc("input format"),
> +    cl::values(clEnumValN(ConvertFormats::BINARY, "raw",
> +                          "input is in raw binary"),
> +               clEnumValN(ConvertFormats::YAML, "yaml", "input is in yaml")),
> +    cl::sub(Convert));
> +static cl::alias ConvertInputFormat2("i", cl::aliasopt(ConvertInputFormat),
> +                                     cl::desc("Alias for -input-format"),
> +                                     cl::sub(Convert));
> +static cl::opt<ConvertFormats> ConvertOutputFormat(
> +    "output-format", cl::desc("output format"),
> +    cl::values(clEnumValN(ConvertFormats::BINARY, "raw", "output in binary"),
> +               clEnumValN(ConvertFormats::YAML, "yaml", "output in yaml")),
> +    cl::sub(Convert));
> +static cl::alias ConvertOutputFormat2("f", cl::aliasopt(ConvertOutputFormat),
> +                                      cl::desc("Alias for -output-format"),
> +                                      cl::sub(Convert));
> +static cl::opt<std::string>
> +    ConvertOutput("output", cl::value_desc("output file"), cl::init("-"),
> +                  cl::desc("output file; use '-' for stdout"),
> +                  cl::sub(Convert));
> +static cl::alias ConvertOutput2("o", cl::aliasopt(ConvertOutput),
> +                                cl::desc("Alias for -output"),
> +                                cl::sub(Convert));
> +
> +static cl::opt<bool>
> +    ConvertSymbolize("symbolize",
> +                     cl::desc("symbolize function ids from the input log"),
> +                     cl::init(false), cl::sub(Convert));
> +static cl::alias ConvertSymbolize2("y", cl::aliasopt(ConvertSymbolize),
> +                                   cl::desc("Alias for -symbolize"),
> +                                   cl::sub(Convert));
> +
> +static cl::opt<std::string>
> +    ConvertInstrMap("instr_map",
> +                    cl::desc("binary with the instrumentation map, or "
> +                             "a separate instrumentation map"),
> +                    cl::value_desc("binary with xray_instr_map"),
> +                    cl::sub(Convert), cl::init(""));
> +static cl::alias ConvertInstrMap2("m", cl::aliasopt(ConvertInstrMap),
> +                                  cl::desc("Alias for -instr_map"),
> +                                  cl::sub(Convert));
> +static cl::opt<bool> ConvertSortInput(
> +    "sort",
> +    cl::desc("determines whether to sort input log records by timestamp"),
> +    cl::sub(Convert), cl::init(true));
> +static cl::alias ConvertSortInput2("s", cl::aliasopt(ConvertSortInput),
> +                                   cl::desc("Alias for -sort"),
> +                                   cl::sub(Convert));
> +static cl::opt<InstrumentationMapExtractor::InputFormats> InstrMapFormat(
> +    "instr-map-format", cl::desc("format of instrumentation map"),
> +    cl::values(clEnumValN(InstrumentationMapExtractor::InputFormats::ELF, "elf",
> +                          "instrumentation map in an ELF header"),
> +               clEnumValN(InstrumentationMapExtractor::InputFormats::YAML,
> +                          "yaml", "instrumentation map in YAML")),
> +    cl::sub(Convert), cl::init(InstrumentationMapExtractor::InputFormats::ELF));
> +static cl::alias InstrMapFormat2("t", cl::aliasopt(InstrMapFormat),
> +                                 cl::desc("Alias for -instr-map-format"),
> +                                 cl::sub(Convert));
> +
> +using llvm::yaml::MappingTraits;
> +using llvm::yaml::ScalarEnumerationTraits;
> +using llvm::yaml::IO;
> +using llvm::yaml::Output;
> +
> +void TraceConverter::exportAsYAML(const LogReader &Records, raw_ostream &OS) {
> +  YAMLXRayTrace Trace;
> +  const auto &FH = Records.getFileHeader();
> +  Trace.Header = {FH.Version, FH.Type, FH.ConstantTSC, FH.NonstopTSC,
> +                  FH.CycleFrequency};
> +  Trace.Records.reserve(Records.size());
> +  for (const auto &R : Records) {
> +    Trace.Records.push_back({R.RecordType, R.CPU, R.Type, R.FuncId,
> +                             Symbolize ? FuncIdHelper.SymbolOrNumber(R.FuncId)
> +                                       : std::to_string(R.FuncId),
> +                             R.TSC, R.TId});
> +  }
> +  Output Out(OS);
> +  Out << Trace;
> +}
> +
> +void TraceConverter::exportAsRAWv1(const LogReader &Records, raw_ostream &OS) {
> +  // First write out the file header, in the correct endian-appropriate format
> +  // (XRay assumes currently little endian).
> +  support::endian::Writer<support::endianness::little> Writer(OS);
> +  const auto &FH = Records.getFileHeader();
> +  Writer.write(FH.Version);
> +  Writer.write(FH.Type);
> +  uint32_t Bitfield{0};
> +  if (FH.ConstantTSC)
> +    Bitfield |= 1uL;
> +  if (FH.NonstopTSC)
> +    Bitfield |= 1uL << 1;
> +  Writer.write(Bitfield);
> +  Writer.write(FH.CycleFrequency);
> +
> +  // There's 16 bytes of padding at the end of the file header.
> +  static constexpr uint32_t Padding4B = 0;
> +  Writer.write(Padding4B);
> +  Writer.write(Padding4B);
> +  Writer.write(Padding4B);
> +  Writer.write(Padding4B);
> +
> +  // Then write out the rest of the records, still in an endian-appropriate
> +  // format.
> +  for (const auto &R : Records) {
> +    Writer.write(R.RecordType);
> +    Writer.write(R.CPU);
> +    switch (R.Type) {
> +    case RecordTypes::ENTER:
> +      Writer.write(uint8_t{0});
> +      break;
> +    case RecordTypes::EXIT:
> +      Writer.write(uint8_t{1});
> +      break;
> +    }
> +    Writer.write(R.FuncId);
> +    Writer.write(R.TSC);
> +    Writer.write(R.TId);
> +    Writer.write(Padding4B);
> +    Writer.write(Padding4B);
> +    Writer.write(Padding4B);
> +  }
> +}
> +
> +namespace llvm {
> +namespace xray {
> +
> +static CommandRegistration Unused(&Convert, []() -> Error {
> +  // FIXME: Support conversion to BINARY when upgrading XRay trace versions.
> +  int Fd;
> +  auto EC = sys::fs::openFileForRead(ConvertInput, Fd);
> +  if (EC)
> +    return make_error<StringError>(
> +        Twine("Cannot open file '") + ConvertInput + "'", EC);
> +
> +  Error Err = Error::success();
> +  xray::InstrumentationMapExtractor Extractor(ConvertInstrMap, InstrMapFormat,
> +                                              Err);
> +  handleAllErrors(std::move(Err),
> +                  [&](const ErrorInfoBase &E) { E.log(errs()); });
> +
> +  const auto &FunctionAddresses = Extractor.getFunctionAddresses();
> +  symbolize::LLVMSymbolizer::Options Opts(
> +      symbolize::FunctionNameKind::LinkageName, true, true, false, "");
> +  symbolize::LLVMSymbolizer Symbolizer(Opts);
> +  llvm::xray::FuncIdConversionHelper FuncIdHelper(ConvertInstrMap, Symbolizer,
> +                                                  FunctionAddresses);
> +  llvm::xray::TraceConverter TC(FuncIdHelper, ConvertSymbolize);
> +  LogReader::LoaderFunction Loader;
> +  switch (ConvertInputFormat) {
> +  case ConvertFormats::BINARY:
> +    Loader = NaiveLogLoader;
> +    break;
> +  case ConvertFormats::YAML:
> +    Loader = YAMLLogLoader;
> +    break;
> +  }
> +
> +  LogReader Reader(ConvertInput, Err, ConvertSortInput, Loader);
> +  if (Err)
> +    return joinErrors(
> +        make_error<StringError>(
> +            Twine("Failed loading input file '") + ConvertInput + "'.",
> +            std::make_error_code(std::errc::protocol_error)),
> +        std::move(Err));
> +
> +  raw_fd_ostream OS(ConvertOutput, EC,
> +                    ConvertOutputFormat == ConvertFormats::BINARY
> +                        ? sys::fs::OpenFlags::F_None
> +                        : sys::fs::OpenFlags::F_Text);
> +  if (EC)
> +    return make_error<StringError>(
> +        Twine("Cannot open file '") + ConvertOutput + "' for writing.", EC);
> +
> +  switch (ConvertOutputFormat) {
> +  case ConvertFormats::YAML:
> +    TC.exportAsYAML(Reader, OS);
> +    break;
> +  case ConvertFormats::BINARY:
> +    TC.exportAsRAWv1(Reader, OS);
> +    break;
> +  }
> +  return Error::success();
> +});
> +
> +} // namespace xray
> +} // namespace llvm
> 
> Added: llvm/trunk/tools/llvm-xray/xray-converter.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/xray-converter.h?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/xray-converter.h (added)
> +++ llvm/trunk/tools/llvm-xray/xray-converter.h Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,39 @@
> +//===- xray-converter.h - XRay Trace Conversion ---------------------------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// Defines the TraceConverter class for turning binary traces into
> +// human-readable text and vice versa.
> +//
> +//===----------------------------------------------------------------------===//
> +#ifndef LLVM_TOOLS_LLVM_XRAY_XRAY_CONVERTER_H
> +#define LLVM_TOOLS_LLVM_XRAY_XRAY_CONVERTER_H
> +
> +#include "func-id-helper.h"
> +#include "xray-log-reader.h"
> +#include "xray-record.h"
> +
> +namespace llvm {
> +namespace xray {
> +
> +class TraceConverter {
> +  FuncIdConversionHelper &FuncIdHelper;
> +  bool Symbolize;
> +
> +public:
> +  TraceConverter(FuncIdConversionHelper &FuncIdHelper, bool Symbolize = false)
> +      : FuncIdHelper(FuncIdHelper), Symbolize(Symbolize) {}
> +
> +  void exportAsYAML(const LogReader &Records, raw_ostream &OS);
> +  void exportAsRAWv1(const LogReader &Records, raw_ostream &OS);
> +};
> +
> +} // namespace xray
> +} // namespace llvm
> +
> +#endif // LLVM_TOOLS_LLVM_XRAY_XRAY_CONVERTER_H
> 
> Modified: llvm/trunk/tools/llvm-xray/xray-extract.cc
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/xray-extract.cc?rev=291529&r1=291528&r2=291529&view=diff
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/xray-extract.cc (original)
> +++ llvm/trunk/tools/llvm-xray/xray-extract.cc Mon Jan  9 20:38:11 2017
> @@ -162,8 +162,7 @@ llvm::Error LoadBinaryInstrELF(
>               "'.",
>           std::make_error_code(std::errc::executable_format_error));
>     }
> -    auto AlwaysInstrument = Extractor.getU8(&OffsetPtr);
> -    Entry.AlwaysInstrument = AlwaysInstrument != 0;
> +    Entry.AlwaysInstrument = Extractor.getU8(&OffsetPtr) != 0;
> 
>     // We replicate the function id generation scheme implemented in the runtime
>     // here. Ideally we should be able to break it out, or output this map from
> @@ -185,30 +184,82 @@ llvm::Error LoadBinaryInstrELF(
>   return llvm::Error::success();
> }
> 
> +Error LoadYAMLInstrMap(
> +    StringRef Filename, std::deque<SledEntry> &Sleds,
> +    InstrumentationMapExtractor::FunctionAddressMap &InstrMap,
> +    InstrumentationMapExtractor::FunctionAddressReverseMap &FunctionIds) {
> +  int Fd;
> +  if (auto EC = sys::fs::openFileForRead(Filename, Fd))
> +    return make_error<StringError>(
> +        Twine("Failed opening file '") + Filename + "' for reading.", EC);
> +
> +  uint64_t FileSize;
> +  if (auto EC = sys::fs::file_size(Filename, FileSize))
> +    return make_error<StringError>(
> +        Twine("Failed getting size of file '") + Filename + "'.", EC);
> +
> +  std::error_code EC;
> +  sys::fs::mapped_file_region MappedFile(
> +      Fd, sys::fs::mapped_file_region::mapmode::readonly, FileSize, 0, EC);
> +  if (EC)
> +    return make_error<StringError>(
> +        Twine("Failed memory-mapping file '") + Filename + "'.", EC);
> +
> +  std::vector<YAMLXRaySledEntry> YAMLSleds;
> +  Input In(StringRef(MappedFile.data(), MappedFile.size()));
> +  In >> YAMLSleds;
> +  if (In.error())
> +    return make_error<StringError>(
> +        Twine("Failed loading YAML document from '") + Filename + "'.",
> +        In.error());
> +
> +  for (const auto &Y : YAMLSleds) {
> +    InstrMap[Y.FuncId] = Y.Function;
> +    FunctionIds[Y.Function] = Y.FuncId;
> +    Sleds.push_back(
> +        SledEntry{Y.Address, Y.Function, Y.Kind, Y.AlwaysInstrument});
> +  }
> +  return Error::success();
> +}
> +
> } // namespace
> 
> InstrumentationMapExtractor::InstrumentationMapExtractor(std::string Filename,
>                                                          InputFormats Format,
>                                                          Error &EC) {
>   ErrorAsOutParameter ErrAsOutputParam(&EC);
> +  if (Filename.empty()) {
> +    EC = Error::success();
> +    return;
> +  }
>   switch (Format) {
>   case InputFormats::ELF: {
>     EC = handleErrors(
>         LoadBinaryInstrELF(Filename, Sleds, FunctionAddresses, FunctionIds),
> -        [](std::unique_ptr<ErrorInfoBase> E) {
> +        [&](std::unique_ptr<ErrorInfoBase> E) {
>           return joinErrors(
>               make_error<StringError>(
>                   Twine("Cannot extract instrumentation map from '") +
> -                      ExtractInput + "'.",
> +                      Filename + "'.",
>                   std::make_error_code(std::errc::executable_format_error)),
>               std::move(E));
>         });
>     break;
>   }
> -  default:
> -    llvm_unreachable("Input format type not supported yet.");
> +  case InputFormats::YAML: {
> +    EC = handleErrors(
> +        LoadYAMLInstrMap(Filename, Sleds, FunctionAddresses, FunctionIds),
> +        [&](std::unique_ptr<ErrorInfoBase> E) {
> +          return joinErrors(
> +              make_error<StringError>(
> +                  Twine("Cannot load YAML instrumentation map from '") +
> +                      Filename + "'.",
> +                  std::make_error_code(std::errc::wrong_protocol_type)),
> +              std::move(E));
> +        });
>     break;
>   }
> +  }
> }
> 
> void InstrumentationMapExtractor::exportAsYAML(raw_ostream &OS) {
> 
> Added: llvm/trunk/tools/llvm-xray/xray-log-reader.cc
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/xray-log-reader.cc?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/xray-log-reader.cc (added)
> +++ llvm/trunk/tools/llvm-xray/xray-log-reader.cc Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,165 @@
> +//===- xray-log-reader.cc - XRay Log Reader Implementation ----------------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// XRay log reader implementation.
> +//
> +//===----------------------------------------------------------------------===//
> +#include <sys/types.h>
> +
> +#include "xray-log-reader.h"
> +#include "xray-record-yaml.h"
> +#include "llvm/Support/DataExtractor.h"
> +#include "llvm/Support/FileSystem.h"
> +
> +using namespace llvm;
> +using namespace llvm::xray;
> +using llvm::yaml::Input;
> +
> +LogReader::LogReader(
> +    StringRef Filename, Error &Err, bool Sort,
> +    std::function<Error(StringRef, XRayFileHeader &, std::vector<XRayRecord> &)>
> +        Loader) {
> +  ErrorAsOutParameter Guard(&Err);
> +  int Fd;
> +  if (auto EC = sys::fs::openFileForRead(Filename, Fd)) {
> +    Err = make_error<StringError>(
> +        Twine("Cannot read log from '") + Filename + "'", EC);
> +    return;
> +  }
> +  uint64_t FileSize;
> +  if (auto EC = sys::fs::file_size(Filename, FileSize)) {
> +    Err = make_error<StringError>(
> +        Twine("Cannot read log from '") + Filename + "'", EC);
> +    return;
> +  }
> +
> +  std::error_code EC;
> +  sys::fs::mapped_file_region MappedFile(
> +      Fd, sys::fs::mapped_file_region::mapmode::readonly, FileSize, 0, EC);
> +  if (EC) {
> +    Err = make_error<StringError>(
> +        Twine("Cannot read log from '") + Filename + "'", EC);
> +    return;
> +  }
> +
> +  if (auto E = Loader(StringRef(MappedFile.data(), MappedFile.size()),
> +                      FileHeader, Records)) {
> +    Err = std::move(E);
> +    return;
> +  }
> +
> +  if (Sort)
> +    std::sort(
> +        Records.begin(), Records.end(),
> +        [](const XRayRecord &L, const XRayRecord &R) { return L.TSC < R.TSC; });
> +}
> +
> +Error llvm::xray::NaiveLogLoader(StringRef Data, XRayFileHeader &FileHeader,
> +                                 std::vector<XRayRecord> &Records) {
> +  // FIXME: Maybe deduce whether the data is little or big-endian using some
> +  // magic bytes in the beginning of the file?
> +
> +  // First 32 bytes of the file will always be the header. We assume a certain
> +  // format here:
> +  //
> +  //   (2)   uint16 : version
> +  //   (2)   uint16 : type
> +  //   (4)   uint32 : bitfield
> +  //   (8)   uint64 : cycle frequency
> +  //   (16)  -      : padding
> +  //
> +  if (Data.size() < 32)
> +    return make_error<StringError>(
> +        "Not enough bytes for an XRay log.",
> +        std::make_error_code(std::errc::invalid_argument));
> +
> +  if (Data.size() - 32 == 0 || Data.size() % 32 != 0)
> +    return make_error<StringError>(
> +        "Invalid-sized XRay data.",
> +        std::make_error_code(std::errc::invalid_argument));
> +
> +  DataExtractor HeaderExtractor(Data, true, 8);
> +  uint32_t OffsetPtr = 0;
> +  FileHeader.Version = HeaderExtractor.getU16(&OffsetPtr);
> +  FileHeader.Type = HeaderExtractor.getU16(&OffsetPtr);
> +  uint32_t Bitfield = HeaderExtractor.getU32(&OffsetPtr);
> +  FileHeader.ConstantTSC = Bitfield & 1uL;
> +  FileHeader.NonstopTSC = Bitfield & 1uL << 1;
> +  FileHeader.CycleFrequency = HeaderExtractor.getU64(&OffsetPtr);
> +
> +  if (FileHeader.Version != 1)
> +    return make_error<StringError>(
> +        Twine("Unsupported XRay file version: ") + Twine(FileHeader.Version),
> +        std::make_error_code(std::errc::invalid_argument));
> +
> +  // Each record after the header will be 32 bytes, in the following format:
> +  //
> +  //   (2)   uint16 : record type
> +  //   (1)   uint8  : cpu id
> +  //   (1)   uint8  : type
> +  //   (4)   sint32 : function id
> +  //   (8)   uint64 : tsc
> +  //   (4)   uint32 : thread id
> +  //   (12)  -      : padding
> +  for (auto S = Data.drop_front(32); !S.empty(); S = S.drop_front(32)) {
> +    DataExtractor RecordExtractor(S, true, 8);
> +    uint32_t OffsetPtr = 0;
> +    Records.emplace_back();
> +    auto &Record = Records.back();
> +    Record.RecordType = RecordExtractor.getU16(&OffsetPtr);
> +    Record.CPU = RecordExtractor.getU8(&OffsetPtr);
> +    auto Type = RecordExtractor.getU8(&OffsetPtr);
> +    switch (Type) {
> +    case 0:
> +      Record.Type = RecordTypes::ENTER;
> +      break;
> +    case 1:
> +      Record.Type = RecordTypes::EXIT;
> +      break;
> +    default:
> +      return make_error<StringError>(
> +          Twine("Unknown record type '") + Twine(int{Type}) + "'",
> +          std::make_error_code(std::errc::protocol_error));
> +    }
> +    Record.FuncId = RecordExtractor.getSigned(&OffsetPtr, sizeof(int32_t));
> +    Record.TSC = RecordExtractor.getU64(&OffsetPtr);
> +    Record.TId = RecordExtractor.getU32(&OffsetPtr);
> +  }
> +  return Error::success();
> +}
> +
> +Error llvm::xray::YAMLLogLoader(StringRef Data, XRayFileHeader &FileHeader,
> +                                std::vector<XRayRecord> &Records) {
> +
> +  // Load the documents from the MappedFile.
> +  YAMLXRayTrace Trace;
> +  Input In(Data);
> +  In >> Trace;
> +  if (In.error())
> +    return make_error<StringError>("Failed loading YAML Data.", In.error());
> +
> +  FileHeader.Version = Trace.Header.Version;
> +  FileHeader.Type = Trace.Header.Type;
> +  FileHeader.ConstantTSC = Trace.Header.ConstantTSC;
> +  FileHeader.NonstopTSC = Trace.Header.NonstopTSC;
> +  FileHeader.CycleFrequency = Trace.Header.CycleFrequency;
> +
> +  if (FileHeader.Version != 1)
> +    return make_error<StringError>(
> +        Twine("Unsupported XRay file version: ") + Twine(FileHeader.Version),
> +        std::make_error_code(std::errc::invalid_argument));
> +
> +  Records.clear();
> +  std::transform(Trace.Records.begin(), Trace.Records.end(),
> +                 std::back_inserter(Records), [&](const YAMLXRayRecord &R) {
> +                   return XRayRecord{R.RecordType, R.CPU, R.Type,
> +                                     R.FuncId,     R.TSC, R.TId};
> +                 });
> +  return Error::success();
> +}
> 
> Added: llvm/trunk/tools/llvm-xray/xray-log-reader.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/xray-log-reader.h?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/xray-log-reader.h (added)
> +++ llvm/trunk/tools/llvm-xray/xray-log-reader.h Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,57 @@
> +//===- xray-log-reader.h - XRay Log Reader Interface ----------------------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// Define the interface for an XRay log reader. Currently we only support one
> +// version of the log (naive log) with fixed-sized records.
> +//
> +//===----------------------------------------------------------------------===//
> +#ifndef LLVM_TOOLS_LLVM_XRAY_XRAY_LOG_READER_H
> +#define LLVM_TOOLS_LLVM_XRAY_XRAY_LOG_READER_H
> +
> +#include <cstdint>
> +#include <deque>
> +#include <vector>
> +
> +#include "xray-record-yaml.h"
> +#include "xray-record.h"
> +#include "llvm/Support/Error.h"
> +#include "llvm/Support/FileSystem.h"
> +
> +namespace llvm {
> +namespace xray {
> +
> +class LogReader {
> +  XRayFileHeader FileHeader;
> +  std::vector<XRayRecord> Records;
> +
> +  typedef std::vector<XRayRecord>::const_iterator citerator;
> +
> +public:
> +  typedef std::function<Error(StringRef, XRayFileHeader &,
> +                              std::vector<XRayRecord> &)>
> +      LoaderFunction;
> +
> +  LogReader(StringRef Filename, Error &Err, bool Sort, LoaderFunction Loader);
> +
> +  const XRayFileHeader &getFileHeader() const { return FileHeader; }
> +
> +  citerator begin() const { return Records.begin(); }
> +  citerator end() const { return Records.end(); }
> +  size_t size() const { return Records.size(); }
> +};
> +
> +Error NaiveLogLoader(StringRef Data, XRayFileHeader &FileHeader,
> +                     std::vector<XRayRecord> &Records);
> +Error YAMLLogLoader(StringRef Data, XRayFileHeader &FileHeader,
> +                    std::vector<XRayRecord> &Records);
> +
> +} // namespace xray
> +} // namespace llvm
> +
> +#endif // LLVM_TOOLS_LLVM_XRAY_XRAY_LOG_READER_H
> 
> Added: llvm/trunk/tools/llvm-xray/xray-record-yaml.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/xray-record-yaml.h?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/xray-record-yaml.h (added)
> +++ llvm/trunk/tools/llvm-xray/xray-record-yaml.h Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,102 @@
> +//===- xray-record-yaml.h - XRay Record YAML Support Definitions ----------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// Types and traits specialisations for YAML I/O of XRay log entries.
> +//
> +//===----------------------------------------------------------------------===//
> +#ifndef LLVM_TOOLS_LLVM_XRAY_XRAY_RECORD_YAML_H
> +#define LLVM_TOOLS_LLVM_XRAY_XRAY_RECORD_YAML_H
> +
> +#include <type_traits>
> +
> +#include "xray-record.h"
> +#include "llvm/Support/YAMLTraits.h"
> +
> +namespace llvm {
> +namespace xray {
> +
> +struct YAMLXRayFileHeader {
> +  uint16_t Version;
> +  uint16_t Type;
> +  bool ConstantTSC;
> +  bool NonstopTSC;
> +  uint64_t CycleFrequency;
> +};
> +
> +struct YAMLXRayRecord {
> +  uint16_t RecordType;
> +  uint8_t CPU;
> +  RecordTypes Type;
> +  int32_t FuncId;
> +  std::string Function;
> +  uint64_t TSC;
> +  uint32_t TId;
> +};
> +
> +struct YAMLXRayTrace {
> +  YAMLXRayFileHeader Header;
> +  std::vector<YAMLXRayRecord> Records;
> +};
> +
> +using XRayRecordStorage =
> +    std::aligned_storage<sizeof(XRayRecord), alignof(XRayRecord)>::type;
> +
> +} // namespace xray
> +
> +namespace yaml {
> +
> +// YAML Traits
> +// -----------
> +template <> struct ScalarEnumerationTraits<xray::RecordTypes> {
> +  static void enumeration(IO &IO, xray::RecordTypes &Type) {
> +    IO.enumCase(Type, "function-enter", xray::RecordTypes::ENTER);
> +    IO.enumCase(Type, "function-exit", xray::RecordTypes::EXIT);
> +  }
> +};
> +
> +template <> struct MappingTraits<xray::YAMLXRayFileHeader> {
> +  static void mapping(IO &IO, xray::YAMLXRayFileHeader &Header) {
> +    IO.mapRequired("version", Header.Version);
> +    IO.mapRequired("type", Header.Type);
> +    IO.mapRequired("constant-tsc", Header.ConstantTSC);
> +    IO.mapRequired("nonstop-tsc", Header.NonstopTSC);
> +    IO.mapRequired("cycle-frequency", Header.CycleFrequency);
> +  }
> +};
> +
> +template <> struct MappingTraits<xray::YAMLXRayRecord> {
> +  static void mapping(IO &IO, xray::YAMLXRayRecord &Record) {
> +    // FIXME: Make this type actually be descriptive
> +    IO.mapRequired("type", Record.RecordType);
> +    IO.mapRequired("func-id", Record.FuncId);
> +    IO.mapOptional("function", Record.Function);
> +    IO.mapRequired("cpu", Record.CPU);
> +    IO.mapRequired("thread", Record.TId);
> +    IO.mapRequired("kind", Record.Type);
> +    IO.mapRequired("tsc", Record.TSC);
> +  }
> +
> +  static constexpr bool flow = true;
> +};
> +
> +template <> struct MappingTraits<xray::YAMLXRayTrace> {
> +  static void mapping(IO &IO, xray::YAMLXRayTrace &Trace) {
> +    // A trace file contains two parts, the header and the list of all the
> +    // trace records.
> +    IO.mapRequired("header", Trace.Header);
> +    IO.mapRequired("records", Trace.Records);
> +  }
> +};
> +
> +} // namespace yaml
> +} // namespace llvm
> +
> +LLVM_YAML_IS_SEQUENCE_VECTOR(xray::YAMLXRayRecord)
> +
> +#endif // LLVM_TOOLS_LLVM_XRAY_XRAY_RECORD_YAML_H
> 
> Added: llvm/trunk/tools/llvm-xray/xray-record.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-xray/xray-record.h?rev=291529&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-xray/xray-record.h (added)
> +++ llvm/trunk/tools/llvm-xray/xray-record.h Mon Jan  9 20:38:11 2017
> @@ -0,0 +1,55 @@
> +//===- xray-record.h - XRay Trace Record ----------------------------------===//
> +//
> +//                     The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file replicates the record definition for XRay log entries. This should
> +// follow the evolution of the log record versions supported in the compiler-rt
> +// xray project.
> +//
> +//===----------------------------------------------------------------------===//
> +#ifndef LLVM_TOOLS_LLVM_XRAY_XRAY_RECORD_H
> +#define LLVM_TOOLS_LLVM_XRAY_XRAY_RECORD_H
> +
> +#include <cstdint>
> +
> +namespace llvm {
> +namespace xray {
> +
> +struct XRayFileHeader {
> +  uint16_t Version = 0;
> +  uint16_t Type = 0;
> +  bool ConstantTSC;
> +  bool NonstopTSC;
> +  uint64_t CycleFrequency = 0;
> +};
> +
> +enum class RecordTypes { ENTER, EXIT };
> +
> +struct XRayRecord {
> +  uint16_t RecordType;
> +
> +  // The CPU where the thread is running. We assume number of CPUs <= 256.
> +  uint8_t CPU;
> +
> +  // Identifies the type of record.
> +  RecordTypes Type;
> +
> +  // The function ID for the record.
> +  int32_t FuncId;
> +
> +  // Get the full 8 bytes of the TSC when we get the log record.
> +  uint64_t TSC;
> +
> +  // The thread ID for the currently running thread.
> +  uint32_t TId;
> +};
> +
> +} // namespace xray
> +} // namespace llvm
> +
> +#endif // LLVM_TOOLS_LLVM_XRAY_XRAY_RECORD_H
> 
> 
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 163 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170214/7cd579cc/attachment.sig>


More information about the llvm-commits mailing list