[llvm] r291529 - [XRay] Implement `llvm-xray convert` -- trace file conversion

[Dimitry Andric via llvm-commits]

On 14 Feb 2017, at 21:40, Dean Michael Berris <dberris at google.com> wrote:
> 
> On Tue, Feb 14, 2017 at 12:33 PM Dimitry Andric <dimitry at andric.com> wrote:
> Hi Dean,
> 
> I have been trying to figure out one llvm-xray test failure that occurs while testing the release_40 branch (but this same failure also shows up for me on trunk):
> 
> FAIL: LLVM :: tools/llvm-xray/X86/convert-with-debug-syms.txt (30283 of 37661)
> ******************** TEST 'LLVM :: tools/llvm-xray/X86/convert-with-debug-syms.txt' FAILED ********************
> Script:
> --
> /home/dim/obj/llvm-294803-rel40-freebsd12-i386-ninja-rel-1/./bin/llvm-xray convert -m /share/dim/src/llvm/release_40/test
> /tools/llvm-xray/X86/Inputs/elf64-sample-o2.bin -y /share/dim/src/llvm/release_40/test/tools/llvm-xray/X86/Inputs/naive-l
> og-simple.xray -f=yaml -o - 2>&1 | /home/dim/obj/llvm-294803-rel40-freebsd12-i386-ninja-rel-1/./bin/FileCheck /share/dim/
> src/llvm/release_40/test/tools/llvm-xray/X86/convert-with-debug-syms.txt
> --
> Exit Code: 1
> 
> Command Output (stderr):
> --
> /share/dim/src/llvm/release_40/test/tools/llvm-xray/X86/convert-with-debug-syms.txt:13:15: error: expected string not fou
> nd in input
> ; CHECK-NEXT: - { type: 0, func-id: 2, function: {{.*foo.*}}, cpu: 37, thread: 84697, kind: function-enter,
>               ^
> <stdin>:11:2: note: scanning from here
>  - { type: 0, func-id: 2, function: 'foo(void)', cpu: 37, thread: 84697,
>  ^
> <stdin>:19:2: note: possible intended match here
>  - { type: 0, func-id: 3, function: main, cpu: 37, thread: 84697, kind: function-exit,
>  ^
> 
> --
> 
> ********************
> 
> What happens here is that the test/tools/llvm-xray/X86/convert-with-debug-syms.txt test case has the following CHECK-NEXT lines:
> 
> ; CHECK-NEXT:   - { type: 0, func-id: 2, function: {{.*foo.*}}, cpu: 37, thread: 84697, kind: function-enter,
> ; CHECK-NEXT:       tsc: 3315356841454542 }
> 
> However, on my FreeBSD-based systems, the output for the 'foo' and 'bar' lines is formatted slightly differently, wrapping at the 'kind' key instead of the 'tsc' key:
> 
>   - { type: 0, func-id: 2, function: 'foo(void)', cpu: 37, thread: 84697,
>       kind: function-enter, tsc: 3315356841454542 }
> 
> Any idea what causes this?  Maybe the yaml outputter is retrieving the terminal size differently somehow, and thus wrapping it at another margin?
> 
> If so, maybe there is some way of ensuring the output either never wraps, or always wraps at the same margin?
> 
> 
> Interesting. I've not found this before. We're currently just using the defaults for the yaml printer. There could be a way to not pretty-print, but I haven't explored that specifically.

I was just reading include/llvm/Support/YAMLTraits.h, and there it has:

  Output(llvm::raw_ostream &, void *Ctxt = nullptr, int WrapColumn = 70);

and xray-converter.cc instantiates an Output object with only the raw_ostream specified, so it should indeed use the default WrapColumn.

However, when I just ran this on Ubuntu, it *didn't* wrap at 70 columns.  The output was this, instead:

---
header:
  version:         1
  type:            0
  constant-tsc:    true
  nonstop-tsc:     true
  cycle-frequency: 2601000000
records:
  - { type: 0, func-id: 3, function: main, cpu: 37, thread: 84697, kind: function-enter,
      tsc: 3315356841453914 }
  - { type: 0, func-id: 2, function: 'foo()', cpu: 37, thread: 84697, kind: function-enter,
      tsc: 3315356841454542 }
  - { type: 0, func-id: 2, function: 'foo()', cpu: 37, thread: 84697, kind: function-exit,
      tsc: 3315356841454670 }
  - { type: 0, func-id: 1, function: 'bar()', cpu: 37, thread: 84697, kind: function-enter,
      tsc: 3315356841454762 }
  - { type: 0, func-id: 1, function: 'bar()', cpu: 37, thread: 84697, kind: function-exit,
      tsc: 3315356841454802 }
  - { type: 0, func-id: 3, function: main, cpu: 37, thread: 84697, kind: function-exit,
      tsc: 3315356841494828 }
...

It turns out this is apparently influenced by the LANG setting, which is en_US.UTF-8 on Ubuntu, and then the column counting in the YAML Output traits don't work as expected?  With LANG=C, it properly wraps at 70 chars.

I'm not completely sure what is going on here, the output() method looks like this:

void Output::output(StringRef s) {
  Column += s.size();
  Out << s;
}

but does StringRef::size() return something different for UTF-8 data?


> Can you file a bug on this so that I don't forget to have a look? If you have time to actually try and fix it, I'd be more than happy to review a patch for it.

Sure, when I understand what the YAML Outputter and/or StringRef are doing, I will submit a review for you.

-Dimitry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 163 bytes
Desc: Message signed with OpenPGP
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170214/0beab685/attachment.sig>