[all-commits] [llvm/llvm-project] a94fa8: [CSSPGO][llvm-profgen] Parse mmap events from perf...

Fri Nov 20 14:27:12 PST 2020

  Branch: refs/heads/master
  Home:   https://github.com/llvm/llvm-project
  Commit: a94fa8622971fa800d8f82c05b57117ff0372897
      https://github.com/llvm/llvm-project/commit/a94fa8622971fa800d8f82c05b57117ff0372897
  Author: wlei <wlei at fb.com>
  Date:   2020-11-20 (Fri, 20 Nov 2020)

  Changed paths:
    A llvm/docs/CommandGuide/llvm-profgen.rst
    A llvm/test/tools/llvm-profgen/lit.local.cfg
    A llvm/test/tools/llvm-profgen/mmapEvent.test
    A llvm/tools/llvm-profgen/CMakeLists.txt
    A llvm/tools/llvm-profgen/ErrorHandling.h
    A llvm/tools/llvm-profgen/LLVMBuild.txt
    A llvm/tools/llvm-profgen/PerfReader.cpp
    A llvm/tools/llvm-profgen/PerfReader.h
    A llvm/tools/llvm-profgen/ProfiledBinary.h
    A llvm/tools/llvm-profgen/llvm-profgen.cpp

  Log Message:
  -----------
  [CSSPGO][llvm-profgen] Parse mmap events from perf script

This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

As a starter, this change sets up an entry point by introducing PerfReader to load profiled binaries and perf traces(including perf events and perf samples). For the event, here it parses the mmap2 events from perf script to build the loader snaps, which is used to retrieve the image load address in the subsequent perf tracing parsing.

As described in llvm-profgen.rst, the tool being built aims to support multiple input perf data (preprocessed by perf script) as well as multiple input binary images. It should also support dynamic reload/unload shared objects by leveraging the loader snaps being built by this change

Reviewed By: wenlei, wmi

Differential Revision: https://reviews.llvm.org/D89707

  Commit: 32221694cb927b6acc7a8e16af7155e4e31418a4
      https://github.com/llvm/llvm-project/commit/32221694cb927b6acc7a8e16af7155e4e31418a4
  Author: wlei <wlei at fb.com>
  Date:   2020-11-20 (Fri, 20 Nov 2020)

  Changed paths:
    M llvm/docs/CommandGuide/llvm-profgen.rst
    A llvm/test/tools/llvm-profgen/disassemble.s
    M llvm/test/tools/llvm-profgen/lit.local.cfg
    M llvm/test/tools/llvm-profgen/mmapEvent.test
    M llvm/tools/llvm-profgen/CMakeLists.txt
    M llvm/tools/llvm-profgen/ErrorHandling.h
    M llvm/tools/llvm-profgen/LLVMBuild.txt
    M llvm/tools/llvm-profgen/PerfReader.cpp
    A llvm/tools/llvm-profgen/ProfiledBinary.cpp
    M llvm/tools/llvm-profgen/ProfiledBinary.h
    M llvm/tools/llvm-profgen/llvm-profgen.cpp

  Log Message:
  -----------
  [CSSPGO][llvm-profgen] Disassemble text sections

This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

This change enables disassembling the text sections to build various address maps that are potentially used by the virtual unwinder.  A switch `--show-disassembly` is being added to print the disassembly code.

Like the llvm-objdump tool, this change leverages existing LLVM components to parse and disassemble ELF binary files. So far X86 is supported.

Test Plan:

ninja check-llvm

Reviewed By: wmi, wenlei

Differential Revision: https://reviews.llvm.org/D89712

  Commit: 0196b45ceaf8784eae058e6af4fd943f16a2d071
      https://github.com/llvm/llvm-project/commit/0196b45ceaf8784eae058e6af4fd943f16a2d071
  Author: wlei <wlei at fb.com>
  Date:   2020-11-20 (Fri, 20 Nov 2020)

  Changed paths:
    A llvm/test/tools/llvm-profgen/symbolize.ll
    M llvm/tools/llvm-profgen/CMakeLists.txt
    A llvm/tools/llvm-profgen/CallContext.h
    M llvm/tools/llvm-profgen/ProfiledBinary.cpp
    M llvm/tools/llvm-profgen/ProfiledBinary.h

  Log Message:
  -----------
  [CSSPGO][llvm-profgen] Instruction symbolization

This stack of changes introduces `llvm-profgen` utility which generates a profile data file from given perf script data files for sample-based PGO. It’s part of(not only) the CSSPGO work. Specifically to support context-sensitive with/without pseudo probe profile, it implements a series of functionalities including perf trace parsing, instruction symbolization, LBR stack/call frame stack unwinding, pseudo probe decoding, etc. Also high throughput is achieved by multiple levels of sample aggregation and compatible format with one stop is generated at the end. Please refer to: https://groups.google.com/g/llvm-dev/c/1p1rdYbL93s for the CSSPGO RFC.

This change adds the support of instruction symbolization. Given the RVA on an instruction pointer, a full calling context can be printed side-by-side with the disassembly code.
E.g.
```
 Disassembly of section .text [0x0, 0x4a]:

 <funcA>:
     0:	mov	eax, edi                           funcA:0
     2:	mov	ecx, dword ptr [rip]               funcLeaf:2 @ funcA:1
     8:	lea	edx, [rcx + 3]                     fib:2 @ funcLeaf:2 @ funcA:1
     b:	cmp	ecx, 3                             fib:2 @ funcLeaf:2 @ funcA:1
     e:	cmovl	edx, ecx                           fib:2 @ funcLeaf:2 @ funcA:1
    11:	sub	eax, edx                           funcLeaf:2 @ funcA:1
    13:	ret                                        funcA:2
    14:	nop	word ptr cs:[rax + rax]
    1e:	nop

 <funcLeaf>:
    20:	mov	eax, edi                           funcLeaf:1
    22:	mov	ecx, dword ptr [rip]               funcLeaf:2
    28:	lea	edx, [rcx + 3]                     fib:2 @ funcLeaf:2
    2b:	cmp	ecx, 3                             fib:2 @ funcLeaf:2
    2e:	cmovl	edx, ecx                           fib:2 @ funcLeaf:2
    31:	sub	eax, edx                           funcLeaf:2
    33:	ret                                        funcLeaf:3
    34:	nop	word ptr cs:[rax + rax]
    3e:	nop

 <fib>:
    40:	lea	eax, [rdi + 3]                     fib:2
    43:	cmp	edi, 3                             fib:2
    46:	cmovl	eax, edi                           fib:2
    49:	ret                                        fib:8
```

Test Plan:
ninja check-llvm

Reviewed By: wenlei, wmi

Differential Revision: https://reviews.llvm.org/D89715

Compare: https://github.com/llvm/llvm-project/compare/c58df88886e4...0196b45ceaf8