[all-commits] [llvm/llvm-project] ded733: Add a tool for diffing size remarks

[Jessica Paquette via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: ded733bd49ff44cf94bd8a7a821ba37390dc94b1
      https://github.com/llvm/llvm-project/commit/ded733bd49ff44cf94bd8a7a821ba37390dc94b1
  Author: Jessica Paquette <jpaquette at apple.com>
  Date:   2022-02-02 (Wed, 02 Feb 2022)

  Changed paths:
    M llvm/test/CMakeLists.txt
    M llvm/test/lit.cfg.py
    A llvm/test/tools/llvm-remark-size-diff/Inputs/1-func-1-instr-1-stack.yaml
    A llvm/test/tools/llvm-remark-size-diff/Inputs/1-func-2-instr-2-stack.yaml
    A llvm/test/tools/llvm-remark-size-diff/Inputs/2-identical-func-1-instr-1-stack.yaml
    A llvm/test/tools/llvm-remark-size-diff/Inputs/empty-file.yaml
    A llvm/test/tools/llvm-remark-size-diff/Inputs/inconvertible-integer.yaml
    A llvm/test/tools/llvm-remark-size-diff/Inputs/no-instruction-count-remarks.yaml
    A llvm/test/tools/llvm-remark-size-diff/Inputs/unexpected-key.yaml
    A llvm/test/tools/llvm-remark-size-diff/add-remove-func.test
    A llvm/test/tools/llvm-remark-size-diff/empty-file.test
    A llvm/test/tools/llvm-remark-size-diff/inconvertible-integer.test
    A llvm/test/tools/llvm-remark-size-diff/increase-decrease-inst-count.test
    A llvm/test/tools/llvm-remark-size-diff/no-difference.test
    A llvm/test/tools/llvm-remark-size-diff/no-instruction-count-remarks.test
    A llvm/test/tools/llvm-remark-size-diff/unexpected-key.test
    A llvm/tools/llvm-remark-size-diff/CMakeLists.txt
    A llvm/tools/llvm-remark-size-diff/RemarkSizeDiff.cpp

  Log Message:
  -----------
  Add a tool for diffing size remarks

This is a tool which can handle bitstream and YAML remarks. The idea here is to
provide more insight into which functions changed in a benchmark when testing
compiler changes.

E.g. "foo got 20% bigger, so maybe we should look more closely at that."

To use the tool, you can use...

```
$ llvm-remark-size-diff remarks_file_a remarks_file_b --parser=yaml|bitstream
```

... on two remarks files containing at least instruction count remarks. This
will output some data on instruction count change and also other relevant
information such as stack size change from `remarks_file_a` to `remarks_file_b`.

This is a bit of a WIP so I'm happy to change the format etc. Ultimately I think
it'd be best to have some JSON output which could be consumed by another tool.
But some base-level, greppable output is very handy to have anyway.

The format I'm proposing here is

```
<files> <inc/dec in inst count> <fn name> <inst count change> <stack B change>
```

Where the files and increase/decrease are indicated like below:

- `<files>` is one of `++` (file B), `--` (file A), `==` (both)
- `<inc/dec in inst count>` is one of `>` (increase) or `<` (decrease)

This makes it easy to grep for things like "which functions appeared in A but
did not appear in B?" Or "what are all the instruction count decreases?"

Differential Revision: https://reviews.llvm.org/D112940