[all-commits] [llvm/llvm-project] 12c662: [BOLT][Utils] Add nfc-stat-parser.py (#71979)

[Amir Ayupov via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 12c66251ad1f64fef134fee3d0cff7fa8492d175
      https://github.com/llvm/llvm-project/commit/12c66251ad1f64fef134fee3d0cff7fa8492d175
  Author: Amir Ayupov <aaupov at fb.com>
  Date:   2023-11-11 (Sat, 11 Nov 2023)

  Changed paths:
    A bolt/utils/nfc-stat-parser.py

  Log Message:
  -----------
  [BOLT][Utils] Add nfc-stat-parser.py (#71979)

Add a utility to parse output from llvm-bolt-wrapper script and
detect individual and aggregate time/memory swings. The wrapper reports
wall time and peak RSS for each BOLT invocation.

Exit code:
The utility exits with non-zero exit code if any individual test has
time or memory swing larger than `threshold_single` (default 10%), or
the aggregate (geometric mean) swing larger than `threshold_agg`
(default 5%). Short tests where BOLT wall time is less than
`check_longer_than` seconds (default 0.5s) are excluded from threshold
calculation.

Output:
The script prints test results exceeding the individual threshold, and
geomean values if it exceeds aggregate results. In `--verbose` mode all
individual results are printed (short time results are marked with '?').

Example usage:
```
$ cd ~/llvm-build # build folder where NFC testing was invoked
$ python3 ~/llvm-project/bolt/utils/nfc-stat-parser.py \
  --check_longer_than 0.1 `find -name timing.log`
./tools/bolt/test/runtime/X86/exceptions-pic.test/ -88.46% -0.13%
Geomean -19.78% +0.37%
$ echo $?
1
```