[libcxx-commits] [libcxx] [libc++] Add a tool to find when a benchmark changed (PR #233656)

[Louis Dionne via libcxx-commits]

ldionne wrote:

> We're trying to find where the result of a benchmark changes, right? Not the benchmark itself.

The output of the script is something like this:

```
{"after_x": "f08df56d3af3e67ab39ebbfb62be3ed3f86d0c9d", "after_y": 1500.0235982796937, "before_x": "3d7622ea0bd443bb6ccb58d6b33e8cf52a8f0f4e", "before_y": 1048.1198905913648, "change": 0.43115650389323135, "p_value": 0.004975124378109453}
```

That tells you the commits on both sides of the change and their value. You then do this (in parallel) for all the benchmarks.

The main question is that I wonder whether we should have a LLM drive this script directly or if we can add another layer of "deterministic" processing on top to aggregate results, and then have a LLM drive that.

https://github.com/llvm/llvm-project/pull/233656