[all-commits] [llvm/llvm-project] 962339: [lit] Reliable progress indicator and ETA

[Roman Lebedev via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 962339a5eca2c838cb0a3dae6814d7942ccd8ce1
      https://github.com/llvm/llvm-project/commit/962339a5eca2c838cb0a3dae6814d7942ccd8ce1
  Author: Roman Lebedev <lebedev.ri at gmail.com>
  Date:   2021-03-23 (Tue, 23 Mar 2021)

  Changed paths:
    M llvm/utils/lit/lit/ProgressBar.py
    M llvm/utils/lit/lit/display.py
    M llvm/utils/lit/lit/main.py

  Log Message:
  -----------
  [lit] Reliable progress indicator and ETA

Quality of progress bar and ETA in lit has always bothered me.

For example, given `./bin/llvm-lit /repositories/llvm-project/clang/test/CodeGen* -sv`
at 1%, it says it will take 10 more minutes,
at 25%, it says it will take 1.25 more minutes,
at 50%, it says it will take 30 more seconds,
and in the end finishes with `Testing Time: 39.49s`. That's rather wildly unprecise.

Currently, it assumes that every single test will take the same amount of time to run on average.
This is is a somewhat reasonable approximation overall, but it is quite clearly imprecise,
especially in the beginning.

But, we can do better now, after D98179! We now know how long the tests took to run last time.
So we can build a better ETA predictor, by accumulating the time spent already,
the time that will be spent on the tests for which we know the previous time,
and for the test for which we don't have previous time, again use the average time
over the tests for which we know current or previous run time.
It would be better to use median, but i'm wary of the cost that may incur.

Now, on **first** run of `./bin/llvm-lit /repositories/llvm-project/clang/test/CodeGen* -sv`
at 10%, it says it will take 30 seconds,
at 25%, it says it will take 50 more seconds,
at 50%, it says it will take 27 more seconds,
and in the end finishes with `Testing Time: 41.64s`. That's pretty reasonable.

And on second run of `./bin/llvm-lit /repositories/llvm-project/clang/test/CodeGen* -sv`
at 1%, it says it will take 1 minutes,
at 25%, it says it will take 30 more seconds,
at 50%, it says it will take 19 more seconds,
and in the end finishes with `Testing Time: 39.49s`. That's amazing i think!

I think people will love this :)

Reviewed By: yln

Differential Revision: https://reviews.llvm.org/D99073