[PATCH] D99073: [lit] Reliable progress indicator and ETA

[Roman Lebedev via Phabricator via llvm-commits]

lebedev.ri created this revision.
lebedev.ri added reviewers: davezarzycki, jhenderson, jmorse, yln, mehdi_amini.
lebedev.ri added a project: LLVM.
Herald added a subscriber: delcypher.
lebedev.ri requested review of this revision.

Quality of progress bar and ETA in lit has always bothered me.

For example, given `./bin/llvm-lit /repositories/llvm-project/clang/test/CodeGen* -sv`
at 1%, it says it will take 10 more minutes,
at 25%, it says it will take 1.25 more minutes,
at 50%, it says it will take 30 more seconds,
and in the end finishes with `Testing Time: 39.49s`. That's rather wildly unprecise.

Currently, it assumes that every single test will take the same amount of time to run on average.
This is is a somewhat reasonable approximation overall, but it is quite clearly imprecise,
especially in the beginning.

But, we can do better now, after D98179 <https://reviews.llvm.org/D98179>! We now know how long the tests took to run last time.
So we can build a better ETA predictor, by accumulating the time spent already,
the time that will be spent on the tests for which we know the previous time,
and for the test for which we don't have previous time, again use the average time
over the tests for which we know current or previous run time.
It would be better to use median, but i'm wary of the cost that may incur.

Now, on **first** run of `./bin/llvm-lit /repositories/llvm-project/clang/test/CodeGen* -sv`
at 10%, it says it will take 30 seconds,
at 25%, it says it will take 50 more seconds,
at 50%, it says it will take 27 more seconds,
and in the end finishes with `Testing Time: 41.64s`. That's pretty reasonable.

And on second run of `./bin/llvm-lit /repositories/llvm-project/clang/test/CodeGen* -sv`
at 1%, it says it will take 1 minutes,
at 25%, it says it will take 30 more seconds,
at 50%, it says it will take 19 more seconds,
and in the end finishes with `Testing Time: 39.49s`. That's amazing i think!

I think people will love this :)

Note that currently i have not added any test coverage here.
I guess i could try, but i'm not really sure if this is testable..


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D99073

Files:
  llvm/utils/lit/lit/ProgressBar.py
  llvm/utils/lit/lit/display.py
  llvm/utils/lit/lit/main.py

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D99073.332260.patch
Type: text/x-patch
Size: 4694 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210322/84159e39/attachment.bin>