[PATCH] D16318: [LNT] Add a "number of test results" table to the daily report page.

[David Blaikie via llvm-commits]

On Tue, Jan 19, 2016 at 6:34 AM, Kristof Beyls via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> kristof.beyls created this revision.
> kristof.beyls added a reviewer: cmatthews.
> kristof.beyls added a subscriber: llvm-commits.
>
> We've found it useful to have a table on the daily report page which
> reports for each
> machine how many tests were seen on a given day. It has helped us to
> quickly notice
> when a certain machine started producing fewer test/benchmark results for
> some reason.
>
> AFAIK, there isn't another way to quickly notice if a particular machine
> starts producing
> fewer-than-expected benchmark results.
>

I imagine this metric would be confused if there was a day with relatively
few commits, no? (in that case you wouldn't need to run benchmarks because
you'd be up to date)

Perhaps a better/alternative metric would be "commits per benchmark run"?
(this has the opposite effect sometimes, of course - if commit rate spikes
it'll look like the bot slowed down - so perhaps some metric about commit
rate?) This would help catch the underlying concern (or what I assume is
the underlying concern) - when data is insufficiently granular for good
analysis.

(another alternative might be time taken for a report - this will fluctuate
if many new tests are added to the repository, but would otherwise be
independent of commit rate (either too high, or too low))


>
> http://reviews.llvm.org/D16318
>
> Files:
>   lnt/server/reporting/analysis.py
>   lnt/server/reporting/dailyreport.py
>   lnt/server/ui/templates/reporting/daily_report.html
>   tests/server/ui/V4Pages.py
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160119/4a63da80/attachment.html>