[PATCH][LNT] Add ability to filter machines on daily report page + refactor LNT regression tests to enable properly testing it.

[Chris Matthews]

I completely agree with this testing strategy. This is a much better way to test.

I’m fine with an explicit machine list, but I wonder if a more useful way to do this is to have a machine name filter? That way queries like "?filter-machine=AArch64&” could be possible along with an explicit "?filter-machine=lvm-juno-lnt-perf__LNT-AArch64-A53-O3__clang_DEV__aarch64&”.  Just substring matching on the machine name.  That would make it very easy to grab classes of machines.

Please add period to end of comment on patch 6 line 46.

Could you update the command line daily report tool (lnt/lnttool/main.py:304) to also have this functioanltiy? I think the change should be really simple.

Otherwise looks good!

> On May 26, 2015, at 3:41 AM, Kristof Beyls <kristof.beyls at arm.com> wrote:
> 
> Hi Chris,
> 
> I'd like to be able to filter on the daily report page, e.g.
> http://llvm.org/perf/db_default/v4/nts/daily_report/2015/5/21?num_days=7, so
> that it only shows data for a subset of the builders. The previous link
> shows
> many results coming from the penryn builders, which make it hard to just
> investigate the results coming from the juno board.
> 
> 0006-Enable-filtering-of-machines-shown-on-LNT-daily-repo.patch adds support
> for this; the user visible behaviour I went for is adding
> machines={comma-separated list of machine names} to the query string of the
> url, e.g.
> http://llvm.org/perf/db_default/v4/nts/daily_report/2015/5/21?num_days=7&mac
> hines=llvm-juno-lnt-perf__LNT-AArch64-A53-O3__clang_DEV__aarch64.
> 
> To be able to add a proper regression test/unit test for this, I also had to
> refactor the LNT regression tests a bit, which is what the other patches
> contain. In a number of steps, they transform the tests using
> tests/SharedInputs/SmallInstance/lnt.db to using a database constructed from
> human-readable SQL statements, creating the sqlite file on the fly during
> test
> execution. This enables adding per-test records to the database:
> 
> - 0001-For-the-regression-tests-make-temporary-LNT-db-insta.patch
> - 0002-Replace-binary-test-lnt.db-with-human-readable-text-.patch
> - 0003-replace-hexadecimal-encoding-of-blobs-with-textual-r.patch
> - 0004-remove-most-of-the-unnecessary-insert-statement-from.patch
> - 0005-Enable-adding-test-specific-records-to-test-database.patch
> 
> My hope is that this refactoring enables creating new functionality for the
> LNT
> ui in a more Test-Driven Development way, which should make it easier for
> more
> people to contribute to LNT & to reduce the likelihood of regressions in
> LNT.
> 
> What do you think?
> Is the refactoring of the regression tests a step in the right direction?
> Any objections to adding the machine= filter to the daily report page?
> 
> Thanks,
> 
> Kristof
> <0001-For-the-regression-tests-make-temporary-LNT-db-insta.patch><0002-Replace-binary-test-lnt.db-with-human-readable-text-.patch><0003-replace-hexadecimal-encoding-of-blobs-with-textual-r.patch><0004-remove-most-of-the-unnecessary-insert-statement-from.patch><0005-Enable-adding-test-specific-records-to-test-database.patch><0006-Enable-filtering-of-machines-shown-on-LNT-daily-repo.patch>