[Lldb-commits] [PATCH] D12416: Parse dotest.py/dosep.py output, providing general stats and skip reason breakdown by counts.

[Todd Fiala via lldb-commits]

tfiala added a comment.

In http://reviews.llvm.org/D12416#235112, @labath wrote:

> Same question as Zachary. This sounds like a very useful feature and it would be nice to have it integrated into the current test system,


Well, I'd be happy to do that in dosep.py.  There are a couple of challenges:

1. It requires information that is only provided when parsable output mode is invoked in dotest.py.  So it mandates a particular output style that is not necessarily what everyone wants to see.

2. Does everyone always want this output from dosep.py?  I agree with Zachary's earlier comment that having a bunch of options is more painful than not having them, particularly for things that everyone wants.  But this can be verbose.



> instead of making another layer on top of that (it's bad enough we have dosep and dotest already).


The existence of those two is at least partly due to the organic nature in which they developed.  Some energy could be put into that to wrap them together unless dosep.py has grown wildly since I last touched it.

> Parsing the output like this is likely to break the script due to random changes in the other parts.


That probably cuts both ways.  Changing output of a test script (the output I'm adding) seems quite plausible to break other scripts that parse the output of the test infrastructure.

All that aside, if we did want to move this more firmly into dotest/dosep, it currently requires the data to be generated from dotest and collated in dosep.  So it could change like so:

1. Dosep.py could be modified to collect the skip reason data if it detects its presence, and can collate and display if the skip reason info is there.

2. If the data isn't present, dosep.py can just skip presenting the skip reason report piece.

3. We can add a skip reason report suppression flag that skips printing out the skip reason in dosep.py if for some reason somebody doesn't want to see it.  By default it would print skip reason tabulations when available unless this flag was specified.  I see that as an optional task pending anybody really wanting/needing it.

Then the user requirement becomes "add so and such an option to your dotest.py invocation options and you'll get skip counts in your dosep.py output."

Seem reasonable?


http://reviews.llvm.org/D12416