[Lldb-commits] [PATCH] D12416: Parse dotest.py/dosep.py output, providing general stats and skip reason breakdown by counts.

[Zachary Turner via lldb-commits]

On Fri, Aug 28, 2015 at 8:41 AM Todd Fiala <todd.fiala at gmail.com> wrote:

> tfiala added a comment.
>
> In http://reviews.llvm.org/D12416#235112, @labath wrote:
>
> > Same question as Zachary. This sounds like a very useful feature and it
> would be nice to have it integrated into the current test system,
>
>
> Well, I'd be happy to do that in dosep.py.  There are a couple of
> challenges:
>
> 1. It requires information that is only provided when parsable output mode
> is invoked in dotest.py.  So it mandates a particular output style that is
> not necessarily what everyone wants to see.
>
One solution to this that I've thought of is to have a way to launch dotest
in IPC mode.  Piping stdin and stdout is slow because it results in having
to parse text which was not printed with fast consumption in mind.  If
dosep could could launch dotest with an option like
--pipe=dotest_pipe_instance_1 or something, then they could communicate
over that pipe in a structured format like json that is easy to parse.



> > Parsing the output like this is likely to break the script due to random
> changes in the other parts.
>
>
> That probably cuts both ways.  Changing output of a test script (the
> output I'm adding) seems quite plausible to break other scripts that parse
> the output of the test infrastructure.
>
The pipe issue would solve that, but another way to at least alleviate the
pain associated with that is to agree that we print all summary information
at the end, and we try to keep the summary information as stable as
possible.  We still might have to change stuff sometimes, but the benefit
of doing things this way is that we can then agree that everything EXCEPT
the summary information can change for any reason at all, or no reason at
all, and it won't break anyone.  The summary informatino could be printed
in a nice structured format with 1 result on each line, so that new
information could simply be appended, and the only time you'd break
something is if you deleted a statistic.

Thoughts?  It might be more work to do things this way, but I kind of feel
like the complexity of dosep and dotest is getting to the point where it
might be worth considering putting some extra time in to think about things
like this.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-commits/attachments/20150828/36b0991e/attachment.html>