[lldb-dev] test results look typical?

[Todd Fiala via lldb-dev]

On Tue, Aug 25, 2015 at 4:22 PM, <dawn at burble.org> wrote:

> On Tue, Aug 25, 2015 at 04:41:34PM +0000, Zachary Turner wrote:
> > It would be great (and not too difficult) to add skip counts to dosep.  I
> > modified dotest so it formats the result summary in a nice single string
> > that you can regex match to get counts.  It's already matched in dosep,
> but
> > we just aren't pulling out the skip counts.  So it would be very easy to
> > add this.
>
> I would like to see totals from all the dotest.py's RESULTs counts:
>     RESULT: PASSED (4 passes, 0 failures, 0 errors, 0 skipped, 0 expected
> failures, 0 unexpected successes)
> as well as the timeouts from dosep.
>
> Of course, dosep needs to be fixed to count the test cases correctly
> first. :)
> It seems to miss the results from every 4th test suite or so.
>
>
I may dig into that if nobody beats me to it.  I did the original
multiprocessing work on dosep ~1.5 years ago and it may be doing something
goofy.  So far the results have been remarkably stable on the counts for me
over the last 2 days.


> > On Tue, Aug 25, 2015 at 7:41 AM Todd Fiala via lldb-dev <
> > lldb-dev at lists.llvm.org> wrote:
> > > For counting I will probably go back to my old method of parsing the
> > > output of a serial dotest run, since IIRC I can get skip counts
> accurately
> > > there as well.  (Or perhaps that should be added to dosep.py, it's
> been a
> > > while since I last heavily modified that script).
>
> You can get all the counts from running dosep by counting up the results
> from each dotest run.  Collect the output via:
>
>     ./dosep.py -s --options "-v --executable $BLDDIR/bin/lldb" 2>&1 | tee
> test_out.log
>
> Then your totals will be:
>
>     export passes=`grep -E "^RESULT: " test_out.log | sed 's/(//' | awk
> '{count+=$3} END {print count}'` || true
>     export failures=`grep -E "^RESULT:" test_out.log | awk '{count+=$5}
> END {print count}'` || true
>     export errors=`grep -E "^RESULT:" test_out.log | awk '{count+=$7} END
> {print count}'` || true
>     export skips=`grep -E "^RESULT:" test_out.log | awk '{count+=$9} END
> {print count}'` || true
>     [...]
>     export total=`grep -E "^Ran [0-9]+ tests? in" lldb_test_out.log | awk
> '{count+=$2} END {print count}'`
>
>
Great, thanks Dawn!

I'd like to get all the counts into dosep.py at least as an option, but
having something to cross check it with is good (and getting a quick answer
is nice as well, thanks.)
-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150825/4c7fb1c8/attachment.html>