[lldb-dev] LLDB OS X buildbot/testbot details

[Todd Fiala via lldb-dev]

Hi all,

The llvm.org Green Dragon (i.e. Jenkins-based) LLDB OS X buildbot/testbot
has received some improvements today.  The Jenkins build now uses the xUnit
plugin to process xUnit-based test suite results, which are displayed more
usefully on the "build and test" page here
<http://lab.llvm.org:8080/green/job/lldb_build_test/>.

A few notes on some relevant details:

* The summary page allows you to click the test details and see the
pass/fail and timing info for each of the tests

* If the test is in failure/error, the backtrace will show up in the
details for the test method when you drill through the test results.

* The test summary information that you would normally see in the output of
a local test run is available in the bottom of the "Console Output" page
for a given build.  This can be useful  as xUnit/JUnit only really has the
concept of a pass/fail/error, but we also have timeouts, unexpected
successes, exceptional exits, etc. that all have to get mapped to the JUnit
pass/fail/error model.  The JUnit test method run details try to capture
this, but I find it useful from time to time to go to the consule output to
see the "real" results.  (Rerun info is only available on the Console
Output).

* The Jenkins jobs are set up in a 2-level job/sub-job structure.  The link
I sent above is the most useful one to look at because it does the build
and test phase, and that covers 95% of what you want to look at.  BUT, if
you want to check what code is actually contained in the build and test
run, you need to go to the parent job (will be listed under here
<http://lab.llvm.org:8080/green/job/LLDB/>, which lists the main job), then
find its first sub-job called "Acquire Sources".  That will tell you which
changes were synched for this job.  (I may do something about this in the
future since the workflow on this seems suboptimal, but this is how it is
right now and is similar for other LLVM projects on Jenkins).

* Right now the gtests are built, but not run.  I'll be addressing this as
soon as Zachary and I straighten out a failing gtest on OS X.  Those test
results will just show up in the Console Output for the build and test
phase, and will fail the build if they either fail to build or have a test
run failure.

* Currently the equivalent of the python test session directory is not
collected and archived.  I plan to remedy that in the future.  I currently
*do* archive the JUnit.xml output file from the XunitResultsFormatter
output.

Let me know if you have any questions or feedback.

Thanks!
-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160122/c062fa5c/attachment.html>