[lldb-dev] FreeBSD testrun snapshot

[Ed Maste]

After the recent discussions about buildbots I thought it would be
prudent to document the current state of the tests on FreeBSD.

First, results from my local machine,

FreeBSD 10.1-RC2 #14 r272876+6a195a0(stable-10)
LLVM: r222430
Clang: r222429
LLDB: r222372

One test fails (and has no PR yet), when invoked from "ninja check-lldb":

FAIL: LLDB (suite) :: TestValueVarUpdate.py (FreeBSD feynman 10.1-RC2
FreeBSD 10.1-RC2 #14 r272876+6a195a0(stable-10): Mon Oct 20 09:23:12
EDT 2014     emaste at feynman:/tank/emaste/obj/tank/emaste/src/git-stable-10/sys/GENERIC
amd64 amd64)

FAIL: test_with_dwarf_and_process_launch_api
(TestValueVarUpdate.HelloWorldTestCase)
   Test SBValue::GetValueDidChange
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tank/emaste/src/llvm/tools/lldb/test/lldbtest.py", line 331, in wrapper
    return func(self, *args, **kwargs)
  File "/tank/emaste/src/llvm/tools/lldb/test/lldbtest.py", line 382, in wrapper
    return func(self, *args, **kwargs)
  File "/tank/emaste/src/llvm/tools/lldb/test/python_api/value_var_update/TestValueVarUpdate.py",
line 28, in test_with_dwarf_and_process_launch_api
    self.do_test()
  File "/tank/emaste/src/llvm/tools/lldb/test/python_api/value_var_update/TestValueVarUpdate.py",
line 61, in do_test
    self.assertTrue(i.GetValueDidChange(), "GetValueDidChange() is
saying a lie")
AssertionError: False is not True : GetValueDidChange() is saying a lie
Config=x86_64-/usr/bin/clang

The two assertions here are:

        self.assertTrue(i_val != i.GetValueAsUnsigned(0), "GetValue()
is saying a lie")
        self.assertTrue(i.GetValueDidChange(), "GetValueDidChange() is
saying a lie")

which is a bit odd: GetValueAsUnsigned shows that the variable's value
has been updated, but that's not reflected in GetValueDidChange. It
fails both with parallel tests (8 CPUs) and without parallelism.
However, if I invoke this test in isolation (though python dotest.py
-f ...) it passes.

This test is also the only failure I see on the FreeBSD-hosted
buildbot, http://llvm-amd64.freebsd.your.org/b/buildslaves/freebsd-amd64

The same test fails on the LLVM-hosted FreeBSD buildbot,
http://lab.llvm.org:8011/builders/lldb-x86_64-freebsd, along with five
more:

FAIL: LLDB (suite) :: TestCompletion.py (FreeBSD dellpoweredge4
10.1-BETA3 FreeBSD 10.1-BETA3 #0 r272190: Sat Sep 27 22:07:29 PDT 2014
    master at dellpoweredge4:/usr/obj/usr/src/sys/GENERIC amd64 amd64)
FAIL: LLDB (suite) :: TestConcurrentEvents.py (FreeBSD dellpoweredge4
10.1-BETA3 FreeBSD 10.1-BETA3 #0 r272190: Sat Sep 27 22:07:29 PDT 2014
    master at dellpoweredge4:/usr/obj/usr/src/sys/GENERIC amd64 amd64)
FAIL: LLDB (suite) :: TestSharedLib.py (FreeBSD dellpoweredge4
10.1-BETA3 FreeBSD 10.1-BETA3 #0 r272190: Sat Sep 27 22:07:29 PDT 2014
    master at dellpoweredge4:/usr/obj/usr/src/sys/GENERIC amd64 amd64)
FAIL: LLDB (suite) :: TestSharedLibStrippedSymbols.py (FreeBSD
dellpoweredge4 10.1-BETA3 FreeBSD 10.1-BETA3 #0 r272190: Sat Sep 27
22:07:29 PDT 2014
master at dellpoweredge4:/usr/obj/usr/src/sys/GENERIC amd64 amd64)
FAIL: LLDB (suite) :: TestStaticVariables.py (FreeBSD dellpoweredge4
10.1-BETA3 FreeBSD 10.1-BETA3 #0 r272190: Sat Sep 27 22:07:29 PDT 2014
    master at dellpoweredge4:/usr/obj/usr/src/sys/GENERIC amd64 amd64)

Also, I found five core files from test executables after a test run:

test/api/multiple-debuggers/multi-process-drive.core
test/api/multiple-debuggers/testprog.core
test/functionalities/attach_resume/AttachResume.core
test/functionalities/thread/state/a.out.core
test/driver/batch_mode/a.out.core