[lldb-dev] Some API test failures are really opaque/could be improved
David Blaikie via lldb-dev
lldb-dev at lists.llvm.org
Sun Oct 17 20:54:29 PDT 2021
Wondering if anyone else has encountered/dealt with debugging lldb test
failures like the one shown at the end of this email ("AssertionError: 10
!= 5" in "test.assertEqual(process.GetState(), lldb.eStateStopped)" while
checking that a breakpoint was reached)
Is there anything that could be done to improve the debuggability of such
failures? Logging standard output/error from the lldb process or any other
logging it might have? At least for one of these I managed to figure it out
by running lldb directly on the binary and finding that the binary couldn't
find libc++.so (that's a story for another bug/email thread, looks like
maybe lldb libc++ pretty printer tests are using the system installed
libc++, not the just-built libc++ (& thus not running if there is no system
installed libc++)). But my current failure like this seems a bit more
inscrutible and I'm still looking into it.
Looks like dotest.py has some sense of logging (it has a --log-success
option which says preserves the logs even on failure - though the output of
dotest.py, at least for me, has no mention of logs, log files, or where
they might be located, and looking at the source points to some sort of
".log" files... ah, found some)
So, yeah, there do seem to be some Failure.log, SkippedTest.log, etc -
should dotest print something about the paths to those files when it exits
with failure, maybe?
********************
FAIL: lldb-api ::
functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py
(23 of 23)
******************** TEST 'lldb-api ::
functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py'
FAILED ********************
Script:
--
/usr/bin/python3
/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/dotest.py -u
CXXFLAGS -u CFLAGS --env ARCHIVER=/usr/bin/ar --env
OBJCOPY=/usr/bin/objcopy --env
LLVM_LIBS_DIR=/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
--arch x86_64 --build-dir
/usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex
--lldb-module-cache-dir
/usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-lldb/lldb-api
--clang-module-cache-dir
/usr/local/google/home/blaikie/dev/llvm/build/release/lldb-test-build.noindex/module-cache-clang/lldb-api
--executable
/usr/local/google/home/blaikie/dev/llvm/build/release/./bin/lldb --compiler
/usr/local/google/home/blaikie/dev/llvm/build/release/./bin/clang
--dsymutil
/usr/local/google/home/blaikie/dev/llvm/build/release/./bin/dsymutil
--llvm-tools-dir
/usr/local/google/home/blaikie/dev/llvm/build/release/./bin --lldb-libs-dir
/usr/local/google/home/blaikie/dev/llvm/build/release/./lib
/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set
-p TestDataFormatterLibcxxSet.py
--
Exit Code: 1
Command Output (stdout):
--
lldb version 14.0.0 (git at github.com:llvm/llvm-project.git revision
6176fda3f992b5086302b3826aa0636135cc4cc0)
clang revision 6176fda3f992b5086302b3826aa0636135cc4cc0
llvm revision 6176fda3f992b5086302b3826aa0636135cc4cc0
Skipping the following test categories: ['dsym', 'gmodules', 'debugserver',
'objc']
--
Command Output (stderr):
--
UNSUPPORTED: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_ref_and_ptr_dsym
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does
not fall in any category of interest for this run)
FAIL: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_ref_and_ptr_dwarf
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
FAIL: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_ref_and_ptr_dwo
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
UNSUPPORTED: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_ref_and_ptr_gmodules
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does
not fall in any category of interest for this run)
UNSUPPORTED: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_with_run_command_dsym
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does
not fall in any category of interest for this run)
FAIL: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_with_run_command_dwarf
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
FAIL: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_with_run_command_dwo
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
UNSUPPORTED: LLDB
(/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang-x86_64) ::
test_with_run_command_gmodules
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase) (test case does
not fall in any category of interest for this run)
======================================================================
FAIL: test_ref_and_ptr_dwarf
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
Test that the data formatters work on ref and ptr.
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py",
line 1823, in test_method
return attrvalue(self)
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py",
line 128, in test_ref_and_ptr
(self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint(
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 970, in run_to_source_breakpoint
return run_to_breakpoint_do_run(test, target, breakpoint, launch_info,
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 892, in run_to_breakpoint_do_run
test.assertEqual(process.GetState(), lldb.eStateStopped)
AssertionError: 10 != 5
Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang
======================================================================
FAIL: test_ref_and_ptr_dwo
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
Test that the data formatters work on ref and ptr.
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py",
line 1823, in test_method
return attrvalue(self)
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py",
line 128, in test_ref_and_ptr
(self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint(
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 970, in run_to_source_breakpoint
return run_to_breakpoint_do_run(test, target, breakpoint, launch_info,
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 892, in run_to_breakpoint_do_run
test.assertEqual(process.GetState(), lldb.eStateStopped)
AssertionError: 10 != 5
Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang
======================================================================
FAIL: test_with_run_command_dwarf
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
Test that that file and class static variables display correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py",
line 1823, in test_method
return attrvalue(self)
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py",
line 49, in test_with_run_command
(self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint(
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 970, in run_to_source_breakpoint
return run_to_breakpoint_do_run(test, target, breakpoint, launch_info,
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 892, in run_to_breakpoint_do_run
test.assertEqual(process.GetState(), lldb.eStateStopped)
AssertionError: 10 != 5
Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang
======================================================================
FAIL: test_with_run_command_dwo
(TestDataFormatterLibcxxSet.LibcxxSetDataFormatterTestCase)
Test that that file and class static variables display correctly.
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbtest.py",
line 1823, in test_method
return attrvalue(self)
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/test/API/functionalities/data-formatter/data-formatter-stl/libcxx/set/TestDataFormatterLibcxxSet.py",
line 49, in test_with_run_command
(self.target, process, _, bkpt) = lldbutil.run_to_source_breakpoint(
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 970, in run_to_source_breakpoint
return run_to_breakpoint_do_run(test, target, breakpoint, launch_info,
File
"/usr/local/google/home/blaikie/dev/llvm/src/lldb/packages/Python/lldbsuite/test/lldbutil.py",
line 892, in run_to_breakpoint_do_run
test.assertEqual(process.GetState(), lldb.eStateStopped)
AssertionError: 10 != 5
Config=x86_64-/usr/local/google/home/blaikie/dev/llvm/build/release/bin/clang
----------------------------------------------------------------------
Ran 8 tests in 3.035s
RESULT: FAILED (0 passes, 4 failures, 0 errors, 4 skipped, 0 expected
failures, 0 unexpected successes)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20211017/c88276b5/attachment-0001.html>
More information about the lldb-dev
mailing list