[lldb-dev] Running LLDB tests
Johnny Chen
johnny.chen at apple.com
Fri Jan 28 11:13:11 PST 2011
Hi Warren,
If you look in the '2011-01-28-12_17_51' directory, you should see some files explaining the detailed traces
which lead to the failure reporting. For example,
[10:04:58] johnny:/Volumes/data/lldb/svn/trunk/test $ ls -l 2011-01-28-09_57_01
total 64
-rw-r--r-- 1 johnny admin 3171 Jan 28 10:00 ExpectedFailure-TestNamespace.NamespaceTestCase.test_with_dsym_and_run_command.log
-rw-r--r-- 1 johnny admin 3129 Jan 28 10:00 ExpectedFailure-TestNamespace.NamespaceTestCase.test_with_dwarf_and_run_command.log
-rw-r--r-- 1 johnny admin 2173 Jan 28 09:59 ExpectedFailure-TestObjCMethods2.FoundationTestCase2.test_NSArray_expr_commands_with_dsym.log
-rw-r--r-- 1 johnny admin 2130 Jan 28 09:59 ExpectedFailure-TestObjCMethods2.FoundationTestCase2.test_NSArray_expr_commands_with_dwarf.log
-rw-r--r-- 1 johnny admin 2639 Jan 28 09:59 ExpectedFailure-TestObjCMethods2.FoundationTestCase2.test_NSString_expr_commands_with_dsym.log
-rw-r--r-- 1 johnny admin 2596 Jan 28 09:59 ExpectedFailure-TestObjCMethods2.FoundationTestCase2.test_NSString_expr_commands_with_dwarf.log
-rw-r--r-- 1 johnny admin 1492 Jan 28 09:58 Failure-TestClassTypes.ClassTypesTestCase.test_with_dsym_and_expr_parser.log
-rw-r--r-- 1 johnny admin 1450 Jan 28 09:58 Failure-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_expr_parser.log
[10:53:21] johnny:/Volumes/data/lldb/svn/trunk/test $ tail -20 2011-01-28-09_57_01/Failure-TestClassTypes.ClassTypesTestCase.test_with_dwarf_and_expr_parser.log
Expecting start string: Breakpoint created: 1: name = 'C', locations = 1
Not matched
FAIL
Traceback (most recent call last):
File "/Volumes/data/lldb/svn/trunk/test/class_types/TestClassTypes.py", line 53, in test_with_dwarf_and_expr_parser
self.class_types_expr_parser()
File "/Volumes/data/lldb/svn/trunk/test/class_types/TestClassTypes.py", line 165, in class_types_expr_parser
startstr = "Breakpoint created: 1: name = 'C', locations = 1")
File "/Volumes/data/lldb/svn/trunk/test/lldbtest.py", line 802, in expect
msg if msg else EXP_MSG(str, exe))
AssertionError: False is not True : Breakpoint created successfully
To rerun this test, issue the following command from the 'test' directory:
./dotest.py -A i386 -C clang -v -t -f ClassTypesTestCase.test_with_dwarf_and_expr_parser
[10:54:35] johnny:/Volumes/data/lldb/svn/trunk/test $
You can do further debugging from the Python stack traceback to have some idea of how/why the test failed.
It could even be that the Test*.py itself is incorrect. :-)
Right now, the test suite runs as a single Python invocation, so if there is a seg fault, the test suite aborts abruptly.
You can exclude certain test cases from being run while investigating the seg faults, however, by customizing
your own copy of blacklist.py in the test directory. The file contains some example usages.
For lldb loggings, you can look at the lldbLoggings() function within dotest.py file, it currently use some environment
variables to set the logging destination and the logging options. I'll ask Greg whether there are any specific thing to
turn on just for Dwarf reader things.
Thanks.
On Jan 28, 2011, at 10:37 AM, Warren.Paul at nokia.com wrote:
> I've been playing around with LLDB and trying to get it to load some symbol files generated by the toolchains we support. So far so good, but I need to start doing more lookups, etc. and would like to script these tests. I thought I'd start with the LLDB tests and build up from there. But when I run the tests I get mostly failures followed by a crash. :( Am I running them incorrectly, or missing some steps, … ?? Any help would be greatly appreciated. Also, I haven't seen any logging in the Dwarf reader yet but have hit a few asserts. Any tips for having my test scripts report anything the debugger doesn't like or expect in the Dwarf?
>
> Thanks,
> Warren
>
> Here's the output of the tests:
>
> ./dotest.py -v
>
> Session logs for test failures/errors will go into directory '2011-01-28-12_17_51'
> ----------------------------------------------------------------------
> Collected 178 tests
>
> 1: test_nonrunning_command_abbreviations (TestAbbreviations.AbbreviationsTestCase) ...
>
> Hello!
>
> ok
> 2: test_with_dsym (TestAbbreviations.AbbreviationsTestCase) ... FAIL
> 3: test_with_dwarf (TestAbbreviations.AbbreviationsTestCase) ... FAIL
> 4: test_with_dsym (TestAliases.AliasTestCase) ...
>
>
> Whoopee!
>
>
>
>
> hello
>
>
> hello
>
> FAIL
> 5: test_with_dwarf (TestAliases.AliasTestCase) ...
>
>
> Whoopee!
>
>
>
>
> hello
>
>
> hello
>
> FAIL
> 6: test_with_dsym_and_python_api (TestArrayTypes.ArrayTypesTestCase)
> Use Python APIs to inspect variables with array types. ... FAIL
> 7: test_with_dsym_and_run_command (TestArrayTypes.ArrayTypesTestCase)
> Test 'frame variable var_name' on some variables with array types. ... FAIL
> 8: test_with_dwarf_and_python_api (TestArrayTypes.ArrayTypesTestCase)
> Use Python APIs to inspect variables with array types. ... FAIL
> 9: test_with_dwarf_and_run_command (TestArrayTypes.ArrayTypesTestCase)
> Test 'frame variable var_name' on some variables with array types. ... FAIL
> 10: test_with_dsym_and_python_api (TestBitfields.BitfieldsTestCase)
> Use Python APIs to inspect a bitfields variable. ... FAIL
> 11: test_with_dsym_and_run_command (TestBitfields.BitfieldsTestCase)
> Test 'frame variable ...' on a variable with bitfields. ... FAIL
> 12: test_with_dwarf_and_python_api (TestBitfields.BitfieldsTestCase)
> Use Python APIs to inspect a bitfields variable. ... FAIL
> 13: test_with_dwarf_and_run_command (TestBitfields.BitfieldsTestCase)
> Test 'frame variable ...' on a variable with bitfields. ... FAIL
> 14: test_with_dsym (TestBreakpointCommand.BreakpointCommandTestCase)
> Test a sequence of breakpoint command add, list, and remove. ... FAIL
> 15: test_with_dwarf (TestBreakpointCommand.BreakpointCommandTestCase)
> Test a sequence of breakpoint command add, list, and remove. ... FAIL
> 16: test_with_dsym_and_python_api (TestBreakpointConditions.BreakpointConditionsTestCase)
> Use Python APIs to set breakpoint conditions. ... FAIL
> 17: test_with_dsym_and_run_command (TestBreakpointConditions.BreakpointConditionsTestCase)
> Exercise breakpoint condition with 'breakpoint modify -c <expr> id'. ... FAIL
> 18: test_with_dwarf_and_python_api (TestBreakpointConditions.BreakpointConditionsTestCase)
> Use Python APIs to set breakpoint conditions. ... FAIL
> 19: test_with_dwarf_and_run_command (TestBreakpointConditions.BreakpointConditionsTestCase)
> Exercise breakpoint condition with 'breakpoint modify -c <expr> id'. ... FAIL
> 20: test_with_dsym_and_python_api (TestBreakpointIgnoreCount.BreakpointIgnoreCountTestCase)
> Use Python APIs to set breakpoint ignore count. ... FAIL
> 21: test_with_dsym_and_run_command (TestBreakpointIgnoreCount.BreakpointIgnoreCountTestCase)
> Exercise breakpoint ignore count with 'breakpoint set -i <count>'. ... FAIL
> 22: test_with_dwarf_and_python_api (TestBreakpointIgnoreCount.BreakpointIgnoreCountTestCase)
> Use Python APIs to set breakpoint ignore count. ... FAIL
> 23: test_with_dwarf_and_run_command (TestBreakpointIgnoreCount.BreakpointIgnoreCountTestCase)
> Exercise breakpoint ignore count with 'breakpoint set -i <count>'. ... FAIL
> 24: test_with_dsym (TestBreakpointLocations.BreakpointLocationsTestCase)
> Test breakpoint enable/disable for a breakpoint ID with multiple locations. ... FAIL
> 25: test_with_dwarf (TestBreakpointLocations.BreakpointLocationsTestCase)
> Test breakpoint enable/disable for a breakpoint ID with multiple locations. ... FAIL
> 26: test_with_dsym_and_python_api (TestStaticVariables.StaticVariableTestCase)
> Test Python APIs on file and class static variables. ... FAIL
> 27: test_with_dsym_and_run_command (TestStaticVariables.StaticVariableTestCase)
> Test that file and class static variables display correctly. ... FAIL
> 28: test_with_dwarf_and_python_api (TestStaticVariables.StaticVariableTestCase)
> Test Python APIs on file and class static variables. ... FAIL
> 29: test_with_dwarf_and_run_command (TestStaticVariables.StaticVariableTestCase)
> Test that file and class static variables display correctly. ... FAIL
> 30: test_with_dsym_and_expr_parser (TestClassTypes.ClassTypesTestCase)
> Test 'frame variable this' and 'expr this' when stopped inside a constructor. ... FAIL
> 31: test_with_dsym_and_python_api (TestClassTypes.ClassTypesTestCase)
> Use Python APIs to create a breakpoint by (filespec, line). ... FAIL
> 32: test_with_dsym_and_run_command (TestClassTypes.ClassTypesTestCase)
> Test 'frame variable this' when stopped on a class constructor. ... FAIL
> 33: test_with_dwarf_and_expr_parser (TestClassTypes.ClassTypesTestCase)
> Test 'frame variable this' and 'expr this' when stopped inside a constructor. ... FAIL
> 34: test_with_dwarf_and_python_api (TestClassTypes.ClassTypesTestCase)
> Use Python APIs to create a breakpoint by (filespec, line). ... FAIL
> 35: test_with_dwarf_and_run_command (TestClassTypes.ClassTypesTestCase)
> Test 'frame variable this' when stopped on a class constructor. ... FAIL
> 36: test_with_dsym_and_python_api (TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase)
> Disassemble each call frame when stopped on C's constructor. ... FAIL
> 37: test_with_dsym_and_run_command (TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase)
> Disassemble each call frame when stopped on C's constructor. ... FAIL
> 38: test_with_dwarf_and_python_api (TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase)
> Disassemble each call frame when stopped on C's constructor. ... FAIL
> 39: test_with_dwarf_and_run_command (TestClassTypesDisassembly.IterateFrameAndDisassembleTestCase)
> Disassemble each call frame when stopped on C's constructor. ... FAIL
> 40: test_command_source (TestCommandSource.CommandSourceTestCase)
> Test that lldb command "command source" works correctly. ... 2011-01-28
> ok
> 41: test_with_dsym_command (TestConditionalBreak.ConditionalBreakTestCase)
> Simulate a user using lldb commands to break on c() if called from a(). ... FAIL
> 42: test_with_dsym_python (TestConditionalBreak.ConditionalBreakTestCase)
> Exercise some thread and frame APIs to break if c() is called by a(). ... FAIL
> 43: test_with_dwarf_command (TestConditionalBreak.ConditionalBreakTestCase)
> Simulate a user using lldb commands to break on c() if called from a(). ... FAIL
> 44: test_with_dwarf_python (TestConditionalBreak.ConditionalBreakTestCase)
> Exercise some thread and frame APIs to break if c() is called by a(). ... FAIL
> 45: test_with_dsym (TestDeadStrip.DeadStripTestCase)
> Test breakpoint works correctly with dead-code stripping. ... FAIL
> 46: test_with_dwarf (TestDeadStrip.DeadStripTestCase)
> Test breakpoint works correctly with dead-code stripping. ... FAIL
> 47: test_with_dsym (TestEnumTypes.EnumTypesTestCase)
> Test 'image lookup -t days' and check for correct display. ... FAIL
> 48: test_with_dwarf (TestEnumTypes.EnumTypesTestCase)
> Test 'image lookup -t days' and check for correct display. ... FAIL
> 49: test_choice (TestSequenceFunctions.SequenceFunctionsTestCase) ... ok
> 50: test_sample (TestSequenceFunctions.SequenceFunctionsTestCase) ... ok
> 51: test_shuffle (TestSequenceFunctions.SequenceFunctionsTestCase) ... ok
> 52: test_persistent_variables (TestPersistentVariables.PersistentVariablesTestCase)
> Test that lldb persistent variables works correctly. ... FAIL
> 53: test_expr_commands (Test8638051.Radar8638051TestCase)
> The following expression commands should not crash. ... FAIL
> 54: test_evaluate_expression_python (TestExprs.BasicExprCommandsTestCase)
> Test SBFrame.EvaluateExpression() API for evaluating an expression. ... FAIL
> 55: test_expr_commands_can_handle_quotes (TestExprs.BasicExprCommandsTestCase)
> Throw some expression commands with quotes at lldb. ... Stack dump:
> 0. <eof> parser at end of file
> 1. Parse:2:1: Generating code for declaration '$__lldb_valid_pointer_check'
> Segmentation fault
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20110128/faf455e2/attachment.html>
More information about the lldb-dev
mailing list