[lldb-dev] [RFC] Testsuite in lldb & possible future directions

[Pavel Labath via lldb-dev]

On 6 February 2018 at 15:51, Davide Italiano <dccitaliano at gmail.com> wrote:
>
> FWIW, I strongly believe we should all agree on a configuration to run
> tests and standardize on that.
> It's unfortunate that we have two build systems, but there are plans
> to move away from manually generating xcodebuild, as many agree it's a
> terrible maintenance burden.
> So, FWIW, I'll share my conf (I'm on high Sierra):
>
>
> git clone https://github.com/monorepo
> symlink clang -> tools
> symlink lldb -> tools
> symlink libcxx -> projects (this particular one has caused lots of
> trouble for me in the past, and I realized it's undocumented :()
>
> cmake -GNinja -DCMAKE_BUILD_TYPE=Release ../llvm
> ninja check-lldb
>
Right, so I tried following these instructions as precisely as I could.

- The first thing that failed is the libc++ link step (missing -lcxxabi_shared).

So, I added libcxxabi to the build, and tried again.
Aaand, I have to say the situation is much better now: I got two
unexpected successes and one timeout:
UNEXPECTED SUCCESS: test_lldbmi_output_grammar
(tools/lldb-mi/syntax/TestMiSyntax.py)
UNEXPECTED SUCCESS: test_process_interrupt_dsym
(functionalities/thread/state/TestThreadStates.py)
TIMEOUT: test_breakpoint_doesnt_match_file_with_different_case_dwarf
(functionalities/breakpoint/breakpoint_case_sensitivity/TestBreakpointCaseSensitivity.py)

On the second run I got these results:
FAIL: test_launch_in_terminal (functionalities/tty/TestTerminal.py)
UNEXPECTED SUCCESS: test_lldbmi_output_grammar
(tools/lldb-mi/syntax/TestMiSyntax.py)
UNEXPECTED SUCCESS: test_process_interrupt_dwarf
(functionalities/thread/state/TestThreadStates.py)


So, checking out libc++ certainly helped (this definitely needs to be
documented somewhere) a lot. Of these, the MI test seems to be failing
consistently. The rest appear to be flakes. I am attaching the logs
from the second run, but there doesn't appear to be anything
interesting there...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Failure-LaunchInTerminalTestCase-test_launch_in_terminal.log
Type: application/octet-stream
Size: 3191 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20180207/d071b162/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: UnexpectedSuccess-MiSyntaxTestCase-test_lldbmi_output_grammar.log
Type: application/octet-stream
Size: 2607 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20180207/d071b162/attachment-0001.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: UnexpectedSuccess-ThreadStateTestCase-test_process_interrupt_dwarf.log
Type: application/octet-stream
Size: 4576 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20180207/d071b162/attachment-0002.obj>