[lldb-dev] [RFC] Testsuite in lldb & possible future directions

[Davide Italiano via lldb-dev]

On Wed, Feb 7, 2018 at 9:32 AM, Pavel Labath <labath at google.com> wrote:
> On 6 February 2018 at 15:51, Davide Italiano <dccitaliano at gmail.com> wrote:
>>
>> FWIW, I strongly believe we should all agree on a configuration to run
>> tests and standardize on that.
>> It's unfortunate that we have two build systems, but there are plans
>> to move away from manually generating xcodebuild, as many agree it's a
>> terrible maintenance burden.
>> So, FWIW, I'll share my conf (I'm on high Sierra):
>>
>>
>> git clone https://github.com/monorepo
>> symlink clang -> tools
>> symlink lldb -> tools
>> symlink libcxx -> projects (this particular one has caused lots of
>> trouble for me in the past, and I realized it's undocumented :()
>>
>> cmake -GNinja -DCMAKE_BUILD_TYPE=Release ../llvm
>> ninja check-lldb
>>
> Right, so I tried following these instructions as precisely as I could.
>
> - The first thing that failed is the libc++ link step (missing -lcxxabi_shared).
>
> So, I added libcxxabi to the build, and tried again.
> Aaand, I have to say the situation is much better now: I got two
> unexpected successes and one timeout:
> UNEXPECTED SUCCESS: test_lldbmi_output_grammar
> (tools/lldb-mi/syntax/TestMiSyntax.py)
> UNEXPECTED SUCCESS: test_process_interrupt_dsym
> (functionalities/thread/state/TestThreadStates.py)
> TIMEOUT: test_breakpoint_doesnt_match_file_with_different_case_dwarf
> (functionalities/breakpoint/breakpoint_case_sensitivity/TestBreakpointCaseSensitivity.py)
>
> On the second run I got these results:
> FAIL: test_launch_in_terminal (functionalities/tty/TestTerminal.py)
> UNEXPECTED SUCCESS: test_lldbmi_output_grammar
> (tools/lldb-mi/syntax/TestMiSyntax.py)
> UNEXPECTED SUCCESS: test_process_interrupt_dwarf
> (functionalities/thread/state/TestThreadStates.py)
>
>
> So, checking out libc++ certainly helped (this definitely needs to be
> documented somewhere) a lot. Of these, the MI test seems to be failing
> consistently. The rest appear to be flakes. I am attaching the logs
> from the second run, but there doesn't appear to be anything
> interesting there...

Terrific that we're making progress! I plan to take a look at the
`lldb-mi` failure soon, as I can reproduce it here fairly
consistently.

About the others, we've seen
functionalities/breakpoint/breakpoint_case_sensitivity/TestBreakpointCaseSensitivity.py
failing on the bots and I think might be due to a spotlight issue
Adrian found (and fixed).
You might still have `.dSYM` bundles from stale build directories, i.e.

To fix this, you need to wipe out all old build artifacts:

- Inside of the LLDB source tree:
 $ git clean -f -d

- Globally:
 $ find / -name a.out.dSYM -exec rm -rf \{} \;

This s a long shot, but might help you

--
Davide