[lldb-dev] How do I use lit to only run the lldb test suite, now that dotest multiprocessing capabilities have been removed?

[Ted Woodward via lldb-dev]

Thanks Jonas.

Is full support for --param fairly recent? I tried it with a version of our master, based on top-of-tree from about a month ago, and it didn't work quite right. It's passing the dotest args, but it's also generating some args, so I'm seeing odd effects.

Here is my run line:
bin/python bin/llvm-lit /local/mnt/ted/8.4/llvm/lldb/lit/Suite --param 'dotest-args=-A v66 -C /prj/dsp/qdsp6/release/internal/HEXAGON/branch-8.4/linux64/latest/Tools/bin/hexagon-clang --executable /local/scratch/ted/8.4/build/bin/lldb -t -v -f RecursiveTypesTestCase.test_recursive_type_1_dwarf'

I only want to run RecursiveTyepsTestCase.test_recursive_type_1_dwarf, but it's running the whole test suite. Here's a dotest line from the run:

/local/mnt/ted/8.4/build/bin/python /local/mnt/ted/8.4/llvm/lldb/test/dotest.py -q --arch=v66 -s /local/mnt/ted/8.4/build/lldb-test-traces --build-dir /local/mnt/ted/8.4/build/lldb-test-build.noindex -S nm -u CXXFLAGS -u CFLAGS --executable /local/mnt/ted/8.4/build/./bin/lldb --dsymutil /local/mnt/ted/8.4/build/./bin/dsymutil --filecheck /local/mnt/ted/8.4/build/./bin/FileCheck -C /local/mnt/ted/8.4/build/./bin/clang --env ARCHIVER=/usr/bin/ar --env OBJCOPY=/usr/bin/objcopy -A v66 -C /prj/dsp/qdsp6/release/internal/HEXAGON/branch-8.4/linux64/latest/Tools/bin/hexagon-clang --executable /local/scratch/ted/8.4/build/bin/lldb -t -v -f RecursiveTypesTestCase.test_recursive_type_1_dwarf --env LLVM_LIBS_DIR=/local/mnt/ted/8.4/build/./lib /local/mnt/ted/8.4/llvm/lldb/packages/Python/lldbsuite/test/functionalities/breakpoint/debugbreak -p TestDebugBreak.py


It's got both --arch= and -A, -C is set to my build directory clang as well as the clang I told it to use, --executable is set twice, and it's got -f RecursiveTypesTestCase.test_recursive_type_1_dwarf and -p TestDebugBreak.py .

These tests that do a "process launch" (which is most of them) invoke the hexagon simulator, but it was never launched. There was also only 1 testcase built in /local/mnt/ted/8.4/build/lldb-test-build.noindex - types/TestRecursiveTypes.test_recursive_type_1_dwarf .

This does not have the patch that removes multiprocess support from dotest.

Ted

> -----Original Message-----
> From: Jonas Devlieghere <jonas at devlieghere.com>
> Sent: Thursday, August 8, 2019 2:50 PM
> To: Ted Woodward <tedwood at quicinc.com>
> Cc: LLDB <lldb-dev at lists.llvm.org>
> Subject: [EXT] Re: How do I use lit to only run the lldb test suite, now that
> dotest multiprocessing capabilities have been removed?
> 
> Hey Ted,
> 
> 1. You can run just the dotest-tests by pointing lit at the `lit/Suite` directory.
> 2. You can pass arguments to dotest by passing `dotest-args` in --param.
> 
> The invocation would look something like this:
> 
> /path/to/llvm/bin/llvm-lit /path/to/lldb/lit/Suite --param 'dotest-args=--foo --
> bar'
> 
> Hope that helps,
> Jonas
> 
> On Thu, Aug 8, 2019 at 9:31 AM Ted Woodward <tedwood at quicinc.com>
> wrote:
> >
> > RE: https://reviews.llvm.org/D65311
> >
> >
> >
> > Internally we use dotest to run the lldb test suite with various RTOS
> configurations for the test binaries. In these runs we don’t care about the lit
> tests or the unit tests, because they are OS agnostic. We do this by specifying
> the compiler, lldb, and test flavor (static testcase + os, dynamic library testcase
> loaded by an OS image, dynamic library testcase loaded by an OS image
> running the OS’ debug stub).
> >
> >
> >
> > With the multiprocess testrunner removed, how do I have lit:
> >
> > Only run the lldb test suite
> > Run dotest with specific arguments
> >
> >
> >
> > We’re not running cmake; we’re taking an existing tools build and running
> the tests from the source directory using the toolset.