[lldb-dev] skip some tests with "check-lldb"

[Jonas Devlieghere via lldb-dev]

> On Sep 20, 2018, at 2:25 PM, Gábor Márton <martongabesz at gmail.com> wrote:
> 
> Hi Jonas,
> 
> Thanks for the clarification.
> Finally I could skip some of the tests by using a negative lookahead
> regex passed to --filter. I wouldn't say that this is so convenient
> and easy to overview, but works.
> For example, to skip the TestCalculatorMode.py and TestPrintArray.py
> tests I used the following argument (appended to the normal lit
> command line reported by "ninja check-lldb -v" ) :
> --filter '^((?!(TestCalculatorMode\.py|TestPrintArray\.py)).)*$'
> Would be beneficial though to have a --exclude besides --filter to
> avoid using a complex negative lookahead regex.

I have to agree that this looks overly complex. Maybe it's something to bring
up on the llvm-dev mailing list as a general improvement for lit?

> Cheers,
> Gabor
> On Thu, Sep 20, 2018 at 12:53 PM Jonas Devlieghere
> <jdevlieghere at apple.com> wrote:
>> 
>> When using lit as the driver (which is the case for check-lldb), the test
>> granularity is at file-level: lit invokes dotest.py for every test file. You
>> should be able to specify files to skip with lit's --filter.
>> 
>> With regards to threading, lit will schedule one instance of dotest.py on every
>> thread, which processes one test file. Unless you passed specific options for
>> dotest.py, the latter will run the different tests within that one file on
>> different threads as well. (IIRC)
>> 
>> On Sep 19, 2018, at 7:20 PM, Gábor Márton via lldb-dev <lldb-dev at lists.llvm.org> wrote:
>> 
>> That's okay, but is it possible to skip a few tests, when using lit? I was thinking about moving the test files I want to skip, but that has obvious drawbacks. Also --filter does not seem so useful in this case.
>> 
>> On Wed, 19 Sep 2018, 18:46 , <ted.woodward at codeaurora.org> wrote:
>>> 
>>> Unless you pass  --no-multiprocess to dotest, it should detect how many cores your system has and use them.
>>> 
>>> --
>>> Ted Woodward
>>> Qualcomm Innovation Center, Inc.
>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
>>> 
>>> -----Original Message-----
>>> From: lldb-dev <lldb-dev-bounces at lists.llvm.org> On Behalf Of Gábor Márton via lldb-dev
>>> Sent: Wednesday, September 19, 2018 11:04 AM
>>> To: lldb-dev at lists.llvm.org
>>> Subject: Re: [lldb-dev] skip some tests with "check-lldb"
>>> 
>>> I just realized that `dotest.py` has a --thread option. Is that the one which is used during the lit test (`ninja check-lldb`) ?
>>> 
>>> On Wed, Sep 19, 2018 at 6:00 PM Gábor Márton <martongabesz at gmail.com> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> I'd like to skip some tests when I run "ninja check-lldb", because they fail.
>>>> I am on release_70 branch.
>>>> I know I could use dotest.py directly, but that would exercise only one thread.
>>>> Is there a way to execute the tests parallel on all cores and in the
>>>> same time skip some of the tests?
>>>> 
>>>> Thanks,
>>>> Gabor
>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>> 
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
>>