[lldb-dev] skip some tests with "check-lldb"

[Jim Ingham via lldb-dev]

When I am working on some changes and there are tests I know I've broken but don't want to fix just yet, I just edit the test file and skip it using the skip decorators.  You have to remember not to check in the skips when you check in whatever change you are making - but presumably by the time you are ready to submit something you will have fixed the tests anyway.  That's much simpler than trying to monkey with the test runner to do your bidding.

Jim

> On Sep 20, 2018, at 5:25 AM, Gábor Márton via lldb-dev <lldb-dev at lists.llvm.org> wrote:
> 
> Hi Jonas,
> 
> Thanks for the clarification.
> Finally I could skip some of the tests by using a negative lookahead
> regex passed to --filter. I wouldn't say that this is so convenient
> and easy to overview, but works.
> For example, to skip the TestCalculatorMode.py and TestPrintArray.py
> tests I used the following argument (appended to the normal lit
> command line reported by "ninja check-lldb -v" ) :
> --filter '^((?!(TestCalculatorMode\.py|TestPrintArray\.py)).)*$'
> Would be beneficial though to have a --exclude besides --filter to
> avoid using a complex negative lookahead regex.
> 
> Cheers,
> Gabor
> On Thu, Sep 20, 2018 at 12:53 PM Jonas Devlieghere
> <jdevlieghere at apple.com> wrote:
>> 
>> When using lit as the driver (which is the case for check-lldb), the test
>> granularity is at file-level: lit invokes dotest.py for every test file. You
>> should be able to specify files to skip with lit's --filter.
>> 
>> With regards to threading, lit will schedule one instance of dotest.py on every
>> thread, which processes one test file. Unless you passed specific options for
>> dotest.py, the latter will run the different tests within that one file on
>> different threads as well. (IIRC)
>> 
>> On Sep 19, 2018, at 7:20 PM, Gábor Márton via lldb-dev <lldb-dev at lists.llvm.org> wrote:
>> 
>> That's okay, but is it possible to skip a few tests, when using lit? I was thinking about moving the test files I want to skip, but that has obvious drawbacks. Also --filter does not seem so useful in this case.
>> 
>> On Wed, 19 Sep 2018, 18:46 , <ted.woodward at codeaurora.org> wrote:
>>> 
>>> Unless you pass  --no-multiprocess to dotest, it should detect how many cores your system has and use them.
>>> 
>>> --
>>> Ted Woodward
>>> Qualcomm Innovation Center, Inc.
>>> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
>>> 
>>> -----Original Message-----
>>> From: lldb-dev <lldb-dev-bounces at lists.llvm.org> On Behalf Of Gábor Márton via lldb-dev
>>> Sent: Wednesday, September 19, 2018 11:04 AM
>>> To: lldb-dev at lists.llvm.org
>>> Subject: Re: [lldb-dev] skip some tests with "check-lldb"
>>> 
>>> I just realized that `dotest.py` has a --thread option. Is that the one which is used during the lit test (`ninja check-lldb`) ?
>>> 
>>> On Wed, Sep 19, 2018 at 6:00 PM Gábor Márton <martongabesz at gmail.com> wrote:
>>>> 
>>>> Hi,
>>>> 
>>>> I'd like to skip some tests when I run "ninja check-lldb", because they fail.
>>>> I am on release_70 branch.
>>>> I know I could use dotest.py directly, but that would exercise only one thread.
>>>> Is there a way to execute the tests parallel on all cores and in the
>>>> same time skip some of the tests?
>>>> 
>>>> Thanks,
>>>> Gabor
>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>> 
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
>> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev