[lldb-dev] Ubuntu version-based fail/skip

[Zachary Turner via lldb-dev]

I'd prefer to avoid calling the unittest2 functions.  We already do that in
a couple places, but if we could centralize on one place where we call
unittest2 decorators it would really make it easier to customize our own
decorators.  For example, I have a short-term goal of adding an option to
dotest that allows us to treat skips as xfails.  Right now this is
problematic because we have calls into the unittest2 decorators scattered
all over.  If it's in one place, it's very easy to make this change.

I like tamas's solution, and I was planning on adding that anyway.
 basically make @decorateTest() take a new keyword argument called test_fun
default to None.  inside the wrapper function, just run that function if
it's specified and have that be factored into the equation.  Everything
else shoudl just work.  By putting it here, we also get the behavior for
free on either skips or xfails

On Thu, Jan 28, 2016 at 3:03 PM Todd Fiala via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> That could be a reasonable way to do it.  Now that I think about it,
> unittest2 already gives us a generic skip where we can put the logic in
> that we want.  Not sure why that didn't occur to me earlier as I've done
> that very thing in the past. (I think I've conditioned myself to use our
> custom decorators...)
>
> On Mon, Jan 25, 2016 at 5:44 AM, Tamas Berghammer <tberghammer at google.com>
> wrote:
>
>> I think recently we are trying to reduce the number of decorators we are
>> having so adding a few new Ubuntu specific decorators might not be a good
>> idea. My suggestion would be to move on a little bit to the functional
>> programming style with adding a new option to @expetedFailureAll where we
>> can specify a function what have to evaluate to true for the decorator to
>> be considered (and it is evaluated only after all other condition of
>> @expectedFailureAll). Then we can create a free function called
>> getLinuxDistribution what will return the distribution id and then as a
>> final step we can specify a lambda to expetedFailureAll through its new
>> argument what calls getLinuxDistribution and compares it with the right
>> value. I know it is a lot of hoops to jump over to get a distribution
>> specific decorator but I think this approach can handle arbitrarily complex
>> skip/xfail conditions what will help us in the future.
>>
>> What do you think?
>>
>> Thanks,
>> Tamas
>>
>>
>>
>> On Fri, Jan 22, 2016 at 6:31 PM Todd Fiala <todd.fiala at gmail.com> wrote:
>>
>>> Hey all,
>>>
>>> What do you think about having some kind of way of marking the (in this
>>> case, specifically) Ubuntu distribution for fail/skip test decorators?
>>> I've had a few cases where I've needed to mark tests failing on for Ubuntu
>>> where it really was only a particular release of an Ubuntu distribution,
>>> and wasn't specifically the compiler.  (i.e. it was a constellation of more
>>> moving parts that clearly occur on a particular release of an Ubuntu
>>> distribution but not on others, and certainly not generically across all
>>> Linux distributions).
>>>
>>> I'd love to have a way to skip and xfail a test for a specific Ubuntu
>>> distribution release.  I guess it could be done uber-genrically, but with
>>> Linux distributions this can get complicated due to the os/distribution
>>> axes.  So I'd be happy to start off with just having them at a distribution
>>> basis:
>>>
>>> @skipIfUbuntu(version_check_list)  # version_check_list contains one or
>>> more version checks that, if passing, trigger the skip
>>>
>>> @expectedFailureUbuntu(version_check_list)  # similar to above
>>>
>>> Or possibly more usefully,
>>>
>>> @skipIfLinuxDistribution(version_check_list)  # version_check_list
>>> contains one or more version checks that, if passing, trigger the skip,
>>> includes the distribution
>>>
>>> @expectedFailureLinuxDistribution(version_check_list)  # similar to above
>>>
>>>
>>> It's not clear to me how to work in the os=linux, distribution=Ubuntu
>>> into the more generic checks like and get distribution-level version
>>> checking working right otherwise, but I'm open to suggestions.
>>>
>>> The workaround for the short term is to just use blanket-linux @skipIf
>>> and @expectedFailure style calls.
>>>
>>> Thoughts?
>>> --
>>> -Todd
>>>
>>
>
>
> --
> -Todd
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160128/da52964a/attachment.html>