[lldb-dev] problem with TestLoadUnload.py

[Pavel Labath via lldb-dev]

A couple of things here:
- there should be no performance difference between doing build in
setUp and the test function as setUp is called once per test function
- my change was to run have the paralelization at a file level
(previously it was at folder-level). All test functions in a single
file are still run serially.
- also, why do we need libgen.h in the test in the first place? Can we
just remove that?

So, right now it should be possible to move the build call into the
test function without negatively impacting much of anything. However,
I would like it if we can move towards a world, where we can do more
stuff in the setUp() function (or even setUpTestCase()), as that what
they were meant for -- setting up state which is common for multiple
tests. One way to achieve this would be to resurrect our ability to
@skip an entire test class. Right now the situation is that some of
our decorators work on class level, but most of them don't. I don't
think it should not be too hard to fix the decorators to work at class
level as well. Then if the whole test class does not apply in some
situation, we can @skip the whole class and avoid even running the
setUp code.

WDYT?


On 23 February 2018 at 13:08, Ted Woodward <ted.woodward at codeaurora.org> wrote:
> I tried to put @skipIf(...) before setUp, but it didn't work. Currently I
> have the build inside an if, checking for Hexagon. We don't support this use
> of shared libraries, so all tests are skipped.
>
> I certainly don't want to build the testcase 6 times, given that we're
> moving away from that! But we need some general way to skip the build for
> systems that can't build a given testcase. I have many tests skipped when
> running without an OS, because we don't support c++11 in that mode.
>
> --
> Qualcomm Innovation Center, Inc.
> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
> Linux Foundation Collaborative Project
>
>> -----Original Message-----
>> From: aprantl at apple.com [mailto:aprantl at apple.com]
>> Sent: Friday, February 23, 2018 2:47 PM
>> To: Ted Woodward <ted.woodward at codeaurora.org>
>> Cc: Pavel Labath <labath at google.com>
>> Subject: Re: problem with TestLoadUnload.py
>>
>>
>>
>> > On Feb 23, 2018, at 12:08 PM, Ted Woodward
>> <ted.woodward at codeaurora.org> wrote:
>> >
>> > Hi Adrian,
>> >
>> > In r324368 you changed TestLoadUnload.py to do a build in the setUp
>> > function. This seems to be called before the actual testcases are
>> > checked, so a skipped testcase would still call setUp and do the
>> > build. If the build fails (say, on systems that don't have libgen.h),
>> > the test errors out before it can be skipped.
>> >
>> > Would you mind moving the build into each testcase?
>>
>> +Pavel
>>
>> I'm afraid that would cause quite a performance degradation in the case
> where
>> the test is not skipped wince we then need to build the test once for each
> test
>> function. On the other hand since Pavel's change to run each test function
> in
>> parallel we are presumably building the test once per test function
> anyway, so
>> this should not be any more expensive.
>>
>> That said, I wonder if the right solution for your problem wouldn't be to
> not run
>> setUp if the test depending on it is skipped.
>>
>> (Do you mind if I add lldb-dev to the CC list?)
>>
>> -- adrian
>>
>> > Thanks,
>> >
>> > Ted
>> >
>> > --
>> > Qualcomm Innovation Center, Inc.
>> > The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
>> > a Linux Foundation Collaborative Project
>> >
>> >
>
>