[lldb-dev] Test suite rebuilding test executables many times

[Todd Fiala via lldb-dev]

Not sure if this is relevant, but I seem to recall the remote test
execution would spin off each test method run (test case level, not test
suite level) into a new directory.  I don't think that would be inherently
broken by a no-clean scenario but we'd want to make sure it doesn't break.

-Todd

On Wed, Aug 26, 2015 at 7:52 AM, Zachary Turner via lldb-dev <
lldb-dev at lists.llvm.org> wrote:

> I thought of this too and I started prototyping it.
>
> The issue that I ran into is that dsym and dwarf tests can all be xfailed,
> skipped, etc for different reasons, so if there is one method body, you
> need a way to still define the set of conditions under which dsym and dwarf
> tests should run, skip, xfail, timeout, etc.
>
> Do you want to start writing
> @skipIfDwarfAndOsIsLinuxButCompilerIsNotClang?  Because I know I don't
> want to deal with the combinatorial explosion of decorators that would
> result :)
>
> I have some ideas here as well, for example I think we only actually need
> 1 decorator that we can configure via keyword arguments that can handle
> arbitrarily complex scenarios of xfailing, debug infos, etc.  e.g.
> @lldb_test(debug_types="dwo,dwarf", xfail = {...}, skip = {...}).  But
> it's all unrelated to the original problem I'm trying to solve.  So I think
> it would be good to design a solution for that, but to do it separately.
>
> The nice thing about just changing the default from clean to not clean is
> that it's about a 3 line change, has potentially large speed improvements
> across the board, and also fixes bugs.
>
>
>
> On Wed, Aug 26, 2015 at 2:01 AM Pavel Labath <labath at google.com> wrote:
>
>> On 26 August 2015 at 06:14, Zachary Turner via lldb-dev
>> <lldb-dev at lists.llvm.org> wrote:
>> >
>> > I'll wait and see if anyone can remember which tests rebuild binaries on
>> > purpose.  Otherwise I will try to look through them and see if I can
>> figure
>> > it out.
>>
>> TestInferiorChanged is one that I remember.
>>
>> I think this is a good thing to do, but it will need to be done with a
>> steady hand.
>>
>>
>>
>> I was also thinking about the dsym/dwo tests.. Instead of basically
>> having a copy of each test for dwarf and dsym (and soon also dwo), how
>> about having just one test, and have some higher level logic (the test
>> runner) know that it needs to execute each test multiple times. The
>> tests would then just do a buildDefault() (or something) and on the
>> first run it would build normal dwarf, on the second one dsym, etc. If
>> we need to run a test only for some combination of debug infos, we
>> could have @skipIfDsym annotations, like we do for the rest of stuff.
>> I think this will make what Zachary is proposing easier to do, and it
>> will make the test writing less awkard.
>>
>>
>> What do you think? I'm ready to chip in on this if we agree to go down
>> this way...
>>
>> pl
>>
>
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
>


-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150826/f7fa6100/attachment-0001.html>