[lldb-dev] Separating test runner and tests

Fri Dec 11 11:13:48 PST 2015

It just requires running the test file as a python script.

The runner is fired off like this:

if __name__ == "__main__":
    unittest.main()

which is typically added to the bottom of all test files so you can call it
directly.

-Todd

On Fri, Dec 11, 2015 at 11:12 AM, Todd Fiala <todd.fiala at gmail.com> wrote:

> Unittest.
>
> Comes with Python.
>
> On Fri, Dec 11, 2015 at 11:07 AM, Zachary Turner <zturner at google.com>
> wrote:
>
>> Presumably those tests use an entirely different, hand-rolled test
>> running infrastructure?
>>
>> On Fri, Dec 11, 2015 at 10:52 AM Todd Fiala <todd.fiala at gmail.com> wrote:
>>
>>> One thing I want to make sure we can do is have a sane way of storing
>>> and running tests that  test the test execution engine.  Those are tests
>>> that should not run as part of an "lldb test run".  These are tests that
>>> maintainers of the test system run to make sure we're not breaking stuff
>>> when we touch the test system.
>>>
>>> I would be writing more of those if I had a semi-sane way of doing it.
>>>  (Part of the reason I broke out the python-based timeout logic the way I
>>> did, before the major packaging changes, was so I had an obvious spot to
>>> add tests for the process runner logic).
>>>
>>> On Fri, Dec 11, 2015 at 10:03 AM, Todd Fiala <todd.fiala at gmail.com>
>>> wrote:
>>>
>>>> I like it.
>>>>
>>>> On Fri, Dec 11, 2015 at 9:51 AM, Zachary Turner <zturner at google.com>
>>>> wrote:
>>>>
>>>>> Yea wasn't planning on doing this today, just throwing the idea out
>>>>> there.
>>>>>
>>>>> On Fri, Dec 11, 2015 at 9:35 AM Todd Fiala <todd.fiala at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> I'm fine with the idea.
>>>>>>
>>>>>> FWIW the test events model will likely shift a bit, as it is
>>>>>> currently a single sink, whereas I am likely to turn it into a test event
>>>>>> filter chain shortly here.  Formatters still make sense as they'll be the
>>>>>> things at the end of the chain.
>>>>>>
>>>>>> Minor detail, result_formatter.py should be results_formatter.py -
>>>>>> they are ResultsFormatter instances (plural on Results since it transforms
>>>>>> a series of results into coherent reported output).  I'll rename that at
>>>>>> some point in the near future, but if you shift a number of things around,
>>>>>> you can do that.
>>>>>>
>>>>>> I'm just about done with the multi-pass running.  I expect to get an
>>>>>> opt-in version of that running end of day today or worst case on Sunday.
>>>>>> It would be awesome if you can hold off on any significant change like that
>>>>>> until this little bit is done as I'm sure we'll collide, particularly since
>>>>>> this hits dosep.py pretty significantly.
>>>>>>
>>>>>> Thanks!
>>>>>>
>>>>>> -Todd
>>>>>>
>>>>>> On Fri, Dec 11, 2015 at 1:33 AM, Pavel Labath via lldb-dev <
>>>>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>> Sounds like a reasonable thing to do. A couple of tiny remarks:
>>>>>>> - when you do the move, you might as well rename dotest into
>>>>>>> something
>>>>>>> else, just to avoid the "which dotest should I run" type of
>>>>>>> questions...
>>>>>>> - there is nothing that makes it obvious that "engine" is actually a
>>>>>>> "test running engine", as it sits in a sibling folder. OTOH,
>>>>>>> "test_engine" might be too verbose, and messes up tab completion, so
>>>>>>> that might not be a good idea either...
>>>>>>>
>>>>>>> pl
>>>>>>>
>>>>>>>
>>>>>>> On 10 December 2015 at 23:30, Zachary Turner via lldb-dev
>>>>>>> <lldb-dev at lists.llvm.org> wrote:
>>>>>>> > Currently our folder structure looks like this:
>>>>>>> >
>>>>>>> > lldbsuite
>>>>>>> > |-- test
>>>>>>> >     |-- dotest.py
>>>>>>> >     |-- dosep.py
>>>>>>> >     |-- lldbtest.py
>>>>>>> >     |-- ...
>>>>>>> >     |-- functionalities
>>>>>>> >     |-- lang
>>>>>>> >     |-- expression_command
>>>>>>> >     |-- ...
>>>>>>> > etc
>>>>>>> >
>>>>>>> > I've been thinking about organizing it like this instead:
>>>>>>> >
>>>>>>> > lldbsuite
>>>>>>> > |-- test
>>>>>>> >     |-- functionalities
>>>>>>> >     |-- lang
>>>>>>> >     |-- expression_command
>>>>>>> >     |-- ...
>>>>>>> > |-- engine
>>>>>>> >     |-- dotest.py
>>>>>>> >     |-- dosep.py
>>>>>>> >     |-- lldbtest.py
>>>>>>> >     |-- ...
>>>>>>> >
>>>>>>> > Anybody have any thoughts on this?  Good idea or bad idea?  The
>>>>>>> main reason
>>>>>>> > I want to do this is because as we start breaking up some of the
>>>>>>> code, it
>>>>>>> > makes sense to start having some subpackages under the `engine`
>>>>>>> folder (or
>>>>>>> > the `test` folder in our current world).  For example, Todd and I
>>>>>>> have
>>>>>>> > discussed the idea of putting formatter related stuff under a
>>>>>>> `formatters`
>>>>>>> > subpackage.  In the current world, there's no way to differentiate
>>>>>>> between
>>>>>>> > folders which contain tests and folders which contain test
>>>>>>> infrastructure,
>>>>>>> > so when we walk the directory tree looking for tests we end up
>>>>>>> walking a
>>>>>>> > bunch of directories that are used for test infrastructure code
>>>>>>> and not
>>>>>>> > actual tests.  So I like the logical separation this provides --
>>>>>>> having the
>>>>>>> > tests themselves all under a single subpackage.
>>>>>>> >
>>>>>>> > Thoughts?
>>>>>>> >
>>>>>>> > _______________________________________________
>>>>>>> > lldb-dev mailing list
>>>>>>> > lldb-dev at lists.llvm.org
>>>>>>> > http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>> >
>>>>>>> _______________________________________________
>>>>>>> lldb-dev mailing list
>>>>>>> lldb-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> -Todd
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> -Todd
>>>>
>>>
>>>
>>>
>>> --
>>> -Todd
>>>
>>
>
>
> --
> -Todd
>

-- 
-Todd
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20151211/b7ab9532/attachment-0001.html>