[cfe-commits] [clang-tools-extra] r163009 - in /clang-tools-extra/trunk/test: loop-convert/ remove-cstr-calls/

[Daniel Dunbar]

On Friday, August 31, 2012, Chandler Carruth wrote:

> On Fri, Aug 31, 2012 at 12:33 PM, Daniel Dunbar <daniel at zuster.org<javascript:_e({}, 'cvml', 'daniel at zuster.org');>
> > wrote:
>
>> On Fri, Aug 31, 2012 at 12:23 PM, David Blaikie <dblaikie at gmail.com<javascript:_e({}, 'cvml', 'dblaikie at gmail.com');>>
>> wrote:
>> > On Fri, Aug 31, 2012 at 11:53 AM, Daniel Dunbar <daniel at zuster.org<javascript:_e({}, 'cvml', 'daniel at zuster.org');>>
>> wrote:
>> >> On Fri, Aug 31, 2012 at 11:23 AM, David Blaikie <dblaikie at gmail.com<javascript:_e({}, 'cvml', 'dblaikie at gmail.com');>>
>> wrote:
>> >>> Daniel (& anyone else),
>> >>>
>> >>> Do you know if the documentation here (
>> >>> http://llvm.org/docs/TestingGuide.html#rtcustom /
>> >>>
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?view=diff&r1=36059&r2=36060
>> >>> ) is still valid? It doesn't appear to be, since my change below
>> >>> doesn't do particularly interesting quoting/escaping gymnastics &
>> >>> seems to work fine.
>> >>>
>> >>> Any ideas? Should we update the documentation? Does anyone know
>> >>> what/when/how/why this changed?
>> >>
>> >> What is actually going on here is that the LLVM test suite and the
>> >> Clang test suite use a different style of test.
>> >>
>> >> The LLVM test suite is technically written using Tcl syntax, and the
>> >> Clang test suite uses shell syntax. The document in question is
>> >> referring to the LLVM test suite.
>> >>
>> >> The fact that the test suites use different styles is an unfortunate
>> >> and poorly documented historical accident. Ideally we would kill off
>> >> the Tcl style and only use the shell style.
>> >
>> > I see - I think I'd heard some of that in various discussions. Where
>> > is the option for shell V tcl specified?
>>
>> The top level lit.cfg specifies the "test format", e.g.:
>>   config.test_format = lit.formats.ShTest(execute_external)
>>
>> Actually, apparently I lied the LLVM suite has moved over to the Sh
>> style tests and I never noticed:
>>   http://llvm.org/viewvc/llvm-project?view=rev&revision=159525
>>
>> So in fact the document in question is completely out of date.
>>
>
> Sorry for missing the documentation...
>
>
>>
>> >
>> >> Also, what is a "lesser lit-like" test runner?
>> >
>> > In this case, it's a shell script that attempts to transform RUN lines
>> > into a shell script to run a lit test - it works for the basic cases.
>>
>> Dare I ask why you would do this instead of just running lit?
>
>
> We run the tests in a distributed fashion, one test per worker, and the
> start-up overhead of python combined with the lit stuff takes more time
> than most of the test cases by a factor of 10 to 1000. ;] The shell script
> starts up very very fast.
>

Makes sense.

What if lit had a mode to generate all the scripts and then you just farmed
them out? That would avoid having to duplicate any of the script parsing
code.

 - Daniel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20120831/f6986216/attachment.html>