[cfe-commits] [clang-tools-extra] r163009 - in /clang-tools-extra/trunk/test: loop-convert/ remove-cstr-calls/

Nico Weber thakis at chromium.org
Fri Aug 31 13:47:17 PDT 2012


On Fri, Aug 31, 2012 at 1:38 PM, Chandler Carruth <chandlerc at gmail.com>wrote:

> On Fri, Aug 31, 2012 at 1:31 PM, Daniel Dunbar <daniel at zuster.org> wrote:
>
>>
>>
>> On Friday, August 31, 2012, Chandler Carruth wrote:
>>
>>> On Fri, Aug 31, 2012 at 12:33 PM, Daniel Dunbar <daniel at zuster.org>wrote:
>>>
>>>> On Fri, Aug 31, 2012 at 12:23 PM, David Blaikie <dblaikie at gmail.com>
>>>> wrote:
>>>> > On Fri, Aug 31, 2012 at 11:53 AM, Daniel Dunbar <daniel at zuster.org>
>>>> wrote:
>>>> >> On Fri, Aug 31, 2012 at 11:23 AM, David Blaikie <dblaikie at gmail.com>
>>>> wrote:
>>>> >>> Daniel (& anyone else),
>>>> >>>
>>>> >>> Do you know if the documentation here (
>>>> >>> http://llvm.org/docs/TestingGuide.html#rtcustom /
>>>> >>>
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?view=diff&r1=36059&r2=36060
>>>> >>> ) is still valid? It doesn't appear to be, since my change below
>>>> >>> doesn't do particularly interesting quoting/escaping gymnastics &
>>>> >>> seems to work fine.
>>>> >>>
>>>> >>> Any ideas? Should we update the documentation? Does anyone know
>>>> >>> what/when/how/why this changed?
>>>> >>
>>>> >> What is actually going on here is that the LLVM test suite and the
>>>> >> Clang test suite use a different style of test.
>>>> >>
>>>> >> The LLVM test suite is technically written using Tcl syntax, and the
>>>> >> Clang test suite uses shell syntax. The document in question is
>>>> >> referring to the LLVM test suite.
>>>> >>
>>>> >> The fact that the test suites use different styles is an unfortunate
>>>> >> and poorly documented historical accident. Ideally we would kill off
>>>> >> the Tcl style and only use the shell style.
>>>> >
>>>> > I see - I think I'd heard some of that in various discussions. Where
>>>> > is the option for shell V tcl specified?
>>>>
>>>> The top level lit.cfg specifies the "test format", e.g.:
>>>>   config.test_format = lit.formats.ShTest(execute_external)
>>>>
>>>> Actually, apparently I lied the LLVM suite has moved over to the Sh
>>>> style tests and I never noticed:
>>>>   http://llvm.org/viewvc/llvm-project?view=rev&revision=159525
>>>>
>>>> So in fact the document in question is completely out of date.
>>>>
>>>
>>> Sorry for missing the documentation...
>>>
>>>
>>>>
>>>> >
>>>> >> Also, what is a "lesser lit-like" test runner?
>>>> >
>>>> > In this case, it's a shell script that attempts to transform RUN lines
>>>> > into a shell script to run a lit test - it works for the basic cases.
>>>>
>>>> Dare I ask why you would do this instead of just running lit?
>>>
>>>
>>> We run the tests in a distributed fashion, one test per worker, and the
>>> start-up overhead of python combined with the lit stuff takes more time
>>> than most of the test cases by a factor of 10 to 1000. ;] The shell script
>>> starts up very very fast.
>>>
>>
>> Makes sense.
>>
>> What if lit had a mode to generate all the scripts and then you just
>> farmed them out? That would avoid having to duplicate any of the script
>> parsing code.
>>
>
> This would work, although it wouldn't be appreciably better than our
> current arrangement imo.
>
> My long term idea is I would like to split lit into two pieces. One for
> running a test case, and one for managing the suite of tests.
>
> The former would be well suited to a very efficient C++ program. I would
> teach it just enough shell-like syntax to cope with the existing test
> suite, possibly simplifying the shell used in the existing test suite as
> necessary. This would then remove the dependency on bash (which hurts
> windows a lot and makes it  confusing as the bash versions change), make
> the test suite run a small amount faster, and solve our specific problem
> internally.
>
> The latter I'd like to eventually rewrite in C++ as well, specifically as
> part of Ninja, or a plugin, or at least re-using its code to do really
> efficient process management. Ninja is able to keep my 16 cpus between 20%
> and 50% busier than lit is currently, and some experiments show that as the
> # of cpus goes up, this is only going to get worse.
>
> Of course, none of this is "short term". ;]
>

For what it's worth, I wrote a lit-like testrunner in go a while ago. It's
pretty hacky but was able to pass all clang tests. It wasn't all that much
faster than lit (maybe 20%?), and the limiting factor seemed to be
launching processes for the tests. You'd have that overhead in the very
efficient c++ program too I suppose. (
https://github.com/nico/glitch/blob/master/glitch.go )

Nico
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20120831/ddfde35e/attachment.html>


More information about the cfe-commits mailing list