[cfe-dev] RFC: Clang test runner changes

[Argyrios Kyrtzidis]

Hi Daniel,

On Aug 4, 2009, at 11:49 AM, Daniel Dunbar wrote:

> On Tue, Aug 4, 2009 at 11:09 AM, Mike Stump<mrs at apple.com> wrote:
>> On Aug 4, 2009, at 10:02 AM, Shantonu Sen wrote:
>>>
>>> Would it be possible for the .script file to include the PATH change
>>> in its text so it's easier to run the .script file manually to
>>> reproduce failures?
>>
>> I got hit by this just today...  I'd like it as well.  :-)
>>
>
> Yes, I hope to improve this as part of the move to running commands
> internally (without a shell). The commands will probably get the path
> resolved, and the error output will include the full command which
> should reproduce the error when run from the command line. I can also
> have it dump the .script as it does now with resolved paths, on
> failures.
>
> A related problem I would like to solve is that it is impossible to
> tell what failed in tests like:
> --
> RUN: clang ... -o %t
> RUN: grep ... | count 1
> RUN: grep ... | count 1
> ...
> RUN: grep ... | count 1
> --
> (FileCheck helps with this as well, when applicable).

I'd like to use FileCheck in this way:

// CHECK: <t1.c:5:12, t1.c:5:12>
// CHECK: <t1.c:6:12, t1.c:6:12>

Each line will be checked whether it contains the given string, but  
FileCheck will also verify that there are no other lines displayed,  
just 2 lines.

Do you think it's reasonable to add a command-line option for this  
functionality ?

-Argiris