[cfe-commits] [clang-tools-extra] r163009 - in /clang-tools-extra/trunk/test: loop-convert/ remove-cstr-calls/

[David Blaikie]

On Fri, Aug 31, 2012 at 11:53 AM, Daniel Dunbar <daniel at zuster.org> wrote:
> On Fri, Aug 31, 2012 at 11:23 AM, David Blaikie <dblaikie at gmail.com> wrote:
>> Daniel (& anyone else),
>>
>> Do you know if the documentation here (
>> http://llvm.org/docs/TestingGuide.html#rtcustom /
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/TestingGuide.html?view=diff&r1=36059&r2=36060
>> ) is still valid? It doesn't appear to be, since my change below
>> doesn't do particularly interesting quoting/escaping gymnastics &
>> seems to work fine.
>>
>> Any ideas? Should we update the documentation? Does anyone know
>> what/when/how/why this changed?
>
> What is actually going on here is that the LLVM test suite and the
> Clang test suite use a different style of test.
>
> The LLVM test suite is technically written using Tcl syntax, and the
> Clang test suite uses shell syntax. The document in question is
> referring to the LLVM test suite.
>
> The fact that the test suites use different styles is an unfortunate
> and poorly documented historical accident. Ideally we would kill off
> the Tcl style and only use the shell style.

I see - I think I'd heard some of that in various discussions. Where
is the option for shell V tcl specified?

> Also, what is a "lesser lit-like" test runner?

In this case, it's a shell script that attempts to transform RUN lines
into a shell script to run a lit test - it works for the basic cases.

- David

>
>  - Daniel
>
>>
>> Thanks,
>> - David
>>
>> On Fri, Aug 31, 2012 at 10:49 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>> Author: dblaikie
>>> Date: Fri Aug 31 12:49:33 2012
>>> New Revision: 163009
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=163009&view=rev
>>> Log:
>>> Fix tests to be more robust (to older versions of grep, lesser lit-like test runners, etc)
>>>
>>> Seems I had a problem with my version of grep, when run by lit, not supporting
>>> the \s escape. This seems to fix it for me & I'll be getting the buildbots to
>>> run these tests too to keep an eye on them (actually loop-convert tests still
>>> fail when run via a make build, so that'll be addressed in a future commit). I
>>> could use [[:space:]] to generalize over other whitespace but that seemed
>>> unnecessarily verbose when the flexibility wasn't actually required by the
>>> current text of the tests.
>>>
>>> Also I just simplified a lot of the loop-convert tests (removing the
>>> unecessary temp file deletion at the start, removing the unnecessary && for
>>> FileCheck, etc).
>>>
>>> The remove-cstr-calls/basic.cpp changes were necessitated by an out of tree
>>> lit-like test runner that's a bit less fantastic about escaping. They were
>>> modeled on existing tooling test cases in Clang, with thanks to Manuel Klimek
>>> for the pointers.
>>>
>>> Modified:
>>>     clang-tools-extra/trunk/test/loop-convert/array.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/confidence.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/dependency.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/iterator.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/naming.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/negative.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/nesting.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/nocompile.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp
>>>     clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp
>>>     clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/array.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/array.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/array.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/array.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,12 +1,11 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:         && FileCheck -input-file=%t.cpp %s
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>>  // RUN: cp %t.cpp %t.base
>>> -// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out \
>>> -// RUN:         && FileCheck -check-prefix=COUNTONLY -input-file=%T/out %s \
>>> -// RUN:         && diff %t.cpp %t.base
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>> +// RUN: cp %t.base %t.cpp
>>> +// RUN: loop-convert -count-only . %t.cpp -- -I %S/Inputs > %T/out
>>> +// RUN: FileCheck -check-prefix=COUNTONLY -input-file=%T/out %s
>>> +// RUN: diff %t.cpp %t.base
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/confidence.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/confidence.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/confidence.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/confidence.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,9 +1,8 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:         && FileCheck -input-file=%t.cpp %s
>>> -// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs \
>>> -// RUN:         && FileCheck -check-prefix=RISKY -input-file=%t.cpp %s
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>> +// RUN: loop-convert . %t.cpp -A2 -- -I %S/Inputs
>>> +// RUN: FileCheck -check-prefix=RISKY -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/dependency.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/dependency.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/dependency.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/dependency.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,5 +1,4 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>>  // RUN: loop-convert . %t.cpp -- && FileCheck -input-file=%t.cpp %s
>>>
>>>  void f() {
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/iterator.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/iterator.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/iterator.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,9 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:           && FileCheck -input-file=%t.cpp %s
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: rm -rf %t.cpp
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/naming.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/naming.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/naming.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/naming.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,7 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:           && FileCheck -input-file=%t.cpp %s
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/negative-iterator.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,7 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:         && FileCheck -input-file=%t.cpp %s
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/negative-multi-end-call.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,7 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs \
>>> -// RUN:         && FileCheck -input-file=%t.cpp %s
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert -A0 . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray-extra.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,7 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \
>>> -// RUN:         && FileCheck -input-file=%t.cpp %s
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/negative-pseudoarray.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,7 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs \
>>> -// RUN:         && FileCheck -input-file=%t.cpp %s
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert -A1 . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/negative.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/negative.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/negative.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/negative.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,11 +1,9 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %S/Inputs/negative-header.h > \
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %S/Inputs/negative-header.h > \
>>>  // RUN:       %T/negative-header.h
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs/ \
>>> -// RUN:   && FileCheck -input-file=%t.cpp %s \
>>> -// RUN:   && FileCheck -input-file=%T/negative-header.h \
>>> -// RUN:                %S/Inputs/negative-header.h
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs/
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>> +// RUN: FileCheck -input-file=%T/negative-header.h %S/Inputs/negative-header.h
>>>
>>>  #include "negative-header.h"
>>>  #include "structures.h"
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/nesting.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nesting.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/nesting.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/nesting.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,7 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:           && FileCheck -input-file=%t.cpp %s
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/nocompile.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/nocompile.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/nocompile.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/nocompile.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,8 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:         || FileCheck -input-file=%t.cpp %s
>>> -// Note that this test expects the compilation to fail!
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: not loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  void valid() {
>>>    const int arr[5];
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/pseudoarray.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,8 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:           && FileCheck -input-file=%t.cpp %s
>>> -// RUN: rm -rf %t.cpp
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>  #include "structures.h"
>>>
>>>  const int N = 6;
>>>
>>> Modified: clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp (original)
>>> +++ clang-tools-extra/trunk/test/loop-convert/single-iterator.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,9 +1,6 @@
>>> -// RUN: rm -rf %t.cpp
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: loop-convert . %t.cpp -- -I %S/Inputs \
>>> -// RUN:           && FileCheck -input-file=%t.cpp %s
>>> -// RUN: grep -Ev "//\s*[A-Z-]+:" %s > %t.cpp
>>> -// RUN: rm -rf %t.cpp
>>> +// RUN: grep -Ev "// *[A-Z-]+:" %s > %t.cpp
>>> +// RUN: loop-convert . %t.cpp -- -I %S/Inputs
>>> +// RUN: FileCheck -input-file=%t.cpp %s
>>>
>>>  #include "structures.h"
>>>
>>>
>>> Modified: clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp?rev=163009&r1=163008&r2=163009&view=diff
>>> ==============================================================================
>>> --- clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp (original)
>>> +++ clang-tools-extra/trunk/test/remove-cstr-calls/basic.cpp Fri Aug 31 12:49:33 2012
>>> @@ -1,7 +1,9 @@
>>> -// RUN: echo '[{"directory":".","command":"clang++ -c %T/test.cpp","file":"%T/test.cpp"}]' > %T/compile_commands.json
>>> -// RUN: cp "%s" "%T/test.cpp"
>>> -// RUN: remove-cstr-calls "%T" "%T/test.cpp"
>>> -// RUN: cat "%T/test.cpp" | FileCheck %s
>>> +// RUN: rm -rf %t
>>> +// RUN: mkdir %t
>>> +// RUN: echo "[{\"directory\":\".\",\"command\":\"clang++ -c %t/test.cpp\",\"file\":\"%t/test.cpp\"}]" | sed -e 's/\\/\//g' > %t/compile_commands.json
>>> +// RUN: cp "%s" "%t/test.cpp"
>>> +// RUN: remove-cstr-calls "%t" "%t/test.cpp"
>>> +// RUN: cat "%t/test.cpp" | FileCheck %s
>>>  // REQUIRES: shell
>>>  // FIXME: implement a mode for refactoring tools that takes input from stdin
>>>  // and writes output to stdout for easier testing of tools.
>>>
>>>
>>> _______________________________________________
>>> cfe-commits mailing list
>>> cfe-commits at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits