[LLVMdev] How to write a regression test case?

[Changcheng Wang]

yeah,i see.
i thought FileCheck check if the same string in the two file,is it?
the file t.s maybe do not have the same string with %s,so FileCheck
perhaps give a fail.
i am not sure about it.

On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>> the example is more like what i need,it is so nice.
>> but,i am indefinite if "RUN: FileCheck < %t.s  %s" can pass,i
>> understand that t.s was translate from t.ll,%s means read the local
>> source,are they the same?
>
> %t indicates a temporary file for output. ".s" is just a suffix I used
> conventionly.
> You can replace it with any string you like.
>
> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>
>>
>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>> I did an experiment just now with a test case like:
>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>> // RUN: llc %t.ll  -o %t.s
>>> // RUN: FileCheck < %t.s  %s
>>>
>>> ...
>>>
>>> // CHECK: ...
>>>
>>> And it worked.
>>>
>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>
>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>> Hi,Yang
>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>
>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>> #include"stdio.h"
>>>>>>
>>>>>> int main() {
>>>>>>   printf ("Helloworld.\n");
>>>>>>   return 0;
>>>>>> }
>>>>>>
>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>> .......
>>>>>> ;CHECK.....
>>>>>>
>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>
>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>
>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>       clang -emit-llvm -S test.c -o test.ll
>>>>>
>>>>> And the corresponding command for regression test might be:
>>>>>      ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>
>>>> We can write:
>>>> // RUN: clang < %s -emit-llvm -S
>>>> in a .c file to test clang.
>>>>
>>>> And if we use  FileCheck, like:
>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>
>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>
>>>>>
>>>>> Regards.
>>>>>
>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>> llvmdev at cs.uiuc.edu)
>>>>>
>>>>>>
>>>>>> thanks very much!
>>>>>> best wishes
>>>>>>
>>>>>> changcheng
>>>>>>
>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>> Hi, changcheng,
>>>>>>>
>>>>>>> There are following steps to be taken to write a test case:
>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>     like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>> modification on it.
>>>>>>> 3. create a file, say "example.ll"
>>>>>>>     (The name of file does not matter, but the suffix of the file name
>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>     for more details.
>>>>>>>     example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>> directives at the beginning of the file.
>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>> above test case newly added.
>>>>>>>     Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>     llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>> install tree. You can find it under
>>>>>>>     your-build-tree/Release+Asserts/bin/ or
>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>> build
>>>>>>>     configuration.
>>>>>>>
>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>> ; RUN: llc < %s  -march=x86 | FileCheck %s
>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>> on exactly CURRENT file.
>>>>>>>
>>>>>>> Hope it work for you.
>>>>>>> Regards.
>>>>>>>
>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>> hi,all:
>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>> out a entire document about it.
>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>> write a RUN:lines.
>>>>>>>>
>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>
>>>>>>>> best wishes,
>>>>>>>>
>>>>>>>> changcheng
>>>>>>>> _______________________________________________
>>>>>>>> LLVM Developers mailing list
>>>>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> 杨勇勇 (Yang Yongyong)
>>>>
>>>>
>>>>
>>>> --
>>>> 杨勇勇 (Yang Yongyong)
>>>
>>>
>>>
>>> --
>>> 杨勇勇 (Yang Yongyong)
>
>
>
> --
> 杨勇勇 (Yang Yongyong)