[LLVMdev] How to write a regression test case?

[Triple Yang]

Wang,
Attachment is a simple example, you may put it in test/CodeGen, and run with
llvm-lit C90

It worked ok in my llvm 3.2 devel. Let know if you come into any trouble.

Regards.

2012/8/27 Changcheng Wang <changcheng at multicorewareinc.com>:
> hi,Yang:
> if you can give me a examle,which check out a string(such as "abcd")
> from a file(such as aaa.c) with fileCheck?
> i tried to write it but failed.
>
> thanks for your help!
>
> best wished!
>
> changcheng
>
> On Wed, Aug 22, 2012 at 9:30 PM, Triple Yang <triple.yang at gmail.com> wrote:
>> I guess so.
>> FileCheck has powerful extensions than just matching some strings so that
>> it can support complex situations.
>>
>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>> yeah,i see.
>>> i thought FileCheck check if the same string in the two file,is it?
>>> the file t.s maybe do not have the same string with %s,so FileCheck
>>> perhaps give a fail.
>>> i am not sure about it.
>>>
>>> On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>> the example is more like what i need,it is so nice.
>>>>> but,i am indefinite if "RUN: FileCheck < %t.s  %s" can pass,i
>>>>> understand that t.s was translate from t.ll,%s means read the local
>>>>> source,are they the same?
>>>>
>>>> %t indicates a temporary file for output. ".s" is just a suffix I used
>>>> conventionly.
>>>> You can replace it with any string you like.
>>>>
>>>> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>>>>
>>>>>
>>>>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>> I did an experiment just now with a test case like:
>>>>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>>>>> // RUN: llc %t.ll  -o %t.s
>>>>>> // RUN: FileCheck < %t.s  %s
>>>>>>
>>>>>> ...
>>>>>>
>>>>>> // CHECK: ...
>>>>>>
>>>>>> And it worked.
>>>>>>
>>>>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>>>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>>>>
>>>>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>> Hi,Yang
>>>>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>>>>
>>>>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>> #include"stdio.h"
>>>>>>>>>
>>>>>>>>> int main() {
>>>>>>>>>   printf ("Helloworld.\n");
>>>>>>>>>   return 0;
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>> .......
>>>>>>>>> ;CHECK.....
>>>>>>>>>
>>>>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>>>>
>>>>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>>>>
>>>>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>>>>       clang -emit-llvm -S test.c -o test.ll
>>>>>>>>
>>>>>>>> And the corresponding command for regression test might be:
>>>>>>>>      ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>
>>>>>>> We can write:
>>>>>>> // RUN: clang < %s -emit-llvm -S
>>>>>>> in a .c file to test clang.
>>>>>>>
>>>>>>> And if we use  FileCheck, like:
>>>>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>>>>
>>>>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>>>>
>>>>>>>>
>>>>>>>> Regards.
>>>>>>>>
>>>>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>>>>> llvmdev at cs.uiuc.edu)
>>>>>>>>
>>>>>>>>>
>>>>>>>>> thanks very much!
>>>>>>>>> best wishes
>>>>>>>>>
>>>>>>>>> changcheng
>>>>>>>>>
>>>>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>> Hi, changcheng,
>>>>>>>>>>
>>>>>>>>>> There are following steps to be taken to write a test case:
>>>>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>>>>     like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>>>>> modification on it.
>>>>>>>>>> 3. create a file, say "example.ll"
>>>>>>>>>>     (The name of file does not matter, but the suffix of the file name
>>>>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>>>>     for more details.
>>>>>>>>>>     example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>>>>> directives at the beginning of the file.
>>>>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>>>>> above test case newly added.
>>>>>>>>>>     Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>>>>     llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>>>>> install tree. You can find it under
>>>>>>>>>>     your-build-tree/Release+Asserts/bin/ or
>>>>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>>>>> build
>>>>>>>>>>     configuration.
>>>>>>>>>>
>>>>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>>>>> ; RUN: llc < %s  -march=x86 | FileCheck %s
>>>>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>>>>> on exactly CURRENT file.
>>>>>>>>>>
>>>>>>>>>> Hope it work for you.
>>>>>>>>>> Regards.
>>>>>>>>>>
>>>>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>> hi,all:
>>>>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>>>>> out a entire document about it.
>>>>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>>>>> write a RUN:lines.
>>>>>>>>>>>
>>>>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>>>>
>>>>>>>>>>> best wishes,
>>>>>>>>>>>
>>>>>>>>>>> changcheng
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> 杨勇勇 (Yang Yongyong)
>>>>
>>>>
>>>>
>>>> --
>>>> 杨勇勇 (Yang Yongyong)
>>
>>
>>
>> --
>> 杨勇勇 (Yang Yongyong)



-- 
杨勇勇 (Yang Yongyong)