[LLVMdev] How to write a regression test case?

[Triple Yang]

aha, that's quite straightforward.

2012/8/29 Changcheng Wang <changcheng at multicorewareinc.com>:
> hi,yang:
> i had delete the temporary files,sorry to trouble you.
> it is so easy: // RUN: rm ....
>
> Regards
> changcheng
>
> On Wed, Aug 29, 2012 at 2:41 PM, Changcheng Wang
> <changcheng at multicorewareinc.com> wrote:
>> hi,yang:
>> how can i remove the temporary files?
>> if i can remove them,all is ok!
>>
>> yours:
>>
>> changcheng
>>
>> On Wed, Aug 29, 2012 at 2:14 PM, Changcheng Wang
>> <changcheng at multicorewareinc.com> wrote:
>>> hi,yang:
>>> sorry to trouble you another time,i used your example and it run
>>> well,but a problem came,it create severel temporary files i chaged the
>>> RUN: line but failed.
>>> i changed it like this,can you give me some suggestions?
>>>
>>> // RUN: clang %s -S -emit-llvm | llc -march=c | clang -S -emit-llvm |
>>> lli | FileCheck %s
>>>
>>> thanks,
>>> best wishes,
>>>
>>> changcheng
>>>
>>>
>>> On Wed, Aug 29, 2012 at 8:39 AM, Changcheng Wang
>>> <changcheng at multicorewareinc.com> wrote:
>>>> hi,yang:
>>>> i had realized the test.
>>>> thanks for your help!
>>>>
>>>> yours,
>>>> changcheng
>>>>
>>>> On Tue, Aug 28, 2012 at 8:58 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>> That example is a standalone one which means you can run it without
>>>>> extra modification.
>>>>> Feel free to try it.
>>>>>
>>>>> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>> hi,yang:
>>>>>> sorry to trouble you,i have a little question about the example you give me.
>>>>>> the example is a C file,can i run it without translate into a ll file?
>>>>>> the command line maybe like this: llvm-lit ./example.c
>>>>>>
>>>>>
>>>>> Yes, you can. There is no need for extra translation.
>>>>>
>>>>>> as i want to write a test to check cBackend,i want the test :
>>>>>> 1.translate a example.c file to example.ll file with clang.
>>>>>> 2.translate the example.ll file to transExample.c file with cBackend.
>>>>>> 3.compile the transExample.c file to transExample.bc with clang.
>>>>>> 4.run the transExample.bc and check the output.
>>>>>>
>>>>>> can i write the RUN: line into example.c file just like you did?
>>>>>
>>>>> I guess so. Just pay attention to file IO and stdio. If you are not
>>>>> sure about something, redirect IO flows to temporary files.
>>>>>
>>>>> Anyway, feel free to try.
>>>>>
>>>>>>
>>>>>> thanks,
>>>>>> your,
>>>>>> changcheng
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 28, 2012 at 11:40 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>> Wang,
>>>>>>> Attachment is a simple example, you may put it in test/CodeGen, and run with
>>>>>>> llvm-lit C90
>>>>>>>
>>>>>>> It worked ok in my llvm 3.2 devel. Let know if you come into any trouble.
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>> 2012/8/27 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>> hi,Yang:
>>>>>>>> if you can give me a examle,which check out a string(such as "abcd")
>>>>>>>> from a file(such as aaa.c) with fileCheck?
>>>>>>>> i tried to write it but failed.
>>>>>>>>
>>>>>>>> thanks for your help!
>>>>>>>>
>>>>>>>> best wished!
>>>>>>>>
>>>>>>>> changcheng
>>>>>>>>
>>>>>>>> On Wed, Aug 22, 2012 at 9:30 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>> I guess so.
>>>>>>>>> FileCheck has powerful extensions than just matching some strings so that
>>>>>>>>> it can support complex situations.
>>>>>>>>>
>>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>> yeah,i see.
>>>>>>>>>> i thought FileCheck check if the same string in the two file,is it?
>>>>>>>>>> the file t.s maybe do not have the same string with %s,so FileCheck
>>>>>>>>>> perhaps give a fail.
>>>>>>>>>> i am not sure about it.
>>>>>>>>>>
>>>>>>>>>> On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>> the example is more like what i need,it is so nice.
>>>>>>>>>>>> but,i am indefinite if "RUN: FileCheck < %t.s  %s" can pass,i
>>>>>>>>>>>> understand that t.s was translate from t.ll,%s means read the local
>>>>>>>>>>>> source,are they the same?
>>>>>>>>>>>
>>>>>>>>>>> %t indicates a temporary file for output. ".s" is just a suffix I used
>>>>>>>>>>> conventionly.
>>>>>>>>>>> You can replace it with any string you like.
>>>>>>>>>>>
>>>>>>>>>>> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>> I did an experiment just now with a test case like:
>>>>>>>>>>>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>>>>>>>>>>>> // RUN: llc %t.ll  -o %t.s
>>>>>>>>>>>>> // RUN: FileCheck < %t.s  %s
>>>>>>>>>>>>>
>>>>>>>>>>>>> ...
>>>>>>>>>>>>>
>>>>>>>>>>>>> // CHECK: ...
>>>>>>>>>>>>>
>>>>>>>>>>>>> And it worked.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>>>>>>>>>>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>>> Hi,Yang
>>>>>>>>>>>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>>>>>>>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>>>> #include"stdio.h"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> int main() {
>>>>>>>>>>>>>>>>   printf ("Helloworld.\n");
>>>>>>>>>>>>>>>>   return 0;
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>>>>>>>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>>>> .......
>>>>>>>>>>>>>>>> ;CHECK.....
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>>>>>>>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>>>>>>>>>>>       clang -emit-llvm -S test.c -o test.ll
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> And the corresponding command for regression test might be:
>>>>>>>>>>>>>>>      ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We can write:
>>>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S
>>>>>>>>>>>>>> in a .c file to test clang.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And if we use  FileCheck, like:
>>>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>>>>>>>>>>>> llvmdev at cs.uiuc.edu)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thanks very much!
>>>>>>>>>>>>>>>> best wishes
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>>>>>> Hi, changcheng,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There are following steps to be taken to write a test case:
>>>>>>>>>>>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>>>>>>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>>>>>>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>>>>>>>>>>>     like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>>>>>>>>>>>> modification on it.
>>>>>>>>>>>>>>>>> 3. create a file, say "example.ll"
>>>>>>>>>>>>>>>>>     (The name of file does not matter, but the suffix of the file name
>>>>>>>>>>>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>>>>>>>>>>>     for more details.
>>>>>>>>>>>>>>>>>     example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>>>>>>>>>>>> directives at the beginning of the file.
>>>>>>>>>>>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>>>>>>>>>>>> above test case newly added.
>>>>>>>>>>>>>>>>>     Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>>>>>>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>>>>>>>>>>>     llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>>>>>>>>>>>> install tree. You can find it under
>>>>>>>>>>>>>>>>>     your-build-tree/Release+Asserts/bin/ or
>>>>>>>>>>>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>>>>>>>>>>>> build
>>>>>>>>>>>>>>>>>     configuration.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>>>>>>>>>>>> ; RUN: llc < %s  -march=x86 | FileCheck %s
>>>>>>>>>>>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>>>>>>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>>>>>>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>>>>>>>>>>>> on exactly CURRENT file.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hope it work for you.
>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>>>>> hi,all:
>>>>>>>>>>>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>>>>>>>>>>>> out a entire document about it.
>>>>>>>>>>>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>>>>>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>>>>>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>>>>>>>>>>>> write a RUN:lines.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> best wishes,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>>>>>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>>>>>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> 杨勇勇 (Yang Yongyong)



-- 
杨勇勇 (Yang Yongyong)