[LLVMdev] How to write a regression test case?
Changcheng Wang
changcheng at multicorewareinc.com
Mon Aug 27 23:35:23 PDT 2012
hi,yang:
thanks,i replaced "{helloworld}" with "helloworld" as you said,and the
test run well on llvm version 3.0,but fail on llvm 162227,is it right?
llvm version 162227 do not support it?
your,
changcheng
On Tue, Aug 28, 2012 at 2:28 PM, Triple Yang <triple.yang at gmail.com> wrote:
> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>> hi,yang:
>> another question，i write a test like this:
>> /*****test start******/
>> ; RUN: llc -march=c < %s | FileCheck %s
>>
>> ; ModuleID = 'hello.c'
>> target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-
>> f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-
>> n8:16:32:64-S128"
>> target triple = "x86_64-unknown-linux-gnu"
>>
>> @.str = private unnamed_addr constant [12 x i8] c"helloworld\0A\00", align 1
>>
>> define i32 @main() nounwind uwtable {
>> entry:
>> %retval = alloca i32, align 4
>> store i32 0, i32* %retval
>> %call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12 x i8]*
>> @.str, i32 0, i32 0))
>> ret i32 0
>> }
>>
>> declare i32 @printf(i8*, ...)
>> ;
>>
>> ;CHECK: main
>> ;CHECK: printf
>> /******test end*******/
>> i run the example ok,now i what to check if the file llc ouput contain
>> a string "helloworld",so i add a sentence ";CHECK: {helloworld}" after
>> ";CHECK: printf".but it failed.why?
>
> Why do you check "{helloworld}" rather than "helloworld"?
>
> If you are not sure a string pattern does exist in result file,
> compile it manually and check the content before
> adding a CHECK to test it.
>
>>
>> yours,
>> changcheng
>>
>>
>> On Tue, Aug 28, 2012 at 12:38 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>> You are welcome.
>>>
>>> /* example.c */
>>> // RUN: clang %s -S -O0 -emit-llvm -o %t.ll
>>> // RUN: llc %t.ll -O0 -march=x86 -o %t.s
>>> // RUN: FileCheck < %t.s %s
>>>
>>>
>>> // CHECK: fct
>>> unsigned int fct(unsigned int n) {
>>> if(n>0) return fct(n-1)*n;
>>> else return 1;
>>> }
>>> /* end example.c */
>>>
>>>
>>> /* lit.local.cfg */
>>> config.suffixes = ['.c', '.cpp', '.ll']
>>>
>>> targets = set(config.root.targets_to_build.split())
>>> if not 'VPU' in targets:
>>> config.unsupported = True
>>> /* end lit.local.cfg */
>>>
>>>
>>> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>> hi,yang:
>>>> thanks for your help!
>>>> my net is so poor that i can not donwload your attachment.
>>>> would you like to paste here,or send to my other email:200005275 at 163.com?
>>>>
>>>> yours
>>>>
>>>> changcheng
>>>>
>>>>
>>>>
>>>> On Tue, Aug 28, 2012 at 11:43 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>> Wang,
>>>>>
>>>>> Attachment is a simple example, you may put it in test/CodeGen, and run with
>>>>>
>>>>> llvm-lit C90
>>>>>
>>>>> It worked ok in my llvm 3.2 devel. Let know if you run into any trouble.
>>>>>
>>>>> Regards.
>>>>>
>>>>> 2012/8/27 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>> hi,Yang:
>>>>>> if you can give me a examle,which check out a string(such as "abcd")
>>>>>> from a file(such as aaa.c) with fileCheck?
>>>>>> i tried to write it but failed.
>>>>>>
>>>>>> thanks for your help!
>>>>>>
>>>>>> best wished!
>>>>>>
>>>>>> changcheng
>>>>>>
>>>>>> On Wed, Aug 22, 2012 at 9:30 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>> I guess so.
>>>>>>> FileCheck has powerful extensions than just matching some strings so that
>>>>>>> it can support complex situations.
>>>>>>>
>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>> yeah,i see.
>>>>>>>> i thought FileCheck check if the same string in the two file,is it?
>>>>>>>> the file t.s maybe do not have the same string with %s,so FileCheck
>>>>>>>> perhaps give a fail.
>>>>>>>> i am not sure about it.
>>>>>>>>
>>>>>>>> On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>> the example is more like what i need,it is so nice.
>>>>>>>>>> but,i am indefinite if "RUN: FileCheck < %t.s %s" can pass,i
>>>>>>>>>> understand that t.s was translate from t.ll,%s means read the local
>>>>>>>>>> source,are they the same?
>>>>>>>>>
>>>>>>>>> %t indicates a temporary file for output. ".s" is just a suffix I used
>>>>>>>>> conventionly.
>>>>>>>>> You can replace it with any string you like.
>>>>>>>>>
>>>>>>>>> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>> I did an experiment just now with a test case like:
>>>>>>>>>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>>>>>>>>>> // RUN: llc %t.ll -o %t.s
>>>>>>>>>>> // RUN: FileCheck < %t.s %s
>>>>>>>>>>>
>>>>>>>>>>> ...
>>>>>>>>>>>
>>>>>>>>>>> // CHECK: ...
>>>>>>>>>>>
>>>>>>>>>>> And it worked.
>>>>>>>>>>>
>>>>>>>>>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>>>>>>>>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>>>>>>>>>
>>>>>>>>>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>> Hi,Yang
>>>>>>>>>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>>>>>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>> #include"stdio.h"
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> int main() {
>>>>>>>>>>>>>> printf ("Helloworld.\n");
>>>>>>>>>>>>>> return 0;
>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>>>>>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>> .......
>>>>>>>>>>>>>> ;CHECK.....
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>>>>>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>>>>>>>>>
>>>>>>>>>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>>>>>>>>> clang -emit-llvm -S test.c -o test.ll
>>>>>>>>>>>>>
>>>>>>>>>>>>> And the corresponding command for regression test might be:
>>>>>>>>>>>>> ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>>
>>>>>>>>>>>> We can write:
>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S
>>>>>>>>>>>> in a .c file to test clang.
>>>>>>>>>>>>
>>>>>>>>>>>> And if we use FileCheck, like:
>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>>>>>>>>>
>>>>>>>>>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>
>>>>>>>>>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>>>>>>>>>> llvmdev at cs.uiuc.edu)
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> thanks very much!
>>>>>>>>>>>>>> best wishes
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>>>> Hi, changcheng,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> There are following steps to be taken to write a test case:
>>>>>>>>>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>>>>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>>>>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>>>>>>>>> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>>>>>>>>>> modification on it.
>>>>>>>>>>>>>>> 3. create a file, say "example.ll"
>>>>>>>>>>>>>>> (The name of file does not matter, but the suffix of the file name
>>>>>>>>>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>> example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>>>>>>>>>> directives at the beginning of the file.
>>>>>>>>>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>>>>>>>>>> above test case newly added.
>>>>>>>>>>>>>>> Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>>>>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>>>>>>>>> llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>>>>>>>>>> install tree. You can find it under
>>>>>>>>>>>>>>> your-build-tree/Release+Asserts/bin/ or
>>>>>>>>>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>>>>>>>>>> build
>>>>>>>>>>>>>>> configuration.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>>>>>>>>>> ; RUN: llc < %s -march=x86 | FileCheck %s
>>>>>>>>>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>>>>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>>>>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>>>>>>>>>> on exactly CURRENT file.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hope it work for you.
>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>>> hi,all:
>>>>>>>>>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>>>>>>>>>> out a entire document about it.
>>>>>>>>>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>>>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>>>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>>>>>>>>>> write a RUN:lines.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> best wishes,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> 杨勇勇 (Yang Yongyong)
>>>
>>>
>>>
>>> --
>>> 杨勇勇 (Yang Yongyong)
>
>
>
> --
> 杨勇勇 (Yang Yongyong)
More information about the llvm-dev
mailing list