[LLVMdev] How to write a regression test case?
Changcheng Wang
changcheng at multicorewareinc.com
Mon Aug 27 23:41:36 PDT 2012
aha,i add cBackend into llvm version 162227,my test run passed.
it so hapyy!
thanks for all of you!
still a question:
what does "; PR1028" means?
best wishes,
changcheng
On Tue, Aug 28, 2012 at 2:35 PM, Changcheng Wang
<changcheng at multicorewareinc.com> wrote:
> hi,yang:
>
> thanks,i replaced "{helloworld}" with "helloworld" as you said,and the
> test run well on llvm version 3.0,but fail on llvm 162227,is it right?
> llvm version 162227 do not support it?
>
> your,
> changcheng
>
>
>
> On Tue, Aug 28, 2012 at 2:28 PM, Triple Yang <triple.yang at gmail.com> wrote:
>> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>>> hi,yang:
>>> another question，i write a test like this:
>>> /*****test start******/
>>> ; RUN: llc -march=c < %s | FileCheck %s
>>>
>>> ; ModuleID = 'hello.c'
>>> target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-
>>> f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-
>>> n8:16:32:64-S128"
>>> target triple = "x86_64-unknown-linux-gnu"
>>>
>>> @.str = private unnamed_addr constant [12 x i8] c"helloworld\0A\00", align 1
>>>
>>> define i32 @main() nounwind uwtable {
>>> entry:
>>> %retval = alloca i32, align 4
>>> store i32 0, i32* %retval
>>> %call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12 x i8]*
>>> @.str, i32 0, i32 0))
>>> ret i32 0
>>> }
>>>
>>> declare i32 @printf(i8*, ...)
>>> ;
>>>
>>> ;CHECK: main
>>> ;CHECK: printf
>>> /******test end*******/
>>> i run the example ok,now i what to check if the file llc ouput contain
>>> a string "helloworld",so i add a sentence ";CHECK: {helloworld}" after
>>> ";CHECK: printf".but it failed.why?
>>
>> Why do you check "{helloworld}" rather than "helloworld"?
>>
>> If you are not sure a string pattern does exist in result file,
>> compile it manually and check the content before
>> adding a CHECK to test it.
>>
>>>
>>> yours,
>>> changcheng
>>>
>>>
>>> On Tue, Aug 28, 2012 at 12:38 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>> You are welcome.
>>>>
>>>> /* example.c */
>>>> // RUN: clang %s -S -O0 -emit-llvm -o %t.ll
>>>> // RUN: llc %t.ll -O0 -march=x86 -o %t.s
>>>> // RUN: FileCheck < %t.s %s
>>>>
>>>>
>>>> // CHECK: fct
>>>> unsigned int fct(unsigned int n) {
>>>> if(n>0) return fct(n-1)*n;
>>>> else return 1;
>>>> }
>>>> /* end example.c */
>>>>
>>>>
>>>> /* lit.local.cfg */
>>>> config.suffixes = ['.c', '.cpp', '.ll']
>>>>
>>>> targets = set(config.root.targets_to_build.split())
>>>> if not 'VPU' in targets:
>>>> config.unsupported = True
>>>> /* end lit.local.cfg */
>>>>
>>>>
>>>> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>> hi,yang:
>>>>> thanks for your help!
>>>>> my net is so poor that i can not donwload your attachment.
>>>>> would you like to paste here,or send to my other email:200005275 at 163.com?
>>>>>
>>>>> yours
>>>>>
>>>>> changcheng
>>>>>
>>>>>
>>>>>
>>>>> On Tue, Aug 28, 2012 at 11:43 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>> Wang,
>>>>>>
>>>>>> Attachment is a simple example, you may put it in test/CodeGen, and run with
>>>>>>
>>>>>> llvm-lit C90
>>>>>>
>>>>>> It worked ok in my llvm 3.2 devel. Let know if you run into any trouble.
>>>>>>
>>>>>> Regards.
>>>>>>
>>>>>> 2012/8/27 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>> hi,Yang:
>>>>>>> if you can give me a examle,which check out a string(such as "abcd")
>>>>>>> from a file(such as aaa.c) with fileCheck?
>>>>>>> i tried to write it but failed.
>>>>>>>
>>>>>>> thanks for your help!
>>>>>>>
>>>>>>> best wished!
>>>>>>>
>>>>>>> changcheng
>>>>>>>
>>>>>>> On Wed, Aug 22, 2012 at 9:30 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>> I guess so.
>>>>>>>> FileCheck has powerful extensions than just matching some strings so that
>>>>>>>> it can support complex situations.
>>>>>>>>
>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>> yeah,i see.
>>>>>>>>> i thought FileCheck check if the same string in the two file,is it?
>>>>>>>>> the file t.s maybe do not have the same string with %s,so FileCheck
>>>>>>>>> perhaps give a fail.
>>>>>>>>> i am not sure about it.
>>>>>>>>>
>>>>>>>>> On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>> the example is more like what i need,it is so nice.
>>>>>>>>>>> but,i am indefinite if "RUN: FileCheck < %t.s %s" can pass,i
>>>>>>>>>>> understand that t.s was translate from t.ll,%s means read the local
>>>>>>>>>>> source,are they the same?
>>>>>>>>>>
>>>>>>>>>> %t indicates a temporary file for output. ".s" is just a suffix I used
>>>>>>>>>> conventionly.
>>>>>>>>>> You can replace it with any string you like.
>>>>>>>>>>
>>>>>>>>>> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>> I did an experiment just now with a test case like:
>>>>>>>>>>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>>>>>>>>>>> // RUN: llc %t.ll -o %t.s
>>>>>>>>>>>> // RUN: FileCheck < %t.s %s
>>>>>>>>>>>>
>>>>>>>>>>>> ...
>>>>>>>>>>>>
>>>>>>>>>>>> // CHECK: ...
>>>>>>>>>>>>
>>>>>>>>>>>> And it worked.
>>>>>>>>>>>>
>>>>>>>>>>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>>>>>>>>>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>>>>>>>>>>
>>>>>>>>>>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>> Hi,Yang
>>>>>>>>>>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>>>>>>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>>> #include"stdio.h"
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> int main() {
>>>>>>>>>>>>>>> printf ("Helloworld.\n");
>>>>>>>>>>>>>>> return 0;
>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>>>>>>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>>> .......
>>>>>>>>>>>>>>> ;CHECK.....
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>>>>>>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>>>>>>>>>> clang -emit-llvm -S test.c -o test.ll
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And the corresponding command for regression test might be:
>>>>>>>>>>>>>> ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>>>
>>>>>>>>>>>>> We can write:
>>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S
>>>>>>>>>>>>> in a .c file to test clang.
>>>>>>>>>>>>>
>>>>>>>>>>>>> And if we use FileCheck, like:
>>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>>>>>>>>>>
>>>>>>>>>>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>>>>>>>>>>> llvmdev at cs.uiuc.edu)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks very much!
>>>>>>>>>>>>>>> best wishes
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>>>>> Hi, changcheng,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> There are following steps to be taken to write a test case:
>>>>>>>>>>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>>>>>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>>>>>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>>>>>>>>>> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>>>>>>>>>>> modification on it.
>>>>>>>>>>>>>>>> 3. create a file, say "example.ll"
>>>>>>>>>>>>>>>> (The name of file does not matter, but the suffix of the file name
>>>>>>>>>>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>>> example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>>>>>>>>>>> directives at the beginning of the file.
>>>>>>>>>>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>>>>>>>>>>> above test case newly added.
>>>>>>>>>>>>>>>> Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>>>>>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>>>>>>>>>> llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>>>>>>>>>>> install tree. You can find it under
>>>>>>>>>>>>>>>> your-build-tree/Release+Asserts/bin/ or
>>>>>>>>>>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>>>>>>>>>>> build
>>>>>>>>>>>>>>>> configuration.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>>>>>>>>>>> ; RUN: llc < %s -march=x86 | FileCheck %s
>>>>>>>>>>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>>>>>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>>>>>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>>>>>>>>>>> on exactly CURRENT file.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hope it work for you.
>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>>>> hi,all:
>>>>>>>>>>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>>>>>>>>>>> out a entire document about it.
>>>>>>>>>>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>>>>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>>>>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>>>>>>>>>>> write a RUN:lines.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> best wishes,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> 杨勇勇 (Yang Yongyong)
>>>>
>>>>
>>>>
>>>> --
>>>> 杨勇勇 (Yang Yongyong)
>>
>>
>>
>> --
>> 杨勇勇 (Yang Yongyong)
More information about the llvm-dev
mailing list