[LLVMdev] How to write a regression test case?
Triple Yang
triple.yang at gmail.com
Mon Aug 27 23:54:24 PDT 2012
Cong!
I don't know about PR1028. try grep'ing the source codes to find
anything relative.
2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
> aha,i add cBackend into llvm version 162227,my test run passed.
> it so hapyy!
> thanks for all of you!
>
> still a question:
> what does "; PR1028" means?
>
> best wishes,
>
> changcheng
>
> On Tue, Aug 28, 2012 at 2:35 PM, Changcheng Wang
> <changcheng at multicorewareinc.com> wrote:
>> hi,yang:
>>
>> thanks,i replaced "{helloworld}" with "helloworld" as you said,and the
>> test run well on llvm version 3.0,but fail on llvm 162227,is it right?
>> llvm version 162227 do not support it?
>>
>> your,
>> changcheng
>>
>>
>>
>> On Tue, Aug 28, 2012 at 2:28 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>> hi,yang:
>>>> another question，i write a test like this:
>>>> /*****test start******/
>>>> ; RUN: llc -march=c < %s | FileCheck %s
>>>>
>>>> ; ModuleID = 'hello.c'
>>>> target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-
>>>> f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-
>>>> n8:16:32:64-S128"
>>>> target triple = "x86_64-unknown-linux-gnu"
>>>>
>>>> @.str = private unnamed_addr constant [12 x i8] c"helloworld\0A\00", align 1
>>>>
>>>> define i32 @main() nounwind uwtable {
>>>> entry:
>>>> %retval = alloca i32, align 4
>>>> store i32 0, i32* %retval
>>>> %call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12 x i8]*
>>>> @.str, i32 0, i32 0))
>>>> ret i32 0
>>>> }
>>>>
>>>> declare i32 @printf(i8*, ...)
>>>> ;
>>>>
>>>> ;CHECK: main
>>>> ;CHECK: printf
>>>> /******test end*******/
>>>> i run the example ok,now i what to check if the file llc ouput contain
>>>> a string "helloworld",so i add a sentence ";CHECK: {helloworld}" after
>>>> ";CHECK: printf".but it failed.why?
>>>
>>> Why do you check "{helloworld}" rather than "helloworld"?
>>>
>>> If you are not sure a string pattern does exist in result file,
>>> compile it manually and check the content before
>>> adding a CHECK to test it.
>>>
>>>>
>>>> yours,
>>>> changcheng
>>>>
>>>>
>>>> On Tue, Aug 28, 2012 at 12:38 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>> You are welcome.
>>>>>
>>>>> /* example.c */
>>>>> // RUN: clang %s -S -O0 -emit-llvm -o %t.ll
>>>>> // RUN: llc %t.ll -O0 -march=x86 -o %t.s
>>>>> // RUN: FileCheck < %t.s %s
>>>>>
>>>>>
>>>>> // CHECK: fct
>>>>> unsigned int fct(unsigned int n) {
>>>>> if(n>0) return fct(n-1)*n;
>>>>> else return 1;
>>>>> }
>>>>> /* end example.c */
>>>>>
>>>>>
>>>>> /* lit.local.cfg */
>>>>> config.suffixes = ['.c', '.cpp', '.ll']
>>>>>
>>>>> targets = set(config.root.targets_to_build.split())
>>>>> if not 'VPU' in targets:
>>>>> config.unsupported = True
>>>>> /* end lit.local.cfg */
>>>>>
>>>>>
>>>>> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>> hi,yang:
>>>>>> thanks for your help!
>>>>>> my net is so poor that i can not donwload your attachment.
>>>>>> would you like to paste here,or send to my other email:200005275 at 163.com?
>>>>>>
>>>>>> yours
>>>>>>
>>>>>> changcheng
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Aug 28, 2012 at 11:43 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>> Wang,
>>>>>>>
>>>>>>> Attachment is a simple example, you may put it in test/CodeGen, and run with
>>>>>>>
>>>>>>> llvm-lit C90
>>>>>>>
>>>>>>> It worked ok in my llvm 3.2 devel. Let know if you run into any trouble.
>>>>>>>
>>>>>>> Regards.
>>>>>>>
>>>>>>> 2012/8/27 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>> hi,Yang:
>>>>>>>> if you can give me a examle,which check out a string(such as "abcd")
>>>>>>>> from a file(such as aaa.c) with fileCheck?
>>>>>>>> i tried to write it but failed.
>>>>>>>>
>>>>>>>> thanks for your help!
>>>>>>>>
>>>>>>>> best wished!
>>>>>>>>
>>>>>>>> changcheng
>>>>>>>>
>>>>>>>> On Wed, Aug 22, 2012 at 9:30 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>> I guess so.
>>>>>>>>> FileCheck has powerful extensions than just matching some strings so that
>>>>>>>>> it can support complex situations.
>>>>>>>>>
>>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>> yeah,i see.
>>>>>>>>>> i thought FileCheck check if the same string in the two file,is it?
>>>>>>>>>> the file t.s maybe do not have the same string with %s,so FileCheck
>>>>>>>>>> perhaps give a fail.
>>>>>>>>>> i am not sure about it.
>>>>>>>>>>
>>>>>>>>>> On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>> the example is more like what i need,it is so nice.
>>>>>>>>>>>> but,i am indefinite if "RUN: FileCheck < %t.s %s" can pass,i
>>>>>>>>>>>> understand that t.s was translate from t.ll,%s means read the local
>>>>>>>>>>>> source,are they the same?
>>>>>>>>>>>
>>>>>>>>>>> %t indicates a temporary file for output. ".s" is just a suffix I used
>>>>>>>>>>> conventionly.
>>>>>>>>>>> You can replace it with any string you like.
>>>>>>>>>>>
>>>>>>>>>>> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>> I did an experiment just now with a test case like:
>>>>>>>>>>>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>>>>>>>>>>>> // RUN: llc %t.ll -o %t.s
>>>>>>>>>>>>> // RUN: FileCheck < %t.s %s
>>>>>>>>>>>>>
>>>>>>>>>>>>> ...
>>>>>>>>>>>>>
>>>>>>>>>>>>> // CHECK: ...
>>>>>>>>>>>>>
>>>>>>>>>>>>> And it worked.
>>>>>>>>>>>>>
>>>>>>>>>>>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>>>>>>>>>>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>>> Hi,Yang
>>>>>>>>>>>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>>>>>>>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>>>> #include"stdio.h"
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> int main() {
>>>>>>>>>>>>>>>> printf ("Helloworld.\n");
>>>>>>>>>>>>>>>> return 0;
>>>>>>>>>>>>>>>> }
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>>>>>>>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>>>>> .......
>>>>>>>>>>>>>>>> ;CHECK.....
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>>>>>>>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>>>>>>>>>>> clang -emit-llvm -S test.c -o test.ll
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> And the corresponding command for regression test might be:
>>>>>>>>>>>>>>> ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We can write:
>>>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S
>>>>>>>>>>>>>> in a .c file to test clang.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> And if we use FileCheck, like:
>>>>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>>>>>>>>>>>> llvmdev at cs.uiuc.edu)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> thanks very much!
>>>>>>>>>>>>>>>> best wishes
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>>>>>> Hi, changcheng,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> There are following steps to be taken to write a test case:
>>>>>>>>>>>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>>>>>>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>>>>>>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>>>>>>>>>>> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>>>>>>>>>>>> modification on it.
>>>>>>>>>>>>>>>>> 3. create a file, say "example.ll"
>>>>>>>>>>>>>>>>> (The name of file does not matter, but the suffix of the file name
>>>>>>>>>>>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>>>>> example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>>>>>>>>>>>> directives at the beginning of the file.
>>>>>>>>>>>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>>>>>>>>>>>> above test case newly added.
>>>>>>>>>>>>>>>>> Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>>>>>>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>>>>>>>>>>> llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>>>>>>>>>>>> install tree. You can find it under
>>>>>>>>>>>>>>>>> your-build-tree/Release+Asserts/bin/ or
>>>>>>>>>>>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>>>>>>>>>>>> build
>>>>>>>>>>>>>>>>> configuration.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>>>>>>>>>>>> ; RUN: llc < %s -march=x86 | FileCheck %s
>>>>>>>>>>>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>>>>>>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>>>>>>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>>>>>>>>>>>> on exactly CURRENT file.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hope it work for you.
>>>>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>>>>> hi,all:
>>>>>>>>>>>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>>>>>>>>>>>> out a entire document about it.
>>>>>>>>>>>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>>>>>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>>>>>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>>>>>>>>>>>> write a RUN:lines.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> best wishes,
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>>>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> 杨勇勇 (Yang Yongyong)
>>>
>>>
>>>
>>> --
>>> 杨勇勇 (Yang Yongyong)
--
杨勇勇 (Yang Yongyong)
More information about the llvm-dev
mailing list