[LLVMdev] How to write a regression test case?
Changcheng Wang
changcheng at multicorewareinc.com
Mon Aug 27 22:28:01 PDT 2012
hi,yang:
another question，i write a test like this:
/*****test start******/
; RUN: llc -march=c < %s | FileCheck %s
; ModuleID = 'hello.c'
target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-
f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-
n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"
@.str = private unnamed_addr constant [12 x i8] c"helloworld\0A\00", align 1
define i32 @main() nounwind uwtable {
entry:
%retval = alloca i32, align 4
store i32 0, i32* %retval
%call = call i32 (i8*, ...)* @printf(i8* getelementptr inbounds ([12 x i8]*
@.str, i32 0, i32 0))
ret i32 0
}
declare i32 @printf(i8*, ...)
;
;CHECK: main
;CHECK: printf
/******test end*******/
i run the example ok,now i what to check if the file llc ouput contain
a string "helloworld",so i add a sentence ";CHECK: {helloworld}" after
";CHECK: printf".but it failed.why?
yours,
changcheng
On Tue, Aug 28, 2012 at 12:38 PM, Triple Yang <triple.yang at gmail.com> wrote:
> You are welcome.
>
> /* example.c */
> // RUN: clang %s -S -O0 -emit-llvm -o %t.ll
> // RUN: llc %t.ll -O0 -march=x86 -o %t.s
> // RUN: FileCheck < %t.s %s
>
>
> // CHECK: fct
> unsigned int fct(unsigned int n) {
> if(n>0) return fct(n-1)*n;
> else return 1;
> }
> /* end example.c */
>
>
> /* lit.local.cfg */
> config.suffixes = ['.c', '.cpp', '.ll']
>
> targets = set(config.root.targets_to_build.split())
> if not 'VPU' in targets:
> config.unsupported = True
> /* end lit.local.cfg */
>
>
> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>> hi,yang:
>> thanks for your help!
>> my net is so poor that i can not donwload your attachment.
>> would you like to paste here,or send to my other email:200005275 at 163.com?
>>
>> yours
>>
>> changcheng
>>
>>
>>
>> On Tue, Aug 28, 2012 at 11:43 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>> Wang,
>>>
>>> Attachment is a simple example, you may put it in test/CodeGen, and run with
>>>
>>> llvm-lit C90
>>>
>>> It worked ok in my llvm 3.2 devel. Let know if you run into any trouble.
>>>
>>> Regards.
>>>
>>> 2012/8/27 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>> hi,Yang:
>>>> if you can give me a examle,which check out a string(such as "abcd")
>>>> from a file(such as aaa.c) with fileCheck?
>>>> i tried to write it but failed.
>>>>
>>>> thanks for your help!
>>>>
>>>> best wished!
>>>>
>>>> changcheng
>>>>
>>>> On Wed, Aug 22, 2012 at 9:30 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>> I guess so.
>>>>> FileCheck has powerful extensions than just matching some strings so that
>>>>> it can support complex situations.
>>>>>
>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>> yeah,i see.
>>>>>> i thought FileCheck check if the same string in the two file,is it?
>>>>>> the file t.s maybe do not have the same string with %s,so FileCheck
>>>>>> perhaps give a fail.
>>>>>> i am not sure about it.
>>>>>>
>>>>>> On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>> the example is more like what i need,it is so nice.
>>>>>>>> but,i am indefinite if "RUN: FileCheck < %t.s %s" can pass,i
>>>>>>>> understand that t.s was translate from t.ll,%s means read the local
>>>>>>>> source,are they the same?
>>>>>>>
>>>>>>> %t indicates a temporary file for output. ".s" is just a suffix I used
>>>>>>> conventionly.
>>>>>>> You can replace it with any string you like.
>>>>>>>
>>>>>>> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>> I did an experiment just now with a test case like:
>>>>>>>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>>>>>>>> // RUN: llc %t.ll -o %t.s
>>>>>>>>> // RUN: FileCheck < %t.s %s
>>>>>>>>>
>>>>>>>>> ...
>>>>>>>>>
>>>>>>>>> // CHECK: ...
>>>>>>>>>
>>>>>>>>> And it worked.
>>>>>>>>>
>>>>>>>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>>>>>>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>>>>>>>
>>>>>>>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>> Hi,Yang
>>>>>>>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>>>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>>>>>>>
>>>>>>>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>> #include"stdio.h"
>>>>>>>>>>>>
>>>>>>>>>>>> int main() {
>>>>>>>>>>>> printf ("Helloworld.\n");
>>>>>>>>>>>> return 0;
>>>>>>>>>>>> }
>>>>>>>>>>>>
>>>>>>>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>>>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>> .......
>>>>>>>>>>>> ;CHECK.....
>>>>>>>>>>>>
>>>>>>>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>>>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>>>>>>>
>>>>>>>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>>>>>>>
>>>>>>>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>>>>>>> clang -emit-llvm -S test.c -o test.ll
>>>>>>>>>>>
>>>>>>>>>>> And the corresponding command for regression test might be:
>>>>>>>>>>> ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>
>>>>>>>>>> We can write:
>>>>>>>>>> // RUN: clang < %s -emit-llvm -S
>>>>>>>>>> in a .c file to test clang.
>>>>>>>>>>
>>>>>>>>>> And if we use FileCheck, like:
>>>>>>>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>>>>>>>
>>>>>>>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Regards.
>>>>>>>>>>>
>>>>>>>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>>>>>>>> llvmdev at cs.uiuc.edu)
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> thanks very much!
>>>>>>>>>>>> best wishes
>>>>>>>>>>>>
>>>>>>>>>>>> changcheng
>>>>>>>>>>>>
>>>>>>>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>> Hi, changcheng,
>>>>>>>>>>>>>
>>>>>>>>>>>>> There are following steps to be taken to write a test case:
>>>>>>>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>>>>>>> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>>>>>>>> modification on it.
>>>>>>>>>>>>> 3. create a file, say "example.ll"
>>>>>>>>>>>>> (The name of file does not matter, but the suffix of the file name
>>>>>>>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>> example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>>>>>>>> directives at the beginning of the file.
>>>>>>>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>>>>>>>> above test case newly added.
>>>>>>>>>>>>> Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>>>>>>> llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>>>>>>>> install tree. You can find it under
>>>>>>>>>>>>> your-build-tree/Release+Asserts/bin/ or
>>>>>>>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>>>>>>>> build
>>>>>>>>>>>>> configuration.
>>>>>>>>>>>>>
>>>>>>>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>>>>>>>> ; RUN: llc < %s -march=x86 | FileCheck %s
>>>>>>>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>>>>>>>> on exactly CURRENT file.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Hope it work for you.
>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>> hi,all:
>>>>>>>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>>>>>>>> out a entire document about it.
>>>>>>>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>>>>>>>> write a RUN:lines.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> best wishes,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> --
>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> 杨勇勇 (Yang Yongyong)
>>>
>>>
>>>
>>> --
>>> 杨勇勇 (Yang Yongyong)
>
>
>
> --
> 杨勇勇 (Yang Yongyong)
More information about the llvm-dev
mailing list