[LLVMdev] How to write a regression test case?
Changcheng Wang
changcheng at multicorewareinc.com
Tue Aug 28 23:14:10 PDT 2012
hi,yang:
sorry to trouble you another time,i used your example and it run
well,but a problem came,it create severel temporary files i chaged the
RUN: line but failed.
i changed it like this,can you give me some suggestions?
// RUN: clang %s -S -emit-llvm | llc -march=c | clang -S -emit-llvm |
lli | FileCheck %s
thanks,
best wishes,
changcheng
On Wed, Aug 29, 2012 at 8:39 AM, Changcheng Wang
<changcheng at multicorewareinc.com> wrote:
> hi,yang:
> i had realized the test.
> thanks for your help!
>
> yours,
> changcheng
>
> On Tue, Aug 28, 2012 at 8:58 PM, Triple Yang <triple.yang at gmail.com> wrote:
>> That example is a standalone one which means you can run it without
>> extra modification.
>> Feel free to try it.
>>
>> 2012/8/28 Changcheng Wang <changcheng at multicorewareinc.com>:
>>> hi,yang:
>>> sorry to trouble you,i have a little question about the example you give me.
>>> the example is a C file,can i run it without translate into a ll file?
>>> the command line maybe like this: llvm-lit ./example.c
>>>
>>
>> Yes, you can. There is no need for extra translation.
>>
>>> as i want to write a test to check cBackend,i want the test :
>>> 1.translate a example.c file to example.ll file with clang.
>>> 2.translate the example.ll file to transExample.c file with cBackend.
>>> 3.compile the transExample.c file to transExample.bc with clang.
>>> 4.run the transExample.bc and check the output.
>>>
>>> can i write the RUN: line into example.c file just like you did?
>>
>> I guess so. Just pay attention to file IO and stdio. If you are not
>> sure about something, redirect IO flows to temporary files.
>>
>> Anyway, feel free to try.
>>
>>>
>>> thanks,
>>> your,
>>> changcheng
>>>
>>>
>>> On Tue, Aug 28, 2012 at 11:40 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>> Wang,
>>>> Attachment is a simple example, you may put it in test/CodeGen, and run with
>>>> llvm-lit C90
>>>>
>>>> It worked ok in my llvm 3.2 devel. Let know if you come into any trouble.
>>>>
>>>> Regards.
>>>>
>>>> 2012/8/27 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>> hi,Yang:
>>>>> if you can give me a examle,which check out a string(such as "abcd")
>>>>> from a file(such as aaa.c) with fileCheck?
>>>>> i tried to write it but failed.
>>>>>
>>>>> thanks for your help!
>>>>>
>>>>> best wished!
>>>>>
>>>>> changcheng
>>>>>
>>>>> On Wed, Aug 22, 2012 at 9:30 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>> I guess so.
>>>>>> FileCheck has powerful extensions than just matching some strings so that
>>>>>> it can support complex situations.
>>>>>>
>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>> yeah,i see.
>>>>>>> i thought FileCheck check if the same string in the two file,is it?
>>>>>>> the file t.s maybe do not have the same string with %s,so FileCheck
>>>>>>> perhaps give a fail.
>>>>>>> i am not sure about it.
>>>>>>>
>>>>>>> On Wed, Aug 22, 2012 at 3:32 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>> 2012/8/22 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>> the example is more like what i need,it is so nice.
>>>>>>>>> but,i am indefinite if "RUN: FileCheck < %t.s %s" can pass,i
>>>>>>>>> understand that t.s was translate from t.ll,%s means read the local
>>>>>>>>> source,are they the same?
>>>>>>>>
>>>>>>>> %t indicates a temporary file for output. ".s" is just a suffix I used
>>>>>>>> conventionly.
>>>>>>>> You can replace it with any string you like.
>>>>>>>>
>>>>>>>> See entry "tmp" in http://llvm.org/docs/TestingGuide.html#rtvars
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Aug 22, 2012 at 1:55 PM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>> I did an experiment just now with a test case like:
>>>>>>>>>> // RUN: clang %s -S -emit-llvm -o %t.ll
>>>>>>>>>> // RUN: llc %t.ll -o %t.s
>>>>>>>>>> // RUN: FileCheck < %t.s %s
>>>>>>>>>>
>>>>>>>>>> ...
>>>>>>>>>>
>>>>>>>>>> // CHECK: ...
>>>>>>>>>>
>>>>>>>>>> And it worked.
>>>>>>>>>>
>>>>>>>>>> It seems clang does not take stdin as its input, so "clang < %s" fails.
>>>>>>>>>> Meanwhile, both "llc < %t.ll" and "llc %t.ll" work.
>>>>>>>>>>
>>>>>>>>>> 2012/8/22 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>> 2012/8/21 Triple Yang <triple.yang at gmail.com>:
>>>>>>>>>>>> 2012/8/21 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>> Hi,Yang
>>>>>>>>>>>>> thanks for your entire answer,i will do it follow you.
>>>>>>>>>>>>> still another question puzzled me:i write a hello.c file like this:
>>>>>>>>>>>>>
>>>>>>>>>>>>> //RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>> #include"stdio.h"
>>>>>>>>>>>>>
>>>>>>>>>>>>> int main() {
>>>>>>>>>>>>> printf ("Helloworld.\n");
>>>>>>>>>>>>> return 0;
>>>>>>>>>>>>> }
>>>>>>>>>>>>>
>>>>>>>>>>>>> in fact, i want to thanslate it to a hello.ll file like this:
>>>>>>>>>>>>> ;RUN: llc -march=c < %s | FileCheck %s
>>>>>>>>>>>>> .......
>>>>>>>>>>>>> ;CHECK.....
>>>>>>>>>>>>>
>>>>>>>>>>>>> but,when i thanslate the hello.c with clang,the sentence "//RUN: llc
>>>>>>>>>>>>> -march=c < %s | FileCheck %s" was treated as comment.
>>>>>>>>>>>>>
>>>>>>>>>>>>> it samed the method translating hello.c to hello.ll is wrong,isnot it?
>>>>>>>>>>>>
>>>>>>>>>>>> To translate a .c file to a .ll file, you should wirte, for example,
>>>>>>>>>>>> clang -emit-llvm -S test.c -o test.ll
>>>>>>>>>>>>
>>>>>>>>>>>> And the corresponding command for regression test might be:
>>>>>>>>>>>> ; RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>>
>>>>>>>>>>> We can write:
>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S
>>>>>>>>>>> in a .c file to test clang.
>>>>>>>>>>>
>>>>>>>>>>> And if we use FileCheck, like:
>>>>>>>>>>> // RUN: clang < %s -emit-llvm -S | FileCheck %s
>>>>>>>>>>> make sure there is some "// CHECK:" in that file, or lit will report a failure.
>>>>>>>>>>>
>>>>>>>>>>> See utils/lit/lit/ExampleTests/Clang for more details.
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Regards.
>>>>>>>>>>>>
>>>>>>>>>>>> (PS. When you reply to this email, make sure you send a copy to
>>>>>>>>>>>> llvmdev at cs.uiuc.edu)
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> thanks very much!
>>>>>>>>>>>>> best wishes
>>>>>>>>>>>>>
>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Tue, Aug 21, 2012 at 10:46 AM, Triple Yang <triple.yang at gmail.com> wrote:
>>>>>>>>>>>>>> Hi, changcheng,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There are following steps to be taken to write a test case:
>>>>>>>>>>>>>> 1. create a directory in test, say "test/XXX"
>>>>>>>>>>>>>> 2. create a file named lit.local.cfg, which is a configuration file.
>>>>>>>>>>>>>> The straightforward way is to copy an existed one,
>>>>>>>>>>>>>> like "test/CodeGen/SPARC/lit.local.cfg". You might need make some
>>>>>>>>>>>>>> modification on it.
>>>>>>>>>>>>>> 3. create a file, say "example.ll"
>>>>>>>>>>>>>> (The name of file does not matter, but the suffix of the file name
>>>>>>>>>>>>>> DOES matter. See config.suffixes = [ ... ] in lit.local.cfg)
>>>>>>>>>>>>>> for more details.
>>>>>>>>>>>>>> example.ll includes normal content of LLVM IR, and the ";RUN ..."
>>>>>>>>>>>>>> directives at the beginning of the file.
>>>>>>>>>>>>>> 4. run all regression test cases with "make check", which includes
>>>>>>>>>>>>>> above test case newly added.
>>>>>>>>>>>>>> Or you might want to run just that case with "llvm-lit test/XXX"
>>>>>>>>>>>>>> or "llvm-lit test/XXX/example.ll"
>>>>>>>>>>>>>> llvm-lit resides in BUILD tree, distinguished from source tree and
>>>>>>>>>>>>>> install tree. You can find it under
>>>>>>>>>>>>>> your-build-tree/Release+Asserts/bin/ or
>>>>>>>>>>>>>> your-build-tree/Debug+Asserts/bin/ or likewise, which depends on your
>>>>>>>>>>>>>> build
>>>>>>>>>>>>>> configuration.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> By the way, in the following ";RUN ..." directive
>>>>>>>>>>>>>> ; RUN: llc < %s -march=x86 | FileCheck %s
>>>>>>>>>>>>>> %s indicates current file under test. I guess the first %s stands for
>>>>>>>>>>>>>> the input for llc, while the second and last %s specifies
>>>>>>>>>>>>>> what to check using "; CHECK ..." directives since you write them down
>>>>>>>>>>>>>> on exactly CURRENT file.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hope it work for you.
>>>>>>>>>>>>>> Regards.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2012/8/20 Changcheng Wang <changcheng at multicorewareinc.com>:
>>>>>>>>>>>>>>> hi,all:
>>>>>>>>>>>>>>> i really want to how to write a regression test case,but i do not find
>>>>>>>>>>>>>>> out a entire document about it.
>>>>>>>>>>>>>>> By reading LLVM Testing Infrastructure Guide and FileCheck - Flexible
>>>>>>>>>>>>>>> pattern matching file verifier, i only have a idea,but it is not
>>>>>>>>>>>>>>> enough for me to write a test case.i need more detail about how to
>>>>>>>>>>>>>>> write a RUN:lines.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> thanks for you scan,wish your letter.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> best wishes,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> changcheng
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> LLVM Developers mailing list
>>>>>>>>>>>>>>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
>>>>>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> --
>>>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> --
>>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> --
>>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> 杨勇勇 (Yang Yongyong)
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> 杨勇勇 (Yang Yongyong)
>>>>
>>>>
>>>>
>>>> --
>>>> 杨勇勇 (Yang Yongyong)
>>
>>
>>
>> --
>> 杨勇勇 (Yang Yongyong)
More information about the llvm-dev
mailing list