[LLVMdev] Proposal for GSoC project for improving llvm-test testsuite

[Devang Patel]

On Mar 18, 2008, at 9:52 PM, Rajika Kumarasiri wrote:

> hello everybody,
> I would like to improve the llvm-test suite[1] as a part of GSoC  
> 2008 program for LLVM.
> I have few concerns/problems regarding this, please give me your  
> feedbacks and suggestions to come up with the proper proposal.
>
> Goal: Improve the llvm-test testsuite.

There are two parts :
1) llvm-project/llvm/test in svn. This part checks correctness and  
features.
2) Whole program testsuite, which is located at llvm-project/test- 
suite. This part includes benchmarks and applications.

>
>
> Idea: Extend the llvm-test testsuite to include new programs and  
> benchmarks[2]. Test programs should be more CPU intensive  with few  
> library dependencies  at the end which are delivered in source format.
>
> I have following few problems, please help me to understand them.
>
> 1. Before everything I need to clarify something.
> Normally we add tests in a software  to check the correctness of  
> that software code base.

This is done in 1)st part.

> Here in this project llvm trying to specific on third party software  
> codes. So I am not sure how a test be structured within llvm to  
> check llvm code base correctness using those third party codes(This  
> is what I understood, please correct me if I am wrong). I'd glad if  
> I can get more information on this.

When LLVM fails to build third party code, you want to derive small  
test cases to reproduce such failures. This small and independent  
tests are added in 1st part.

> 2. As described llvm testsuite program should be more CPU intensive  
> programs, is there is any reason for that

The goal here (in 2)nd part) is to measure the performance of the code  
generated by LLVM.

> and how  a llvm test suite program can make more CPU intensive, any  
> points that should consider to write a more CPU intensive test  
> program for llvm?

IIUC, You do not want to write new test suite program here. You want  
to adopt existing programs and benchmarks and update them such that  
they fit in 2)nd part.

Let's take BioPerf as an example. You want to take this existing  
benchmark and appropriately update its makefiles so that it builds on  
2)nd part of llvm test-suite. I am not familiar with this benchmark  
itself, but it uses many external libraries the you want to minimize  
library use if possible. Finally, the benchmark is likely to use some  
sort of input data to measure performance. You want to adjust input  
data set such that it does not take hours to run this benchmark and at  
the same time it takes enough time to notice performance changes.

> 3. Since we need to avoid many library dependencies, we need to  
> reduce dependencies to standard C/C++ libraries and llvm libraries  
> itself.Is this correct ? If so any points that need to consider  
> writing tests which has few library decencies?
>
> 4. What a test program should cover?
> As I read in the testsuite guide [3], test programs in llvm-test  
> covers two things in general. It check for a particular LLVM feature  
> or trigger a specific bug in LLVM. So in addition to that any other  
> point that need to consider, along with the structure of a test case  
> would be helpful
> And also as described test programs are code fragments and whole  
> programs. I think in here also I may need to write either of them  
> depending of the benchmark trying[2].
>
> 5. Finally if I am going cover test programs for[3], may be setting  
> few targets( targets in the sense that I am writing test programs  
> for A variety of C++ benchmarks,BioPerf,LLC bench etc.., ) would be  
> help to define the time line. May be more important one can be done  
> in the first touch.


>
>
> Please guide me on these. Please correct me if I am wrong in  
> anything. Thanks in advance!
>
> Me:
> I am a computer science undergraduate[4]. I'm interested in compiler  
> technology and have experience in C/C++ programming.My resume [5].
>
>
> [1] - http://llvm.org/OpenProjects.html#llvmtest
> [2] - http://nondot.org/sabre/LLVMNotes/#benchmarks
> [3] - http://llvm.org/docs/TestingGuide.html#org
> [4] - http://www.cse.mrt.ac.lk/
> [5] - http://rajikacc.googlepages.com/resume_rajika.pdf
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

-
Devang



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20080319/5bdcd6b2/attachment.html>