[LLVMdev] Proposal for GSoC project for improving llvm-test testsuite
dpatel at apple.com
Wed Mar 19 11:12:18 PDT 2008
On Mar 18, 2008, at 9:52 PM, Rajika Kumarasiri wrote:
> hello everybody,
> I would like to improve the llvm-test suite as a part of GSoC
> 2008 program for LLVM.
> I have few concerns/problems regarding this, please give me your
> feedbacks and suggestions to come up with the proper proposal.
> Goal: Improve the llvm-test testsuite.
There are two parts :
1) llvm-project/llvm/test in svn. This part checks correctness and
2) Whole program testsuite, which is located at llvm-project/test-
suite. This part includes benchmarks and applications.
> Idea: Extend the llvm-test testsuite to include new programs and
> benchmarks. Test programs should be more CPU intensive with few
> library dependencies at the end which are delivered in source format.
> I have following few problems, please help me to understand them.
> 1. Before everything I need to clarify something.
> Normally we add tests in a software to check the correctness of
> that software code base.
This is done in 1)st part.
> Here in this project llvm trying to specific on third party software
> codes. So I am not sure how a test be structured within llvm to
> check llvm code base correctness using those third party codes(This
> is what I understood, please correct me if I am wrong). I'd glad if
> I can get more information on this.
When LLVM fails to build third party code, you want to derive small
test cases to reproduce such failures. This small and independent
tests are added in 1st part.
> 2. As described llvm testsuite program should be more CPU intensive
> programs, is there is any reason for that
The goal here (in 2)nd part) is to measure the performance of the code
generated by LLVM.
> and how a llvm test suite program can make more CPU intensive, any
> points that should consider to write a more CPU intensive test
> program for llvm?
IIUC, You do not want to write new test suite program here. You want
to adopt existing programs and benchmarks and update them such that
they fit in 2)nd part.
Let's take BioPerf as an example. You want to take this existing
benchmark and appropriately update its makefiles so that it builds on
2)nd part of llvm test-suite. I am not familiar with this benchmark
itself, but it uses many external libraries the you want to minimize
library use if possible. Finally, the benchmark is likely to use some
sort of input data to measure performance. You want to adjust input
data set such that it does not take hours to run this benchmark and at
the same time it takes enough time to notice performance changes.
> 3. Since we need to avoid many library dependencies, we need to
> reduce dependencies to standard C/C++ libraries and llvm libraries
> itself.Is this correct ? If so any points that need to consider
> writing tests which has few library decencies?
> 4. What a test program should cover?
> As I read in the testsuite guide , test programs in llvm-test
> covers two things in general. It check for a particular LLVM feature
> or trigger a specific bug in LLVM. So in addition to that any other
> point that need to consider, along with the structure of a test case
> would be helpful
> And also as described test programs are code fragments and whole
> programs. I think in here also I may need to write either of them
> depending of the benchmark trying.
> 5. Finally if I am going cover test programs for, may be setting
> few targets( targets in the sense that I am writing test programs
> for A variety of C++ benchmarks,BioPerf,LLC bench etc.., ) would be
> help to define the time line. May be more important one can be done
> in the first touch.
> Please guide me on these. Please correct me if I am wrong in
> anything. Thanks in advance!
> I am a computer science undergraduate. I'm interested in compiler
> technology and have experience in C/C++ programming.My resume .
>  - http://llvm.org/OpenProjects.html#llvmtest
>  - http://nondot.org/sabre/LLVMNotes/#benchmarks
>  - http://llvm.org/docs/TestingGuide.html#org
>  - http://www.cse.mrt.ac.lk/
>  - http://rajikacc.googlepages.com/resume_rajika.pdf
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the llvm-dev