[LLVMdev] [Polly] Set up performance tester for GSOC2013 FastPolly project

[Star Tan]

At 2013-06-09 15:09:54,"Tobias Grosser" <tobias at grosser.es> wrote:

>On 06/06/2013 11:17 AM, Star Tan wrote:
>> Hi Tobias,
>>
>>
>> I am recently trying to set up the performance tester for FastPolly project. According to your suggestion, I plan to use the LNT infrastructure to set up the performance tester. For this purpose, I think I should do this job in three steps:
>>
>>
>> First, I will add PolyBench to LLVM test-suite since PolyBench is the critical benchmarks for FastPolly. I have adjust the PolyBench-c-3.2 so we can put it into LLVM test-suite/SingleSource/Benchmarks/.  Please find the attached PolyBench_for_LLVM_test_suite.tgz.
>
>Polybench should already be part of the LLVM test suite. Please have a 
>source at SingleSource/Benchmarks/Polybench/
I see. Yes, the Polybench has been added to LLVM test suite. My previous investigation is based on test-suite-3.0, which is not up-to-date.

>
>> Second, I think I should select some benchmarks for FastPolly testing. Running all LLVM test-suite benchmarks is very time-consuming and unnecessary. My idea is still to run the mediebench  and PolyBench using LNT infrastructure. Please find the attached Prliminary PolyBench evaluation results (LNT_PolyBench.html) using LNT infrastructure.
>
>I believe in the end we should run all benchmarks. It is important to
>see the performance over a wider range of input programs. To get started 
>you can obviously focus on specific programs or problems.
>
OK, I can run all benchmark once every half month. In other testing, we can mainly focus on Polybench and mediabench.


>> Third, I think I should evaluate the compile-time and running time in both Polly-enabling case and Polly-disabling case. Currently, I am not clear how to configurate LNT so it can test the same benchmark using different parameters in the same runtime test, but I will investigate it.
>
>The LNT infrastructure will provide you with compile-time and run-time
>statistics. We should set up the tester with 'clang -O3' and 
>'polly-clang -O3' and compare the compile-time of both runs against each 
>other.
>
>> Do you think it is a good way to set up a performance tester for the FastPolly project? If you agree with me, I will try to commit the PolyBench to llvm test-suite in the next step.
>
>You can skip the step with committing the test suite.
>
>A the moment I do not have dedicated machines available to set up public 
>performance testers, so we need to rely on Sebastian's internal testers 
>or your own experiments. I may be able to get some machines in the not 
>so distance future. Until we reach this point, I propose you make 
>yourself familiar with LNT, the LLVM test suite and possibly also the 
>buildbot system we have. If you have a machine where you can run this 
>locally, this would already be a good start.
Thanks. Yes, I will run all tests on my own computer recently.
>
>Cheers,
>Tobias
>
>
Star Tan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130609/27c6de50/attachment.html>