[LLVMdev] I would like to merge PARSEC into test-suite
criswell at uiuc.edu
Wed Jul 21 11:57:47 PDT 2010
Daniel Dunbar wrote:
> On Tue, Jul 20, 2010 at 1:30 AM, Owen Anderson <resistor at mac.com> wrote:
>> On Jul 20, 2010, at 1:18 AM, Patrick Simmons wrote:
>>> It is open-source and redistributable, and I have added LICENSE.TXT
>>> files to all the pieces I want to merge. These are blackscholes,
>>> canneal, dedup, fluidanimate, freqmine, streamcluster, and swaptions. I
>>> will disable the tests by default on the initial merge and test
>>> thoroughly on Linux and MacOS before enabling them. May I please commit
>>> my changes directly to test-suite?
> Awesome! Can we see a patch?
Hi! I'm the person that asked Patrick to email llvmdev about
integrating PARSEC. We needed to make PARSEC work with test-suite
because we use test-suite as the foundation of our testing
infrastructure, and since we've gone through the trouble to make it
work, we might as well let others benefit from our effort.
I advised Patrick to *not* send a patch because it would be very large
and mostly contain PARSEC source code. However, I've since realized
that we can place it on our web site and send out a URL for it.
Patrick, I'll let you know how to do that.
> The PARSEC webpage mentions that the benchmark suite is quite large.
> Does your patch actually commit a specific version to the test-suite,
> or is it setup like the externals tests where the user is expected to
> download PARSEC on the side if they wish to test it?
We can integrate PARSEC either as a regular internal test with its
source code inside the test suite or as an external test like SPEC,
depending on what people think is best. We wanted to know if there were
objections to either approach.
>> I'm not convinced that the PARSEC tests are appropriate for addition the LLVM testsuite, because they are multithreaded tests. The testsuite needs to be low-volatility in terms of performance variation, as well as easy to verify correctness, and easy for the compiler developers to debug miscompilations. I don't think PARSEC is a good match on these fronts.
> I don't agree, more tests are always better. We can always invent
> infrastructure to ignore the performance of certain tests if that
> becomes an issue.
As an aside, I'll offer up my opinion on the purpose of test-suite:
IMHO, the test suite is for the LLVM community. Benchmarking LLVM is
one of its uses. However, I think the test suite is also useful for
stress testing LLVM's correctness and for research (e.g., we use the
SPEC and MultiSource/Benchmark tests in our papers). As long as it does
not grow too large, I think test-suite should continue to fill these
needs. If it does grow too large (or some tests cause problems), we can
split it into several smaller test suites tailored to different
subgroups of the community.
So, with that said, are there still objections to integrating it as an
internal test? If not, I'll ask Patrick to create a patch, and we'll
send a URL for it. If there are still objections, would integrating it
as an external test (like SPEC) be okay?
-- John T.
> - Daniel
>> LLVM Developers mailing list
>> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu http://llvm.cs.uiuc.edu
More information about the llvm-dev