[LLVMdev] LNT BenchmarkGame

[Hal Finkel]

----- Original Message -----
> From: "Daniel Dunbar" <daniel at zuster.org>
> To: "Renato Golin" <renato.golin at linaro.org>
> Cc: "Hal Finkel" <hfinkel at anl.gov>, "Marshall Clow" <mclow.lists at gmail.com>, "LLVM Dev" <llvmdev at cs.uiuc.edu>
> Sent: Tuesday, March 12, 2013 12:30:12 PM
> Subject: Re: [LLVMdev] LNT BenchmarkGame
> 
> 
> 
> 
> 
> 
> 
> On Tue, Mar 12, 2013 at 10:23 AM, Renato Golin <
> renato.golin at linaro.org > wrote:
> 
> 
> 
> 
> On 12 March 2013 16:48, Daniel Dunbar < daniel at zuster.org > wrote:
> 
> 
> 
> 
> 
> 
> The former mode is historically what the test suite did, the latter
> mode is substantially faster (and independent of bugs in the native
> CC).
> 
> 
> 
> Yes, I agree this is better for many cases, but not for all.
> Implementing RNG that is good enough for the tests' purposes, fast
> enough not to steal the benchmarks' hot spots and does not use
> target/library-specific code is not trivial.
> 
> 
> This is not true, all one needs to do is replace existing srand(),
> rand() with some specific platforms version (and those are usually
> very simple RNGs). If the code is already using srand()/rand() then
> there is no reason to assume somehow the benchmark is worse if it
> always used the FreeBSD one, say, as opposed to a platform specific
> one.

+1

There are a couple of example implementations here which are only a few lines long:
http://wiki.osdev.org/Random_Number_Generator

 -Hal

> 
> - Daniel
> 
> 
> 
> 
> 
> 
> 
> I think that, in this particular case, having bugs in GCC is far less
> problematic than assuming fixed outputs.
> 
> 
> I've tried USE_REFERENCE_OUTPUT := 0 on the Makefile, but the
> test.log still prints it as 1 (and fails).
> 
> 
> cheers,
> --renato
>