[LLVMdev] LNT BenchmarkGame

[Daniel Dunbar]

On Tue, Mar 12, 2013 at 10:23 AM, Renato Golin <renato.golin at linaro.org>wrote:

> On 12 March 2013 16:48, Daniel Dunbar <daniel at zuster.org> wrote:
>
>> The former mode is historically what the test suite did, the latter mode
>> is substantially faster (and independent of bugs in the native CC).
>>
>
> Yes, I agree this is better for many cases, but not for all. Implementing
> RNG that is good enough for the tests' purposes, fast enough not to steal
> the benchmarks' hot spots and does not use target/library-specific code is
> not trivial.
>

This is not true, all one needs to do is replace existing srand(), rand()
with some specific platforms version (and those are usually very simple
RNGs). If the code is already using srand()/rand() then there is no reason
to assume somehow the benchmark is worse if it always used the FreeBSD one,
say, as opposed to a platform specific one.

 - Daniel

I think that, in this particular case, having bugs in GCC is far less
> problematic than assuming fixed outputs.
>
> I've tried USE_REFERENCE_OUTPUT := 0 on the Makefile, but the test.log
> still prints it as 1 (and fails).
>
> cheers,
> --renato
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20130312/4bcc2d5e/attachment.html>