[cfe-dev] [llvm-dev] test-suite: a new proposal for how to move forward to make "test-suite" more automatic, more flexible, and more maintainable, especially WRT reference outputs

[Sebastian Pop via cfe-dev]

On Thu, Oct 6, 2016 at 11:38 AM, Renato Golin <renato.golin at linaro.org> wrote:
> On 6 October 2016 at 16:11, Sebastian Pop via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
>> There is a very easy way to output only one double from the polybench:
>> - compile the kernel with fp-contract=off and -fno-fast-math
>
> Sebastian, please stop crossing the wires. This is a separate discussion.

We need to get deterministic output for all possible combinations of
CFLAGS the users will compile the test-suite with.

>
>
>> - add a "+" reduction loop of all the elements in the output array
>> (also compiled with strict FP computations such that the output is
>> deterministic)
>
> addition can saturate/overflow and lose precision, especially if we
> have hundreds of thousands of results or if the type is float, not
> double. Whatever the aggregation function we use has to be meaningful.

Agreed.
I'm also fine using any stable hashing function and link polybench
tests against that.