[llvm-dev] LLVM LNT floating point performance tests on X86 - using the llvm-test-suite benchmarks

[Blower, Melanie I via llvm-dev]

Using march=native as suggested by @Florian Hahn<mailto:florian_hahn at apple.com> I can see 20 failing tests, thanks!

I looked at the source for SingleSource/Benchmarks/Polybench/linear-algebra/kernels/trmm which is one of the failing tests and noticed that there is #pragma STDC FP_CONTRACT OFF in many places, but not in the function kernel_trmm

Perhaps this function was overlooked for that pragma because the FMA is syntactically not obvious due to the += operation?  So I added the pragma and reran that test, and now that test is passing.

Would it be acceptable to modify trmm with this change?

From: Florian Hahn <florian_hahn at apple.com>
Sent: Wednesday, May 19, 2021 12:25 PM
To: Blower, Melanie I <melanie.blower at intel.com>; llvm-dev <llvm-dev at lists.llvm.org>
Cc: Johannes Doerfert <johannesdoerfert at gmail.com>; spatel+llvm at rotateright.com; hal.finkel.llvm at gmail.com
Subject: Re: LLVM LNT floating point performance tests on X86 - using the llvm-test-suite benchmarks




On May 19, 2021, at 17:08, Blower, Melanie I <melanie.blower at intel.com<mailto:melanie.blower at intel.com>> wrote:

What I'm trying to do is to determine whether the patch I'm submitting is going to cause benchmarking problems that force the patch to be reverted--since that happened the last time I committed the patch (several months ago).


IIUC the problem with the patch was not runtime performance, but unexpected results causing some tests/benchmarks to fail.

I think you might need to select a CPU on X86 to cause the mis-compares. Below the commands I used to reproduce the failure of MultiSource/Applications/oggenc/oggenc. Note `-march=native`. Without that, the test passes. There are a couple of other failures as well.


> cmake -G Ninja \
    -DCMAKE_C_COMPILER=/path/to/bin/clang \
    -DCMAKE_C_FLAGS="-O3 -march=native" -DCMAKE_CXX_FLAGS="-O3 -march=native" \
    Path/to/llvm-test-suite

> ninja MultiSource/Applications/oggenc/oggenc

> llvm-lit MultiSource/Applications/oggenc/
-- Testing: 1 tests, 1 workers --
FAIL: test-suite :: MultiSource/Applications/oggenc/oggenc.test (1 of 1)
********************
Failed Tests (1):
  test-suite :: MultiSource/Applications/oggenc/oggenc.test
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210519/d0788f07/attachment.html>