<html><head><style type='text/css'>p { margin: 0; }</style></head><body><div style='font-family: arial,helvetica,sans-serif; font-size: 10pt; color: #000000'><br><hr id="zwchr"><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><b>From: </b>"Mehdi Amini" <mehdi.amini@apple.com><br><b>To: </b>"Hal Finkel" <hfinkel@anl.gov><br><b>Cc: </b>"Renato Golin" <renato.golin@linaro.org>, "Sebastian Paul Pop" <s.pop@samsung.com>, "llvm-dev" <llvm-dev@lists.llvm.org>, "Matthias Braun" <matze@braunis.de>, "Clang Dev" <cfe-dev@lists.llvm.org>, "nd" <nd@arm.com>, "Abe Skolnik" <a.skolnik@samsung.com><br><b>Sent: </b>Wednesday, October 12, 2016 1:29:12 PM<br><b>Subject: </b>Re: [llvm-dev] [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"<br><br>
<br class=""><div><blockquote class=""><div class="">On Oct 12, 2016, at 7:05 AM, Hal Finkel via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class=""><hr id="zwchr"></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><blockquote style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class="">From: "Renato Golin" <<a href="mailto:renato.golin@linaro.org" class="" target="_blank">renato.golin@linaro.org</a>><br class="">To: "Sebastian Pop" <<a href="mailto:sebpop.llvm@gmail.com" class="" target="_blank">sebpop.llvm@gmail.com</a>><br class="">Cc: "Hal Finkel" <<a href="mailto:hfinkel@anl.gov" class="" target="_blank">hfinkel@anl.gov</a>>, "Sebastian Paul Pop" <<a href="mailto:s.pop@samsung.com" class="" target="_blank">s.pop@samsung.com</a>>, "llvm-dev" <<a href="mailto:llvm-dev@lists.llvm.org" class="" target="_blank">llvm-dev@lists.llvm.org</a>>,<br class="">"Matthias Braun" <<a href="mailto:matze@braunis.de" class="" target="_blank">matze@braunis.de</a>>, "Clang Dev" <<a href="mailto:cfe-dev@lists.llvm.org" class="" target="_blank">cfe-dev@lists.llvm.org</a>>, "nd" <<a href="mailto:nd@arm.com" class="" target="_blank">nd@arm.com</a>>, "Abe Skolnik"<br class=""><<a href="mailto:a.skolnik@samsung.com" class="" target="_blank">a.skolnik@samsung.com</a>><br class="">Sent: Wednesday, October 12, 2016 8:35:16 AM<br class="">Subject: Re: [test-suite] making polybench/symm succeed with "-Ofast" and "-ffp-contract=on"<br class=""><br class="">On 12 October 2016 at 14:26, Sebastian Pop <<a href="mailto:sebpop.llvm@gmail.com" class="" target="_blank">sebpop.llvm@gmail.com</a>><br class="">wrote:<br class=""><blockquote class="">Correct me if I misunderstood: you would be ok changing the<br class="">reference output to exactly match the output of "-O0<br class="">-ffp-contract=off".<br class=""></blockquote><br class="">No, that's not at all what I said.<br class=""><br class="">Matching identical outputs to FP tests makes no sense because there's<br class="">*always* an error bar.<br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">This is something we need to understand. No, there's not always an error bar. With FMA formation and without non-IEEE-compliant optimizations (i.e. fast-math), the optimized answer should be identical to the non-optimized answer. </span></div></blockquote><div><br class=""></div><div id="DWT10366">Can you clarify: in my mind the F in FMA is for “fused”, i.e. no intermediate truncation, i.e. not the same numerical result. But you imply the opposite above?</div></div></blockquote>Sorry, I did not type that correctly. I meant *without* FMA formation and without non-IEEE-compliant optimizations (i.e. fast-math), the <span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">optimized answer should be identical to the non-optimized answer. <br><br> -Hal<br></span><br><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px; color: rgb(0, 0, 0); font-weight: normal; font-style: normal; text-decoration: none; font-family: Helvetica,Arial,sans-serif; font-size: 12pt;"><div><div></div><div><br class=""></div><div><br class=""></div><div>— </div><div>Mehdi</div><div><br class=""></div><br class=""><blockquote class=""><div class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">If these don't match, then we should understand why. This used to be a large problem because of fp80-related issues on x86 processors, but even on x86 if we stick to SSE (etc.) FP instructions, this is not an issue any more. We still do see cross-system discrepancies sometimes because of differences in denormal handling, but on the same system that should be consistent (aside, perhaps, from compiler-level constant-folding issues).</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">-Hal</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><blockquote style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><br class="">The output of O0, O1, O2, O3, Ofast, Os, Oz should all be within the<br class="">boundaries of an average and its associated error bar.<br class=""><br class="">By understanding what's the *expected* output and its associated<br class="">error<br class="">range we can accurately predict what will be the correct<br class="">reference_output and the tolerance for each individual test.<br class=""><br class="">Your solution 2 "works" because you're doing the matching yourself,<br class="">in<br class="">the code, and for that, you pay the penalty of running it twice. But<br class="">it's not easy to control the tolerance, nor it's stable for all<br class="">platforms where we don't yet run the test suite.<br class=""><br class="">My original proposal, and what I'm still proposing here, is to<br class="">understand the tests and make them right, by giving them proper<br class="">references and tolerances. If the output is too large, reduce/sample<br class="">in a way that doesn't increase the error ranges too much, enough to<br class="">keep the tolerance low, so we can still catch bugs in the FP<br class="">transformations.<br class=""><br class="">cheers,<br class="">--renato<br class=""><br class=""></blockquote><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">--<span class="Apple-converted-space"> </span></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">Hal Finkel</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">Lead, Compiler Technology and Programming Languages</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">Leadership Computing Facility</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">Argonne National Laboratory</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">_______________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; float: none; display: inline ! important;" class="">LLVM Developers mailing list</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class="" target="_blank">llvm-dev@lists.llvm.org</a><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px;" class="" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></blockquote><br><br><br>-- <br><div><span name="x"></span>Hal Finkel<br>Lead, Compiler Technology and Programming Languages<br>Leadership Computing Facility<br>Argonne National Laboratory<span name="x"></span><br></div></div></body></html>