<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";
mso-fareast-language:EN-GB;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Verdana","sans-serif";
color:#1F497D;
font-weight:normal;
font-style:normal;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Sorry, I read your description of the problem
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">“</span>While the results for iomp5 are far better on darwin than those for gomp on darwin, we still are lagging behind the performance of gomp using futex
on linux.”<o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">as being that libiomp5.so was underperforming on Linux because we’re not using futex there, so I was explaining how we could do that.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">I now grok that what you’re saying is that you’d like to see performance on Darwin (without futexes) that is faster than on Linux (with futexes).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">So, I suggest trying the TAS lock, (KMP_LOCK_KIND=tas on Darwin).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Depending on what you think OpenMP is used for, though, locks may be irrelevant. If you look at the latest SPECOMP codes, there are none that use locks (down
from the previous version that had a couple).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">In HPC locks should be rare and heavily contended locks absent completely. (Because if there are heavily contended locks in a significant part of the code,
it won’t perform well anyway, so doesn’t qualify for the “HPC” name </span><span style="font-size:10.0pt;font-family:Wingdings;color:#1F497D">J</span><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">-- Jim<br>
<br>
James Cownie <james.h.cownie@intel.com><br>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Tel: +44 117 9071438</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Jack Howarth [mailto:howarth.mailing.lists@gmail.com]
<br>
<b>Sent:</b> Wednesday, May 28, 2014 3:15 PM<br>
<b>To:</b> Cownie, James H<br>
<b>Cc:</b> openmp-dev@dcs-maillist2.engr.illinois.edu<br>
<b>Subject:</b> Re: [Openmp-dev] initial clang-omp/openmp benchmarking<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">James,<o:p></o:p></p>
<div>
<p class="MsoNormal"> The files I used (with the shell script adjusted for the compiler of course) are attached to the gcc <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">bugzilla at <a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61333">https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61333</a>. As for using futex, I am focused on<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">using darwin so that isn't really an option.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> Jack<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">ps Attached is an archive with a set of openmp open source demos that I found. There might be a <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">better example to benchmark locks in there than heated_plate_openmp.c and heated_plate_gcc.sh,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Wed, May 28, 2014 at 9:27 AM, Cownie, James H <<a href="mailto:james.h.cownie@intel.com" target="_blank">james.h.cownie@intel.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Without looking at the benchmark code (do you have a link to it?), given the description it sounds
as if it is omp_lock_t intensive.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">If so you can explicitly use FUTEX locks with libiomp5.so on Linux by setting the environment variable
KMP_LOCK_KIND=futex.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">You may also want to play with KMP_LOCK_KIND=tas, which uses a “test and test-and-set” lock.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">The choice of a default lock implementation is not trivial, since some lock benchmarks (such as that
in EPCC) are for heavily contended locks, whereas many codes have lightly contended locks and benefit from simpler (and unfair) lock implementations.</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">-- Jim<br>
<br>
James Cownie <<a href="mailto:james.h.cownie@intel.com" target="_blank">james.h.cownie@intel.com</a>><br>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)</span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Tel:
<a href="tel:%2B44%20117%209071438" target="_blank">+44 117 9071438</a></span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:openmp-dev-bounces@cs.uiuc.edu" target="_blank">openmp-dev-bounces@cs.uiuc.edu</a> [mailto:<a href="mailto:openmp-dev-bounces@cs.uiuc.edu" target="_blank">openmp-dev-bounces@cs.uiuc.edu</a>]
<b>On Behalf Of </b>Jack Howarth<br>
<b>Sent:</b> Wednesday, May 28, 2014 2:22 PM<br>
<b>To:</b> <a href="mailto:openmp-dev@dcs-maillist2.engr.illinois.edu" target="_blank">
openmp-dev@dcs-maillist2.engr.illinois.edu</a><br>
<b>Subject:</b> [Openmp-dev] initial clang-omp/openmp benchmarking</span><o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> I've done some initial benchmarking of openmp performance using the<br>
clang compiler from our fink llvm34-3.4.1-0e packaging which has the<br>
current openmp trunk svn built against the llvm/compiler-rt/clang 3.4.1<br>
with a back port of current clang-omp from github applied. The results for<br>
the heated_plate_openmp.c demo code compiled and run with the<br>
heated_plate_gcc.sh shell script revealed some interesting results. The<br>
demo code is run at one, two and four OMP processes. Ratioing these<br>
timings to the one OMP process timing shows the following on a 16-core <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">MacPro on darwin13…<br>
<br>
1:1.90:3.31 for FSF gcc 4.8.3<br>
<br>
1:1.90:3.30 for FSF gcc 4.9.0<br>
<br>
1:1.99:3.71 for clang 3.4.1 with openmp and merged clang-omp<br>
<br>
this compares to the results on a 24-core Fedora 15 linux box<br>
<br>
1:1.99:3.92 for FSF gcc 4.6.3<br>
<br>
1:1.99:3.93 for FSF gcc 4.8 branch svn<br>
<br>
I've filed <a href="https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61333" target="_blank">
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61333</a> on the<br>
reduced performance of gomp on darwin compared to iomp5 on darwin and<br>
gomp on linux. Their response was that darwin's use of pthread_mutex calls<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">rather than futex was the cause in gomp and that we should be using linux.<br>
While the results for iomp5 are far better on darwin than those for<br>
gomp on darwin, we still are lagging behind the performance of gomp using<br>
futex on linux. FYI, the heated_plate_openmp.c and heated_plate_gcc.sh <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">are attached to PR 61333.<br>
Jack<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p>---------------------------------------------------------------------<br>
Intel Corporation (UK) Limited<br>
Registered No. 1134945 (England)<br>
Registered Office: Pipers Way, Swindon SN3 1RJ<br>
VAT No: 860 2173 47<o:p></o:p></p>
<p>This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p>---------------------------------------------------------------------<br>
Intel Corporation (UK) Limited<br>
Registered No. 1134945 (England)<br>
Registered Office: Pipers Way, Swindon SN3 1RJ<br>
VAT No: 860 2173 47</p>
<p>This e-mail and any attachments may contain confidential material for<br>
the sole use of the intended recipient(s). Any review or distribution<br>
by others is strictly prohibited. If you are not the intended<br>
recipient, please contact the sender and delete all copies.</p></body>
</html>