<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Gisha;
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Gisha",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.EmailStyle19
{mso-style-type:personal;
font-family:"Gisha",sans-serif;
color:#4D7479;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Gisha",sans-serif;
color:#4D7479;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">Hi Hal,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">Some comments below.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">Thanks!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">Terry<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Hal Finkel [mailto:hfinkel@anl.gov] <br>
<b>Sent:</b> Sunday, April 12, 2015 10:49 AM<br>
<b>To:</b> Wilmarth, Terry L<br>
<b>Cc:</b> openmp-dev@dcs-maillist2.engr.illinois.edu<br>
<b>Subject:</b> Re: [Openmp-dev] Hierarchy generation patch<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Hi Terry,<br>
<br>
- bool uninitialized;<br>
+ volatile kmp_int8 uninitialized; // 0=initialized, 1=uninitialized, 2=initialization in progress<br>
<br>
Can you please define these in an enum so we can use names instead of the numbers directly (even if the variable itself is a volatile int). I'd rather that than having to remember what the values mean.</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#4D7479"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">[Terry L Wilmarth] I’m pretty actively refactoring code in this area. Can this happen in a later change?<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black"><br>
<br>
+ numPerLevel[0] = 4;<br>
+ numPerLevel[1] = nprocs/4;<br>
+ if (nprocs%4) numPerLevel[1]++;<br>
+<br>
+ for (int i=noaff_maxLevels-1; i>=0; --i) // count non-empty levels to get depth<br>
+ if (numPerLevel[i] != 1 || noaff_depth > 1) // only count one top-level '1'<br>
+ noaff_depth++;<br>
+<br>
+ kmp_uint32 branch = 4;<br>
+ if (numPerLevel[0] == 1) branch = nprocs/4;<br>
+ if (branch<4) branch=4;<br>
+ for (kmp_uint32 d=0; d<noaff_depth-1; ++d) { // optimize hierarchy width<br>
+ while (numPerLevel[d] > branch || (d==0 && numPerLevel[d]>4)) { // max 4 on level 0!<br>
+ if (numPerLevel[d] & 1) numPerLevel[d]++;<br>
+ numPerLevel[d] = numPerLevel[d] >> 1;<br>
+ if (numPerLevel[d+1] == 1) noaff_depth++;<br>
+ numPerLevel[d+1] = numPerLevel[d+1] << 1;<br>
}<br>
+ if(numPerLevel[0] == 1) {<br>
+ branch = branch >> 1;<br>
+ if (branch<4) branch = 4;<br>
+ }<br>
+ }<br>
<br>
I realize that this is a pre-existing issue, but what's so special about 4? Can we add a comment explaining that, and maybe make a constant (max_branch or similar) to hold this special number? Is this just a heuristic value used when we have affinity enabled
but don't actually have affinity information?</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:#4D7479"><o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">[Terry L Wilmarth] Yes, this is just a way to put a hierarchy in place when someone changes the default barrier to ‘hierarchical’ on
a machine where there is no affinity. The branching factor of 4 with 4 leaves gives a simple but similar-performing behavior to the hyper barrier’s default settings. I can add comments and constants along with my other refactoring.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">My refactoring consists of moving all the hierarchy stuff outside of the #ifdef for affinity presence, since I need it in general to
handle cases with and without affinity. I think it should look a lot better then, and take fewer LOC.<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black"><br>
Thanks again,<br>
Hal</span><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479"><o:p></o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">
<hr size="2" width="100%" align="center" id="zwchr">
</span></div>
<blockquote style="border:none;border-left:solid #1010FF 1.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-family:"Helvetica",sans-serif;color:black">From:
</span></b><span style="font-family:"Helvetica",sans-serif;color:black">"Terry L Wilmarth" <<a href="mailto:terry.l.wilmarth@intel.com">terry.l.wilmarth@intel.com</a>><br>
<b>To: </b>"Terry L Wilmarth" <<a href="mailto:terry.l.wilmarth@intel.com">terry.l.wilmarth@intel.com</a>>,
<a href="mailto:openmp-dev@dcs-maillist2.engr.illinois.edu">openmp-dev@dcs-maillist2.engr.illinois.edu</a><br>
<b>Sent: </b>Friday, April 10, 2015 4:08:01 PM<br>
<b>Subject: </b>Re: [Openmp-dev] Hierarchy generation patch</span><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479">*<b>ping</b>*</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:#4D7479"> </span><span style="color:black"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0in 0in 0in;border-color:-moz-use-text-color -moz-use-text-color">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black">
<a href="mailto:openmp-dev-bounces@cs.uiuc.edu">openmp-dev-bounces@cs.uiuc.edu</a> [<a href="mailto:openmp-dev-bounces@cs.uiuc.edu">mailto:openmp-dev-bounces@cs.uiuc.edu</a>]
<b>On Behalf Of </b>Wilmarth, Terry L<br>
<b>Sent:</b> Friday, April 03, 2015 11:57 AM<br>
<b>To:</b> <a href="mailto:openmp-dev@dcs-maillist2.engr.illinois.edu">openmp-dev@dcs-maillist2.engr.illinois.edu</a><br>
<b>Subject:</b> [Openmp-dev] Hierarchy generation patch<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:black">Here’s another patch to the hierarchy generation:</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Gisha",sans-serif;color:black">1. hierarchy_gen_fixed.patch: These are improvements to generation of the hierarchy used by hierarchical barrier. They improve how the generation reacts to affinity
set to none, or disabled, or no affinity available, or oversubscription.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:"Gisha",sans-serif;color:black">--<br>
Terry L. Wilmarth<br>
<a href="mailto:terry.l.wilmarth@intel.com" target="_blank">terry.l.wilmarth@intel.com</a> 217/403-4251<br>
Intel/SSG/DPD/TCAR/RAD/Threading Runtimes </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:12.0pt;font-family:"Helvetica",sans-serif;color:black"><br>
_______________________________________________<br>
Openmp-dev mailing list<br>
<a href="mailto:Openmp-dev@dcs-maillist2.engr.illinois.edu">Openmp-dev@dcs-maillist2.engr.illinois.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/openmp-dev">http://lists.cs.uiuc.edu/mailman/listinfo/openmp-dev</a><o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black"><br>
<br>
<br>
-- <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:black">Hal Finkel<br>
Assistant Computational Scientist<br>
Leadership Computing Facility<br>
Argonne National Laboratory<o:p></o:p></span></p>
</div>
</div>
</div>
</body>
</html>