[Openmp-dev] Hierarchy generation patch

[Hal Finkel]

----- Original Message -----

> From: "Terry L Wilmarth" <terry.l.wilmarth at intel.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: openmp-dev at dcs-maillist2.engr.illinois.edu
> Sent: Monday, April 13, 2015 1:27:08 PM
> Subject: RE: [Openmp-dev] Hierarchy generation patch

> Hi Hal,
> Some comments below.
> Thanks!
> Terry

> From: Hal Finkel [mailto:hfinkel at anl.gov]
> Sent: Sunday, April 12, 2015 10:49 AM
> To: Wilmarth, Terry L
> Cc: openmp-dev at dcs-maillist2.engr.illinois.edu
> Subject: Re: [Openmp-dev] Hierarchy generation patch

> Hi Terry,

> - bool uninitialized;
> + volatile kmp_int8 uninitialized; // 0=initialized, 1=uninitialized,
> 2=initialization in progress

> Can you please define these in an enum so we can use names instead of
> the numbers directly (even if the variable itself is a volatile
> int). I'd rather that than having to remember what the values mean.
> [Terry L Wilmarth] I’m pretty actively refactoring code in this area.
> Can this happen in a later change?
Yes, doing this in follow-up is fine. 

> + numPerLevel[0] = 4;
> + numPerLevel[1] = nprocs/4;
> + if (nprocs%4) numPerLevel[1]++;
> +
> + for (int i=noaff_maxLevels-1; i>=0; --i) // count non-empty levels
> to get depth
> + if (numPerLevel[i] != 1 || noaff_depth > 1) // only count one
> top-level '1'
> + noaff_depth++;
> +
> + kmp_uint32 branch = 4;
> + if (numPerLevel[0] == 1) branch = nprocs/4;
> + if (branch<4) branch=4;
> + for (kmp_uint32 d=0; d<noaff_depth-1; ++d) { // optimize hierarchy
> width
> + while (numPerLevel[d] > branch || (d==0 && numPerLevel[d]>4)) { //
> max 4 on level 0!
> + if (numPerLevel[d] & 1) numPerLevel[d]++;
> + numPerLevel[d] = numPerLevel[d] >> 1;
> + if (numPerLevel[d+1] == 1) noaff_depth++;
> + numPerLevel[d+1] = numPerLevel[d+1] << 1;
> }
> + if(numPerLevel[0] == 1) {
> + branch = branch >> 1;
> + if (branch<4) branch = 4;
> + }
> + }

> I realize that this is a pre-existing issue, but what's so special
> about 4? Can we add a comment explaining that, and maybe make a
> constant (max_branch or similar) to hold this special number? Is
> this just a heuristic value used when we have affinity enabled but
> don't actually have affinity information?
> [Terry L Wilmarth] Yes, this is just a way to put a hierarchy in
> place when someone changes the default barrier to ‘hierarchical’ on
> a machine where there is no affinity. The branching factor of 4 with
> 4 leaves gives a simple but similar-performing behavior to the hyper
> barrier’s default settings. I can add comments and constants along
> with my other refactoring.
> My refactoring consists of moving all the hierarchy stuff outside of
> the #ifdef for affinity presence, since I need it in general to
> handle cases with and without affinity. I think it should look a lot
> better then, and take fewer LOC.
That makes sense; please proceed. :-) 

-Hal 

> Thanks again,
> Hal
> ----- Original Message -----

> > From: "Terry L Wilmarth" < terry.l.wilmarth at intel.com >
> 
> > To: "Terry L Wilmarth" < terry.l.wilmarth at intel.com >,
> > openmp-dev at dcs-maillist2.engr.illinois.edu
> 
> > Sent: Friday, April 10, 2015 4:08:01 PM
> 
> > Subject: Re: [Openmp-dev] Hierarchy generation patch
> 
> > * ping *
> 

> > From: openmp-dev-bounces at cs.uiuc.edu [
> > mailto:openmp-dev-bounces at cs.uiuc.edu ] On Behalf Of Wilmarth,
> > Terry
> > L
> 
> > Sent: Friday, April 03, 2015 11:57 AM
> 
> > To: openmp-dev at dcs-maillist2.engr.illinois.edu
> 
> > Subject: [Openmp-dev] Hierarchy generation patch
> 

> > Here’s another patch to the hierarchy generation:
> 
> > 1. hierarchy_gen_fixed.patch: These are improvements to generation
> > of
> > the hierarchy used by hierarchical barrier. They improve how the
> > generation reacts to affinity set to none, or disabled, or no
> > affinity available, or oversubscription.
> 
> > --
> 
> > Terry L. Wilmarth
> 
> > terry.l.wilmarth at intel.com 217/403-4251
> 
> > Intel/SSG/DPD/TCAR/RAD/Threading Runtimes
> 

> > _______________________________________________
> 
> > Openmp-dev mailing list
> 
> > Openmp-dev at dcs-maillist2.engr.illinois.edu
> 
> > http://lists.cs.uiuc.edu/mailman/listinfo/openmp-dev
> 
> --

> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
-- 

Hal Finkel 
Assistant Computational Scientist 
Leadership Computing Facility 
Argonne National Laboratory 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/openmp-dev/attachments/20150413/58c5a8f8/attachment.html>