[Openmp-dev] OpenMP code and thread priority on OSX

[Cownie, James H via Openmp-dev]

> Can you possibly point me where exactly in the runtime code?

% cd runtime/src
% grep pthread_create *.*
kmp_i18n.h:        KMP_SYSFAIL( "pthread_create", status );
kmp_i18n.h:        KMP_CHECK_SYSFAIL( "pthread_create", status );
z_Linux_util.c:            status = pthread_create( & handle, & thread_attr, __kmp_launch_worker, (void *) th );
z_Linux_util.c:                KMP_SYSFAIL( "pthread_create", status );
z_Linux_util.c:    status = pthread_create( &handle, & thread_attr, __kmp_launch_monitor, (void *) th );
z_Linux_util.c:        KMP_SYSFAIL( "pthread_create", status );

(Yup, z_Linux_util.c is counterintuitive...)

> What do you mean by "not designed for concurrency, but for parallelism" in this specific use context?
Concurrency is handling multiple asynchronous events, and can be a useful way to structure code even when one has only a single hardware thread. For instance to execute callbacks when the user presses on buttons in a GUI, or a network packet arrives.

Parallelism is using many hardware threads to reduce the time to solution of a single problem. It is futile if you only have one hardware thread.

OpenMP is designed for parallelism. It works best when it controls all the hardware and there is nothing else going on. In your case there clearly are other things going on (otherwise changing the priority wouldn't be necessary). In such an environment OpenMP may not work well. 
Amongst the reasons are
1) Many openMP codes use static work distribution. That assumes both that the work is evenly distributed between threads, and that the threads execute at the same speed. If a thread is stolen away by the OS that second assumption is false. At which point all other threads will have to wait at the next join or barrier for the laggard to arrive.
2) Even if you use dynamic scheduling, the definition of OpenMP barriers (and join) is that all the threads must arrive at the barrier, not that all the work has to be complete. So if the laggard is still not executing OpenMP code even though all the work is complete all the other threads still have to wait.

-- Jim

James Cownie <james.h.cownie at intel.com>
SSG/DPD/TCAR (Technical Computing, Analyzers and Runtimes)
Tel: +44 117 9071438


-----Original Message-----
From: Stéphane Letz [mailto:letz at grame.fr] 
Sent: Thursday, November 26, 2015 1:07 PM
To: Cownie, James H
Cc: LLVM-OpenMP (openmp-dev at lists.llvm.org)
Subject: Re: [Openmp-dev] OpenMP code and thread priority on OSX


Le 26 nov. 2015 à 13:54, "Cownie, James H" <james.h.cownie at intel.com> a écrit :

> The OpenMP runtime does not manipulate thread priorities
> A first skim through https://developer.apple.com/library/mac/documentation/Darwin/Conceptual/KernelProgramming/scheduler/scheduler.html doesn't make it clear to me whether threads inherit their priority from their creator.
> 
> Since OpenMP threads are created using pthread_create, that page suggests you may need to use pthread_setschedparam inside each thread to bump its priority.

The thing is that on OSX audio threads are actually "THREAD_TIME_CONSTRAINT_POLICY" threads, and this priority cannot be setup using Posix API, but needs "thread_policy_set" kind of API (as you can see in the URL you gave).

> (Or you may be able to hack the runtime to pass the priority as an attribute at thread creation time).

Can you possibly point me where exactly in the runtime code?

And would this kind of "inheritance policy" be better dynamically setup? I mean is this a requirement that could be needed for more use-cases to be part on the part of the official code?

> 
> BEWARE, though, you will almost certainly also want to change the default sleep behaviour of your OpenMP threads. By default they will spin for 200ms before sleeping in the kernel, which is unlikely to be helpful to the overall user experience if they're at high-priority! (See KMP_BLOCKTIME).

Ok thanks.
> 
> Overall OpenMP is not designed for concurrency, but for parallelism, and you may find this a hard thing to make work.
> 
> -- Jim
> 

What do you mean by "not designed for concurrency, but for parallelism" in this specific use context?

Thanks.

Stéphane Letz

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.