[cfe-dev] [libc++] Working on the parallel STL algorithms

[Hal Finkel via cfe-dev]

On 05/17/2017 09:50 AM, Bryce Lelbach wrote:
> Some thoughts:
>
> * I pretty much agree with Hal - his vision for what we should do here
> closely matches mine.
> * I agree with Chris that we may want to target the Intel OMP Runtime
> interface - but Hal is right that this may be sub-optimal.
> Additionally, it may be quite tricky to do correctly.
> * I am a little concerned that it will be hard to de-couple the
> "parallel engine" and "algorithms" components of the library if we're
> using OpenMP pragmas. I am also a little concerned about using pragmas
> because they don't always interact well with C++ abstractions.
> * I want to make sure we end up with one generic "algorithms" core and
> O(3) backends that libc++ supports/maintains, and an interface that
> 3rd parties can implement to plug their "parallel engines" into
> libc++'s algorithms. I'm a little worried that the pragma-based OpenMP
> one might be tightly coupled (although 3rd parties could plug in by
> implement the OpenMP runtime API, that'd be sub-optimal).

I don't see why you have that concern. We have plenty of existing 
parallel abstraction libraries (e.g. Kokkos) that have multiple 
backends, including OpenMP, and don't have this problem. You can 
certainly have a set of templates that implement bulk task dispatch, 
etc. in terms of OpenMP and a set of templates that implement that 
interface in terms of other things. The key thing here is to reuse the 
thread pool and get the added compiler optimizations that the pragmas 
can imply (when OpenMP is available). To be clear, I'm not proposing 
that we put OpenMP pragmas on the algorithm implementations themselves, 
only use it for an underlying parallelism-execution provider.

What is true, AFAIK, is that the abstractions don't play well with 
OpenMP pragmas that take variable names (reductions, for example) in a 
general way. Thus, for example, it is not straightforward/possible to 
have a variadic template that takes a list of variables on which to 
perform a reduction and transform that into a loop with a pragma with 
the right list of variables in a reduction clause (in general). You can 
handle specific cases (e.g. one reduction variable for specific 
operations). For the general case, we need to do our own thing using 
underlying primitives (which is what we'd need to do for the non-OpenMP 
implementations as well). I don't even know that we run into this 
problem with the current set of standardized algorithms.

  -Hal

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory