[cfe-dev] [RFC] Add SYCL programming model support.

Thu Feb 7 12:34:17 PST 2019

On 2/7/19 1:55 PM, Bader, Alexey via cfe-dev wrote:

>> Let me check that I understand this question correctly. Are you asking about implementation of pointer classes representing pointers to different address spaces?

> As for address spaces I think you can shortcut by mapping to OpenCL address spaces indeed. Although I don't know why address space qualifiers wouldn't be used directly actually? At some point I would like to enable address spaces without "__" prefix btw to allow porting OpenCL C code to C++. Not sure if it can create issues for SYCL then. But it's worth making this clear now.

One of the goals of SYCL design is to allow developers to compile and run SYCL code even if the compiler toolchain doesn’t support acceleration through OpenCL. Unfortunately "OpenCL address spaces" is not C++ standard feature (yet :)), so if we expose them to the user, the program written with these extensions will not be supported by other C++ compiler like GCC or MSVC. Using standard API allows us to utilize all sorts of extensions for API implementation and emulate them with standard C++ if extensions are not available.

It is plausible to assume that it should be easier for C++ developers to adopt new functionality through standard C++ concepts like class/function rather than through language extensions.

> However, I think for SYCL address spaces is just one example of much broader picture? What about all other language features that are wrapped into the libraries? For example the code in SemaOverload.cpp of this commit illustrates that you need a tight coupling between compiler and library.

https://github.com/intel/llvm/commit/03354a29868b79a30e6fb2c8311bb409a8cc2346#diff-811283eaa55fa65f65713fdd7ecaf4aa

We switched to using "function attributes" in later commit https://github.com/llvm/llvm-project/commit/120b4b509d758e27c17111eaa0398b4cecf7575a. Basically SYCL runtime marks functions supposed to be offloaded to the target device with special function attribute (similar to OpenCL `kernel` attribute) and compiler doesn't rely on particular library function names.

One the other hand there are other places where similar dependencies exist. For instance, typical SYCL kernel function captures "accessor" parameters, which provides "view" on the data accessed the by the device code. This accessor class contains a pointer this data and it's initialized on the host. To pass C++ class with a pointer to memory from the host to accelerator we need either:

1. system to support some sort of virtual memory, so the target know how to handle host pointers

2. some cooperation between the compiler and runtime on converting host pointers to target pointers

As OpenCL implementation is not guaranteed to support option (1), we implemented option (2) and current implementation relies on SYCL class method names from the standard, but I guess this might be not the best option. I am going to send a separate email to discuss this topic in more details.

>> I need better understand the “OpenCL C++ route” and how it’s aligned with SYCL design philosophy, which tries to enable programing of accelerators via “extension-free” standard C++.

>As for OpenCL we are just enabling C++ functionality to work in OpenCL. That would mean all the library based language features from OpenCL C++ won't be implemented.

By "library based language features" you mean OpenCL specific data types. Right? If so, I think SYCL can re-use OpenCL C++ implementation by outlining "device code" from the single source and then treating it as a OpenCL C++ program.

>Btw, I feel there is a little contradiction here because if you can just use “extension-free” standard C++ then you wouldn't need to modify Clang?

SYCL code is supposed to be valid C++ and should work with any C++11 compiler, but when we compile it "in SYCL mode" the device code can be offloaded to OpenCL accelerator. This feature requires Clang modifications to enable offloading of the "device part" inside a single source, enforce additional target restrictions for this "device part" (e.g. OpenCL devices typically do not support function pointers), lowering device code to the format accepted by OpenCL runtime (e.g. binary, SPIR-V).

>> Current implementation relies on existing “LLVM to SPIR-V” translator [1]. It’s integrated as an external tool into the toolchain for SYCL. We would like LLVM to have native support of SPIR-V format, so we rely on your work here.

> Just to understand: do you plan to add a SPIR-V triple to Clang&LLVM and a special action into Clang that would invoke the translator after generation of IR? If yes I would quite like to see it done more generically in Clang and not just for SYCL. But I think there were some concerns from the other members of LLVM. I will let them comment if it's still the case.

I agree that SPIR-V support must be added not only for SYCL, but for OpenCL C++ too, as it's necessary part of OpenCL C++ compiler toolchain. I think other extensions/APIs might benefit from having native SPIR-V support in LLVM (e.g. OpenMP/Vulkan).

IIRC, the latest discussion ended with a request to build a community around the translator tool. IMHO, we have the community for a long time, but it's not vocal in the LLVM mailing lists and not visible for LLVM community (I can blame myself too :)). I'm aware of multiple projects using this tool to offload computation to OpenCL accelerators and I'll try to provide the evidence in dedicated mailing thread.

First, let me say that I support this effort to add SYCL support to Clang/LLVM. Having a standard, single-source accelerator programming model will be important to the HPC ecosystem and beyond.

Are you thinking about supporting SYCL only via lowering to SPIR-V, or also via direct invocation of appropriate hardware backends? One thing that worries me is that SPIR-V does not support function pointers, and while SYCL doesn't either (or virtual functions, as noted on pg 16, ch 2 of the SYCL 1.2.1 spec), given our experience with other accelerator programming models, I'm not sure how many of our applications would find SYCL an appealing model without this support. Thus, while I think that SYCL is an interesting model, and I know a number of developers interested in learning more about it, being trapped into this restriction by a SPIR-V funnel seems highly undesirable. It seems like this could undesirably limit our ability to support extensions of this kind. Support for inline assembly is another important feature that seems like it might have trouble passing through a SPIR-V layer. There might be other SPIR-V restrictions that pose a similar problem.

I know that ComputeCPP from CodePlay supports some kind of direct-to-PTX path in their LLVM/Clang fork, for example. This is certainly a feature that is important to our current/planned evaluation work regarding SYCL.

Thanks again,

Hal

>> We re-used the OpenCL C++ compiler component here to emit LLVM IR for the “LLVM to SPIR-V” translator. For instance, this pass adjusts accelerator specific data types to the format recognized by the translator [2]. I’m open to the suggestions how to improve the format, so we don’t need “adjusting passes”.

>Just to be more specific I guess you mean the OpenCL C++ prototype compiler here (which is quite different from the implementation in mainline Clang)? Can you explain what kind of adjustments you are trying to make and why the approach from OpenCL C wouldn't apply in your case?

Sure. OpenCL C approach for built-in functions is "Itanuim C++ ABI mangled names in global name space". This doesn't work for OpenCL C++/SYCL as these built-ins collide with user functions. We re-use existing prototype to speed-up SYCL development, but according to my understanding it might violate some LLVM guidelines for extending LLVM IR.

>> Anyway https://reviews.llvm.org/D57768 is not related to these topics and it’s aligned with existing CUDA/OpenMP functionality.

>As I wrote, these comments are not to the review but they are conceptually important aspects that the community should align on. It might be good to have a concrete plan before starting to work on something?

I'll write a design document to provide more details on how things are done.

--------------------------------------------------------------------
Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park,
17 Krylatskaya Str., Bldg 4, Moscow 121614,
Russian Federation

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

_______________________________________________
cfe-dev mailing list
cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190207/658a1222/attachment.html>