[cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

Mon Jul 11 09:48:18 PDT 2016

Hi Sam,

I think we can use macro to enable new types and built-in function that doesn't require native support by clang.

Here is quote from OpenCL extension specification (page 9):

Every extension which affects the OpenCL language semantics, syntax or adds built-in functions to the language must create a preprocessor #define that matches the extension name string.
This #define would be available in the language if and only if the extension is supported on a given implementation.

Example:
An extension which adds the extension string "cl_khr_3d_image_writes" should also add a preprocessor #define called cl_khr_3d_image_writes. A kernel can now use this preprocessor #define to do something like:

#ifdef cl_khr_3d_image_writes
    // do something using the extension
#else
   // do something else or #error!
#endif

So there should be no problem for users to use #ifdef cl_vendor_new_extension to condition their own code.

What benefits do you see in using pragma instead of preprocessing directives?

Thanks,
Alexey

-----Original Message-----
From: Liu, Yaxun (Sam) [mailto:Yaxun.Liu at amd.com] 
Sent: Friday, July 8, 2016 11:21 PM
To: Bader, Alexey <alexey.bader at intel.com>; Anastasia Stulova <Anastasia.Stulova at arm.com>; Sumner, Brian <Brian.Sumner at amd.com>; C Bergstrom <cbergstrom at pathscale.com>
Cc: cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Pan, Xiuli <xiuli.pan at intel.com>; nd <nd at arm.com>; Jeroen Ketema <j.ketema at imperial.ac.uk>
Subject: RE: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

If the concern is that using `#pragma OPENCL EXTENSION name : begin` may pollute the original OPENCL EXTENSION pragma. How about 

#pragma clang_opencl_extension name : begin version #pragma clang_opencl_extension name : end

And we can put this feature under an OpenCL extension clang_opencl_extension and disabled by default, so only those who enables it can use these pragmas.

Sam

-----Original Message-----
From: Liu, Yaxun (Sam)
Sent: Friday, July 8, 2016 1:44 PM
To: 'Bader, Alexey' <alexey.bader at intel.com>; Anastasia Stulova <Anastasia.Stulova at arm.com>; Sumner, Brian <Brian.Sumner at amd.com>; C Bergstrom <cbergstrom at pathscale.com>
Cc: cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Pan, Xiuli <xiuli.pan at intel.com>; nd <nd at arm.com>; Jeroen Ketema <j.ketema at imperial.ac.uk>
Subject: RE: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

It may not be a good idea since users may want to use #ifdef cl_vendor_new_extension to condition their own code.

Sam

-----Original Message-----
From: Bader, Alexey [mailto:alexey.bader at intel.com]
Sent: Friday, July 8, 2016 1:31 PM
To: Liu, Yaxun (Sam) <Yaxun.Liu at amd.com>; Anastasia Stulova <Anastasia.Stulova at arm.com>; Sumner, Brian <Brian.Sumner at amd.com>; C Bergstrom <cbergstrom at pathscale.com>
Cc: cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Pan, Xiuli <xiuli.pan at intel.com>; nd <nd at arm.com>; Jeroen Ketema <j.ketema at imperial.ac.uk>
Subject: RE: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

Hi Sam,

Could preprocessor be utilized to diagnose invalid use of such extensions?

For instance, library developer would implement library header with the following code:

#ifdef cl_vendor_new_extension

typedef global struct type1;
void func1();
void func2();

#endif

User is required to pass -Dcl_vendor_new_extension if the intension is to use new extension.
If define is not set, but type1 or func1/func2 are present in the code - clang will report errors.

Thanks,
Alexey

-----Original Message-----
From: Liu, Yaxun (Sam) [mailto:Yaxun.Liu at amd.com]
Sent: Wednesday, July 6, 2016 7:11 PM
To: Anastasia Stulova <Anastasia.Stulova at arm.com>; Bader, Alexey <alexey.bader at intel.com>; Sumner, Brian <Brian.Sumner at amd.com>; C Bergstrom <cbergstrom at pathscale.com>
Cc: cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Pan, Xiuli <xiuli.pan at intel.com>; nd <nd at arm.com>; Jeroen Ketema <j.ketema at imperial.ac.uk>
Subject: RE: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

Sorry for the delayed response.

If an OpenCL extension defines types and builtin functions. When this extension is disabled, we should diagnose the use of such types and builtin functions. If we did not do that already, that means our diagnostics need to be improved.

If we want to implement that purely in the compiler, we need to define the types and builtin functions of all extensions in Clang. As we know there are some limitations in the types that can be used in the builtin td file, so most likely these builtin functions definitions need to be added programmatically in Clang, which is a lengthy and error-prone process, and definitely not friendly to the library developers.

This justifies a better way to define the OpenCL extension and its associated types and builtin functions.

For certain types and builtin functions which require special semantics, implementation in Clang itself is unavoidable. However, for most extensions, probably we only need to diagnose the availability of the types and builtin functions associated with an extension, whereas the types and builtin functions themselves do not need special diagnostics. In such case defining them in Clang programmatically is unnecessary. We only need to have a way to tell Clang that these types and builtin functions are associated with an OpenCL extension.

Since these types and builtin functions are defined in header files, so naturally we can use pragma to do that.

I propose a modification of the previous proposal which allows us to associate types and function declarations with an OpenCL extension, e.g.

#pragma OpenCL EXTENSION new_extension : begin optional_available_opencl_version

Typedef global struct   type1;
Void func1();
Void func2();

#pragma OpenCL EXTENSION new_extension : end

Basically the types and builtin functions defined between #pragma OpenCL EXTENSION name : begin and #pragma OpenCL EXTENSION name : end associates with the extension so that we can diagnose invalid uses of them.

I also added an optional argument for the pragma to indicate the earliest available OpenCL version of this extension so that we can diagnose incompatible OpenCL version for the extension.

Sam

-----Original Message-----
From: Anastasia Stulova [mailto:Anastasia.Stulova at arm.com]
Sent: Tuesday, June 28, 2016 10:11 AM
To: Bader, Alexey <alexey.bader at intel.com>; Sumner, Brian <Brian.Sumner at amd.com>; C Bergstrom <cbergstrom at pathscale.com>
Cc: Liu, Yaxun (Sam) <Yaxun.Liu at amd.com>; cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Pan, Xiuli <xiuli.pan at intel.com>; nd <nd at arm.com>; Jeroen Ketema <j.ketema at imperial.ac.uk>
Subject: RE: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

> I don't this it's required to have compiler directive for every OpenCL extension. It makes sense to have compiler directive only for extensions that modify OpenCL kernel language semantics.
I agree with you here, but I guess it's mainly required for compatibility reasons although most of vendors use Clang anyways. But yes, some extensions from the Spec don't seem to be useful for compiler indeed. Wondering if it should be discussed with Khronos?

Anastasia

-----Original Message-----
From: Bader, Alexey [mailto:alexey.bader at intel.com]
Sent: 27 June 2016 19:34
To: Sumner, Brian; C Bergstrom
Cc: Anastasia Stulova; Liu, Yaxun (Sam); cfe-dev (cfe-dev at lists.llvm.org); Pan, Xiuli; nd; Jeroen Ketema
Subject: RE: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

Hi Brian,

> Alexey, this proposal does change the behavior of the compiler.  It informs clang that there is an OpenCL extension that can be enabled and disabled whose name was not known at the time clang was built.  After being registered, clang must process enable and disable pragmas for the new extension exactly as it does the built-in KHR extensions.

What I meant is that I don't quite understand how it's useful developers. Clang is supposed to check for errors and produce LLVM IR for the back-end.

The only value I see that this registering allows developers use "#pragma ... enable" and suppress clang's warning about unsupported extension. But since clang doesn't provide any additional error checking for the registered extension nor produce different LLVM IR, I don't see any reason to use "#pragma ... enable" in the first place. As an alternative solution we can introduce a compiler knob that will disable this diagnostics.

My understanding is that the only useful OpenCL extension supported by clang are 'cl_khr_fp16', 'cl_khr_fp64' and 'cl_khr_*_atomics', since they enable compilation of half/double/atomic types. The rest are useless for the front-end compiler. If developer enable 'cl_khr_gl_sharing' in the OpenCL program - it will have zero effect on compilation results.

I don't this it's required to have compiler directive for every OpenCL extension. It makes sense to have compiler directive only for extensions that modify OpenCL kernel language semantics.

Thanks,
Alexey

-----Original Message-----
From: Sumner, Brian [mailto:Brian.Sumner at amd.com]
Sent: Monday, June 27, 2016 5:29 PM
To: C Bergström <cbergstrom at pathscale.com>; Bader, Alexey <alexey.bader at intel.com>
Cc: Anastasia Stulova <Anastasia.Stulova at arm.com>; Liu, Yaxun (Sam) <Yaxun.Liu at amd.com>; cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Pan, Xiuli <xiuli.pan at intel.com>; nd <nd at arm.com>; Jeroen Ketema <j.ketema at imperial.ac.uk>
Subject: RE: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

Christopher,  we can certainly modify clang, and have.  But then we have to carry those modifications around and try to get them into the clang code base.  Is it OK for clang to carry around every OpenCL vendor's vendor specific additions?  AMD has quite a few extensions, and so do Intel, Qualcomm, and others.   Wouldn't it be better for clang to provide a mechanism to allow extensions to be defined in a way that avoids clang having to carry around each and every one of those vendor extensions?

Alexey, this proposal does change the behavior of the compiler.  It informs clang that there is an OpenCL extension that can be enabled and disabled whose name was not known at the time clang was built.  After being registered, clang must process enable and disable pragmas for the new extension exactly as it does the built-in KHR extensions.

Brian

-----Original Message-----
From: C Bergström [mailto:cbergstrom at pathscale.com]
Sent: Monday, June 27, 2016 2:56 AM
To: Bader, Alexey
Cc: Anastasia Stulova; Sumner, Brian; Liu, Yaxun (Sam); cfe-dev (cfe-dev at lists.llvm.org); Pan, Xiuli; nd; Jeroen Ketema
Subject: Re: [cfe-dev] [RFC][OpenCL] Allow users to add supported OpenCL extensions by pragma

I don't care 2-ways about OpenCL, but adding non-standard pragma is a slippery slope. Please don't add extensions because some vendor is incapable of doing slight modifications to the source. If instead of a pragma you offered a build configuration, macros or something else to make this easier that I could totally understand.

On Mon, Jun 27, 2016 at 3:43 PM, Bader, Alexey via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> Hi,
>
>
>
> Sorry for being late to the discussion, but I didn’t get the use case 
> of the proposed feature.
>
> Why vendor/library developer would need compiler to support for some 
> pragma that doesn’t change to behavior of the compiler?
>
>
>
> Thanks,
>
> Alexey
>
>
>
> From: Anastasia Stulova [mailto:Anastasia.Stulova at arm.com]
> Sent: Tuesday, June 14, 2016 6:56 PM
> To: Sumner, Brian <Brian.Sumner at amd.com>; Liu, Yaxun (Sam) 
> <Yaxun.Liu at amd.com>; cfe-dev (cfe-dev at lists.llvm.org) 
> <cfe-dev at lists.llvm.org>; Bader, Alexey <alexey.bader at intel.com>; Pan, 
> Xiuli <xiuli.pan at intel.com>
> Cc: Jeroen Ketema <j.ketema at imperial.ac.uk>; nd <nd at arm.com>
> Subject: RE: [RFC][OpenCL] Allow users to add supported OpenCL 
> extensions by pragma
>
>
>
> Thanks, Brian!
>
>
>
> Yes, perhaps we could try to make the extension list fully dynamic. 
> Although I would imagine it’s still useful to have standard extensions 
> from the Spec directly in Clang to be available for the cases without 
> standard header include.
>
>
>
> Cheers,
>
> Anastasia
>
>
>
> From: Sumner, Brian [mailto:Brian.Sumner at amd.com]
> Sent: 14 June 2016 16:18
> To: Anastasia Stulova; Liu, Yaxun (Sam); cfe-dev 
> (cfe-dev at lists.llvm.org); Bader, Alexey (alexey.bader at intel.com); Pan, 
> Xiuli
> Cc: Jeroen Ketema; nd
> Subject: RE: [RFC][OpenCL] Allow users to add supported OpenCL 
> extensions by pragma
>
>
>
> Hi Anastasia,
>
>
>
> To answer your last question, this would be specific to clang.  It 
> will allow those of us using clang for OpenCL languages to implement 
> many of our vendor specific extensions without any other changes to 
> clang.  I expect a few of the current KHR extensions that do not 
> involve new or special types could be moved into opencl-c.h using this mechanism if desired as well.
>
>
>
> Thanks,
>
> Brian
>
>
>
> From: Anastasia Stulova [mailto:Anastasia.Stulova at arm.com]
> Sent: Monday, June 13, 2016 11:25 AM
> To: Liu, Yaxun (Sam); cfe-dev (cfe-dev at lists.llvm.org); Bader, Alexey 
> (alexey.bader at intel.com); Pan, Xiuli
> Cc: Sumner, Brian; Jeroen Ketema; nd
> Subject: RE: [RFC][OpenCL] Allow users to add supported OpenCL 
> extensions by pragma
>
>
>
> Interesting idea! Just to be clear, you are suggesting to add parsing 
> of the following into Clang:
>
>
>
> #pragma OPENCL EXTENSION the_new_extension_name : register
>
>
>
> Which would add custom the_new_extension_name to the list of known and 
> supported OpenCL extensions dynamically?
>
>
>
> Normally, target triple that combines RT and specific hardware type 
> would be used for those purposes.
>
> But I believe it shouldn’t be too complicated to add this pragma 
> considering that OpenCL already has similar ones.
>
>
>
> This approach could offer some degree of flexibility to the library 
> implementations,
>
> which would otherwise have to add separate triple representing their 
> library ABIs.
>
>
>
> Would this be allowed in any OpenCL code as some sort of Clang 
> extension or is there a plan to add this into Spec?
>
>
>
> Thanks,
>
> Anastasia
>
>
>
> From: Liu, Yaxun (Sam) [mailto:Yaxun.Liu at amd.com]
> Sent: 10 June 2016 19:40
> To: cfe-dev (cfe-dev at lists.llvm.org); Anastasia Stulova; Bader, Alexey 
> (alexey.bader at intel.com); Pan, Xiuli
> Cc: Sumner, Brian; Jeroen Ketema
> Subject: [RFC][OpenCL] Allow users to add supported OpenCL extensions 
> by pragma
>
>
>
> Currently Clang defines supported OpenCL extensions for each target 
> based on triple and CPU. As a default configuration this seems sufficient.
>
>
>
> However if vendors and library developers want to add support of their 
> own extensions, they have to modify Clang, which is very inconvenient.
>
>
>
> We need a flexible and expressive way to add supported extensions for 
> vendors and library developers.
>
>
>
> Brian has a proposal which introduces a pragma to add supported extensions:
>
>
>
> #pragma OPENCL EXTENSION the_new_extension_name : register
>
>
>
> This pragma tells clang the name of a new OpenCL extension and request 
> that it process it just like any other OpenCL extension, e.g.
> recognize
>
>
>
> #pragma OPENCL EXTENSION the_new_extension_name : enable/disable
>
>
>
> in subsequent code.
>
>
>
> Since this pragma can be used in the header file of vendors or library 
> developers’ OpenCL implementations, it can provide flexible and 
> expressive way to represent supported extensions when combined with 
> other preprocessor constructs.
>
>
>
> Any feedbacks? Thanks.
>
>
>
> Sam
>
>
>
>
>
>
>
>
> --------------------------------------------------------------------
> Joint Stock Company Intel A/O
> Registered legal address: Krylatsky Hills Business Park,
> 17 Krylatskaya Str., Bldg 4, Moscow 121614, Russian Federation
>
> This e-mail and any attachments may contain confidential material for 
> the sole use of the intended recipient(s). Any review or distribution 
> by others is strictly prohibited. If you are not the intended 
> recipient, please contact the sender and delete all copies.
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>

--------------------------------------------------------------------
Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park,
17 Krylatskaya Str., Bldg 4, Moscow 121614, Russian Federation

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

--------------------------------------------------------------------
Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park,
17 Krylatskaya Str., Bldg 4, Moscow 121614, Russian Federation

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

--------------------------------------------------------------------
Joint Stock Company Intel A/O
Registered legal address: Krylatsky Hills Business Park,
17 Krylatskaya Str., Bldg 4, Moscow 121614,
Russian Federation

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.