[cfe-dev] [RFC] Re-use OpenCL address space attributes for SYCL
Bader, Alexey via cfe-dev
cfe-dev at lists.llvm.org
Mon Jul 27 09:18:31 PDT 2020
> > I don't think (2) deal with language semantics. I assume we both talking about
> > the same case when variable declaration is not explicitly annotated with address
> > space attribute. According to language semantics such objects are allocated in
> > generic address space, but the problem is that most OpenCL implementations have
> > problems with consuming SPIR-V files with global variables in generic address
> > space. As an alternative to CodeGen changes we can consider handling this issue
> > in SPIR-V translator tool.
>
>
> I am not really a CodeGen expert, maybe it will be ok. I think it's better if you discuss
> it with John McCall or someone who is more experienced with CodeGen architecture.
>
> Why don't you just do regular address space deduction in Sema and then cast the
> deduced address space to generic straight after? You already add similar casts for
> pointers that are annotated with address spaces through the user code, right?
> This approach will probably allow to reuse the logic from OpenCL and simplify CodeGen.
I don't see how it can be done without breaking C++ semantics demonstrated in
https://reviews.llvm.org/D80932#2073542.
> > This change is need to keep the types in LLVM IR consistent. Regular C++ user
> > code usually doesn't have address space annotations, so if memory references and
> > pointers are "generic". When allocation is done in named address space, we must
> > add address space cast to keep LLVM pointer types aligned.
>
>
> I feel that your design is slightly different to what address space attributes were intended
> for. The address spaces were introduced for embedded C and other dialects where the
> same logic applies. The address space is added into a type qualifer. This binds an object to
> certain memory segments and therefore the only valid conversions for different addresses
> are either using explicit casts or implicit conversions in operands of operations, similar to
> regular qualifiers or arithmetic conversions. There are no unexpected address space
> conversions from explicit address spaces to generic/default otherwise.
>
> It feels to me that what you actually need semantically is a flat memory. Then the
> embedded C model is just overkill. I feel the address space attribute might just not be
> a good conceptual fit for your design. Have you considered adding a new custom
> attribute to annotate pointer variable classes or variables with memory segments
> without propagating this into a type qualifier?
Yes. Originally we used separate attributes, but we replaced them with OpenCL
attributes. At some point we had OpenCL "parsed attribute representation" with
new semantic attributes - https://github.com/intel/llvm/pull/968/.
I can rebase the patch and upload it to Phabricator if it okay (and add new
parsed representation if needed).
>
> I imagine it would be pretty easy to implement in the frontend as you just need to propagate
> this to IR. Then your middle-end passes can use this annotation to remap from
> default/generic address space into any exact one. I think you can even achieve higher flexibility
> by using such annotation only as some sort of a hint and allow an optimizer to choose alternative
> memory regions if it can result in higher performance.
>
> > Feel free to join today's sync meeting at 9AM PT to have an online discussion.
>
> Thanks, but sorry it was short notice. Also I think it's good to use LLVM channels so we can
> keep everyone else in the loop and also record information for future reference during the
> code review or other documentation purposes.
Agree. Our sync meetings are not intended to replace LLVM channels, but rather
supplement them. In some cases phone calls are more efficient than email chains.
We keep notes from these syncs here:
https://github.com/intel/llvm/wiki/SYCL-upstreaming-working-group-meeting-notes,
so that information is also available for reference.
Thanks,
Alexey
From: Anastasia Stulova <Anastasia.Stulova at arm.com>
Sent: Friday, July 24, 2020 6:20 PM
To: Bader, Alexey <alexey.bader at intel.com>; cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; rjmccall at apple.com
Cc: nd <nd at arm.com>
Subject: Re: [RFC] Re-use OpenCL address space attributes for SYCL
> As SPIR-V doesn't allow casts between constant and generic pointers, SYCL
> implementation doesn't use OpenCL constant address space attribute. "const"
> qualified "global" address space attribute is used instead.
FYI in OpenCL C such conversions are disallowed and since address spaces are
preserved in AST, any such conversion in the source will be diagnosed and
rejected. Constant address space indicates a memory region where read-only
data are to be placed for efficient accesses, so it is not quite the same as global
memory.
> It's not "address spaces" per se, but how OpenCL mode implements them.
> Victor did a good job covering this question in this comment:
> https://reviews.llvm.org/D80932#2073542
I have replied to Victor's comment: https://reviews.llvm.org/D80932#2074792
> What languages do you think might be impacted if we enable this change
> unconditionally? Are there modes other than OpenCL and SYCL targeting SPIR?
Yes, some toolchains use it for standard C and C++ compilations to create
libraries to run on GPU-like targets. Generally, we should not limit SPIR to
OpenCL or SYCL. Clang design is made flexible to allow multiple targets
to support multiple languages. We shouldn't come up with an implementation
that will limit choices for future development.
> I don't think (2) deal with language semantics. I assume we both talking about
> the same case when variable declaration is not explicitly annotated with address
> space attribute. According to language semantics such objects are allocated in
> generic address space, but the problem is that most OpenCL implementations have
> problems with consuming SPIR-V files with global variables in generic address
> space. As an alternative to CodeGen changes we can consider handling this issue
> in SPIR-V translator tool.
I am not really a CodeGen expert, maybe it will be ok. I think it's better if you discuss
it with John McCall or someone who is more experienced with CodeGen architecture.
Why don't you just do regular address space deduction in Sema and then cast the
deduced address space to generic straight after? You already add similar casts for
pointers that are annotated with address spaces through the user code, right?
This approach will probably allow to reuse the logic from OpenCL and simplify CodeGen.
Alternatively, I imagine you could add a simple transformation pass to remap address
spaces If you move this logic into the translator then I guess your compilation
will only work for SPIR-V?
> This change is need to keep the types in LLVM IR consistent. Regular C++ user
> code usually doesn't have address space annotations, so if memory references and
> pointers are "generic". When allocation is done in named address space, we must
> add address space cast to keep LLVM pointer types aligned.
I feel that your design is slightly different to what address space attributes were intended
for. The address spaces were introduced for embedded C and other dialects where the
same logic applies. The address space is added into a type qualifer. This binds an object to
certain memory segments and therefore the only valid conversions for different addresses
are either using explicit casts or implicit conversions in operands of operations, similar to
regular qualifiers or arithmetic conversions. There are no unexpected address space
conversions from explicit address spaces to generic/default otherwise.
It feels to me that what you actually need semantically is a flat memory. Then the
embedded C model is just overkill. I feel the address space attribute might just not be
a good conceptual fit for your design. Have you considered adding a new custom
attribute to annotate pointer variable classes or variables with memory segments
without propagating this into a type qualifier?
I imagine it would be pretty easy to implement in the frontend as you just need to propagate
this to IR. Then your middle-end passes can use this annotation to remap from
default/generic address space into any exact one. I think you can even achieve higher flexibility
by using such annotation only as some sort of a hint and allow an optimizer to choose alternative
memory regions if it can result in higher performance.
> Feel free to join today's sync meeting at 9AM PT to have an online discussion.
Thanks, but sorry it was short notice. Also I think it's good to use LLVM channels so we can
keep everyone else in the loop and also record information for future reference during the
code review or other documentation purposes.
________________________________
From: Bader, Alexey <alexey.bader at intel.com<mailto:alexey.bader at intel.com>>
Sent: 20 July 2020 13:22
To: Anastasia Stulova <Anastasia.Stulova at arm.com<mailto:Anastasia.Stulova at arm.com>>; cfe-dev (cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>) <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>; rjmccall at apple.com<mailto:rjmccall at apple.com> <rjmccall at apple.com<mailto:rjmccall at apple.com>>
Cc: nd <nd at arm.com<mailto:nd at arm.com>>
Subject: RE: [RFC] Re-use OpenCL address space attributes for SYCL
Hi Anastasia,
Sorry for the delay.
> > The main difference with OpenCL mode is that SYCL
> > mode (similar to other single-source GPU programming modes like
> > OpenMP/CUDA/HIP)
> > keeps "default" address space for the declaration without address space
> > attribute annotations.
>
> Just FYI in C++ mode, Clang implements default/generic address space as
> specified in embedded C (ISO/IEC TR 18037) s5.1 - 5.3.
>
> "When not specified otherwise, objects are allocated by default in a generic
> address space, which corresponds to the single address space of ISO/IEC
> 9899:1999."
>
> "Objects are allocated in one or more address spaces. A unique generic address
> space always exists. Every address space other than the generic one has a unique
> name in the form of an identifier. Address spaces other than the generic one are
> called named address spaces. An object is always completely allocated into at
> least one address space. Unless otherwise specified, objects are allocated in
> the generic address space."
>
> It feels to me this is the model you intend to follow?
After reading the document I don't see major conflicts with our SYCL
implementation.
> If you use OpenCL address
> space attributes outside of OpenCL mode there is limited logic that you will
> inherit. For example deduction of address spaces wouldn't work but conversions
> or generation to IR should work fine. It generally sounds like a viable approach
> but OpenCL however used Default (no address space) as private AS for a very long
> time and there are still a number of places where this assumption is inherent in
> the implementation. This is not entirely strange as Default is use by many
> languages for automatic storage anyway. My worry is there could be difficulties
> in reusing the OpenCL address space model due to this.
>
> Btw can you elaborate on your implementation of constant addr space?
As SPIR-V doesn't allow casts between constant and generic pointers, SYCL
implementation doesn't use OpenCL constant address space attribute. "const"
qualified "global" address space attribute is used instead.
>
> > This keeps the code shared between the host and device
> > semantically-correct for both compilers: regular C++ host compiler and SYCL
> > compiler.
>
> Sorry perhaps I am not following this thought but can you explain how
> address spaces make code semantically incorrect?
It's not "address spaces" per se, but how OpenCL mode implements them.
Victor did a good job covering this question in this comment:
https://reviews.llvm.org/D80932#2073542
Example form this comment of valid C++ function, which is not valid in OpenCL
mode:
```c++
template<typename T1, typename T2>
struct is_same {
static constexpr int value = 0;
};
template<typename T>
struct is_same<T, T> {
static constexpr int value = 1;
};
void foo(int p) {
static_assert(is_same<decltype(p), int>::value, "int is not an int?"); // Fails: p is '__private int' != 'int'
static_assert(is_same<decltype(&p), int*>::value, "int* is not an int*?"); // Fails: p is '__private int*' != '__generic int*'
}
```
>
> > To make all pointers without an explicit address space qualifier to be
> > pointers
> > in generic address space, we updated SPIR target address space map, which
> > currently maps default pointers to "private" address space.
>
> The address space map in Clang is not specific to pointer types. How do you
> make it work for pointers only?
I don't think we did anything specific to apply this change to pointers only.
Pointers provided here as an example to demonstrate the impact of the change in
LLVM IR representation for SPIR target.
>
> > We made this change
> > specific to SYCL by adding SYCL environment component to the Triple to avoid
> > impact on other modes targeting SPIR target (e.g. OpenCL). We would be glad to
> > see get a feedback from the community if changing this mapping is applicable
> > for all the modes and additional specialization can be avoided (e.g.
> > [AMDGPU](https://github.com/llvm/llvm-project/blob/master/clang/lib/Basic/Targets/AMDGPU.cpp#L329)
> > maps default to "generic" address space with a couple of exceptions).
>
> Ok, does it mean that you map Default address space to OpenCL generic?
> Please note that Default address space is used outside of OpenCL for all
> other languages so remapping this unconditionally will have a wider impact.
Current implementation applies different mapping only when "sycldevice"
environment is set in target triple.
https://github.com/bader/llvm/pull/18/files#diff-d62fb2e1d8c597ce59fd10e018f6fb77R61
What languages do you think might be impacted if we enable this change
unconditionally? Are there modes other than OpenCL and SYCL targeting SPIR?
>
> > There are a few cases when CodeGen assigns non-default address space:
> >
> > 1. For declaration explicitly annotated with address space attribute
>
> This is generally how CodeGen works mapping language address spaces to target
> address spaces. Is there something different you do here for SYCL?
No.
>
> > 2. Variables with static storage duration and string literals are allocated in
> > global address space unless specific address space it specified.
> > 3. Variables with automatic storage durations are allocated in private address
> > space. It's current compiler behavior and it doesn't require additional
> > changes.
>
> We already have this logic for OpenCL in Sema. I am not an expert in CodeGen but
> I believe its primary task is to map language constructs onto the target specific IR
> i.e. map from AST into IR. However, you are making it dial with language semantic
> instead i.e. add missing AST logic such as address space attribute. I believe there
> are good reasons to have layering architecture that separates various concerns.
> What drives your decision for moving this logic into CodeGen?
I don't think (2) deal with language semantics. I assume we both talking about
the same case when variable declaration is not explicitly annotated with address
space attribute. According to language semantics such objects are allocated in
generic address space, but the problem is that most OpenCL implementations have
problems with consuming SPIR-V files with global variables in generic address
space. As an alternative to CodeGen changes we can consider handling this issue
in SPIR-V translator tool.
>
> > For (2) and (3) cases, once "default" pointer to such variable is obtained, it
> > is immediately addrspacecast'ed to generic, because a user does not (and
> > should not) specify address space for pointers in source code.
>
> Can you explain why you need this cast?
This change is need to keep the types in LLVM IR consistent. Regular C++ user
code usually doesn't have address space annotations, so if memory references and
pointers are "generic". When allocation is done in named address space, we must
add address space cast to keep LLVM pointer types aligned.
> Can user not specify address spaces using
> pointer classes that map into address space attributed types i.e. ending up with
> pointer with address spaces originating from the user code?
Yes.
Feel free to join today's sync meeting at 9AM PT to have an online discussion.
Thanks,
Alexey
From: Anastasia Stulova <Anastasia.Stulova at arm.com<mailto:Anastasia.Stulova at arm.com>>
Sent: Thursday, July 9, 2020 2:51 PM
To: Bader, Alexey <alexey.bader at intel.com<mailto:alexey.bader at intel.com>>; cfe-dev (cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>) <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>; rjmccall at apple.com<mailto:rjmccall at apple.com>
Cc: nd <nd at arm.com<mailto:nd at arm.com>>
Subject: Re: [RFC] Re-use OpenCL address space attributes for SYCL
Hi Alexey,
Thanks for the clarification.
> SYCL compiler re-use generic support for these attributes as is and modifies
> Sema and CodeGen libraries.
Can you elaborate on your modifications in Sema and CodeGen, please?
> The main difference with OpenCL mode is that SYCL
> mode (similar to other single-source GPU programming modes like
> OpenMP/CUDA/HIP)
> keeps "default" address space for the declaration without address space
> attribute annotations.
Just FYI in C++ mode, Clang implements default/generic address space as
specified in embedded C (ISO/IEC TR 18037) s5.1 - 5.3.
"When not specified otherwise, objects are allocated by default in a generic
address space, which corresponds to the single address space of ISO/IEC
9899:1999."
"Objects are allocated in one or more address spaces. A unique generic address
space always exists. Every address space other than the generic one has a unique
name in the form of an identifier. Address spaces other than the generic one are
called named address spaces. An object is always completely allocated into at
least one address space. Unless otherwise specified, objects are allocated in
the generic address space."
It feels to me this is the model you intend to follow? If you use OpenCL address
space attributes outside of OpenCL mode there is limited logic that you will
inherit. For example deduction of address spaces wouldn't work but conversions
or generation to IR should work fine. It generally sounds like a viable approach
but OpenCL however used Default (no address space) as private AS for a very long
time and there are still a number of places where this assumption is inherent in
the implementation. This is not entirely strange as Default is use by many
languages for automatic storage anyway. My worry is there could be difficulties
in reusing the OpenCL address space model due to this.
Btw can you elaborate on your implementation of constant addr space?
> This keeps the code shared between the host and device
> semantically-correct for both compilers: regular C++ host compiler and SYCL
> compiler.
Sorry perhaps I am not following this thought but can you explain how
address spaces make code semantically incorrect?
> To make all pointers without an explicit address space qualifier to be
> pointers
> in generic address space, we updated SPIR target address space map, which
> currently maps default pointers to "private" address space.
The address space map in Clang is not specific to pointer types. How do you
make it work for pointers only?
> We made this change
> specific to SYCL by adding SYCL environment component to the Triple to avoid
> impact on other modes targeting SPIR target (e.g. OpenCL). We would be glad to
> see get a feedback from the community if changing this mapping is applicable
> for all the modes and additional specialization can be avoided (e.g.
> [AMDGPU](https://github.com/llvm/llvm-project/blob/master/clang/lib/Basic/Targets/AMDGPU.cpp#L329)
> maps default to "generic" address space with a couple of exceptions).
Ok, does it mean that you map Default address space to OpenCL generic?
Please note that Default address space is used outside of OpenCL for all
other languages so remapping this unconditionally will have a wider impact.
> There are a few cases when CodeGen assigns non-default address space:
>
> 1. For declaration explicitly annotated with address space attribute
This is generally how CodeGen works mapping language address spaces to target
address spaces. Is there something different you do here for SYCL?
> 2. Variables with static storage duration and string literals are allocated in
> global address space unless specific address space it specified.
> 3. Variables with automatic storage durations are allocated in private address
> space. It's current compiler behavior and it doesn't require additional
> changes.
We already have this logic for OpenCL in Sema. I am not an expert in CodeGen but
I believe its primary task is to map language constructs onto the target specific IR
i.e. map from AST into IR. However, you are making it dial with language semantic
instead i.e. add missing AST logic such as address space attribute. I believe there
are good reasons to have layering architecture that separates various concerns.
What drives your decision for moving this logic into CodeGen?
> For (2) and (3) cases, once "default" pointer to such variable is obtained, it
> is immediately addrspacecast'ed to generic, because a user does not (and
> should not) specify address space for pointers in source code.
Can you explain why you need this cast? Can user not specify address spaces using
pointer classes that map into address space attributed types i.e. ending up with
pointer with address spaces originating from the user code?
Cheers,
Anastasia
________________________________
From: Bader, Alexey <alexey.bader at intel.com<mailto:alexey.bader at intel.com>>
Sent: 26 June 2020 13:04
To: cfe-dev (cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>) <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>; Anastasia Stulova <Anastasia.Stulova at arm.com<mailto:Anastasia.Stulova at arm.com>>; rjmccall at apple.com<mailto:rjmccall at apple.com> <rjmccall at apple.com<mailto:rjmccall at apple.com>>
Subject: [RFC] Re-use OpenCL address space attributes for SYCL
Hi,
We would like to re-use OpenCL address space attributes for SYCL to target
SPIR-V format and enable efficient memory access on GPUs.
```c++
__attribute__((opencl_global))
__attribute__((opencl_local))
__attribute__((opencl_private))
```
The first patch enabling conversion between pointers annotated with OpenCL
address space attribute and "default" pointers is being reviewed here
https://reviews.llvm.org/D80932.
Before moving further with the implementation we would like to discuss two
questions raised in review comments (https://reviews.llvm.org/D80932#2085848).
## Using attributes to annotate memory allocations
Introduction section of SYCL-1.2.1 specification describes multiple compilation
flows intended by the design:
> SYCL is designed to allow a compilation flow where the source file is passed
> through multiple different compilers, including a standard C++ host compiler
> of the developer's choice, and where the resulting application combines the
> results of these compilation passes. This is distinct from a single-source
> flow that might use language extensions that preclude the use of a standard
> host compiler. The SYCL standard does not preclude the use of a single
> compiler flow, but is designed to not require it.
>
> The advantages of this design are two-fold. First, it offers better
> integration with existing tool chains. An application that already builds
> using a chosen compiler can continue to do so when SYCL code is added. Using
> the SYCL tools on a source file within a project will both compile for an
> OpenCL device and let the same source file be compiled using the same host
> compiler that the rest of the project is compiled with. Linking and library
> relationships are unaffected. This design simplifies porting of pre-existing
> applications to SYCL. Second, the design allows the optimal compiler to be
> chosen for each device where different vendors may provide optimized
> tool-chains.
>
> SYCL is designed to be as close to standard C++ as possible. In practice,
> this means that as long as no dependence is created on SYCL's integration
> with OpenCL, a standard C++ compiler can compile the SYCL programs and they
> will run correctly on host CPU. Any use of specialized low-level features
> can be masked using the C preprocessor in the same way that
> compiler-specific intrinsics may be hidden to ensure portability between
> different host compilers.
Following this approach, SYCL uses C++ templates to represent pointers to
disjoint memory regions on an accelerator to enable compilation with standard
C++ toolchain and SYCL compiler toolchain.
For instance:
```c++
// CPU/host implementation
template <typename T, address_space AS> class multi_ptr {
T *data; // ignore address space parameter on CPU
public:
T *get_pointer() { return data; }
}
// check that SYCL mode is ON and we can use non-standard annotations
#if defined(__SYCL_DEVICE_ONLY__)
// GPU/accelerator implementation
template <typename T, address_space AS> class multi_ptr {
// GetAnnotatedPointer<T, global>::type == "__attribute__((opencl_global)) T"
using pointer_t = typename GetAnnotatedPointer<T, AS>::type *;
pointer_t data;
public:
pointer_t get_pointer() { return data; }
}
#endif
```
User can use `multi_ptr` class as regular user-defined type in regular C++ code:
```c++
int *UserFunc(multi_ptr<int, global> ptr) {
/// ...
return ptr.get_pointer();
}
```
Depending on the compiler mode `multi_ptr` will either annotate internal data
with address space attribute or not.
## Implementation details
OpenCL attributes are handled by Parser in all modes. OpenCL mode has specific
logic in Sema and CodeGen components for these attributes.
SYCL compiler re-use generic support for these attributes as is and modifies
Sema and CodeGen libraries. The main difference with OpenCL mode is that SYCL
mode (similar to other single-source GPU programming modes like OpenMP/CUDA/HIP)
keeps "default" address space for the declaration without address space
attribute annotations. This keeps the code shared between the host and device
semantically-correct for both compilers: regular C++ host compiler and SYCL
compiler.
To make all pointers without an explicit address space qualifier to be pointers
in generic address space, we updated SPIR target address space map, which
currently maps default pointers to "private" address space. We made this change
specific to SYCL by adding SYCL environment component to the Triple to avoid
impact on other modes targeting SPIR target (e.g. OpenCL). We would be glad to
see get a feedback from the community if changing this mapping is applicable for
all the modes and additional specialization can be avoided (e.g.
[AMDGPU](https://github.com/llvm/llvm-project/blob/master/clang/lib/Basic/Targets/AMDGPU.cpp#L329)
maps default to "generic" address space with a couple of exceptions).
There are a few cases when CodeGen assigns non-default address space:
1. For declaration explicitly annotated with address space attribute
2. Variables with static storage duration and string literals are allocated in
global address space unless specific address space it specified.
3. Variables with automatic storage durations are allocated in private address
space. It's current compiler behavior and it doesn't require additional
changes.
For (2) and (3) cases, once "default" pointer to such variable is obtained, it
is immediately addrspacecast'ed to generic, because a user does not (and should
not) specify address space for pointers in source code.
A draft patch containing complete change-set is available
[here](https://github.com/bader/llvm/pull/18/).
Does this approach seem reasonable?
Thanks,
Alexey
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20200727/2fcb31eb/attachment-0001.html>
More information about the cfe-dev
mailing list