[cfe-dev] [RFC] Unified offloading option for CUDA/HIP/OpenMP

Sat Dec 12 13:05:57 PST 2020

[AMD Public Use]

When I talk about device arch, I mean the option passed by -mcpu. My understanding is that <sub> is intended for a variation of the <arch> in triple, which covers a set of CPU’s. I don’t think it is proper to treat each of -mcpu option as a variation of the <arch>.

Sam

From: Alexey Bataev <a.bataev at hotmail.com>
Sent: Saturday, December 12, 2020 11:46 AM
To: Liu, Yaxun (Sam) <Yaxun.Liu at amd.com>
Cc: cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Lieberman, Ron <Ron.Lieberman at amd.com>; georgakoudis1 at llnl.gov; jdoerfert at anl.gov; Artem Belevich <tra at google.com>; Searles, Mark <Mark.Searles at amd.com>; Chan, SiuChi <siuchi.chan at amd.com>
Subject: Re: [RFC] Unified offloading option for CUDA/HIP/OpenMP

[CAUTION: External Email]
I still don’t understand why do we need kind but -offload-kind=kind looks much better to me.
As to archs, I think the arch can be included in triple  <arch><sub>-<vendor>-<sys>-<abi>

You can specify sub target `amdgcngfx906-amd-amdhsa` but you need to add subarchs for triple class. How about this?

https://clang.llvm.org/docs/CrossCompilation.html<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fclang.llvm.org%2Fdocs%2FCrossCompilation.html&data=04%7C01%7CYaxun.Liu%40amd.com%7C31982767621948ab68d808d89ebd6daa%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637433884031446571%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=16lsQnCYmYjpxklqfcokC0UEnMVgdsV%2FYRO84F7mxWk%3D&reserved=0>

Best regards,
Alexey Bataev

12 дек. 2020 г., в 10:51, Liu, Yaxun (Sam) <Yaxun.Liu at amd.com<mailto:Yaxun.Liu at amd.com>> написал(а):
[AMD Public Use]

The offload kind is intended for situations where offload kind cannot be inferred from input file type, e.g. bundled LLVM bitcode or bundled assembly file. If it is redundant to specify it multiple times, we may consider a separate -offload-kind= option. I understand OpenMP can use -fopenmp to indicate OpenMP offload kind, but HIP may need an -offload-kind=hip option since it currently does not have one.

`-offload=target1,target2,... -offload-target1=“-march xxx -opt1” -offload-target2=“-march yyy -opt2 -opt3” does not well work for CUDA/HIP since CUDA/HIP can have multiple device archs for the same target.

How about -offload=target1,arch1,opt1a,opt1b -offload=target2,arch2,opt2a,opt2b ?

Thanks.

Sam

-----Original Message-----
From: Alexey Bataev <a.bataev at hotmail.com<mailto:a.bataev at hotmail.com>>
Sent: Saturday, December 12, 2020 9:40 AM
To: Liu, Yaxun (Sam) <Yaxun.Liu at amd.com<mailto:Yaxun.Liu at amd.com>>
Cc: cfe-dev (cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>) <cfe-dev at lists.llvm.org<mailto:cfe-dev at lists.llvm.org>>; Lieberman, Ron <Ron.Lieberman at amd.com<mailto:Ron.Lieberman at amd.com>>; georgakoudis1 at llnl.gov<mailto:georgakoudis1 at llnl.gov>; jdoerfert at anl.gov<mailto:jdoerfert at anl.gov>; Artem Belevich <tra at google.com<mailto:tra at google.com>>; Searles, Mark <Mark.Searles at amd.com<mailto:Mark.Searles at amd.com>>; Chan, SiuChi <siuchi.chan at amd.com<mailto:siuchi.chan at amd.com>>
Subject: Re: [RFC] Unified offloading option for CUDA/HIP/OpenMP

[CAUTION: External Email]

Why do we need “kind” here? We know the kind already, if -fopenmp is used - kind is omp, if .hip is compiled - kind is hip.
Adding arch also does not look good. We may need to pass some extra params. Better to have something like `-offload=target1,target2,... -offload-target1=“-march xxx -opt1” -offload-target2=“-march yyy -opt2 -opt3” ...`.

Best regards,
Alexey Bataev

12 дек. 2020 г., в 09:11, Liu, Yaxun (Sam) <Yaxun.Liu at amd.com<mailto:Yaxun.Liu at amd.com>> написал(а):

[AMD Public Use]

Currently CUDA/HIP and OpenMP has different offloading options, e.g.

clang++  -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx900 test.cpp

clang++ -offload-arch=gfx906 test.hip

Our users request to have a concise way to specify offloading options for OpenMP. Ideally, one option to convey offloading kind, offloading triple, and offloading device arch.

On the other hand, there are some limitations of the current offloading option for CUDA/HIP:

1. It does not specify offloading kind whereas relies on file type to infer offloading kind. If input file is not CUDA/HIP source code (e.g. bundled LLVM bit code), there needs a way to specify offloading kind.

2. It does not specify offloading target triple whereas relies on device arch to infer target triple. As HIP is ported to different targets, there needs a way to specify offloading target triple.

In summary, a unified offloading option is preferred, which conveys offloading kind, offloading target triple and offloading device arch.

I would like to propose to either have a new option or extend the existing -offload-arch option for that, in the format kind-triple-arch, e.g.

-offload=omp-amd-gfx900

-offload=hip-amd-gfx906

Whereas kind and triple can be abbreviations for conciseness, e.g. omp expands to openmp, amd expands to amdgcn-amd-amdhsa. Arch can be omitted, in which case clang will use the default arch for the triple.

Your feedbacks are welcome.

Thanks.

Sam

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20201212/5fd218b9/attachment-0001.html>