[cfe-dev] [RFC] Unified offloading option for CUDA/HIP/OpenMP

[Liu, Yaxun (Sam) via cfe-dev]

[AMD Public Use]

The offload kind is intended for situations where offload kind cannot be inferred from input file type, e.g. bundled LLVM bitcode or bundled assembly file. If it is redundant to specify it multiple times, we may consider a separate -offload-kind= option. I understand OpenMP can use -fopenmp to indicate OpenMP offload kind, but HIP may need an -offload-kind=hip option since it currently does not have one.

`-offload=target1,target2,... -offload-target1=“-march xxx -opt1” -offload-target2=“-march yyy -opt2 -opt3” does not well work for CUDA/HIP since CUDA/HIP can have multiple device archs for the same target.

How about -offload=target1,arch1,opt1a,opt1b -offload=target2,arch2,opt2a,opt2b ?

Thanks.

Sam

-----Original Message-----
From: Alexey Bataev <a.bataev at hotmail.com> 
Sent: Saturday, December 12, 2020 9:40 AM
To: Liu, Yaxun (Sam) <Yaxun.Liu at amd.com>
Cc: cfe-dev (cfe-dev at lists.llvm.org) <cfe-dev at lists.llvm.org>; Lieberman, Ron <Ron.Lieberman at amd.com>; georgakoudis1 at llnl.gov; jdoerfert at anl.gov; Artem Belevich <tra at google.com>; Searles, Mark <Mark.Searles at amd.com>; Chan, SiuChi <siuchi.chan at amd.com>
Subject: Re: [RFC] Unified offloading option for CUDA/HIP/OpenMP

[CAUTION: External Email]

Why do we need “kind” here? We know the kind already, if -fopenmp is used - kind is omp, if .hip is compiled - kind is hip.
Adding arch also does not look good. We may need to pass some extra params. Better to have something like `-offload=target1,target2,... -offload-target1=“-march xxx -opt1” -offload-target2=“-march yyy -opt2 -opt3” ...`.

Best regards,
Alexey Bataev

> 12 дек. 2020 г., в 09:11, Liu, Yaxun (Sam) <Yaxun.Liu at amd.com> написал(а):
>
> [AMD Public Use]
>
> Currently CUDA/HIP and OpenMP has different offloading options, e.g.
>
> clang++  -fopenmp -fopenmp-targets=amdgcn-amd-amdhsa -Xopenmp-target=amdgcn-amd-amdhsa -march=gfx900 test.cpp
>
> clang++ -offload-arch=gfx906 test.hip
>
> Our users request to have a concise way to specify offloading options for OpenMP. Ideally, one option to convey offloading kind, offloading triple, and offloading device arch.
>
> On the other hand, there are some limitations of the current offloading option for CUDA/HIP:
>
> 1. It does not specify offloading kind whereas relies on file type to infer offloading kind. If input file is not CUDA/HIP source code (e.g. bundled LLVM bit code), there needs a way to specify offloading kind.
>
> 2. It does not specify offloading target triple whereas relies on device arch to infer target triple. As HIP is ported to different targets, there needs a way to specify offloading target triple.
>
> In summary, a unified offloading option is preferred, which conveys offloading kind, offloading target triple and offloading device arch.
>
> I would like to propose to either have a new option or extend the existing -offload-arch option for that, in the format kind-triple-arch, e.g.
>
> -offload=omp-amd-gfx900
>
> -offload=hip-amd-gfx906
>
> Whereas kind and triple can be abbreviations for conciseness, e.g. omp expands to openmp, amd expands to amdgcn-amd-amdhsa. Arch can be omitted, in which case clang will use the default arch for the triple.
>
> Your feedbacks are welcome.
>
> Thanks.
>
> Sam
>