[PATCH] D97340: [HIP] Support Spack packages

[Harmen Stoppels via Phabricator via cfe-commits]

haampie added a comment.
Herald added a subscriber: foad.

Hi Yaxunl,

> The patch should not cause circular dependency on HIP or device library.

I'm not saying this patch introduces a circular dependency, I'm saying you are trying to solve an already existing circular dependency (clang needs device libs at runtime, but device libs need llvm to compile).

Let's talk about my PR here: https://github.com/spack/spack/pull/23859. It's building rocm-device-libs as part of llvm-amdgpu by configuring llvm with `-DLLVM_EXTERNAL_PROJECTS=device-libs`.

If you checkout that pr, run `spack install hip at 4.2.0`, what you get is get is:

  $ spack find -p llvm-amdgpu at 4.2.0
  llvm-amdgpu at 4.2.0  /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi

And indeed the bitcode is there:

  $ find /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi -iname '*.bc'
  /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi/amdgcn/bitcode/oclc_isa_version_1033.bc
  /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi/amdgcn/bitcode/ocml.bc
  /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi/amdgcn/bitcode/hip.bc
  ...

Now when I used this `--print-rocm-search-dirs` flag on clang without other flags, what I see is:

  $ /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi/bin/clang++ --print-rocm-search-dirs -x hip hi.cc 
  ROCm installation search path (Spack 4.2.0): /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0
  ROCm installation search path: /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi/lib/clang/12.0.0
  ROCm installation search path: /opt/rocm
  ...
  clang-12: error: cannot find ROCm device library. Provide its path via --rocm-path or --rocm-device-lib-path, or pass -nogpulib to build without ROCm device library.

Now can you make it such that clang will search the llvm prefix itself?

In fact, if you just reduces ALL 3 search paths to just CMAKE_INSTALL_PREFIX path, it will find the correct bitcode files independent of whether you're using spack or whether you're bundling the rocm packages the traditional way in with the shared prefix in /opt/rocm... So, what I'd like to see is this:

  $ /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi/bin/clang++ --print-rocm-search-dirs -x hip hi.cc 
  ROCm installation search path: /spack/opt/spack/linux-ubuntu20.04-zen2/gcc-10.2.0/llvm-amdgpu-4.2.0-a7jwajhh2cmn7p5djyx42lpcdfluk7wi/

and

  $ /opt/rocm/bin/clang++ --print-rocm-search-dirs -x hip hi.cc 
  ROCm installation search path: /opt/rocm

Doesn't that make a whole lot more sense than informing llvm about spack?

Finally, it doesn't settle locating hipcc (nor `.hipVersion), but that shouldn't be a real issue from the point of view of a spack user, since spack already sets the `-I` and `-L` flags for you when you make your package depend on `hip`.

Finally,


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D97340/new/

https://reviews.llvm.org/D97340