[all-commits] [llvm/llvm-project] 47b7c9: [libc] Rework the GPU build to be a regular target...

[Joseph Huber via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 47b7c91abe7af3133a591aa2e73fffa30826f986
      https://github.com/llvm/llvm-project/commit/47b7c91abe7af3133a591aa2e73fffa30826f986
  Author: Joseph Huber <huberjn at outlook.com>
  Date:   2024-02-22 (Thu, 22 Feb 2024)

  Changed paths:
    M clang/lib/Driver/ToolChains/CommonArgs.cpp
    M clang/test/Driver/openmp-offload-gpu.c
    M libc/CMakeLists.txt
    M libc/cmake/modules/LLVMLibCArchitectures.cmake
    M libc/cmake/modules/LLVMLibCCheckMPFR.cmake
    M libc/cmake/modules/LLVMLibCCompileOptionRules.cmake
    M libc/cmake/modules/LLVMLibCHeaderRules.cmake
    M libc/cmake/modules/LLVMLibCLibraryRules.cmake
    M libc/cmake/modules/LLVMLibCObjectRules.cmake
    M libc/cmake/modules/LLVMLibCTestRules.cmake
    M libc/cmake/modules/prepare_libc_gpu_build.cmake
    M libc/docs/gpu/using.rst
    M libc/include/CMakeLists.txt
    M libc/lib/CMakeLists.txt
    M libc/src/__support/File/CMakeLists.txt
    M libc/src/__support/GPU/CMakeLists.txt
    M libc/src/__support/OSUtil/CMakeLists.txt
    M libc/src/__support/RPC/CMakeLists.txt
    M libc/src/math/CMakeLists.txt
    M libc/src/math/gpu/vendor/CMakeLists.txt
    M libc/src/stdio/CMakeLists.txt
    M libc/src/stdlib/CMakeLists.txt
    M libc/src/string/CMakeLists.txt
    M libc/startup/gpu/CMakeLists.txt
    M libc/startup/gpu/amdgpu/CMakeLists.txt
    M libc/startup/gpu/nvptx/CMakeLists.txt
    M libc/test/CMakeLists.txt
    M libc/test/IntegrationTest/CMakeLists.txt
    M libc/test/UnitTest/CMakeLists.txt
    M libc/test/src/__support/CMakeLists.txt
    M libc/test/src/__support/CPP/CMakeLists.txt
    M libc/test/src/__support/File/CMakeLists.txt
    M libc/test/src/errno/CMakeLists.txt
    M libc/test/src/math/CMakeLists.txt
    M libc/test/src/math/smoke/CMakeLists.txt
    M libc/test/src/stdio/CMakeLists.txt
    M libc/test/src/stdlib/CMakeLists.txt
    M libc/test/utils/UnitTest/CMakeLists.txt
    M libc/utils/CMakeLists.txt
    M libc/utils/MPFRWrapper/CMakeLists.txt
    M libc/utils/gpu/CMakeLists.txt
    M libc/utils/gpu/loader/CMakeLists.txt
    M libc/utils/gpu/loader/amdgpu/CMakeLists.txt
    M libc/utils/gpu/loader/nvptx/CMakeLists.txt
    M libc/utils/gpu/server/CMakeLists.txt
    M llvm/CMakeLists.txt
    M llvm/cmake/modules/HandleLLVMOptions.cmake
    M llvm/runtimes/CMakeLists.txt
    M openmp/libomptarget/CMakeLists.txt
    M openmp/libomptarget/plugins-nextgen/common/CMakeLists.txt
    M openmp/libomptarget/plugins-nextgen/common/src/RPC.cpp
    M openmp/libomptarget/test/lit.cfg

  Log Message:
  -----------
  [libc] Rework the GPU build to be a regular target (#81921)

Summary:
This is a massive patch because it reworks the entire build and
everything that depends on it. This is not split up because various bots
would fail otherwise. I will attempt to describe the necessary changes
here.

This patch completely reworks how the GPU build is built and targeted.
Previously, we used a standard runtimes build and handled both NVPTX and
AMDGPU in a single build via multi-targeting. This added a lot of
divergence in the build system and prevented us from doing various
things like building for the CPU / GPU at the same time, or exporting
the startup libraries or running tests without a full rebuild.

The new appraoch is to handle the GPU builds as strict cross-compiling
runtimes. The first step required
https://github.com/llvm/llvm-project/pull/81557 to allow the `LIBC`
target to build for the GPU without touching the other targets. This
means that the GPU uses all the same handling as the other builds in
`libc`.

The new expected way to build the GPU libc is with
`LLVM_LIBC_RUNTIME_TARGETS=amdgcn-amd-amdhsa;nvptx64-nvidia-cuda`.

The second step was reworking how we generated the embedded GPU library
by moving it into the library install step. Where we previously had one
`libcgpu.a` we now have `libcgpu-amdgpu.a` and `libcgpu-nvptx.a`. This
patch includes the necessary clang / OpenMP changes to make that not
break the bots when this lands.

We unfortunately still require that the NVPTX target has an `internal`
target for tests. This is because the NVPTX target needs to do LTO for
the provided version (The offloading toolchain can handle it) but cannot
use it for the native toolchain which is used for making tests.

This approach is vastly superior in every way, allowing us to treat the
GPU as a standard cross-compiling target. We can now install the GPU
utilities to do things like use the offload tests and other fun things.

Some certain utilities need to be built with 
`--target=${LLVM_HOST_TRIPLE}` as well. I think this is a fine
workaround as we
will always assume that the GPU `libc` is a cross-build with a
functioning host.

Depends on https://github.com/llvm/llvm-project/pull/81557



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications