[all-commits] [llvm/llvm-project] b8239a: [OpenMP] Add support for extracting device code in...

Mon Jan 31 20:12:02 PST 2022

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: b8239af0eeedfa67d9fac28d20685b7a09e2935a
      https://github.com/llvm/llvm-project/commit/b8239af0eeedfa67d9fac28d20685b7a09e2935a
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/CMakeLists.txt
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Add support for extracting device code in linker wrapper

This patchs add support for extracting device offloading code from the
linker's input files. If the file contains a section with the name
`.llvm.offloading.<triple>.<arch>` it will be extracted to a new
temporary file to be linked. Addtionally, the host file containing it
will have the section stripped so it does not remain in the executable
once linked.

Depends on D116544

Differential Revision: https://reviews.llvm.org/D116545

  Commit: d0744585f9ea842ca97855b2e002fddc893c4a92
      https://github.com/llvm/llvm-project/commit/d0744585f9ea842ca97855b2e002fddc893c4a92
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [Clang] Initial support for linking offloading code in tool

This patch adds the initial support for linking NVPTX offloading code
using the clang-linker-wrapper tool. This uses the extracted device
files and runs `nvlink` on them. Currently this is then passed to the
existing toolchain for creating linkable OpenMP offloading programs
using `clang-offload-wrapper` and compiling it manually using `llc`.
More work is required to support LTO, Bitcode linking, AMDGPU, and x86
offloading.

Depends on D116545

Differential Revision: https://reviews.llvm.org/D116627

  Commit: 0e82c7553be95343f1cb62946eb0b84bd26f1563
      https://github.com/llvm/llvm-project/commit/0e82c7553be95343f1cb62946eb0b84bd26f1563
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Search for static libraries in offload linker tool

This patch adds support for searching through the linker library paths
to identify static libraries that may contain device code. If device
code is present it will be extracted. This should ideally fully support
static linking with OpenMP offloading.

Depends on D116627

Differential Revision: https://reviews.llvm.org/D116675

  Commit: c732c3df749b3d5a127708c05656a226ed5753ab
      https://github.com/llvm/llvm-project/commit/c732c3df749b3d5a127708c05656a226ed5753ab
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/lib/Driver/Driver.cpp
    M clang/lib/Driver/ToolChains/Clang.cpp
    M clang/tools/clang-linker-wrapper/CMakeLists.txt
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Initial Implementation of LTO and bitcode linking in linker wrapper

This patch implements the fist support for handling LTO in the
offloading pipeline. The flag `-foffload-lto` is used to control if
bitcode is embedded into the device. If bitcode is found in the device,
the extracted files will be sent to the LTO pipeline to be linked and
sent to the backend. This implementation does not separately link the
device bitcode libraries yet.

Depends on D116675

Differential Revision: https://reviews.llvm.org/D116975

  Commit: 3762111aa9608fce12c4f938bfef2b38aed766dd
      https://github.com/llvm/llvm-project/commit/3762111aa9608fce12c4f938bfef2b38aed766dd
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/lib/Driver/ToolChains/AMDGPUOpenMP.cpp
    M clang/lib/Driver/ToolChains/Clang.cpp
    M clang/lib/Driver/ToolChains/Cuda.cpp
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Link the bitcode library late for device LTO

Summary:
This patch adds support for linking the OpenMP device bitcode library
late when doing LTO. This simply passes it in as an additional device
file when doing the final device linking phase with LTO. This has the
advantage that we don't link it multiple times, and the device
references do not get inlined and prevent us from doing needed OpenMP
optimizations when we have visiblity of the whole module.
Fix some failings where the implicit conversion of an Error to an
Expected triggered the deleted copy constructor.

Depends on D116675

Differential revision: https://reviews.llvm.org/D117048

  Commit: f28c3153ee6d93aa07610682519bdf0ea93755b4
      https://github.com/llvm/llvm-project/commit/f28c3153ee6d93aa07610682519bdf0ea93755b4
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/lib/Driver/ToolChains/Clang.cpp
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Add support for embedding bitcode images in wrapper tool

Summary;
This patch adds support for embedding device images in the linker
wrapper tool. This will be used for performing JIT functionality in the
future.

Depends on D117048

Differential Revision: https://reviews.llvm.org/D117049

  Commit: cb7cfaec71850d0a6bc6e00729f0ef6107c51419
      https://github.com/llvm/llvm-project/commit/cb7cfaec71850d0a6bc6e00729f0ef6107c51419
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/lib/Driver/ToolChains/Clang.cpp
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Add extra flag handling to linker wrapper

This patch adds support for a few extra flags in the linker wrapper,
such as debugging flags, verbose output, and passing arguments to ptxas. We also
now forward pass remarks to the LLVM backend so they will show up in the LTO
passes.

Depends on D117049

Differential Revision: https://reviews.llvm.org/D117156

  Commit: ce16ca3c7419fc6cec43d0dae5e9f62df05faf6b
      https://github.com/llvm/llvm-project/commit/ce16ca3c7419fc6cec43d0dae5e9f62df05faf6b
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Add support for linking AMDGPU images

This patch adds support for linking AMDGPU images using the LLD binary.
AMDGPU files are always bitcode images and will always use the LTO
backend. Additionally we now pass the default architecture found with
the `amdgpu-arch` tool to the argument list.

Depends on D117156

Differential Revision: https://reviews.llvm.org/D117246

  Commit: 46d019041cd924d8bdf69c818bbe13f6cd37f466
      https://github.com/llvm/llvm-project/commit/46d019041cd924d8bdf69c818bbe13f6cd37f466
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Improve symbol resolution for OpenMP Offloading LTO

This patch improves the symbol resolution done for LTO with offloading
applications. The symbol resolution done here allows the LTO backend to
internalize more functions. The symbol resoltion done is a simplified
view that does not take into account various options like `--wrap` or
`--dyanimic-list` and always assumes we are creating a shared object.
The actual target may be an executable, but semantically it is used as a
shared object because certain objects need to be visible outside of the
executable when they are read by the OpenMP plugin.

Depends on D117246

Differential Revision: https://reviews.llvm.org/D118155

  Commit: a47b1cf306fb7494d493784c25dafb2dd6084b99
      https://github.com/llvm/llvm-project/commit/a47b1cf306fb7494d493784c25dafb2dd6084b99
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Embed bitcode after optimizations instead of linking

Summary:
Various changes to the linker wrapper, and the bitcode embedding is not
done after the optimizations have run rather than after linking is done.
This saves time when doing JIT.

  Commit: bf499c58af3d3a96661f5c7ef81b264eac575541
      https://github.com/llvm/llvm-project/commit/bf499c58af3d3a96661f5c7ef81b264eac575541
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/lib/Driver/ToolChains/Clang.cpp
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Implement save temps functionality in linker wrapper

Summary:
This patch implements the `-save-temps` flag for the linker wrapper.
This allows the user to inspect the intermeditary outpout that the
linker wrapper creates.

  Commit: 58dc981e082df9db2de1cbd867fa6d73e8eea214
      https://github.com/llvm/llvm-project/commit/58dc981e082df9db2de1cbd867fa6d73e8eea214
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Include the executable name in the temporary files

Summary:
This parses the executable name out of the linker arguments so we can
use it to give more informative temporary file names and so we don't
accidentally use it for device linking.

  Commit: 9375f1563e87e115cc8f9d052760d68460889f4e
      https://github.com/llvm/llvm-project/commit/9375f1563e87e115cc8f9d052760d68460889f4e
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/lib/Driver/ToolChains/Clang.cpp
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Cleanup the Linker Wrapper

Summary:
Various changes and cleanup for the Linker Wrapper tool.

  Commit: eb6ddf288cd0ec58be8ff6c0840c440a5f31dd0b
      https://github.com/llvm/llvm-project/commit/eb6ddf288cd0ec58be8ff6c0840c440a5f31dd0b
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/CMakeLists.txt
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp

  Log Message:
  -----------
  [OpenMP] Replace sysmtem call to `llc` with target machine

Summary:
This patch replaces the system call to the `llc` binary with a library
call to the target machine interface. This should be faster than
relying on an external system call to compile the final wrapper binary.

Differential Revision: https://reviews.llvm.org/D118197

  Commit: 19fac745e3226fbe9db4d3eab106fdaf19d9019d
      https://github.com/llvm/llvm-project/commit/19fac745e3226fbe9db4d3eab106fdaf19d9019d
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M clang/tools/clang-linker-wrapper/CMakeLists.txt
    M clang/tools/clang-linker-wrapper/ClangLinkerWrapper.cpp
    A clang/tools/clang-linker-wrapper/OffloadWrapper.cpp
    A clang/tools/clang-linker-wrapper/OffloadWrapper.h

  Log Message:
  -----------
  [OpenMP] Remove call to 'clang-offload-wrapper' binary

Summary:
This patch removes the system call to the `clang-offload-wrapper` tool
by replicating its functionality in a new file. This improves
performance and makes the future wrapping functionality easier to
change.

Differential Revision: https://reviews.llvm.org/D118198

  Commit: 9d3a47576c1c122891fe88ac13d8a3a131544c34
      https://github.com/llvm/llvm-project/commit/9d3a47576c1c122891fe88ac13d8a3a131544c34
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M llvm/lib/Passes/PassBuilderPipelines.cpp
    M llvm/test/Other/new-pm-lto-defaults.ll

  Log Message:
  -----------
  [PassBuilder] Add OpenMPOpt to default LTO pipeline

The LTO support for OpenMP offloading allows us to run the OpenMPOpt
pass during the LTO pipeline. This patch introduces an early run of the
Module pass and a late run of the CGSCC pass. These are quick no-ops if
there is no OpenMP in the module.

Depends on D118198

Differential Revision: https://reviews.llvm.org/D118611

  Commit: 0ac799b5c977b7fea7f27d0f9086d16eb105f630
      https://github.com/llvm/llvm-project/commit/0ac799b5c977b7fea7f27d0f9086d16eb105f630
  Author: Joseph Huber <jhuber6 at vols.utk.edu>
  Date:   2022-01-31 (Mon, 31 Jan 2022)

  Changed paths:
    M openmp/libomptarget/CMakeLists.txt
    M openmp/libomptarget/plugins/amdgpu/CMakeLists.txt
    M openmp/libomptarget/plugins/cuda/CMakeLists.txt
    M openmp/libomptarget/test/lit.cfg
    A openmp/libomptarget/test/offloading/static_linking.c
    M openmp/libomptarget/test/unified_shared_memory/api.c

  Log Message:
  -----------
  [Libomptarget] Run GPU offloading tests using the new drvier

This patch adds a new target to the tests to run using the new driver as
the method for generating offloading code.

Depends on D116541

Differential Revision: https://reviews.llvm.org/D118637

Compare: https://github.com/llvm/llvm-project/compare/5b2c3d7bbcda...0ac799b5c977