[all-commits] [llvm/llvm-project] cf50f4: [mlir] Link mlir_runner_utils statically into cuda...

[Christian Sigg via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: cf50f4f764566a78da8c0551f853118fe604d8d7
      https://github.com/llvm/llvm-project/commit/cf50f4f764566a78da8c0551f853118fe604d8d7
  Author: Christian Sigg <csigg at google.com>
  Date:   2021-01-20 (Wed, 20 Jan 2021)

  Changed paths:
    M mlir/include/mlir/ExecutionEngine/CRunnerUtils.h
    M mlir/lib/ExecutionEngine/CMakeLists.txt
    M mlir/test/mlir-cuda-runner/all-reduce-and.mlir
    M mlir/test/mlir-cuda-runner/all-reduce-max.mlir
    M mlir/test/mlir-cuda-runner/all-reduce-min.mlir
    M mlir/test/mlir-cuda-runner/all-reduce-op.mlir
    M mlir/test/mlir-cuda-runner/all-reduce-or.mlir
    M mlir/test/mlir-cuda-runner/all-reduce-region.mlir
    M mlir/test/mlir-cuda-runner/all-reduce-xor.mlir
    M mlir/test/mlir-cuda-runner/gpu-to-cubin.mlir
    M mlir/test/mlir-cuda-runner/multiple-all-reduce.mlir
    M mlir/test/mlir-cuda-runner/shuffle.mlir
    M mlir/test/mlir-cuda-runner/two-modules.mlir
    M mlir/test/mlir-rocm-runner/gpu-to-hsaco.mlir
    M mlir/test/mlir-rocm-runner/two-modules.mlir
    M mlir/test/mlir-rocm-runner/vecadd.mlir
    M mlir/test/mlir-rocm-runner/vector-transferops.mlir
    M mlir/tools/mlir-cuda-runner/CMakeLists.txt
    M mlir/tools/mlir-rocm-runner/CMakeLists.txt

  Log Message:
  -----------
  [mlir] Link mlir_runner_utils statically into cuda/rocm-runtime-wrappers.

The runtime-wrappers depend on LLVMSupport, pulling in static initialization code (e.g. command line arguments). Dynamically loading multiple such libraries results in ODR violoations.

So far this has not been an issue, but in D94421, I would like to load both the async-runtime and the cuda-runtime-wrappers as part of a cuda-runner integration test. When doing this, code that asserts that an option category is only registered once fails (note that I've only experienced this in Google's bazel where the async-runtime depends on LLVMSupport, but a similar issue would happen in cmake if more than one runtime-wrapper starts to depend on LLVMSupport).

The underlying issue is that we have a mix of static and dynamic linking. If all dependencies were loaded as shared objects (i.e. if LLVMSupport was linked dynamically to the runtime wrappers), each dependency would only get loaded once. However, linking dependencies dynamically would require special attention to paths (one could dynamically load the dependencies first given explicit paths). The simpler approach seems to be to link all dependencies statically into a single shared object.

This change basically applies the same logic that we have in the c_runner_utils: we have a shared object target that can be loaded dynamically, and we have a static library target that can be linked to other runtime-wrapper shared object targets.

Reviewed By: herhut

Differential Revision: https://reviews.llvm.org/D94399