[clang] Partial revert "[HIP] Fix -mllvm option for device lld linker" (PR #80202)

[Yaxun Liu via cfe-commits]

https://github.com/yxsamliu created https://github.com/llvm/llvm-project/pull/80202

This partially reverts commit aa964f157f9b50fab3895afbfda6e0915cf6bb4a because it caused perf regressions in rccl due to drop of -mllvm -amgpu-kernarg-preload-count=16 from the linker step. Pontentially it could cause similar regressions for other HIP apps using -mllvm options with -fgpu-rdc.

Fixes: SWDEV-443345

>From 29caa4f3bdfef8f53f63b6713123fe55114999bd Mon Sep 17 00:00:00 2001
From: "Yaxun (Sam) Liu" <yaxun.liu at amd.com>
Date: Wed, 31 Jan 2024 16:06:23 -0500
Subject: [PATCH] Partial revert "[HIP] Fix -mllvm option for device lld
 linker"

This partially reverts commit aa964f157f9b50fab3895afbfda6e0915cf6bb4a
because it caused perf regressions in rccl due to drop of
-mllvm -amgpu-kernarg-preload-count=16 from the linker step.
Pontentially it could cause similar regressions for other
HIP apps using -mllvm options with -fgpu-rdc.

Fixes: SWDEV-443345
---
 clang/lib/Driver/ToolChains/HIPAMD.cpp    |  5 +++++
 clang/test/Driver/hip-toolchain-mllvm.hip | 12 ++++++------
 2 files changed, 11 insertions(+), 6 deletions(-)

diff --git a/clang/lib/Driver/ToolChains/HIPAMD.cpp b/clang/lib/Driver/ToolChains/HIPAMD.cpp
index ccb36a6c846c8..f9854a4840092 100644
--- a/clang/lib/Driver/ToolChains/HIPAMD.cpp
+++ b/clang/lib/Driver/ToolChains/HIPAMD.cpp
@@ -142,6 +142,11 @@ void AMDGCN::Linker::constructLldCommand(Compilation &C, const JobAction &JA,
   if (IsThinLTO)
     LldArgs.push_back(Args.MakeArgString("-plugin-opt=-force-import-all"));
 
+  for (const Arg *A : Args.filtered(options::OPT_mllvm)) {
+    LldArgs.push_back(
+        Args.MakeArgString(Twine("-plugin-opt=") + A->getValue(0)));
+  }
+
   if (C.getDriver().isSaveTempsEnabled())
     LldArgs.push_back("-save-temps");
 
diff --git a/clang/test/Driver/hip-toolchain-mllvm.hip b/clang/test/Driver/hip-toolchain-mllvm.hip
index 194cecc40db58..110d1c9b7fd33 100644
--- a/clang/test/Driver/hip-toolchain-mllvm.hip
+++ b/clang/test/Driver/hip-toolchain-mllvm.hip
@@ -1,9 +1,9 @@
 // REQUIRES: x86-registered-target
 // REQUIRES: amdgpu-registered-target
 
-// Check only -Xoffload-linker -mllvm=* options are passed
-// to device lld linker.
-// -mllvm options are passed to clang only.
+// Check -Xoffload-linker -mllvm=* options are passed
+// to device lld linker only.
+// -mllvm options are passed to clang and device lld linker.
 
 // RUN: %clang -### --target=x86_64-linux-gnu \
 // RUN:   --offload-arch=gfx803 --offload-arch=gfx900 \
@@ -33,15 +33,15 @@
 // CHECK-SAME: "-aux-triple" "x86_64-unknown-linux-gnu"
 // CHECK-SAME: {{.*}} "-target-cpu" "gfx803"
 // CHECK-SAME: {{.*}} "-mllvm" "-unroll-count=10" {{.*}}
-// CHECK: [[LLD:".*lld.*"]] {{.*}}"-m" "elf64_amdgpu"{{.*}} "-plugin-opt=-inline-threshold=100"
+// CHECK: [[LLD:".*lld.*"]] {{.*}}"-m" "elf64_amdgpu"{{.*}} "-plugin-opt=-unroll-count=10"{{.*}} "-plugin-opt=-inline-threshold=100"
 
 // CHECK: [[CLANG]] "-cc1" "-triple" "amdgcn-amd-amdhsa"
 // CHECK-SAME: "-aux-triple" "x86_64-unknown-linux-gnu"
 // CHECK-SAME: {{.*}} "-target-cpu" "gfx900"
 // CHECK-SAME: {{.*}} "-mllvm" "-unroll-count=10" {{.*}}
-// CHECK: [[LLD:".*lld.*"]] {{.*}} "-plugin-opt=-inline-threshold=100"
+// CHECK: [[LLD:".*lld.*"]] {{.*}} "-plugin-opt=-unroll-count=10"{{.*}} "-plugin-opt=-inline-threshold=100"
 
 // NEG-NOT: {{".*opt"}}
 // NEG-NOT: {{".*llc"}}
-// NEG-NOT: "-plugin-opt=-unroll-count=10"
+// NEG-NOT: "-m" "elf_x86_64"{{.*}} "-plugin-opt=-unroll-count=10"
 // NEG-NOT: "-m" "elf_x86_64"{{.*}} "-plugin-opt=-inline-threshold=100"