[clang] [llvm] Tentative fix for not removing newly internal functions (PR #106146)

[Greg Roth via cfe-commits]

https://github.com/pow2clk created https://github.com/llvm/llvm-project/pull/106146

Functions are not removed even when made internal by DXILFinalizeLinkage The removal code is called from alwaysinliner and globalopt, which are invoked too early to remove functions made internal by this pass.

This adds a check similar to that in alwaysinliner that removes trivially dead functions after being marked internal. It refactors that code a bit to make it simpler including reversing what is stored in the work queue.

Not sure how to test this. To test all the interactions between alwaysinliner, DXILfinalizelinkage and any other optimization passes, it kinda needs to be end-to-end.

Fixes #106139

>From 20bf1f85d8aa2786ecf874203b9759aa42be9627 Mon Sep 17 00:00:00 2001
From: Greg Roth <grroth at microsoft.com>
Date: Sun, 25 Aug 2024 12:00:03 -0600
Subject: [PATCH] Tentative fix for not removing newly internal functions

Functions are not removed even when made internal by DXILFinalizeLinkage
The removal code is called from alwaysinliner and globalopt, which are
invoked too early to remove functions made internal by this pass.

This adds a check similar to that in alwaysinliner that removes
trivially dead functions after being marked internal. It refactors
that code a bit to make it simpler including reversing what is
stored in the work queue.

Not sure how to test this. To test all the interactions between
alwaysinliner, DXILfinalizelinkage and any other optimization passes,
it kinda needs to be end-to-end.

Fixes #106139
---
 .../CodeGenHLSL/remove-internal-unused.hlsl     | 17 +++++++++++++++++
 llvm/lib/Target/DirectX/DXILFinalizeLinkage.cpp | 15 ++++++++-------
 2 files changed, 25 insertions(+), 7 deletions(-)
 create mode 100644 clang/test/CodeGenHLSL/remove-internal-unused.hlsl

diff --git a/clang/test/CodeGenHLSL/remove-internal-unused.hlsl b/clang/test/CodeGenHLSL/remove-internal-unused.hlsl
new file mode 100644
index 00000000000000..6ec08060e24dd2
--- /dev/null
+++ b/clang/test/CodeGenHLSL/remove-internal-unused.hlsl
@@ -0,0 +1,17 @@
+// RUN: %clang_dxc -T cs_6_0 %s | Filecheck %s
+
+// Verify that internal linkage unused functions are removed
+
+RWBuffer<unsigned> buf;
+
+// CHECK-NOT: define{{.*}}donothing
+void donothing() {
+     buf[1] = 1; // never called, does nothing!
+}
+
+
+[numthreads(1,1,1)]
+[shader("compute")]
+void main() {
+     buf[0] = 0;// I'm doing something!!!     
+}
\ No newline at end of file
diff --git a/llvm/lib/Target/DirectX/DXILFinalizeLinkage.cpp b/llvm/lib/Target/DirectX/DXILFinalizeLinkage.cpp
index c02eb768cdf49b..6508258cdd197a 100644
--- a/llvm/lib/Target/DirectX/DXILFinalizeLinkage.cpp
+++ b/llvm/lib/Target/DirectX/DXILFinalizeLinkage.cpp
@@ -18,19 +18,20 @@
 using namespace llvm;
 
 static bool finalizeLinkage(Module &M) {
-  SmallPtrSet<Function *, 8> EntriesAndExports;
+  SmallPtrSet<Function *, 8> Funcs;
 
   // Find all entry points and export functions
   for (Function &EF : M.functions()) {
-    if (!EF.hasFnAttribute("hlsl.shader") && !EF.hasFnAttribute("hlsl.export"))
+    if (EF.hasFnAttribute("hlsl.shader") || EF.hasFnAttribute("hlsl.export"))
       continue;
-    EntriesAndExports.insert(&EF);
+    Funcs.insert(&EF);
   }
 
-  for (Function &F : M.functions()) {
-    if (F.getLinkage() == GlobalValue::ExternalLinkage &&
-        !EntriesAndExports.contains(&F)) {
-      F.setLinkage(GlobalValue::InternalLinkage);
+  for (Function *F : Funcs) {
+    if (F->getLinkage() == GlobalValue::ExternalLinkage) {
+      F->setLinkage(GlobalValue::InternalLinkage);
+      if (F->isDefTriviallyDead())
+	M.getFunctionList().erase(F);
     }
   }