[llvm] [DSE] Don't use initializes on byval argument (PR #126259)

[Nikita Popov via llvm-commits]

https://github.com/nikic created https://github.com/llvm/llvm-project/pull/126259

There are two ways we can fix this problem, depending on how the semantics of byval and initializes should interact:

* Don't infer initializes on byval arguments. initializes on byval refers to the original caller memory (or having both attributes is made a verifier error).
* Infer initializes on byval, but don't use it in DSE. initializes on byval refers to the callee copy. This matches the semantics of readonly on byval. This is slightly more powerful, for example, we could do a backend optimization where byval + initializes will allocate the full size of byval on the stack but not copy over the parts covered by initializes.

I went with the second variant here, skipping byval + initializes in DSE (FunctionAttrs already doesn't propagate initializes past byval). I'm open to going in the other direction though.

Fixes https://github.com/llvm/llvm-project/issues/126181.

>From 2678d359bc8ff9db63c27667b9c2a14e5c3685d9 Mon Sep 17 00:00:00 2001
From: Nikita Popov <npopov at redhat.com>
Date: Fri, 7 Feb 2025 17:26:06 +0100
Subject: [PATCH] [DSE] Don't use initializes on byval argument

There are two ways we can fix this problem, depending on how the
semantics of byval and initializes should interact:

* Don't infer initializes on byval arguments. initializes on byval
  refers to the original caller memory (or having both attributes
  is made a verifier error).
* Infer initializes on byval, but don't use it in DSE. initializes
  on byval refers to the callee copy. This matches the semantics of
  readonly on byval. This is slightly more powerful, for example,
  we could do a backend optimization where byval + initializes will
  allocate the full size of byval on the stack but not copy over
  the parts covered by initializes.

I went with the second variant here, skipping byval + initializes
in DSE (FunctionAttrs already doesn't propagate initializes past
byval). I'm open to going in the other direction though.

Fixes https://github.com/llvm/llvm-project/issues/126181.
---
 llvm/docs/LangRef.rst                              |  4 ++++
 .../lib/Transforms/Scalar/DeadStoreElimination.cpp |  4 +++-
 .../DeadStoreElimination/inter-procedural.ll       | 14 ++++++++++++++
 3 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 5f0ae5ce8614c63..39ea405ea3d6e37 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -1707,6 +1707,10 @@ Currently, only the following parameter attributes are defined:
     and negative values are allowed in case the argument points partway into
     an allocation. An empty list is not allowed.
 
+    On a ``byval`` argument, ``initializes`` refers to the given parts of the
+    callee copy being overwritten. A ``byval`` callee can never initialize the
+    original caller memory passed to the ``byval`` argument.
+
 ``dead_on_unwind``
     At a high level, this attribute indicates that the pointer argument is dead
     if the call unwinds, in the sense that the caller will not depend on the
diff --git a/llvm/lib/Transforms/Scalar/DeadStoreElimination.cpp b/llvm/lib/Transforms/Scalar/DeadStoreElimination.cpp
index 05b4f176bfc31cc..38454053b039e1f 100644
--- a/llvm/lib/Transforms/Scalar/DeadStoreElimination.cpp
+++ b/llvm/lib/Transforms/Scalar/DeadStoreElimination.cpp
@@ -2283,7 +2283,9 @@ DSEState::getInitializesArgMemLoc(const Instruction *I) {
   for (unsigned Idx = 0, Count = CB->arg_size(); Idx < Count; ++Idx) {
     ConstantRangeList Inits;
     Attribute InitializesAttr = CB->getParamAttr(Idx, Attribute::Initializes);
-    if (InitializesAttr.isValid())
+    // initializes on byval arguments refers to the callee copy, not the
+    // original memory the caller passed in.
+    if (InitializesAttr.isValid() && !CB->isByValArgument(Idx))
       Inits = InitializesAttr.getValueAsConstantRangeList();
 
     Value *CurArg = CB->getArgOperand(Idx);
diff --git a/llvm/test/Transforms/DeadStoreElimination/inter-procedural.ll b/llvm/test/Transforms/DeadStoreElimination/inter-procedural.ll
index e590c5bf4004afd..5f8ab56c22754d4 100644
--- a/llvm/test/Transforms/DeadStoreElimination/inter-procedural.ll
+++ b/llvm/test/Transforms/DeadStoreElimination/inter-procedural.ll
@@ -338,3 +338,17 @@ define i16 @global_var_alias() {
   ret i16 %l
 }
 
+declare void @byval_fn(ptr byval(i32) initializes((0, 4)) %am)
+
+define void @test_byval() {
+; CHECK-LABEL: @test_byval(
+; CHECK-NEXT:    [[A:%.*]] = alloca i32, align 4
+; CHECK-NEXT:    store i32 0, ptr [[A]], align 4
+; CHECK-NEXT:    call void @byval_fn(ptr [[A]])
+; CHECK-NEXT:    ret void
+;
+  %a = alloca i32
+  store i32 0, ptr %a
+  call void @byval_fn(ptr %a)
+  ret void
+}