[llvm-branch-commits] [llvm] [DAGCombiner] Drop `StartingIndex` in `calculateByteProvider` (PR #244905)

via llvm-branch-commits llvm-branch-commits at lists.llvm.org
Sat Nov 21 08:16:53 PST 2026


llvmorg-github-actions[bot] wrote:


<!--LLVM PR SUMMARY COMMENT-->

@llvm/pr-subscribers-llvm-selectiondag

Author: lijinpei-amd

<details>
<summary>Changes</summary>

The function comment of `calculateByteProvider` claims:

"StartingIndex is the byte position we are trying to provide for the
LoadCombine."

This is wrong:
- When not SHL, `StartingIndex` tracks the `StartingIndex` of the
  current call of `calculateByteProvider`.
- When SHL, it tracks `Index`.
Since `Index` can be changed by `SHL` and `BSWAP`, this makes
`StartingIndex` track some intermediate `Index` in the chain of
recursion, which is different from the byte position.

Also, the comment claims:
"If these do not match, then we can not combine the vector loads."
This is also wrong, since MatchLoadCombine already checks that the bytes
are adjacent in memory and that the combined load is allowed.

Drop the check of `StartingIndex` in `EXTRACT_VECTOR_ELT`, and only
check that `VectorIndex` is within the vector type's known minimum
element count. This recovers some missed optimizations.

Assisted-by: Claude Opus 5.5 <noreply@<!-- -->anthropic.com>



---
Full diff: https://github.com/llvm/llvm-project/pull/244905.diff


3 Files Affected:

- (modified) llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp (+18-32) 
- (modified) llvm/test/CodeGen/AArch64/load-combine.ll (+78) 
- (modified) llvm/test/CodeGen/AMDGPU/load-local-i16.ll (+7-7) 


``````````diff
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index 3ebbbe4846a4b..073319a330719 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -9776,15 +9776,12 @@ SDValue DAGCombiner::MatchRotate(SDValue LHS, SDValue RHS, const SDLoc &DL,
 /// contribute to the byte ultimately come from the ExtractVectorElements of the
 /// Load, we don't care if the Load has uses other than ExtractVectorElements,
 /// because those operations are independent from the pattern to be combined.
-/// For vector loads, we simply care that the ByteProviders are adjacent
-/// positions of the same vector, and their index matches the byte that is being
-/// provided. This is captured by the \p VectorIndex algorithm. \p VectorIndex
-/// is the index used in an ExtractVectorElement, and \p StartingIndex is the
-/// byte position we are trying to provide for the LoadCombine. If these do
-/// not match, then we can not combine the vector loads. \p Index uses the
-/// byte position we are trying to provide for and is matched against the
-/// shl and load size. The \p Index algorithm ensures the requested byte is
-/// provided for by the pattern, and the pattern does not over provide bytes.
+/// For vector loads, \p VectorIndex is the index used in an
+/// ExtractVectorElement, which MatchLoadCombine uses to check that the
+/// provided bytes are adjacent in memory. \p Index uses the byte position we
+/// are trying to provide for and is matched against the shl and load size. The
+/// \p Index algorithm ensures the requested byte is provided for by the
+/// pattern, and the pattern does not over provide bytes.
 ///
 ///
 /// The supported LoadCombine pattern for vector loads is as follows
@@ -9808,7 +9805,7 @@ using SDByteProvider = ByteProvider<SDNode *>;
 static std::optional<SDByteProvider>
 calculateByteProvider(SDValue Op, unsigned Index, unsigned Depth,
                       std::optional<uint64_t> VectorIndex,
-                      unsigned StartingIndex = 0, uint8_t *ByteMask = nullptr) {
+                      uint8_t *ByteMask = nullptr) {
 
   // Typical i64 by i8 pattern requires recursion up to 8 calls depth
   if (Depth == 10)
@@ -9835,11 +9832,11 @@ calculateByteProvider(SDValue Op, unsigned Index, unsigned Depth,
   switch (Op.getOpcode()) {
   case ISD::OR: {
     auto LHS = calculateByteProvider(Op->getOperand(0), Index, Depth + 1,
-                                     VectorIndex, StartingIndex, ByteMask);
+                                     VectorIndex, ByteMask);
     if (!LHS)
       return std::nullopt;
     auto RHS = calculateByteProvider(Op->getOperand(1), Index, Depth + 1,
-                                     VectorIndex, StartingIndex, ByteMask);
+                                     VectorIndex, ByteMask);
     if (!RHS)
       return std::nullopt;
 
@@ -9866,7 +9863,7 @@ calculateByteProvider(SDValue Op, unsigned Index, unsigned Depth,
     return Index < ByteShift
                ? SDByteProvider::getConstantZero()
                : calculateByteProvider(Op->getOperand(0), Index - ByteShift,
-                                       Depth + 1, VectorIndex, Index, ByteMask);
+                                       Depth + 1, VectorIndex, ByteMask);
   }
   case ISD::ANY_EXTEND:
   case ISD::SIGN_EXTEND:
@@ -9883,12 +9880,11 @@ calculateByteProvider(SDValue Op, unsigned Index, unsigned Depth,
                        SDByteProvider::getConstantZero())
                  : std::nullopt;
     return calculateByteProvider(NarrowOp, Index, Depth + 1, VectorIndex,
-                                 StartingIndex, ByteMask);
+                                 ByteMask);
   }
   case ISD::BSWAP:
     return calculateByteProvider(Op->getOperand(0), ByteWidth - Index - 1,
-                                 Depth + 1, VectorIndex, StartingIndex,
-                                 ByteMask);
+                                 Depth + 1, VectorIndex, ByteMask);
   case ISD::AND: {
     // Constants are canonicalized to the RHS of AND, so only operand 1 needs
     // to be checked.
@@ -9903,7 +9899,7 @@ calculateByteProvider(SDValue Op, unsigned Index, unsigned Depth,
       return SDByteProvider::getConstantZero();
 
     auto Result = calculateByteProvider(Op->getOperand(0), Index, Depth + 1,
-                                        VectorIndex, StartingIndex, ByteMask);
+                                        VectorIndex, ByteMask);
     if (!Result)
       return std::nullopt;
 
@@ -9923,6 +9919,8 @@ calculateByteProvider(SDValue Op, unsigned Index, unsigned Depth,
     VectorIndex = OffsetOp->getZExtValue();
 
     SDValue NarrowOp = Op->getOperand(0);
+    if (*VectorIndex >= NarrowOp.getValueType().getVectorMinNumElements())
+      return std::nullopt;
     unsigned NarrowBitWidth = NarrowOp.getScalarValueSizeInBits();
     if (NarrowBitWidth % 8 != 0)
       return std::nullopt;
@@ -9932,19 +9930,8 @@ calculateByteProvider(SDValue Op, unsigned Index, unsigned Depth,
     if (Index >= NarrowByteWidth)
       return std::nullopt;
 
-    // Check to see if the position of the element in the vector corresponds
-    // with the byte we are trying to provide for. In the case of a vector of
-    // i8, this simply means the VectorIndex == StartingIndex. For non i8 cases,
-    // the element will provide a range of bytes. For example, if we have a
-    // vector of i16s, each element provides two bytes (V[1] provides byte 2 and
-    // 3).
-    if (*VectorIndex * NarrowByteWidth > StartingIndex)
-      return std::nullopt;
-    if ((*VectorIndex + 1) * NarrowByteWidth <= StartingIndex)
-      return std::nullopt;
-
     return calculateByteProvider(Op->getOperand(0), Index, Depth + 1,
-                                 VectorIndex, StartingIndex, ByteMask);
+                                 VectorIndex, ByteMask);
   }
   case ISD::LOAD: {
     auto L = cast<LoadSDNode>(Op.getNode());
@@ -10306,9 +10293,8 @@ SDValue DAGCombiner::MatchLoadCombine(SDNode *N) {
   SmallVector<uint8_t, 8> ByteMasks(ByteWidth, 0xFF);
   unsigned ZeroExtendedBytes = 0;
   for (int i = ByteWidth - 1; i >= 0; --i) {
-    auto P =
-        calculateByteProvider(SDValue(N, 0), i, 0, /*VectorIndex*/ std::nullopt,
-                              /*StartingIndex*/ i, &ByteMasks[i]);
+    auto P = calculateByteProvider(SDValue(N, 0), i, 0,
+                                   /*VectorIndex*/ std::nullopt, &ByteMasks[i]);
     if (!P)
       return SDValue();
 
diff --git a/llvm/test/CodeGen/AArch64/load-combine.ll b/llvm/test/CodeGen/AArch64/load-combine.ll
index b9af60f311562..6c9f2cb5d5353 100644
--- a/llvm/test/CodeGen/AArch64/load-combine.ll
+++ b/llvm/test/CodeGen/AArch64/load-combine.ll
@@ -771,6 +771,56 @@ define void @short_vector_elt1_to_i16_bswap_strict_align(ptr %in, ptr %out, ptr
   ret void
 }
 
+; x1 = x0
+define void @short_vector_nested_shl_to_i32(ptr %in, ptr %out, ptr %p) {
+; CHECK-LABEL: short_vector_nested_shl_to_i32:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldr w8, [x0]
+; CHECK-NEXT:    str w8, [x1]
+; CHECK-NEXT:    ret
+  %ld = load <4 x i8>, ptr %in, align 4
+  %lo = load i16, ptr %in, align 4
+
+  %e3 = extractelement <4 x i8> %ld, i32 2
+  %e4 = extractelement <4 x i8> %ld, i32 3
+
+  %z2 = zext i8 %e3 to i16
+  %z3 = zext i8 %e4 to i16
+
+  %s3 = shl nuw i16 %z3, 8
+  %hi = or i16 %s3, %z2
+
+  %lo.z = zext i16 %lo to i32
+  %hi.z = zext i16 %hi to i32
+  %hi.s = shl nuw i32 %hi.z, 16
+  %i = or i32 %hi.s, %lo.z
+
+  store i32 %i, ptr %out
+  ret void
+}
+
+; x1 = x0[2:3]
+define void @short_vector_high_half_to_i16(ptr %in, ptr %out, ptr %p) {
+; CHECK-LABEL: short_vector_high_half_to_i16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ldrh w8, [x0, #2]
+; CHECK-NEXT:    strh w8, [x1]
+; CHECK-NEXT:    ret
+  %ld = load <4 x i8>, ptr %in, align 4
+
+  %e3 = extractelement <4 x i8> %ld, i32 2
+  %e4 = extractelement <4 x i8> %ld, i32 3
+
+  %z2 = zext i8 %e3 to i16
+  %z3 = zext i8 %e4 to i16
+
+  %s3 = shl nuw i16 %z3, 8
+  %i = or i16 %s3, %z2
+
+  store i16 %i, ptr %out
+  ret void
+}
+
 ; x1 = x0
 define void @scalable_vector_to_i32(ptr %in, ptr %out, ptr %p) #0 {
 ; CHECK-LABEL: scalable_vector_to_i32:
@@ -948,5 +998,33 @@ define void @scalable_vector_to_i64(ptr %in, ptr %out, ptr %p) #0 {
   ret void
 }
 
+; x1 = x0[4:5], past the known minimum size of the vector load
+define void @scalable_vector_elt4_to_i16(ptr %in, ptr %out, ptr %p) #0 {
+; CHECK-LABEL: scalable_vector_elt4_to_i16:
+; CHECK:       // %bb.0:
+; CHECK-NEXT:    ptrue p0.s
+; CHECK-NEXT:    ld1b { z0.s }, p0/z, [x0]
+; CHECK-NEXT:    mov z1.s, z0.s[4]
+; CHECK-NEXT:    mov z0.s, z0.s[5]
+; CHECK-NEXT:    fmov w8, s1
+; CHECK-NEXT:    fmov w9, s0
+; CHECK-NEXT:    orr w8, w8, w9, lsl #8
+; CHECK-NEXT:    strh w8, [x1]
+; CHECK-NEXT:    ret
+  %ld = load <vscale x 4 x i8>, ptr %in, align 4
+
+  %e5 = extractelement <vscale x 4 x i8> %ld, i32 4
+  %e6 = extractelement <vscale x 4 x i8> %ld, i32 5
+
+  %z4 = zext i8 %e5 to i16
+  %z5 = zext i8 %e6 to i16
+
+  %s5 = shl nuw i16 %z5, 8
+  %i = or i16 %s5, %z4
+
+  store i16 %i, ptr %out
+  ret void
+}
+
 attributes #0 = { "target-features"="+sve" }
 attributes #1 = { "target-features"="+strict-align" }
diff --git a/llvm/test/CodeGen/AMDGPU/load-local-i16.ll b/llvm/test/CodeGen/AMDGPU/load-local-i16.ll
index 9eee7b73344a6..0b4e4fec3cc57 100644
--- a/llvm/test/CodeGen/AMDGPU/load-local-i16.ll
+++ b/llvm/test/CodeGen/AMDGPU/load-local-i16.ll
@@ -152,18 +152,18 @@ define amdgpu_kernel void @local_load_v3i16(ptr addrspace(3) %out, ptr addrspace
 ; EG-LABEL: local_load_v3i16:
 ; EG:       ; %bb.0: ; %entry
 ; EG-NEXT:    ALU 11, @2, KC0[CB0:0-32], KC1[]
-; EG-NEXT:     ADD_INT * T0.W, KC0[2].Z, literal.x,
-; EG-NEXT:    4(5.605194e-45), 0(0.000000e+00)
-; EG-NEXT:     LDS_USHORT_READ_RET * OQAP, T0.W
-; EG-NEXT:     MOV T0.X, OQAP,
-; EG-NEXT:     ADD_INT * T0.W, KC0[2].Y, literal.x,
-; EG-NEXT:    4(5.605194e-45), 0(0.000000e+00)
-; EG-NEXT:     LDS_SHORT_WRITE * T0.W, T0.X,
 ; EG-NEXT:     MOV * T0.W, KC0[2].Z,
 ; EG-NEXT:     LDS_READ_RET * OQAP, T0.W
 ; EG-NEXT:     MOV T0.X, OQAP,
+; EG-NEXT:     ADD_INT * T0.W, KC0[2].Z, literal.x,
+; EG-NEXT:    4(5.605194e-45), 0(0.000000e+00)
+; EG-NEXT:     LDS_USHORT_READ_RET * OQAP, T0.W
+; EG-NEXT:     MOV T0.Y, OQAP,
 ; EG-NEXT:     MOV * T0.W, KC0[2].Y,
 ; EG-NEXT:     LDS_WRITE * T0.W, T0.X,
+; EG-NEXT:     ADD_INT * T0.W, KC0[2].Y, literal.x,
+; EG-NEXT:    4(5.605194e-45), 0(0.000000e+00)
+; EG-NEXT:     LDS_SHORT_WRITE * T0.W, T0.Y,
 ; EG-NEXT:    RETURN
 entry:
   %ld = load <3 x i16>, ptr addrspace(3) %in

``````````

</details>


https://github.com/llvm/llvm-project/pull/244905


More information about the llvm-branch-commits mailing list