[llvm] [SelectionDAG] Fix unsafe cases for loop.dependence.{war/raw}.mask (PR #168565)
Benjamin Maxwell via llvm-commits
llvm-commits at lists.llvm.org
Tue Nov 25 05:49:43 PST 2025
================
@@ -1692,23 +1699,32 @@ void DAGTypeLegalizer::SplitVecRes_BITCAST(SDNode *N, SDValue &Lo,
Hi = DAG.getNode(ISD::BITCAST, dl, HiVT, Hi);
}
+/// Split a loop dependence mask.
+/// This is done by creating a high and low mask, each of half the vector
+/// length. A select of the high mask and a predicate of all zeroes is needed to
+/// guarantee that the high mask is safe. A case where simply producing a high
+/// mask without the select is unsafe, is when the difference between the two
+/// pointers is less than half the vector length, e.g. ptrA = 0 and ptrB 3 when
+/// the vector length is 32.
+/// The full 32xi1 mask should be three active lanes and the rest inactive,
+/// however when half the vector length is added to ptrA to produce the high
+/// mask, the difference between ptrA and ptrB is now -13, which will result
+/// in a mask with all lanes active. The select will guard against this case
+/// by choosing a mask of all inactive lanes when ptrA + VL/2 >= ptrB.
void DAGTypeLegalizer::SplitVecRes_LOOP_DEPENDENCE_MASK(SDNode *N, SDValue &Lo,
SDValue &Hi) {
SDLoc DL(N);
EVT LoVT, HiVT;
std::tie(LoVT, HiVT) = DAG.GetSplitDestVTs(N->getValueType(0));
SDValue PtrA = N->getOperand(0);
SDValue PtrB = N->getOperand(1);
- Lo = DAG.getNode(N->getOpcode(), DL, LoVT, PtrA, PtrB, N->getOperand(2));
-
- unsigned EltSize = N->getConstantOperandVal(2);
- unsigned Offset = EltSize * HiVT.getVectorMinNumElements();
- SDValue Addend = HiVT.isScalableVT()
- ? DAG.getVScale(DL, MVT::i64, APInt(64, Offset))
- : DAG.getConstant(Offset, DL, MVT::i64);
- PtrA = DAG.getNode(ISD::ADD, DL, MVT::i64, PtrA, Addend);
- Hi = DAG.getNode(N->getOpcode(), DL, HiVT, PtrA, PtrB, N->getOperand(2));
+ Lo = DAG.getNode(N->getOpcode(), DL, LoVT, PtrA, PtrB, N->getOperand(2),
+ N->getOperand(3));
----------------
MacDue wrote:
nit: Can you label the unnamed parameters
```suggestion
Lo = DAG.getNode(N->getOpcode(), DL, LoVT, PtrA, PtrB, /*ElementSizeInBytes=*/N->getOperand(2),
/*LaneOffset=*/N->getOperand(3));
```
https://github.com/llvm/llvm-project/pull/168565
More information about the llvm-commits
mailing list