[llvm] [X86] Keep scalar bf16/f16 selects in vector registers (PR #224218)
via llvm-commits
llvm-commits at lists.llvm.org
Tue Sep 29 01:08:22 PDT 2026
================
@@ -48981,6 +48981,81 @@ static SDValue combineSelect(SDNode *N, SelectionDAG &DAG,
bool CondConstantVector = ISD::isBuildVectorOfConstantSDNodes(Cond.getNode());
unsigned EltBitWidth = VT.getScalarSizeInBits();
+ // Soft bf16/f16 scalar selects do a VSELECT in vector registers instead
+ // of a scalar CMOV, to avoid a GPR round-trip. Skip constant operands
+ // (cheaper as immediates) and compare-driven conds (CMOV already reuses
+ // the flags).
+ if (N->getOpcode() == ISD::SELECT && !CondVT.isVector() &&
+ Subtarget.hasSSE2() && !isIntOrFPConstant(LHS) &&
+ !isIntOrFPConstant(RHS)) {
+ // Only worth it if both operands already live in a vector register
+ auto IsBitcastFromGPR = [](SDValue Op) {
+ return Op.getOpcode() == ISD::BITCAST &&
+ Op.getOperand(0).getValueType().isScalarInteger();
+ };
+ SDValue F16LHS, F16RHS;
+ if (!VT.isVector() && isSoftF16(VT, Subtarget)) {
+ if (!IsBitcastFromGPR(LHS) || !IsBitcastFromGPR(RHS)) {
+ F16LHS = DAG.getBitcast(MVT::f16, LHS);
+ F16RHS = DAG.getBitcast(MVT::f16, RHS);
+ }
+ } else if (VT == MVT::i16 && LHS.getOpcode() == ISD::BITCAST &&
+ RHS.getOpcode() == ISD::BITCAST) {
+ MVT SVT = LHS.getOperand(0).getSimpleValueType();
+ if ((SVT == MVT::f16 || SVT == MVT::bf16) &&
+ SVT == RHS.getOperand(0).getSimpleValueType()) {
+ F16LHS = DAG.getBitcast(MVT::f16, LHS.getOperand(0));
+ F16RHS = DAG.getBitcast(MVT::f16, RHS.getOperand(0));
+ }
+ }
+
+ SDValue CondRoot = Cond;
+ while (CondRoot.getOpcode() == ISD::AND ||
----------------
tfzee wrote:
So technically I can't. However it is only used for checking if it comes from a compare whose flags cmove can reuse. So it is more like a heuristic to check profitability and not used in the actual transform. The actual rewrite still uses Cond and shouldn't be able to generate anything unsafe from this peeking itself.
https://github.com/llvm/llvm-project/pull/224218
More information about the llvm-commits
mailing list