[llvm] [GlobalISel][AArch64][AMDGPU] Don't narrow saturating fp-to-int conversions (PR #215567)
via llvm-commits
llvm-commits at lists.llvm.org
Tue Aug 11 06:46:02 PDT 2026
llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-backend-amdgpu
Author: Nathaniel McCallum (npmccallum)
<details>
<summary>Changes</summary>
A saturating float-to-int conversion clamps to the bounds of its **result**
type. Narrowing one to a legal integer width therefore saturates to the wrong
bounds, however small the range of the source is. Both AArch64 and AMDGPU did
that, producing wrong code with no diagnostic, because the generic legalizer
helper offered the transformation without any check and got the signedness wrong
as well.
## The generic problem
`LegalizerHelper::narrowScalar` dispatched `G_FPTOSI_SAT`/`G_FPTOUI_SAT` to
`narrowScalarFPTOI` alongside their non-saturating counterparts. That helper
converts at the narrower width and extends, which is sound only for the plain
opcodes, where out-of-range results are poison. It also derived signedness with
```cpp
bool IsSigned = MI.getOpcode() == TargetOpcode::G_FPTOSI;
```
false for `G_FPTOSI_SAT`, so a signed saturating result was zero-extended rather
than sign-extended — turning every negative result into a large positive one.
## AArch64
The legalizer narrowed any result wider than 64 bits to `i64` when the source
was a 16-bit float, reasoning that the range of an `f16` fits in an `i17`:
| call | produced | expected |
|---|---|---|
| `fptosi.sat.i128.f16(-1.0)` | `0x0..0FFFFFFFFFFFFFFFF` | `-1` |
| `fptosi.sat.i128.f16(inf)` | largest `i64` | largest `i128` |
| `fptosi.sat.i128.f16(-inf)` | `0x0..08000000000000000` | smallest `i128` |
| `fptoui.sat.i128.f16(inf)` | largest `u64` | largest `u128` |
Present since the opcodes were added in feac761f3797. AArch64 selects GlobalISel
at `-O0`, so that is where it is most readily seen, but the defect is in the
selector rather than in any optimization level.
## AMDGPU
Two independent places, both assuming a narrower width is interchangeable:
- the legalizer narrowed `{i64, f16}` to `i32` and zero-filled the upper half,
so `fptosi.sat.i64.f16(-1.0)` gave `0x00000000FFFFFFFF` and `(inf)` gave the
largest `i32`;
- `LowerFP_TO_INT_SAT` built a node with an `i64` result but an `i32` saturation
width. That form is legal — integer type legalization relies on it — but it
saturates to `i32` bounds and extends, which is a different operation. For
`bfloat` this is not confined to infinities: `bfloat` carries the exponent
range of `f32`, so ordinary finite values that fit in an `i64` were clamped to
the largest `i32`.
Introduced in 447f1e43bb9a. That commit is in the llvmorg-23.1.0-rc1 and rc2
tags but has not appeared in a final release.
## Structure
Seven commits: three baseline test commits and four fixes. Each newly added test
is committed first with the current, wrong result and a FIXME, per the precommit
workflow in `TestingGuide.md`. No fix commit introduces a test — each one only
changes expectations that already exist. The AMDGPU GlobalISel fix has no
baseline commit of its own: it is covered by the pre-existing scalar and vector
sat tests, and the new bfloat cases cannot cover it because
`bf16-conversions.ll` has no GlobalISel RUN lines.
```
[AMDGPU] Add saturating bfloat to i64 conversion tests (NFC)
[AMDGPU] Don't narrow saturating fp-to-int conversions in the legalizer
[AMDGPU] Fix saturation width in LowerFP_TO_INT_SAT
[AArch64][GlobalISel] Add saturating f16 to 128-bit legalizer test (NFC)
[AArch64][GlobalISel] Fix saturating f16 to 128-bit conversions
[GlobalISel] Add narrowScalar test for saturating fp-to-int (NFC)
[GlobalISel] Don't narrow saturating fp-to-int conversions
```
The generic commit comes last because it must: with either target rule still
present it breaks that target — AArch64 falls back to SelectionDAG and its
`CHECK-GI` lines stop matching, AMDGPU fails to legalize outright. Every commit
builds and passes on its own.
## Notes for reviewers
**On the size of the test diffs.** They are large but narrow. In
`AMDGPU/fptosi-sat-vector.ll`, 4 of 59 functions change and every added line
is a `CHECK` line. Three multipliers stack in those four: the miscompiled
sequence was about three instructions because it skipped the bounds checks,
and a correct 128-bit saturating conversion needs the convert plus three
comparisons — low bound, high bound, and unordered for NaN — with a select
per 32-bit half; the vector cases scalarize into eight independent
conversions; and the file has eight RUN lines, so each emitted instruction is
recorded eight times. The growth is confined to the `v8f16 -> v8i64` and
`v4f16 -> v4i64` shapes.
**The refusal is covered by a unit test**, not by a lit test: after the target
fixes no target requests the narrowing, so nothing in codegen reaches it.
`LegalizerHelperTest.cpp` calls `narrowScalar` on `G_FPTOSI_SAT`/`G_FPTOUI_SAT`
directly; the preceding commit records that it currently returns `Legalized`,
and the fix flips both assertions to `UnableToLegalize`. The legalizer lit test
that pins the lowered form is in the AArch64 commit, where it changes behaviour.
**Removing a legalization action** would break a target whose rule set has no
`lower` fallback. No in-tree target is affected: AArch64 has `.lowerIf`, AMDGPU
has `.lower()`.
**The near-identical rule on the non-saturating builders is left in place** and
is correct: every finite `f16` fits in an `i17`, out-of-range results are poison
for the plain opcodes, and the helper extends those correctly.
**bfloat is affected on AMDGPU but not AArch64.** Verified against the unfixed
compiler; AArch64 lowers `bfloat` correctly already.
## Testing
`check-llvm` is clean across all targets. Each commit was checked out, built and
run against the AArch64 and AMDGPU suites individually.
Separately, all 65536 `f16` and all 65536 `bfloat` bit patterns were converted
to `i64`, `u64`, `i128` and `u128` under both GlobalISel and SelectionDAG,
executed natively on aarch64, and compared against a reference computed in
exact rational arithmetic: 1,048,576 conversions, no mismatches.
Every fix here works by removing a target special case so that the generic
expansion runs, and that expansion is what the sweep above exercises. What is
left untested by execution is therefore each target's instruction selection of
the generic nodes, not the saturation logic itself.
I have no AMD hardware, so nothing in this series was executed on an AMDGPU
target; its evidence is `check-llvm` plus legalized MIR carrying the same
smallest/largest `i64` clamps and unordered-NaN compare as the AArch64 lowering
that was executed. If a reviewer can run the AMDGPU cases on hardware, I would
welcome the confirmation.
## AI disclosure
Per the LLVM AI Tool Use Policy: this change was developed with AI assistance.
An AI agent performed the investigation, wrote the patches and test updates, and
ran the validation described above; a second AI agent reviewed the series
against the project's conventions, and its findings were incorporated. All of
it was directed and reviewed by me, and I am responsible for the content.
---
Patch is 489.82 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/215567.diff
16 Files Affected:
- (modified) llvm/docs/GlobalISel/GenericOpcode.rst (+6-1)
- (modified) llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp (+9-2)
- (modified) llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp (-7)
- (modified) llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp (-9)
- (modified) llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp (+1-2)
- (added) llvm/test/CodeGen/AArch64/GlobalISel/legalize-fptosi-sat.mir (+45)
- (modified) llvm/test/CodeGen/AArch64/fptosi-sat-scalar.ll (+72-8)
- (modified) llvm/test/CodeGen/AArch64/fptosi-sat-vector.ll (+463-117)
- (modified) llvm/test/CodeGen/AArch64/fptoui-sat-scalar.ll (+36-8)
- (modified) llvm/test/CodeGen/AArch64/fptoui-sat-vector.ll (+278-117)
- (modified) llvm/test/CodeGen/AMDGPU/bf16-conversions.ll (+118)
- (modified) llvm/test/CodeGen/AMDGPU/fptosi-sat-scalar.ll (+355-31)
- (modified) llvm/test/CodeGen/AMDGPU/fptosi-sat-vector.ll (+4061-543)
- (modified) llvm/test/CodeGen/AMDGPU/fptoui-sat-scalar.ll (+233-44)
- (modified) llvm/test/CodeGen/AMDGPU/fptoui-sat-vector.ll (+2207-496)
- (modified) llvm/unittests/CodeGen/GlobalISel/LegalizerHelperTest.cpp (+29)
``````````diff
diff --git a/llvm/docs/GlobalISel/GenericOpcode.rst b/llvm/docs/GlobalISel/GenericOpcode.rst
index d2552b6c3a3f3..caf09b830acfe 100644
--- a/llvm/docs/GlobalISel/GenericOpcode.rst
+++ b/llvm/docs/GlobalISel/GenericOpcode.rst
@@ -575,7 +575,12 @@ Convert between integer and floating point.
G_FPTOSI_SAT, G_FPTOUI_SAT
^^^^^^^^^^^^^^^^^^^^^^^^^^
-Saturating convert between integer and floating point.
+Saturating convert between integer and floating point. Values outside the range
+of the result type saturate to its bounds, and NaN converts to zero.
+
+The saturation bounds are those of the result type, so these cannot be narrowed:
+converting at a narrower width saturates to the wrong bounds even when every
+finite value of the source fits in it.
G_FABS
^^^^^^
diff --git a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
index a6c5a267c87db..0bf7e52513ddf 100644
--- a/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
+++ b/llvm/lib/CodeGen/GlobalISel/LegalizerHelper.cpp
@@ -2122,10 +2122,14 @@ LegalizerHelper::LegalizeResult LegalizerHelper::narrowScalar(MachineInstr &MI,
Observer.changedInstr(MI);
return Legalized;
}
- case TargetOpcode::G_FPTOUI:
- case TargetOpcode::G_FPTOSI:
case TargetOpcode::G_FPTOUI_SAT:
case TargetOpcode::G_FPTOSI_SAT:
+ // A saturating conversion clamps to the bounds of its result type, so it
+ // cannot be performed at a narrower width however small the range of the
+ // source is. lower() expands it and narrows the conversion inside.
+ return UnableToLegalize;
+ case TargetOpcode::G_FPTOUI:
+ case TargetOpcode::G_FPTOSI:
return narrowScalarFPTOI(MI, TypeIdx, NarrowTy);
case TargetOpcode::G_FPEXT:
if (TypeIdx != 0)
@@ -7472,6 +7476,9 @@ LegalizerHelper::narrowScalarFPTOI(MachineInstr &MI, unsigned TypeIdx,
// If all finite floats fit into the narrowed integer type, we can just swap
// out the result type. This is practically only useful for conversions from
// half to at least 16-bits, so just handle the one case.
+ //
+ // This is only valid because out-of-range results are poison here; see the
+ // saturating opcodes in narrowScalar().
if (SrcTy.getScalarType() != LLT::scalar(16) ||
NarrowTy.getScalarSizeInBits() < (IsSigned ? 17u : 16u))
return UnableToLegalize;
diff --git a/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp b/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp
index 2263712120de8..992d36558711d 100644
--- a/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp
+++ b/llvm/lib/Target/AArch64/GISel/AArch64LegalizerInfo.cpp
@@ -974,13 +974,6 @@ AArch64LegalizerInfo::AArch64LegalizerInfo(const AArch64Subtarget &ST)
// Handle types larger than i64 by scalarizing/lowering.
.scalarizeIf(scalarOrEltWiderThan(0, 64), 0)
.scalarizeIf(scalarOrEltWiderThan(1, 64), 1)
- // The range of a fp16 value fits into an i17, so we can lower the width
- // to i64.
- .narrowScalarIf(
- [=](const LegalityQuery &Query) {
- return Query.Types[1] == f16 && Query.Types[0].getSizeInBits() > 64;
- },
- changeTo(0, i64))
.lowerIf(::any(scalarWiderThan(0, 64), scalarWiderThan(1, 64)), 0)
.moreElementsToNextPow2(0)
.widenScalarToNextPow2(0, /*MinSize=*/32)
diff --git a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
index 962988ff97e39..5de9b6cbcd5a0 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPUISelLowering.cpp
@@ -3935,15 +3935,6 @@ SDValue AMDGPUTargetLowering::LowerFP_TO_INT_SAT(const SDValue Op,
// SatWidth == DstWidth or SatWidth > 32
- // Saturate at i32 for i64 dst and f16/bf16 src (will invoke f16 promotion
- // below)
- if (DstVT == MVT::i64 &&
- (SrcVT == MVT::f16 || SrcVT == MVT::bf16 ||
- (SrcVT == MVT::f32 && Src.getOpcode() == ISD::FP16_TO_FP))) {
- const SDValue Int32VTOp = DAG.getValueType(MVT::i32);
- return DAG.getNode(OpOpcode, DL, DstVT, Src, Int32VTOp);
- }
-
// Promote f16/bf16 src to f32 for i32 conversion
if (DstVT == MVT::i32 && (SrcVT == MVT::f16 || SrcVT == MVT::bf16)) {
SDValue PromotedSrc = DAG.getNode(ISD::FP_EXTEND, DL, MVT::f32, Src);
diff --git a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
index 513a6487b39a2..01a90205197c9 100644
--- a/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
+++ b/llvm/lib/Target/AMDGPU/AMDGPULegalizerInfo.cpp
@@ -1258,8 +1258,7 @@ AMDGPULegalizerInfo::AMDGPULegalizerInfo(const GCNSubtarget &ST_,
auto &FPToISat = getActionDefinitionsBuilder({G_FPTOSI_SAT, G_FPTOUI_SAT})
.legalFor({{I32, F32}, {I32, F64}, {I16, F32}})
.legalFor(ST.has16BitInsts(), {{I16, F16}})
- .legalFor(ST.hasVCvtPkIU16F32(), {{V2I16, V2F32}})
- .narrowScalarFor({{I64, F16}}, changeElementSizeTo(0, I32));
+ .legalFor(ST.hasVCvtPkIU16F32(), {{V2I16, V2F32}});
// If available, widen width <16 to i16, intead of i32 so v_cvt_i16/u16_f16 can be used.
if (ST.has16BitInsts())
diff --git a/llvm/test/CodeGen/AArch64/GlobalISel/legalize-fptosi-sat.mir b/llvm/test/CodeGen/AArch64/GlobalISel/legalize-fptosi-sat.mir
new file mode 100644
index 0000000000000..1c714d410c1bb
--- /dev/null
+++ b/llvm/test/CodeGen/AArch64/GlobalISel/legalize-fptosi-sat.mir
@@ -0,0 +1,45 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 6
+# RUN: llc -mtriple=aarch64 -mattr=+fullfp16 -run-pass=legalizer %s -o - | FileCheck %s
+
+# The saturation bounds of G_FPTOSI_SAT are those of its result type, so this is
+# lowered rather than narrowed to a legal integer width.
+
+---
+name: fptosi_sat_s128_s16
+tracksRegLiveness: true
+body: |
+ bb.0:
+ liveins: $h0
+
+ ; CHECK-LABEL: name: fptosi_sat_s128_s16
+ ; CHECK: liveins: $h0
+ ; CHECK-NEXT: {{ $}}
+ ; CHECK-NEXT: [[COPY:%[0-9]+]]:_(f16) = COPY $h0
+ ; CHECK-NEXT: [[FPTOSI:%[0-9]+]]:_(i64) = G_FPTOSI [[COPY]](f16)
+ ; CHECK-NEXT: [[C:%[0-9]+]]:_(i64) = G_CONSTANT i64 63
+ ; CHECK-NEXT: [[ASHR:%[0-9]+]]:_(i64) = G_ASHR [[FPTOSI]], [[C]](i64)
+ ; CHECK-NEXT: [[C1:%[0-9]+]]:_(f16) = G_FCONSTANT half -6.550400e+04
+ ; CHECK-NEXT: [[FCMP:%[0-9]+]]:_(i32) = G_FCMP floatpred(ult), [[COPY]](f16), [[C1]]
+ ; CHECK-NEXT: [[C2:%[0-9]+]]:_(i64) = G_CONSTANT i64 0
+ ; CHECK-NEXT: [[C3:%[0-9]+]]:_(i64) = G_CONSTANT i64 -9223372036854775808
+ ; CHECK-NEXT: [[SELECT:%[0-9]+]]:_(i64) = G_SELECT [[FCMP]](i32), [[C2]], [[FPTOSI]]
+ ; CHECK-NEXT: [[SELECT1:%[0-9]+]]:_(i64) = G_SELECT [[FCMP]](i32), [[C3]], [[ASHR]]
+ ; CHECK-NEXT: [[C4:%[0-9]+]]:_(f16) = G_FCONSTANT half 6.550400e+04
+ ; CHECK-NEXT: [[FCMP1:%[0-9]+]]:_(i32) = G_FCMP floatpred(ogt), [[COPY]](f16), [[C4]]
+ ; CHECK-NEXT: [[C5:%[0-9]+]]:_(i64) = G_CONSTANT i64 -1
+ ; CHECK-NEXT: [[C6:%[0-9]+]]:_(i64) = G_CONSTANT i64 9223372036854775807
+ ; CHECK-NEXT: [[SELECT2:%[0-9]+]]:_(i64) = G_SELECT [[FCMP1]](i32), [[C5]], [[SELECT]]
+ ; CHECK-NEXT: [[SELECT3:%[0-9]+]]:_(i64) = G_SELECT [[FCMP1]](i32), [[C6]], [[SELECT1]]
+ ; CHECK-NEXT: [[FCMP2:%[0-9]+]]:_(i32) = G_FCMP floatpred(uno), [[COPY]](f16), [[COPY]]
+ ; CHECK-NEXT: [[SELECT4:%[0-9]+]]:_(i64) = G_SELECT [[FCMP2]](i32), [[C2]], [[SELECT2]]
+ ; CHECK-NEXT: [[SELECT5:%[0-9]+]]:_(i64) = G_SELECT [[FCMP2]](i32), [[C2]], [[SELECT3]]
+ ; CHECK-NEXT: $x0 = COPY [[SELECT4]](i64)
+ ; CHECK-NEXT: $x1 = COPY [[SELECT5]](i64)
+ ; CHECK-NEXT: RET_ReallyLR implicit $x0, implicit $x1
+ %0:_(f16) = COPY $h0
+ %1:_(i128) = G_FPTOSI_SAT %0
+ %2:_(i64), %3:_(i64) = G_UNMERGE_VALUES %1
+ $x0 = COPY %2
+ $x1 = COPY %3
+ RET_ReallyLR implicit $x0, implicit $x1
+...
diff --git a/llvm/test/CodeGen/AArch64/fptosi-sat-scalar.ll b/llvm/test/CodeGen/AArch64/fptosi-sat-scalar.ll
index f95d338198c4d..921e87f15c272 100644
--- a/llvm/test/CodeGen/AArch64/fptosi-sat-scalar.ll
+++ b/llvm/test/CodeGen/AArch64/fptosi-sat-scalar.ll
@@ -877,14 +877,46 @@ define i100 @test_signed_i100_f16(half %f) nounwind {
; CHECK-GI-CVT-LABEL: test_signed_i100_f16:
; CHECK-GI-CVT: // %bb.0:
; CHECK-GI-CVT-NEXT: fcvt s0, h0
-; CHECK-GI-CVT-NEXT: mov x1, xzr
-; CHECK-GI-CVT-NEXT: fcvtzs x0, s0
+; CHECK-GI-CVT-NEXT: mov w9, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: mov w10, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: movk w9, #51071, lsl #16
+; CHECK-GI-CVT-NEXT: movk w10, #18303, lsl #16
+; CHECK-GI-CVT-NEXT: fmov s1, w9
+; CHECK-GI-CVT-NEXT: fcvtzs x8, s0
+; CHECK-GI-CVT-NEXT: fcmp s0, s1
+; CHECK-GI-CVT-NEXT: fmov s1, w10
+; CHECK-GI-CVT-NEXT: mov x10, #34359738368 // =0x800000000
+; CHECK-GI-CVT-NEXT: asr x9, x8, #63
+; CHECK-GI-CVT-NEXT: csel x8, xzr, x8, lt
+; CHECK-GI-CVT-NEXT: csel x9, x10, x9, lt
+; CHECK-GI-CVT-NEXT: fcmp s0, s1
+; CHECK-GI-CVT-NEXT: mov x10, #34359738367 // =0x7ffffffff
+; CHECK-GI-CVT-NEXT: csinv x8, x8, xzr, le
+; CHECK-GI-CVT-NEXT: csel x9, x10, x9, gt
+; CHECK-GI-CVT-NEXT: fcmp s0, s0
+; CHECK-GI-CVT-NEXT: csel x0, xzr, x8, vs
+; CHECK-GI-CVT-NEXT: csel x1, xzr, x9, vs
; CHECK-GI-CVT-NEXT: ret
;
; CHECK-GI-FP16-LABEL: test_signed_i100_f16:
; CHECK-GI-FP16: // %bb.0:
-; CHECK-GI-FP16-NEXT: fcvtzs x0, h0
-; CHECK-GI-FP16-NEXT: mov x1, xzr
+; CHECK-GI-FP16-NEXT: adrp x8, .LCPI28_1
+; CHECK-GI-FP16-NEXT: fcvtzs x9, h0
+; CHECK-GI-FP16-NEXT: ldr h1, [x8, :lo12:.LCPI28_1]
+; CHECK-GI-FP16-NEXT: adrp x8, .LCPI28_0
+; CHECK-GI-FP16-NEXT: fcmp h0, h1
+; CHECK-GI-FP16-NEXT: ldr h1, [x8, :lo12:.LCPI28_0]
+; CHECK-GI-FP16-NEXT: mov x8, #34359738368 // =0x800000000
+; CHECK-GI-FP16-NEXT: asr x10, x9, #63
+; CHECK-GI-FP16-NEXT: csel x9, xzr, x9, lt
+; CHECK-GI-FP16-NEXT: csel x8, x8, x10, lt
+; CHECK-GI-FP16-NEXT: fcmp h0, h1
+; CHECK-GI-FP16-NEXT: mov x10, #34359738367 // =0x7ffffffff
+; CHECK-GI-FP16-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-FP16-NEXT: csel x8, x10, x8, gt
+; CHECK-GI-FP16-NEXT: fcmp h0, h0
+; CHECK-GI-FP16-NEXT: csel x0, xzr, x9, vs
+; CHECK-GI-FP16-NEXT: csel x1, xzr, x8, vs
; CHECK-GI-FP16-NEXT: ret
%x = call i100 @llvm.fptosi.sat.i100.f16(half %f)
ret i100 %x
@@ -919,14 +951,46 @@ define i128 @test_signed_i128_f16(half %f) nounwind {
; CHECK-GI-CVT-LABEL: test_signed_i128_f16:
; CHECK-GI-CVT: // %bb.0:
; CHECK-GI-CVT-NEXT: fcvt s0, h0
-; CHECK-GI-CVT-NEXT: mov x1, xzr
-; CHECK-GI-CVT-NEXT: fcvtzs x0, s0
+; CHECK-GI-CVT-NEXT: mov w9, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: mov w10, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: movk w9, #51071, lsl #16
+; CHECK-GI-CVT-NEXT: movk w10, #18303, lsl #16
+; CHECK-GI-CVT-NEXT: fmov s1, w9
+; CHECK-GI-CVT-NEXT: fcvtzs x8, s0
+; CHECK-GI-CVT-NEXT: fcmp s0, s1
+; CHECK-GI-CVT-NEXT: fmov s1, w10
+; CHECK-GI-CVT-NEXT: mov x10, #-9223372036854775808 // =0x8000000000000000
+; CHECK-GI-CVT-NEXT: asr x9, x8, #63
+; CHECK-GI-CVT-NEXT: csel x8, xzr, x8, lt
+; CHECK-GI-CVT-NEXT: csel x9, x10, x9, lt
+; CHECK-GI-CVT-NEXT: fcmp s0, s1
+; CHECK-GI-CVT-NEXT: mov x10, #9223372036854775807 // =0x7fffffffffffffff
+; CHECK-GI-CVT-NEXT: csinv x8, x8, xzr, le
+; CHECK-GI-CVT-NEXT: csel x9, x10, x9, gt
+; CHECK-GI-CVT-NEXT: fcmp s0, s0
+; CHECK-GI-CVT-NEXT: csel x0, xzr, x8, vs
+; CHECK-GI-CVT-NEXT: csel x1, xzr, x9, vs
; CHECK-GI-CVT-NEXT: ret
;
; CHECK-GI-FP16-LABEL: test_signed_i128_f16:
; CHECK-GI-FP16: // %bb.0:
-; CHECK-GI-FP16-NEXT: fcvtzs x0, h0
-; CHECK-GI-FP16-NEXT: mov x1, xzr
+; CHECK-GI-FP16-NEXT: adrp x8, .LCPI29_1
+; CHECK-GI-FP16-NEXT: fcvtzs x9, h0
+; CHECK-GI-FP16-NEXT: ldr h1, [x8, :lo12:.LCPI29_1]
+; CHECK-GI-FP16-NEXT: adrp x8, .LCPI29_0
+; CHECK-GI-FP16-NEXT: fcmp h0, h1
+; CHECK-GI-FP16-NEXT: ldr h1, [x8, :lo12:.LCPI29_0]
+; CHECK-GI-FP16-NEXT: mov x8, #-9223372036854775808 // =0x8000000000000000
+; CHECK-GI-FP16-NEXT: asr x10, x9, #63
+; CHECK-GI-FP16-NEXT: csel x9, xzr, x9, lt
+; CHECK-GI-FP16-NEXT: csel x8, x8, x10, lt
+; CHECK-GI-FP16-NEXT: fcmp h0, h1
+; CHECK-GI-FP16-NEXT: mov x10, #9223372036854775807 // =0x7fffffffffffffff
+; CHECK-GI-FP16-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-FP16-NEXT: csel x8, x10, x8, gt
+; CHECK-GI-FP16-NEXT: fcmp h0, h0
+; CHECK-GI-FP16-NEXT: csel x0, xzr, x9, vs
+; CHECK-GI-FP16-NEXT: csel x1, xzr, x8, vs
; CHECK-GI-FP16-NEXT: ret
%x = call i128 @llvm.fptosi.sat.i128.f16(half %f)
ret i128 %x
diff --git a/llvm/test/CodeGen/AArch64/fptosi-sat-vector.ll b/llvm/test/CodeGen/AArch64/fptosi-sat-vector.ll
index 145aef9123a4e..fcceef66470c7 100644
--- a/llvm/test/CodeGen/AArch64/fptosi-sat-vector.ll
+++ b/llvm/test/CodeGen/AArch64/fptosi-sat-vector.ll
@@ -3237,37 +3237,123 @@ define <4 x i100> @test_signed_v4f16_v4i100(<4 x half> %f) {
; CHECK-GI-CVT-LABEL: test_signed_v4f16_v4i100:
; CHECK-GI-CVT: // %bb.0:
; CHECK-GI-CVT-NEXT: // kill: def $d0 killed $d0 def $q0
-; CHECK-GI-CVT-NEXT: mov h1, v0.h[1]
-; CHECK-GI-CVT-NEXT: mov h2, v0.h[2]
-; CHECK-GI-CVT-NEXT: mov x1, xzr
-; CHECK-GI-CVT-NEXT: mov h3, v0.h[3]
-; CHECK-GI-CVT-NEXT: fcvt s0, h0
-; CHECK-GI-CVT-NEXT: mov x3, xzr
-; CHECK-GI-CVT-NEXT: mov x5, xzr
-; CHECK-GI-CVT-NEXT: mov x7, xzr
-; CHECK-GI-CVT-NEXT: fcvt s1, h1
-; CHECK-GI-CVT-NEXT: fcvt s2, h2
+; CHECK-GI-CVT-NEXT: fcvt s2, h0
+; CHECK-GI-CVT-NEXT: mov w9, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: mov h3, v0.h[1]
+; CHECK-GI-CVT-NEXT: movk w9, #51071, lsl #16
+; CHECK-GI-CVT-NEXT: mov x11, #34359738368 // =0x800000000
+; CHECK-GI-CVT-NEXT: mov h5, v0.h[2]
+; CHECK-GI-CVT-NEXT: fmov s1, w9
+; CHECK-GI-CVT-NEXT: mov w9, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: mov h0, v0.h[3]
+; CHECK-GI-CVT-NEXT: movk w9, #18303, lsl #16
+; CHECK-GI-CVT-NEXT: fcvtzs x8, s2
+; CHECK-GI-CVT-NEXT: fmov s4, w9
; CHECK-GI-CVT-NEXT: fcvt s3, h3
-; CHECK-GI-CVT-NEXT: fcvtzs x0, s0
-; CHECK-GI-CVT-NEXT: fcvtzs x2, s1
-; CHECK-GI-CVT-NEXT: fcvtzs x4, s2
-; CHECK-GI-CVT-NEXT: fcvtzs x6, s3
+; CHECK-GI-CVT-NEXT: fcmp s2, s1
+; CHECK-GI-CVT-NEXT: fcvt s0, h0
+; CHECK-GI-CVT-NEXT: asr x10, x8, #63
+; CHECK-GI-CVT-NEXT: csel x8, xzr, x8, lt
+; CHECK-GI-CVT-NEXT: fcvtzs x12, s3
+; CHECK-GI-CVT-NEXT: csel x9, x11, x10, lt
+; CHECK-GI-CVT-NEXT: fcmp s2, s4
+; CHECK-GI-CVT-NEXT: mov x10, #34359738367 // =0x7ffffffff
+; CHECK-GI-CVT-NEXT: csinv x8, x8, xzr, le
+; CHECK-GI-CVT-NEXT: csel x9, x10, x9, gt
+; CHECK-GI-CVT-NEXT: fcmp s2, s2
+; CHECK-GI-CVT-NEXT: fcvt s2, h5
+; CHECK-GI-CVT-NEXT: csel x0, xzr, x8, vs
+; CHECK-GI-CVT-NEXT: csel x1, xzr, x9, vs
+; CHECK-GI-CVT-NEXT: fcmp s3, s1
+; CHECK-GI-CVT-NEXT: asr x8, x12, #63
+; CHECK-GI-CVT-NEXT: csel x9, xzr, x12, lt
+; CHECK-GI-CVT-NEXT: csel x8, x11, x8, lt
+; CHECK-GI-CVT-NEXT: fcmp s3, s4
+; CHECK-GI-CVT-NEXT: fcvtzs x12, s2
+; CHECK-GI-CVT-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-CVT-NEXT: csel x8, x10, x8, gt
+; CHECK-GI-CVT-NEXT: fcmp s3, s3
+; CHECK-GI-CVT-NEXT: csel x2, xzr, x9, vs
+; CHECK-GI-CVT-NEXT: csel x3, xzr, x8, vs
+; CHECK-GI-CVT-NEXT: fcmp s2, s1
+; CHECK-GI-CVT-NEXT: asr x8, x12, #63
+; CHECK-GI-CVT-NEXT: csel x9, xzr, x12, lt
+; CHECK-GI-CVT-NEXT: csel x8, x11, x8, lt
+; CHECK-GI-CVT-NEXT: fcmp s2, s4
+; CHECK-GI-CVT-NEXT: fcvtzs x12, s0
+; CHECK-GI-CVT-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-CVT-NEXT: csel x8, x10, x8, gt
+; CHECK-GI-CVT-NEXT: fcmp s2, s2
+; CHECK-GI-CVT-NEXT: csel x4, xzr, x9, vs
+; CHECK-GI-CVT-NEXT: csel x5, xzr, x8, vs
+; CHECK-GI-CVT-NEXT: fcmp s0, s1
+; CHECK-GI-CVT-NEXT: asr x8, x12, #63
+; CHECK-GI-CVT-NEXT: csel x9, xzr, x12, lt
+; CHECK-GI-CVT-NEXT: csel x8, x11, x8, lt
+; CHECK-GI-CVT-NEXT: fcmp s0, s4
+; CHECK-GI-CVT-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-CVT-NEXT: csel x8, x10, x8, gt
+; CHECK-GI-CVT-NEXT: fcmp s0, s0
+; CHECK-GI-CVT-NEXT: csel x6, xzr, x9, vs
+; CHECK-GI-CVT-NEXT: csel x7, xzr, x8, vs
; CHECK-GI-CVT-NEXT: ret
;
; CHECK-GI-FP16-LABEL: test_signed_v4f16_v4i100:
; CHECK-GI-FP16: // %bb.0:
; CHECK-GI-FP16-NEXT: // kill: def $d0 killed $d0 def $q0
-; CHECK-GI-FP16-NEXT: mov h1, v0.h[1]
-; CHECK-GI-FP16-NEXT: mov h2, v0.h[2]
-; CHECK-GI-FP16-NEXT: mov x1, xzr
-; CHECK-GI-FP16-NEXT: mov h3, v0.h[3]
-; CHECK-GI-FP16-NEXT: fcvtzs x0, h0
-; CHECK-GI-FP16-NEXT: mov x3, xzr
-; CHECK-GI-FP16-NEXT: mov x5, xzr
-; CHECK-GI-FP16-NEXT: mov x7, xzr
-; CHECK-GI-FP16-NEXT: fcvtzs x2, h1
-; CHECK-GI-FP16-NEXT: fcvtzs x4, h2
-; CHECK-GI-FP16-NEXT: fcvtzs x6, h3
+; CHECK-GI-FP16-NEXT: adrp x8, .LCPI64_1
+; CHECK-GI-FP16-NEXT: fcvtzs x9, h0
+; CHECK-GI-FP16-NEXT: mov h2, v0.h[1]
+; CHECK-GI-FP16-NEXT: ldr h1, [x8, :lo12:.LCPI64_1]
+; CHECK-GI-FP16-NEXT: adrp x8, .LCPI64_0
+; CHECK-GI-FP16-NEXT: mov x10, #34359738368 // =0x800000000
+; CHECK-GI-FP16-NEXT: ldr h3, [x8, :lo12:.LCPI64_0]
+; CHECK-GI-FP16-NEXT: mov x11, #34359738367 // =0x7ffffffff
+; CHECK-GI-FP16-NEXT: mov h4, v0.h[2]
+; CHECK-GI-FP16-NEXT: fcmp h0, h1
+; CHECK-GI-FP16-NEXT: asr x8, x9, #63
+; CHECK-GI-FP16-NEXT: fcvtzs x12, h2
+; CHECK-GI-FP16-NEXT: csel x9, xzr, x9, lt
+; CHECK-GI-FP16-NEXT: csel x8, x10, x8, lt
+; CHECK-GI-FP16-NEXT: fcmp h0, h3
+; CHECK-GI-FP16-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-FP16-NEXT: csel x8, x11, x8, gt
+; CHECK-GI-FP16-NEXT: fcmp h0, h0
+; CHECK-GI-FP16-NEXT: mov h0, v0.h[3]
+; CHECK-GI-FP16-NEXT: csel x0, xzr, x9, vs
+; CHECK-GI-FP16-NEXT: csel x1, xzr, x8, vs
+; CHECK-GI-FP16-NEXT: fcmp h2, h1
+; CHECK-GI-FP16-NEXT: asr x8, x12, #63
+; CHECK-GI-FP16-NEXT: csel x9, xzr, x12, lt
+; CHECK-GI-FP16-NEXT: csel x8, x10, x8, lt
+; CHECK-GI-FP16-NEXT: fcmp h2, h3
+; CHECK-GI-FP16-NEXT: fcvtzs x12, h4
+; CHECK-GI-FP16-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-FP16-NEXT: csel x8, x11, x8, gt
+; CHECK-GI-FP16-NEXT: fcmp h2, h2
+; CHECK-GI-FP16-NEXT: csel x2, xzr, x9, vs
+; CHECK-GI-FP16-NEXT: csel x3, xzr, x8, vs
+; CHECK-GI-FP16-NEXT: fcmp h4, h1
+; CHECK-GI-FP16-NEXT: asr x8, x12, #63
+; CHECK-GI-FP16-NEXT: csel x9, xzr, x12, lt
+; CHECK-GI-FP16-NEXT: csel x8, x10, x8, lt
+; CHECK-GI-FP16-NEXT: fcmp h4, h3
+; CHECK-GI-FP16-NEXT: fcvtzs x12, h0
+; CHECK-GI-FP16-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-FP16-NEXT: csel x8, x11, x8, gt
+; CHECK-GI-FP16-NEXT: fcmp h4, h4
+; CHECK-GI-FP16-NEXT: csel x4, xzr, x9, vs
+; CHECK-GI-FP16-NEXT: csel x5, xzr, x8, vs
+; CHECK-GI-FP16-NEXT: fcmp h0, h1
+; CHECK-GI-FP16-NEXT: asr x8, x12, #63
+; CHECK-GI-FP16-NEXT: csel x9, xzr, x12, lt
+; CHECK-GI-FP16-NEXT: csel x8, x10, x8, lt
+; CHECK-GI-FP16-NEXT: fcmp h0, h3
+; CHECK-GI-FP16-NEXT: csinv x9, x9, xzr, le
+; CHECK-GI-FP16-NEXT: csel x8, x11, x8, gt
+; CHECK-GI-FP16-NEXT: fcmp h0, h0
+; CHECK-GI-FP16-NEXT: csel x6, xzr, x9, vs
+; CHECK-GI-FP16-NEXT: csel x7, xzr, x8, vs
; CHECK-GI-FP16-NEXT: ret
%x = call <4 x i100> @llvm.fptosi.sat.v4f16.v4i100(<4 x half> %f)
ret <4 x i100> %x
@@ -3377,37 +3463,123 @@ define <4 x i128> @test_signed_v4f16_v4i128(<4 x half> %f) {
; CHECK-GI-CVT-LABEL: test_signed_v4f16_v4i128:
; CHECK-GI-CVT: // %bb.0:
; CHECK-GI-CVT-NEXT: // kill: def $d0 killed $d0 def $q0
-; CHECK-GI-CVT-NEXT: mov h1, v0.h[1]
-; CHECK-GI-CVT-NEXT: mov h2, v0.h[2]
-; CHECK-GI-CVT-NEXT: mov x1, xzr
-; CHECK-GI-CVT-NEXT: mov h3, v0.h[3]
-; CHECK-GI-CVT-NEXT: fcvt s0, h0
-; CHECK-GI-CVT-NEXT: mov x3, xzr
-; CHECK-GI-CVT-NEXT: mov x5, xzr
-; CHECK-GI-CVT-NEXT: mov x7, xzr
-; CHECK-GI-CVT-NEXT: fcvt s1, h1
-; CHECK-GI-CVT-NEXT: fcvt s2, h2
+; CHECK-GI-CVT-NEXT: fcvt s2, h0
+; CHECK-GI-CVT-NEXT: mov w9, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: mov h3, v0.h[1]
+; CHECK-GI-CVT-NEXT: movk w9, #51071, lsl #16
+; CHECK-GI-CVT-NEXT: mov x11, #-9223372036854775808 // =0x8000000000000000
+; CHECK-GI-CVT-NEXT: mov h5, v0.h[2]
+; CHECK-GI-CVT-NEXT: fmov s1, w9
+; CHECK-GI-CVT-NEXT: mov w9, #57344 // =0xe000
+; CHECK-GI-CVT-NEXT: mov h0, v0.h[3]
+; CHECK-GI-CVT-NEXT: movk w9, #18303, lsl #16
+; CHECK-GI-CVT-NEXT: fcvtzs x8, s2
+; CHECK-G...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/215567
More information about the llvm-commits
mailing list