[all-commits] [llvm/llvm-project] e477a9: [AMDGPU] Re-fold masked hi16 pack into v_or_b32_sd...

[michaelselehov via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: e477a9950ed219a02f4b33b35c4bee24f4c87b00
      https://github.com/llvm/llvm-project/commit/e477a9950ed219a02f4b33b35c4bee24f4c87b00
  Author: michaelselehov <michael.selehov at amd.com>
  Date:   2026-07-30 (Thu, 30 Jul 2026)

  Changed paths:
    M llvm/lib/Target/AMDGPU/SIPeepholeSDWA.cpp
    M llvm/test/CodeGen/AMDGPU/GlobalISel/add.vni16.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/fpow.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.abs.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.atomic.dim.a16.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.gather4.a16.dim.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.getresinfo.a16.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.2darraymsaa.a16.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.load.3d.a16.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.sample.cd.g16.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/llvm.amdgcn.image.sample.g16.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/saddsat.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/select-to-fmin-fmax.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/ssubsat.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/uaddsat.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/usubsat.ll
    M llvm/test/CodeGen/AMDGPU/GlobalISel/vni8-across-blocks.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.1024bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.128bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.256bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.320bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.32bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.48bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.512bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.576bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.640bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.64bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.704bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.768bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.832bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.896bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.960bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgcn.bitcast.96bit.ll
    M llvm/test/CodeGen/AMDGPU/amdgpu-codegenprepare-idiv.ll
    M llvm/test/CodeGen/AMDGPU/buffer-fat-pointer-atomicrmw-fadd.ll
    M llvm/test/CodeGen/AMDGPU/buffer-fat-pointer-atomicrmw-fmax.ll
    M llvm/test/CodeGen/AMDGPU/buffer-fat-pointer-atomicrmw-fmin.ll
    M llvm/test/CodeGen/AMDGPU/buffer-fat-pointers-contents-legalization.ll
    M llvm/test/CodeGen/AMDGPU/build-vector-packed-partial-undef.ll
    M llvm/test/CodeGen/AMDGPU/constant-address-space-32bit.ll
    M llvm/test/CodeGen/AMDGPU/ctlz_zero_poison.ll
    M llvm/test/CodeGen/AMDGPU/cvt_f32_ubyte.ll
    M llvm/test/CodeGen/AMDGPU/dagcombine-fmul-sel.ll
    M llvm/test/CodeGen/AMDGPU/fmaximum.ll
    M llvm/test/CodeGen/AMDGPU/fminimum.ll
    M llvm/test/CodeGen/AMDGPU/fmul-to-ldexp.ll
    M llvm/test/CodeGen/AMDGPU/fneg-fabs.bf16.ll
    M llvm/test/CodeGen/AMDGPU/fneg.bf16.ll
    M llvm/test/CodeGen/AMDGPU/fptosi-sat-vector.ll
    M llvm/test/CodeGen/AMDGPU/fptoui-sat-vector.ll
    M llvm/test/CodeGen/AMDGPU/fptrunc.v2f16.no.fast.math.ll
    M llvm/test/CodeGen/AMDGPU/freeze.ll
    M llvm/test/CodeGen/AMDGPU/frem.ll
    M llvm/test/CodeGen/AMDGPU/idot4u.ll
    M llvm/test/CodeGen/AMDGPU/idot8s.ll
    M llvm/test/CodeGen/AMDGPU/idot8u.ll
    M llvm/test/CodeGen/AMDGPU/integer-mad-patterns.ll
    M llvm/test/CodeGen/AMDGPU/llvm.amdgcn.image.sample.g16.a16.dim.ll
    M llvm/test/CodeGen/AMDGPU/llvm.amdgcn.image.sample.noret.ll
    M llvm/test/CodeGen/AMDGPU/llvm.fptrunc.round.ll
    M llvm/test/CodeGen/AMDGPU/llvm.maximum.f16.ll
    M llvm/test/CodeGen/AMDGPU/llvm.minimum.f16.ll
    M llvm/test/CodeGen/AMDGPU/llvm.modf.ll
    M llvm/test/CodeGen/AMDGPU/mad-mix-lo.ll
    M llvm/test/CodeGen/AMDGPU/packed-op-sel.ll
    M llvm/test/CodeGen/AMDGPU/permute_i8.ll
    M llvm/test/CodeGen/AMDGPU/scalar_to_vector.ll
    A llvm/test/CodeGen/AMDGPU/sdwa-peephole-lshl-or-hi16pack.ll
    A llvm/test/CodeGen/AMDGPU/sdwa-peephole-lshl-or-hi16pack.mir
    M llvm/test/CodeGen/AMDGPU/sdwa-peephole.ll
    M llvm/test/CodeGen/AMDGPU/sminmax.v2i16.ll
    M llvm/test/CodeGen/AMDGPU/strict_fsub.f16.ll
    M llvm/test/CodeGen/AMDGPU/v_sat_pk_u8_i16.ll

  Log Message:
  -----------
  [AMDGPU] Re-fold masked hi16 pack into v_or_b32_sdwa (#210735)

Follow-up to #206058, which selects the (hi << 16) | (z & 0xffff)
high-half pack to a fused v_lshl_or_b32. When z's high bits are not
known zero this needs an explicit v_and_b32 0xffff, and since
v_lshl_or_b32 has no SDWA form the mask can no longer fold into a single
v_or_b32_sdwa (src1_sel:WORD_0) as it did before -- a small but real
perf hole (~5% on a VALU-bound char4 kernel on gfx942).

Add a SIPeepholeSDWA rewrite of the masked pack back into v_lshlrev_b32
+ v_or_b32_sdwa, erasing the dead v_and. It fires only on the masked
form, so the mask-free #206058 win is preserved, and it never increases
the instruction count.

Assisted-by: Claude Opus



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications