[all-commits] [llvm/llvm-project] adfb3b: [X86] Add vXi64 MULHU/MULHS lowering, keeping full...

[Rito Takeuchi via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: adfb3be9d098e4c12895957f9ac3fc9d892b45a9
      https://github.com/llvm/llvm-project/commit/adfb3be9d098e4c12895957f9ac3fc9d892b45a9
  Author: Rito Takeuchi <licht-t at outlook.jp>
  Date:   2026-07-08 (Wed, 08 Jul 2026)

  Changed paths:
    M llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
    M llvm/lib/Target/X86/X86ISelLowering.cpp
    A llvm/test/CodeGen/X86/mulhu-v4i64-umul-lohi-guard.ll
    M llvm/test/CodeGen/X86/srem-vector-lkk.ll
    M llvm/test/CodeGen/X86/urem-vector-lkk.ll
    M llvm/test/CodeGen/X86/vector-idiv-sdiv-256.ll
    M llvm/test/CodeGen/X86/vector-idiv-sdiv-512.ll
    M llvm/test/CodeGen/X86/vector-idiv-udiv-256.ll
    M llvm/test/CodeGen/X86/vector-idiv-udiv-512.ll

  Log Message:
  -----------
  [X86] Add vXi64 MULHU/MULHS lowering, keeping full-width products scalar (#206983)

Based on the discussion in the PR #169819. 
This lands the unsigned (`MULHU`) and signed (`MULHS`)
`vXi64` high-multiply lowering, plus a guard so full 128-bit products
stay scalar.

### What this does

1. **Lower `ISD::MULHU` for `v4i64`/`v8i64`** via `forceExpandMultiply`
(a `vpmuludq` schoolbook), as in #169819.
2. **Lower `ISD::MULHS` for `v4i64`/`v8i64`, gated on AVX512DQ.** The
signed low multiply is `vpmullq`, so `MULHS` is only marked `Custom`
when DQ (and VL for `v4i64`) is available.
3. **Guard the full-width-product case.** When the *low* half of the
product is also used (a `wyhash`-style `lo ^ hi`), vectorizing just the
high half is redundant with the low half's wide multiply. So, this guard
let a single scalar multiply per lane yield both halves, likewise the
unpatched target already did.
This is done by marking `UMUL_LOHI`/`SMUL_LOHI` `v4i64`/`v8i64` `Custom`
(unrolled), which lets the existing DAGCombiner check decline to narrow
`trunc(srl(mul(ext,ext)))` to `MULHU`/`MULHS` when the wide multiply's
low half is used.

### Performance

Same-IR A/B (identical IR, stock vs patched `llc`), EC2 metal,
per-target `-march`/`-mtune`, turbo off, one pinned core. ns/elem,
`stock → patched (ratio)`. Full matrix:
https://github.com/llvm/llvm-project/pull/206983#issuecomment-4871421534.

| kernel (AVX-512 256b) | Cascade | Ice Lake | SPR | Zen 4 |
|---|---|---|---|---|
| `udiv` (MULHU) | 1.71x | 1.71x | 1.78x | 1.88x |
| `div` (MULHS) | 1.18x | 1.25x | 1.31x | 2.04x |
| full-128 `lo ^ hi` (guarded) | 1.00x | 1.00x | 1.00x | 1.00x |

### Testing

- New `test/CodeGen/X86/mulhu-v4i64-umul-lohi-guard.ll`:
`umul_lohi_both_halves` (both halves used → scalarizes) and
`mulhu_high_only` (high only → vectorizes), on AVX2 and AVX-512.
- Existing `vector-idiv-{udiv,sdiv}-{256,512}.ll`, `urem-vector-lkk.ll`,
`srem-vector-lkk.ll` show the `MULHU`/`MULHS` vectorization (the signed
ones only on AVX512DQ RUN lines).
- No regressions across `test/CodeGen/X86`.

### AI Usage Disclosure
This PR was prepared with the assistance of Claude Code.

Fixes #37771



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications