[all-commits] [llvm/llvm-project] 3e24a3: [SelectionDAG] Optimize 32-bit udiv with 33-bit ma...

[MITSUNARI Shigeo via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 3e24a393571e83e8564562b2ec84fe701648da2f
      https://github.com/llvm/llvm-project/commit/3e24a393571e83e8564562b2ec84fe701648da2f
  Author: MITSUNARI Shigeo <herumi at nifty.com>
  Date:   2026-03-06 (Fri, 06 Mar 2026)

  Changed paths:
    M llvm/include/llvm/Support/DivisionByConstantInfo.h
    M llvm/lib/CodeGen/SelectionDAG/TargetLowering.cpp
    M llvm/lib/Support/DivisionByConstantInfo.cpp
    M llvm/test/CodeGen/AArch64/rem-by-const.ll
    M llvm/test/CodeGen/AArch64/sve-streaming-mode-fixed-length-int-div.ll
    A llvm/test/CodeGen/AArch64/udiv-const-optimization.ll
    M llvm/test/CodeGen/AArch64/urem-lkk.ll
    M llvm/test/CodeGen/RISCV/div-by-constant.ll
    A llvm/test/CodeGen/RISCV/udiv-const-optimization.ll
    M llvm/test/CodeGen/RISCV/urem-lkk.ll
    M llvm/test/CodeGen/X86/fold-loop-of-urem.ll
    A llvm/test/CodeGen/X86/udiv-const-optimization.ll
    M llvm/test/CodeGen/X86/urem-lkk.ll

  Log Message:
  -----------
  [SelectionDAG] Optimize 32-bit udiv with 33-bit magic constants on 64-bit targets (#181288)

This PR optimizes 32-bit unsigned division by constants when the magic
constant is 33 bits (IsAdd=true case in UnsignedDivisionByConstantInfo)
on 64-bit targets.

## Overview

Compiler optimization for constant division of `uint32_t` variables
(such as `x / 7`) is based on the method
proposed by Granlund and Montgomery in 1994 (hereafter referred to as
the GM method).
However, the GM method for the IsAdd=true case was optimized for 32-bit
CPUs, not 64-bit CPUs.

This patch provides optimizations specifically for 64-bit CPUs (such as
x86_64 and Apple M-series).
A simple benchmark demonstrates over 60% speedup on both Intel Xeon and
Apple M4 processors.

## The GM Method

The GM method for `x / 7` can be expressed in C code as follows,
where the constants `c` and `a` are magic numbers determined by the
divisor:

```cpp
uint32_t udiv_original(uint32_t x) {
    uint64_t v = x * c;
    v >>= 32;
    uint32_t t = uint32_t(x) - uint32_t(v);
    t >>= 1;
    t += uint32_t(v);
    t >>= a - 33;
    return t;
}
```

For example, division by 7 on x86_64 generates 7 instructions:

```asm
movl    %edi, %eax
imulq   $613566757, %rax, %rax
shrq    $32, %rax
subl    %eax, %edi
shrl    %edi
addl    %edi, %eax
shrl    $2, %eax
```

## Proposed Solution

This patch generates the following optimized code:

```cpp
uint32_t udiv_optimized(uint32_t x) {
    uint128_t v = uint128_t(x) * ((c + 0x100000000) << (64 - a));
    return uint32_t(v >> 64);
}
```

Since a 64-bit right shift of a 128-bit variable extracts the upper 64
bits,
this code eliminates the need for shifts after multiplication.

The implementation pre-shifts the 33-bit magic constant `c = 2^32 +
Magic` left by `(64-a)` bits
and uses the high 64 bits of a 64 x 64 -> 128 bit multiplication
directly.
This eliminates the add/sub/shift sequence.

After optimization, division by 7 becomes 4 instructions (or 3 with
BMI2):

```asm
# Standard (4 instructions)
movl    %edi, %eax
movabsq $2635249153617166336, %rcx
mulq    %rcx
movq    %rdx, %rax

# With BMI2 (3 instructions)
movl    %edi, %edx
movabsq $2635249153617166336, %rax
mulxq   %rax, %rax, %rax
```



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications