[all-commits] [llvm/llvm-project] d402c6: [X86] Fold atomic loads into compares (#221290)

[Andrew Gaul via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: d402c6191ad4051a3cb534dae125dfdf1f304889
      https://github.com/llvm/llvm-project/commit/d402c6191ad4051a3cb534dae125dfdf1f304889
  Author: Andrew Gaul <andrew at gaul.org>
  Date:   2026-09-08 (Tue, 08 Sep 2026)

  Changed paths:
    M llvm/lib/Target/X86/X86ISelLowering.cpp
    M llvm/lib/Target/X86/X86InstrCompiler.td
    M llvm/test/CodeGen/X86/PR40322.ll
    A llvm/test/CodeGen/X86/atomic-cmp-fold.ll
    M llvm/test/CodeGen/X86/atomic-unordered.ll

  Log Message:
  -----------
  [X86] Fold atomic loads into compares (#221290)

A compare of an atomic load kept the load in a register (mov + cmp/test)
where a compare of a regular load folds the load into the compare's
memory operand. The folded form performs a single full-width load, and
on x86 an ordinary load already provides every ordering up to seq_cst,
so folding is legal for any non-extending atomic load.

Add isel patterns folding an atomic load into CMP (mi/mr/rm forms across
i8/i16/i32/i64). Compare-to-zero already reaches these via EmitTest;
nonzero and register compares are emitted as a flag-only X86ISD::SUB (to
enable CSE with a real subtract) whose load the peephole then folds. The
peephole cannot move an ordered access, so those never folded for atomic
loads. Emit X86ISD::CMP instead when an operand is a foldable atomic
load: it has no non-atomic sibling to CSE with anyway.

x86lint reports 5854 "load foldable into compare" sites in a release
build of Firefox's libxul.so; sampling shows the majority are
atomic-load compares of the kind this folds -- mozilla::Atomic and
std::atomic loads, MOZ_LOG level checks (an Atomic<LogLevel>), and Rust
core::sync::atomic and log-crate level checks.

---------

Co-authored-by: Claude Fable 5 <noreply at anthropic.com>



To unsubscribe from these emails, change your notification settings at https://github.com/llvm/llvm-project/settings/notifications