[llvm] [X86] Record the enclosed register in X86DomainReassignment::buildClosure (PR #202534)

[via llvm-commits]

https://github.com/mbhade-amd created https://github.com/llvm/llvm-project/pull/202534


buildClosure recorded the seed register Reg in the function-wide EnclosedEdges map on every worklist iteration instead of CurReg, the register actually being added to the closure. EnclosedEdges therefore only ever contained the seed of each closure.

The driver loop in runOnMachineFunction skips registers already present in EnclosedEdges before starting a new closure. Because only seeds were recorded, every non-seed member of an already-built closure looked like a fresh seed, so a redundant closure was built for it and then immediately discarded by the EnclosedInstrs cross-closure check. The emitted code is unchanged; the pass just performed redundant work proportional to closure size.

Key EnclosedEdges by CurReg so each enclosed register is recorded once.

This was found as part of @jlebar's X86 LLVM bug hunt / FuzzX effort:
https://github.com/SemiAnalysisAI/FuzzX/tree/master/x86/bugs/007-domain-reassignment-wrong-enclosed-key

cc @jlebar

>From 3ad9da96695715fe3ed25e9584f5cc7b2002ba4c Mon Sep 17 00:00:00 2001
From: mbhade <mbhade at amd.com>
Date: Tue, 9 Jun 2026 13:10:22 +0530
Subject: [PATCH] [X86] Record the enclosed register in
 X86DomainReassignment::buildClosure

buildClosure recorded the seed register Reg in the function-wide
EnclosedEdges map on every worklist iteration instead of CurReg, the
register actually being added to the closure. EnclosedEdges therefore
only ever contained the seed of each closure.

The driver loop in runOnMachineFunction skips registers already present
in EnclosedEdges before starting a new closure. Because only seeds were
recorded, every non-seed member of an already-built closure looked like a
fresh seed, so a redundant closure was built for it and then immediately
discarded by the EnclosedInstrs cross-closure check. The emitted code is
unchanged; the pass just performed redundant work proportional to closure
size.

Key EnclosedEdges by CurReg so each enclosed register is recorded once.
---
 llvm/lib/Target/X86/X86DomainReassignment.cpp | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/llvm/lib/Target/X86/X86DomainReassignment.cpp b/llvm/lib/Target/X86/X86DomainReassignment.cpp
index a9f68393aa3e2..dae1cd2640378 100644
--- a/llvm/lib/Target/X86/X86DomainReassignment.cpp
+++ b/llvm/lib/Target/X86/X86DomainReassignment.cpp
@@ -553,7 +553,7 @@ void X86DomainReassignmentImpl::buildClosure(Closure &C, Register Reg) {
     // Register already in this closure.
     if (!C.insertEdge(CurReg))
       continue;
-    EnclosedEdges[Reg] = C.getID();
+    EnclosedEdges[CurReg] = C.getID();
 
     MachineInstr *DefMI = MRI->getVRegDef(CurReg);
     if (!encloseInstr(C, DefMI))