[llvm] [RISCV] Add CodeGen only version of conditional branch instructions t… (PR #87834)

Craig Topper via llvm-commits llvm-commits at lists.llvm.org
Fri Apr 5 14:58:33 PDT 2024


https://github.com/topperc created https://github.com/llvm/llvm-project/pull/87834

…hat are not relaxed by MC layer.

This is a possible fix for #87127.

As described in that bug -O0 implicitly adds -mrelax-all. This causes all branches to be relaxed by the MC layer to a short branch over a longer jump. On large functions, this can increase code size so much that we need to relax an unconditional branch to an indirect branch. The MC layer is not able to create an indirect branch since it needs a scratch register.

I believe this -O0 implying -mrelax-all behavior is a compile time optimization to prevent iterative relaxing fragments at -O0. By relaxing all, they will resolve in two passes through layout.

RISC-V uses the BranchRelaxation pass so we don't need MC layer relaxation for correctness on compiled code. With 1 exception, we do need to relax compressed branches to uncompressed branches.

This patch proposes to add unrelaxable CodeGenOnly branches so that the MC layer will ignore them. This keeps the compile time benefit of -mrelax-all without the size bloat.

Alternatively, we could add some flag from CodeGen to MC to disable the relaxation, but that has some layering challengs and may interact poorly with inline assembly.

Note there are still broken cases even with this patch. Relaxing compressed branches for -mrelax-all can still increase the size too much. We could maybe make BranchRelaxation aware of -mrelax-all to prevent this.

>From 630ee9804752bf606a66e2e2cfb9b2841d1e3822 Mon Sep 17 00:00:00 2001
From: Craig Topper <craig.topper at sifive.com>
Date: Fri, 5 Apr 2024 14:39:24 -0700
Subject: [PATCH] [RISCV] Add CodeGen only version of conditional branch
 instructions that are not relaxed by MC layer.

This is a possible fix for #87127.

As described in that bug -O0 implicitly adds -mrelax-all. This causes
all branches to be relaxed by the MC layer to a short branch over a longer
jump. On large functions, this can increase code size so much that we need
to relax an unconditional branch to an indirect branch. The MC layer is not
able to create an indirect branch since it needs a scratch register.

I believe this -O0 implying -mrelax-all behavior is a compile time optimization
to prevent iterative relaxing fragments at -O0. By relaxing all, they will resolve
in two passes through layout.

RISC-V uses the BranchRelaxation pass so we don't need MC layer relaxation for
correctness on compiled code. With 1 exception, we do need to relax compressed
branches to uncompressed branches.

This patch proposes to add unrelaxable CodeGenOnly branches so that the MC
layer will ignore them. This keeps the compile time benefit of -mrelax-all
without the size bloat.

Alternatively, we could add some flag from CodeGen to MC to disable the relaxation,
but that has some layering challengs and may interact poorly with inline assembly.

Note there are still broken cases even with this patch. Relaxing compressed
branches for -mrelax-all can still increase the size too much. We could maybe make
BranchRelaxation aware of -mrelax-all to prevent this.
---
 .../RISCV/MCTargetDesc/RISCVAsmBackend.cpp    | 11 +++++
 llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp     | 10 ++---
 .../RISCV/RISCVExpandAtomicPseudoInsts.cpp    | 16 +++----
 llvm/lib/Target/RISCV/RISCVISelLowering.cpp   |  4 +-
 llvm/lib/Target/RISCV/RISCVInstrInfo.cpp      | 36 +++++++--------
 llvm/lib/Target/RISCV/RISCVInstrInfo.td       | 40 +++++++++++++----
 llvm/lib/Target/RISCV/RISCVInstrInfoC.td      | 25 ++++++++++-
 .../RISCV/RISCVRedundantCopyElimination.cpp   |  4 +-
 .../instruction-select/brcond-rv32.mir        | 26 +++++------
 .../instruction-select/brcond-rv64.mir        | 26 +++++------
 .../jump-table-brjt-medium-rv64.mir           |  2 +-
 .../jump-table-brjt-pic-rv32.mir              |  2 +-
 .../jump-table-brjt-pic-rv64.mir              |  2 +-
 .../jump-table-brjt-rv32.mir                  |  4 +-
 .../jump-table-brjt-small-rv64.mir            |  2 +-
 .../instruction-select/phi-rv32.mir           |  4 +-
 .../instruction-select/phi-rv64.mir           |  4 +-
 .../CodeGen/RISCV/MachineSink-implicit-x0.mir |  2 +-
 llvm/test/CodeGen/RISCV/branch-opt.mir        |  4 +-
 llvm/test/CodeGen/RISCV/copy-frameindex.mir   |  4 +-
 .../test/CodeGen/RISCV/float-select-verify.ll |  8 ++--
 llvm/test/CodeGen/RISCV/rvv/copyprop.mir      |  4 +-
 .../test/CodeGen/RISCV/rvv/emergency-slot.mir |  4 +-
 .../RISCV/rvv/vsetvli-insert-crossbb.mir      | 44 +++++++++----------
 .../test/CodeGen/RISCV/rvv/vsetvli-insert.mir |  8 ++--
 .../rvv/wrong-stack-offset-for-rvv-object.mir | 28 ++----------
 .../RISCV/select-optimize-multiple.mir        | 14 +++---
 .../RISCV/zcmp-prolog-epilog-crash.mir        |  4 +-
 28 files changed, 189 insertions(+), 153 deletions(-)

diff --git a/llvm/lib/Target/RISCV/MCTargetDesc/RISCVAsmBackend.cpp b/llvm/lib/Target/RISCV/MCTargetDesc/RISCVAsmBackend.cpp
index 182a9c1544fd25..ceb42802b21b89 100644
--- a/llvm/lib/Target/RISCV/MCTargetDesc/RISCVAsmBackend.cpp
+++ b/llvm/lib/Target/RISCV/MCTargetDesc/RISCVAsmBackend.cpp
@@ -183,7 +183,9 @@ void RISCVAsmBackend::relaxInstruction(MCInst &Inst,
   default:
     llvm_unreachable("Opcode not expected!");
   case RISCV::C_BEQZ:
+  case RISCV::PseudoC_BEQZ:
   case RISCV::C_BNEZ:
+  case RISCV::PseudoC_BNEZ:
   case RISCV::C_J:
   case RISCV::C_JAL: {
     [[maybe_unused]] bool Success = RISCVRVC::uncompress(Res, Inst, STI);
@@ -358,9 +360,18 @@ unsigned RISCVAsmBackend::getRelaxedOpcode(unsigned Op) const {
     return RISCV::BEQ;
   case RISCV::C_BNEZ:
     return RISCV::BNE;
+  // Compression in RISCVAsmPrinter is not aware of branch distance so we need
+  // to uncompress the PseudoC_BEQZ/BNEZ here.
+  case RISCV::PseudoC_BEQZ:
+    return RISCV::PseudoBEQ;
+  case RISCV::PseudoC_BNEZ:
+    return RISCV::PseudoBNE;
   case RISCV::C_J:
   case RISCV::C_JAL: // fall through.
     return RISCV::JAL;
+  // NOTE: We do not relax the CodeGenOnly PseudoBEQ and friends to
+  // PseudoLongBEQ here. They were already relaxed by the BranchRelaxation pass
+  // as needed.
   case RISCV::BEQ:
     return RISCV::PseudoLongBEQ;
   case RISCV::BNE:
diff --git a/llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp b/llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp
index 5bf594c0b5eae3..a01aa26f2a13cd 100644
--- a/llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp
+++ b/llvm/lib/Target/RISCV/RISCVAsmPrinter.cpp
@@ -541,7 +541,7 @@ void RISCVAsmPrinter::LowerKCFI_CHECK(const MachineInstr &MI) {
   // Compare the hashes and trap if there's a mismatch.
   MCSymbol *Pass = OutContext.createTempSymbol();
   EmitToStreamer(*OutStreamer,
-                 MCInstBuilder(RISCV::BEQ)
+                 MCInstBuilder(RISCV::PseudoBEQ)
                      .addReg(ScratchRegs[0])
                      .addReg(ScratchRegs[1])
                      .addExpr(MCSymbolRefExpr::create(Pass, OutContext)));
@@ -618,7 +618,7 @@ void RISCVAsmPrinter::EmitHwasanMemaccessSymbols(Module &M) {
     MCSymbol *HandleMismatchOrPartialSym = OutContext.createTempSymbol();
     // X7 contains tag from memory, while X6 contains tag from the pointer
     OutStreamer->emitInstruction(
-        MCInstBuilder(RISCV::BNE)
+        MCInstBuilder(RISCV::PseudoBNE)
             .addReg(RISCV::X7)
             .addReg(RISCV::X6)
             .addExpr(MCSymbolRefExpr::create(HandleMismatchOrPartialSym,
@@ -640,7 +640,7 @@ void RISCVAsmPrinter::EmitHwasanMemaccessSymbols(Module &M) {
                                  MCSTI);
     MCSymbol *HandleMismatchSym = OutContext.createTempSymbol();
     OutStreamer->emitInstruction(
-        MCInstBuilder(RISCV::BGEU)
+        MCInstBuilder(RISCV::PseudoBGEU)
             .addReg(RISCV::X6)
             .addReg(RISCV::X28)
             .addExpr(MCSymbolRefExpr::create(HandleMismatchSym, OutContext)),
@@ -657,7 +657,7 @@ void RISCVAsmPrinter::EmitHwasanMemaccessSymbols(Module &M) {
                                        .addImm(Size - 1),
                                    MCSTI);
     OutStreamer->emitInstruction(
-        MCInstBuilder(RISCV::BGE)
+        MCInstBuilder(RISCV::PseudoBGE)
             .addReg(RISCV::X28)
             .addReg(RISCV::X6)
             .addExpr(MCSymbolRefExpr::create(HandleMismatchSym, OutContext)),
@@ -670,7 +670,7 @@ void RISCVAsmPrinter::EmitHwasanMemaccessSymbols(Module &M) {
         MCInstBuilder(RISCV::LBU).addReg(RISCV::X6).addReg(RISCV::X6).addImm(0),
         MCSTI);
     OutStreamer->emitInstruction(
-        MCInstBuilder(RISCV::BEQ)
+        MCInstBuilder(RISCV::PseudoBEQ)
             .addReg(RISCV::X6)
             .addReg(RISCV::X7)
             .addExpr(MCSymbolRefExpr::create(ReturnSym, OutContext)),
diff --git a/llvm/lib/Target/RISCV/RISCVExpandAtomicPseudoInsts.cpp b/llvm/lib/Target/RISCV/RISCVExpandAtomicPseudoInsts.cpp
index bb772fc5da9224..eddaddbfa9ccaf 100644
--- a/llvm/lib/Target/RISCV/RISCVExpandAtomicPseudoInsts.cpp
+++ b/llvm/lib/Target/RISCV/RISCVExpandAtomicPseudoInsts.cpp
@@ -291,7 +291,7 @@ static void doAtomicBinOpExpansion(const RISCVInstrInfo *TII, MachineInstr &MI,
   BuildMI(LoopMBB, DL, TII->get(getSCForRMW(Ordering, Width, STI)), ScratchReg)
       .addReg(AddrReg)
       .addReg(ScratchReg);
-  BuildMI(LoopMBB, DL, TII->get(RISCV::BNE))
+  BuildMI(LoopMBB, DL, TII->get(RISCV::PseudoBNE))
       .addReg(ScratchReg)
       .addReg(RISCV::X0)
       .addMBB(LoopMBB);
@@ -379,7 +379,7 @@ static void doMaskedAtomicBinOpExpansion(const RISCVInstrInfo *TII,
   BuildMI(LoopMBB, DL, TII->get(getSCForRMW32(Ordering, STI)), ScratchReg)
       .addReg(AddrReg)
       .addReg(ScratchReg);
-  BuildMI(LoopMBB, DL, TII->get(RISCV::BNE))
+  BuildMI(LoopMBB, DL, TII->get(RISCV::PseudoBNE))
       .addReg(ScratchReg)
       .addReg(RISCV::X0)
       .addMBB(LoopMBB);
@@ -498,7 +498,7 @@ bool RISCVExpandAtomicPseudo::expandAtomicMinMaxOp(
     llvm_unreachable("Unexpected AtomicRMW BinOp");
   case AtomicRMWInst::Max: {
     insertSext(TII, DL, LoopHeadMBB, Scratch2Reg, MI.getOperand(6).getReg());
-    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGE))
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::PseudoBGE))
         .addReg(Scratch2Reg)
         .addReg(IncrReg)
         .addMBB(LoopTailMBB);
@@ -506,20 +506,20 @@ bool RISCVExpandAtomicPseudo::expandAtomicMinMaxOp(
   }
   case AtomicRMWInst::Min: {
     insertSext(TII, DL, LoopHeadMBB, Scratch2Reg, MI.getOperand(6).getReg());
-    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGE))
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::PseudoBGE))
         .addReg(IncrReg)
         .addReg(Scratch2Reg)
         .addMBB(LoopTailMBB);
     break;
   }
   case AtomicRMWInst::UMax:
-    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGEU))
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::PseudoBGEU))
         .addReg(Scratch2Reg)
         .addReg(IncrReg)
         .addMBB(LoopTailMBB);
     break;
   case AtomicRMWInst::UMin:
-    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::BGEU))
+    BuildMI(LoopHeadMBB, DL, TII->get(RISCV::PseudoBGEU))
         .addReg(IncrReg)
         .addReg(Scratch2Reg)
         .addMBB(LoopTailMBB);
@@ -539,7 +539,7 @@ bool RISCVExpandAtomicPseudo::expandAtomicMinMaxOp(
   BuildMI(LoopTailMBB, DL, TII->get(getSCForRMW32(Ordering, STI)), Scratch1Reg)
       .addReg(AddrReg)
       .addReg(Scratch1Reg);
-  BuildMI(LoopTailMBB, DL, TII->get(RISCV::BNE))
+  BuildMI(LoopTailMBB, DL, TII->get(RISCV::PseudoBNE))
       .addReg(Scratch1Reg)
       .addReg(RISCV::X0)
       .addMBB(LoopHeadMBB);
@@ -594,7 +594,7 @@ bool tryToFoldBNEOnCmpXchgResult(MachineBasicBlock &MBB,
   }
 
   // Match BNE DestReg, MaskReg.
-  if (MBBI == E || MBBI->getOpcode() != RISCV::BNE)
+  if (MBBI == E || MBBI->getOpcode() != RISCV::PseudoBNE)
     return false;
   Register BNEOp0 = MBBI->getOperand(0).getReg();
   Register BNEOp1 = MBBI->getOperand(1).getReg();
diff --git a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
index 279d8a435a04ca..5aa36b97393359 100644
--- a/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
+++ b/llvm/lib/Target/RISCV/RISCVISelLowering.cpp
@@ -17265,7 +17265,7 @@ static MachineBasicBlock *emitReadCounterWidePseudo(MachineInstr &MI,
       .addImm(HiCounter)
       .addReg(RISCV::X0);
 
-  BuildMI(LoopMBB, DL, TII->get(RISCV::BNE))
+  BuildMI(LoopMBB, DL, TII->get(RISCV::PseudoBNE))
       .addReg(HiReg)
       .addReg(ReadAgainReg)
       .addMBB(LoopMBB);
@@ -17794,7 +17794,7 @@ static MachineBasicBlock *emitFROUND(MachineInstr &MI, MachineBasicBlock *MBB,
     MIB->setFlag(MachineInstr::MIFlag::NoFPExcept);
 
   // Insert branch.
-  BuildMI(MBB, DL, TII.get(RISCV::BEQ))
+  BuildMI(MBB, DL, TII.get(RISCV::PseudoBEQ))
       .addReg(CmpReg)
       .addReg(RISCV::X0)
       .addMBB(DoneMBB);
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp b/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp
index 5582de51b17d19..d950fed07e2cda 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfo.cpp
@@ -876,17 +876,17 @@ static RISCVCC::CondCode getCondFromBranchOpc(unsigned Opc) {
   switch (Opc) {
   default:
     return RISCVCC::COND_INVALID;
-  case RISCV::BEQ:
+  case RISCV::PseudoBEQ:
     return RISCVCC::COND_EQ;
-  case RISCV::BNE:
+  case RISCV::PseudoBNE:
     return RISCVCC::COND_NE;
-  case RISCV::BLT:
+  case RISCV::PseudoBLT:
     return RISCVCC::COND_LT;
-  case RISCV::BGE:
+  case RISCV::PseudoBGE:
     return RISCVCC::COND_GE;
-  case RISCV::BLTU:
+  case RISCV::PseudoBLTU:
     return RISCVCC::COND_LTU;
-  case RISCV::BGEU:
+  case RISCV::PseudoBGEU:
     return RISCVCC::COND_GEU;
   }
 }
@@ -911,17 +911,17 @@ unsigned RISCVCC::getBrCond(RISCVCC::CondCode CC) {
   default:
     llvm_unreachable("Unknown condition code!");
   case RISCVCC::COND_EQ:
-    return RISCV::BEQ;
+    return RISCV::PseudoBEQ;
   case RISCVCC::COND_NE:
-    return RISCV::BNE;
+    return RISCV::PseudoBNE;
   case RISCVCC::COND_LT:
-    return RISCV::BLT;
+    return RISCV::PseudoBLT;
   case RISCVCC::COND_GE:
-    return RISCV::BGE;
+    return RISCV::PseudoBGE;
   case RISCVCC::COND_LTU:
-    return RISCV::BLTU;
+    return RISCV::PseudoBLTU;
   case RISCVCC::COND_GEU:
-    return RISCV::BGEU;
+    return RISCV::PseudoBGEU;
   }
 }
 
@@ -1294,12 +1294,12 @@ bool RISCVInstrInfo::isBranchOffsetInRange(unsigned BranchOp,
   switch (BranchOp) {
   default:
     llvm_unreachable("Unexpected opcode!");
-  case RISCV::BEQ:
-  case RISCV::BNE:
-  case RISCV::BLT:
-  case RISCV::BGE:
-  case RISCV::BLTU:
-  case RISCV::BGEU:
+  case RISCV::PseudoBEQ:
+  case RISCV::PseudoBNE:
+  case RISCV::PseudoBLT:
+  case RISCV::PseudoBGE:
+  case RISCV::PseudoBLTU:
+  case RISCV::PseudoBGEU:
     return isIntN(13, BrOffset);
   case RISCV::JAL:
   case RISCV::PseudoBR:
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfo.td b/llvm/lib/Target/RISCV/RISCVInstrInfo.td
index 2cfad7f7c06110..08076f4b8566cb 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfo.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfo.td
@@ -628,6 +628,17 @@ def BGE  : BranchCC_rri<0b101, "bge">;
 def BLTU : BranchCC_rri<0b110, "bltu">;
 def BGEU : BranchCC_rri<0b111, "bgeu">;
 
+let isCodeGenOnly = 1 in {
+// CodeGenOnly versions of the above that are ignored by MC layer relaxing to
+// PseudoLongB*. We use the BranchRelaxation pass for CodeGen.
+def PseudoBEQ  : BranchCC_rri<0b000, "beq">;
+def PseudoBNE  : BranchCC_rri<0b001, "bne">;
+def PseudoBLT  : BranchCC_rri<0b100, "blt">;
+def PseudoBGE  : BranchCC_rri<0b101, "bge">;
+def PseudoBLTU : BranchCC_rri<0b110, "bltu">;
+def PseudoBGEU : BranchCC_rri<0b111, "bgeu">;
+}
+
 let IsSignExtendingOpW = 1 in {
 def LB  : Load_ri<0b000, "lb">, Sched<[WriteLDB, ReadMemBase]>;
 def LH  : Load_ri<0b001, "lh">, Sched<[WriteLDH, ReadMemBase]>;
@@ -912,6 +923,19 @@ def : InstAlias<"bltz $rs, $offset",
 def : InstAlias<"bgtz $rs, $offset",
                 (BLT      X0, GPR:$rs, simm13_lsb0:$offset)>;
 
+def : InstAlias<"beqz $rs, $offset",
+                (PseudoBEQ GPR:$rs,      X0, simm13_lsb0:$offset)>;
+def : InstAlias<"bnez $rs, $offset",
+                (PseudoBNE GPR:$rs,      X0, simm13_lsb0:$offset)>;
+def : InstAlias<"blez $rs, $offset",
+                (PseudoBGE      X0, GPR:$rs, simm13_lsb0:$offset)>;
+def : InstAlias<"bgez $rs, $offset",
+                (PseudoBGE GPR:$rs,      X0, simm13_lsb0:$offset)>;
+def : InstAlias<"bltz $rs, $offset",
+                (PseudoBLT GPR:$rs,      X0, simm13_lsb0:$offset)>;
+def : InstAlias<"bgtz $rs, $offset",
+                (PseudoBLT      X0, GPR:$rs, simm13_lsb0:$offset)>;
+
 // Always output the canonical mnemonic for the pseudo branch instructions.
 // The GNU tools emit the canonical mnemonic for the branch pseudo instructions
 // as well (e.g. "bgt" will be recognised by the assembler but never printed by
@@ -1412,16 +1436,16 @@ class BrccCompressOpt<CondCode Cond, RVInstB Inst>
           (Inst (XLenVT (ADDI GPR:$lhs, (NegImm simm12:$Constant))),
                 (XLenVT X0), bb:$place)>;
 
-defm : BccPat<SETEQ, BEQ>;
-defm : BccPat<SETNE, BNE>;
-defm : BccPat<SETLT, BLT>;
-defm : BccPat<SETGE, BGE>;
-defm : BccPat<SETULT, BLTU>;
-defm : BccPat<SETUGE, BGEU>;
+defm : BccPat<SETEQ, PseudoBEQ>;
+defm : BccPat<SETNE, PseudoBNE>;
+defm : BccPat<SETLT, PseudoBLT>;
+defm : BccPat<SETGE, PseudoBGE>;
+defm : BccPat<SETULT, PseudoBLTU>;
+defm : BccPat<SETUGE, PseudoBGEU>;
 
 let Predicates = [HasStdExtCOrZca, OptForMinSize] in {
-  def : BrccCompressOpt<SETEQ, BEQ>;
-  def : BrccCompressOpt<SETNE, BNE>;
+  def : BrccCompressOpt<SETEQ, PseudoBEQ>;
+  def : BrccCompressOpt<SETNE, PseudoBNE>;
 }
 
 class LongBccPseudo : Pseudo<(outs),
diff --git a/llvm/lib/Target/RISCV/RISCVInstrInfoC.td b/llvm/lib/Target/RISCV/RISCVInstrInfoC.td
index 18d38348f7214f..a0ef668cd86e70 100644
--- a/llvm/lib/Target/RISCV/RISCVInstrInfoC.td
+++ b/llvm/lib/Target/RISCV/RISCVInstrInfoC.td
@@ -495,6 +495,13 @@ def C_J : RVInst16CJ<0b101, 0b01, (outs), (ins simm12_lsb0:$offset),
 def C_BEQZ : Bcz<0b110, "c.beqz", GPRC>, Sched<[WriteJmp, ReadJmp]>;
 def C_BNEZ : Bcz<0b111, "c.bnez", GPRC>, Sched<[WriteJmp, ReadJmp]>;
 
+// CodeGenOnly versions of the above that compress/uncompress to/from
+// PseudoBEQ/BNE.
+let isCodeGenOnly = 1 in {
+def PseudoC_BEQZ : Bcz<0b110, "c.beqz", GPRC>, Sched<[WriteJmp, ReadJmp]>;
+def PseudoC_BNEZ : Bcz<0b111, "c.bnez", GPRC>, Sched<[WriteJmp, ReadJmp]>;
+}
+
 let hasSideEffects = 0, mayLoad = 0, mayStore = 0 in
 def C_SLLI : RVInst16CI<0b000, 0b10, (outs GPRNoX0:$rd_wb),
                         (ins GPRNoX0:$rd, uimmlog2xlennonzero:$imm),
@@ -956,14 +963,28 @@ def : CompressPat<(JAL X0, simm12_lsb0:$offset),
                   (C_J simm12_lsb0:$offset)>;
 def : CompressPat<(BEQ GPRC:$rs1, X0, simm9_lsb0:$imm),
                   (C_BEQZ GPRC:$rs1, simm9_lsb0:$imm)>;
-let isCompressOnly = true in
+def : CompressPat<(PseudoBEQ GPRC:$rs1, X0, simm9_lsb0:$imm),
+                  (PseudoC_BEQZ GPRC:$rs1, simm9_lsb0:$imm)>;
+let isCompressOnly = true in {
 def : CompressPat<(BEQ X0, GPRC:$rs1, simm9_lsb0:$imm),
                   (C_BEQZ GPRC:$rs1, simm9_lsb0:$imm)>;
+def : CompressPat<(PseudoBEQ X0, GPRC:$rs1, simm9_lsb0:$imm),
+                  (PseudoC_BEQZ GPRC:$rs1, simm9_lsb0:$imm)>;
+}
 def : CompressPat<(BNE GPRC:$rs1, X0, simm9_lsb0:$imm),
                   (C_BNEZ GPRC:$rs1, simm9_lsb0:$imm)>;
-let isCompressOnly = true in
+def : CompressPat<(PseudoBNE GPRC:$rs1, X0, simm9_lsb0:$imm),
+                  (PseudoC_BNEZ GPRC:$rs1, simm9_lsb0:$imm)>;
+def : CompressPat<(BNE GPRC:$rs1, X0, simm9_lsb0:$imm),
+                  (C_BNEZ GPRC:$rs1, simm9_lsb0:$imm)>;
+def : CompressPat<(PseudoBNE GPRC:$rs1, X0, simm9_lsb0:$imm),
+                  (PseudoC_BNEZ GPRC:$rs1, simm9_lsb0:$imm)>;
+let isCompressOnly = true in {
 def : CompressPat<(BNE X0, GPRC:$rs1, simm9_lsb0:$imm),
                   (C_BNEZ GPRC:$rs1, simm9_lsb0:$imm)>;
+def : CompressPat<(PseudoBNE X0, GPRC:$rs1, simm9_lsb0:$imm),
+                  (PseudoC_BNEZ GPRC:$rs1, simm9_lsb0:$imm)>;
+}
 } // Predicates = [HasStdExtCOrZca]
 
 // Quadrant 2
diff --git a/llvm/lib/Target/RISCV/RISCVRedundantCopyElimination.cpp b/llvm/lib/Target/RISCV/RISCVRedundantCopyElimination.cpp
index 61d605fda3f53a..b8333d0fe7ed6c 100644
--- a/llvm/lib/Target/RISCV/RISCVRedundantCopyElimination.cpp
+++ b/llvm/lib/Target/RISCV/RISCVRedundantCopyElimination.cpp
@@ -141,8 +141,8 @@ bool RISCVRedundantCopyElimination::optimizeBlock(MachineBasicBlock &MBB) {
     return false;
 
   MachineBasicBlock::iterator CondBr = PredMBB->getFirstTerminator();
-  assert((CondBr->getOpcode() == RISCV::BEQ ||
-          CondBr->getOpcode() == RISCV::BNE) &&
+  assert((CondBr->getOpcode() == RISCV::PseudoBEQ ||
+          CondBr->getOpcode() == RISCV::PseudoBNE) &&
          "Unexpected opcode");
   assert(CondBr->getOperand(0).getReg() == TargetReg && "Unexpected register");
 
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv32.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv32.mir
index 13c4d49b3c4039..03719a5b6120cc 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv32.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv32.mir
@@ -16,72 +16,72 @@ body:             |
   ; RV32I-NEXT:   [[COPY1:%[0-9]+]]:gpr = COPY $x11
   ; RV32I-NEXT:   [[COPY2:%[0-9]+]]:gpr = COPY $x12
   ; RV32I-NEXT:   [[LW:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BEQ [[LW]], [[COPY]], %bb.14
+  ; RV32I-NEXT:   PseudoBEQ [[LW]], [[COPY]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.1
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.1:
   ; RV32I-NEXT:   [[LW1:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BNE [[LW1]], [[COPY]], %bb.14
+  ; RV32I-NEXT:   PseudoBNE [[LW1]], [[COPY]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.2
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.2:
   ; RV32I-NEXT:   [[LW2:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BLT [[LW2]], [[COPY]], %bb.14
+  ; RV32I-NEXT:   PseudoBLT [[LW2]], [[COPY]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.3
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.3:
   ; RV32I-NEXT:   [[LW3:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BGE [[LW3]], [[COPY]], %bb.14
+  ; RV32I-NEXT:   PseudoBGE [[LW3]], [[COPY]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.4
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.4:
   ; RV32I-NEXT:   [[LW4:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BLTU [[LW4]], [[COPY]], %bb.14
+  ; RV32I-NEXT:   PseudoBLTU [[LW4]], [[COPY]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.5
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.5:
   ; RV32I-NEXT:   [[LW5:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BGEU [[LW5]], [[COPY]], %bb.14
+  ; RV32I-NEXT:   PseudoBGEU [[LW5]], [[COPY]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.6
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.6:
   ; RV32I-NEXT:   [[LW6:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BLT [[COPY]], [[LW6]], %bb.14
+  ; RV32I-NEXT:   PseudoBLT [[COPY]], [[LW6]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.7
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.7:
   ; RV32I-NEXT:   [[LW7:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BGE [[COPY]], [[LW7]], %bb.14
+  ; RV32I-NEXT:   PseudoBGE [[COPY]], [[LW7]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.8
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.8:
   ; RV32I-NEXT:   [[LW8:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BLTU [[COPY]], [[LW8]], %bb.14
+  ; RV32I-NEXT:   PseudoBLTU [[COPY]], [[LW8]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.9
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.9:
   ; RV32I-NEXT:   [[LW9:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BGEU [[COPY]], [[LW9]], %bb.14
+  ; RV32I-NEXT:   PseudoBGEU [[COPY]], [[LW9]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.10
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.10:
   ; RV32I-NEXT:   [[LW10:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
   ; RV32I-NEXT:   [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY2]], 1
-  ; RV32I-NEXT:   BNE [[ANDI]], $x0, %bb.14
+  ; RV32I-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.11
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.11:
   ; RV32I-NEXT:   successors: %bb.14(0x50000000), %bb.12(0x30000000)
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT:   [[LW11:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BGE [[LW11]], $x0, %bb.14
+  ; RV32I-NEXT:   PseudoBGE [[LW11]], $x0, %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.12
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.12:
   ; RV32I-NEXT:   successors: %bb.14(0x30000000), %bb.13(0x50000000)
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT:   [[LW12:%[0-9]+]]:gpr = LW [[COPY1]], 0 :: (volatile load (s32))
-  ; RV32I-NEXT:   BGE $x0, [[LW12]], %bb.14
+  ; RV32I-NEXT:   PseudoBGE $x0, [[LW12]], %bb.14
   ; RV32I-NEXT:   PseudoBR %bb.13
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.13:
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv64.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv64.mir
index de13516e552b12..fd48569f6f1c9e 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv64.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/brcond-rv64.mir
@@ -16,72 +16,72 @@ body:             |
   ; RV64I-NEXT:   [[COPY1:%[0-9]+]]:gpr = COPY $x11
   ; RV64I-NEXT:   [[COPY2:%[0-9]+]]:gpr = COPY $x12
   ; RV64I-NEXT:   [[LD:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BEQ [[LD]], [[COPY]], %bb.14
+  ; RV64I-NEXT:   PseudoBEQ [[LD]], [[COPY]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.1
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.1:
   ; RV64I-NEXT:   [[LD1:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BNE [[LD1]], [[COPY]], %bb.14
+  ; RV64I-NEXT:   PseudoBNE [[LD1]], [[COPY]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.2
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.2:
   ; RV64I-NEXT:   [[LD2:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BLT [[LD2]], [[COPY]], %bb.14
+  ; RV64I-NEXT:   PseudoBLT [[LD2]], [[COPY]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.3
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.3:
   ; RV64I-NEXT:   [[LD3:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BGE [[LD3]], [[COPY]], %bb.14
+  ; RV64I-NEXT:   PseudoBGE [[LD3]], [[COPY]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.4
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.4:
   ; RV64I-NEXT:   [[LD4:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BLTU [[LD4]], [[COPY]], %bb.14
+  ; RV64I-NEXT:   PseudoBLTU [[LD4]], [[COPY]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.5
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.5:
   ; RV64I-NEXT:   [[LD5:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BGEU [[LD5]], [[COPY]], %bb.14
+  ; RV64I-NEXT:   PseudoBGEU [[LD5]], [[COPY]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.6
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.6:
   ; RV64I-NEXT:   [[LD6:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BLT [[COPY]], [[LD6]], %bb.14
+  ; RV64I-NEXT:   PseudoBLT [[COPY]], [[LD6]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.7
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.7:
   ; RV64I-NEXT:   [[LD7:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BGE [[COPY]], [[LD7]], %bb.14
+  ; RV64I-NEXT:   PseudoBGE [[COPY]], [[LD7]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.8
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.8:
   ; RV64I-NEXT:   [[LD8:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BLTU [[COPY]], [[LD8]], %bb.14
+  ; RV64I-NEXT:   PseudoBLTU [[COPY]], [[LD8]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.9
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.9:
   ; RV64I-NEXT:   [[LD9:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BGEU [[COPY]], [[LD9]], %bb.14
+  ; RV64I-NEXT:   PseudoBGEU [[COPY]], [[LD9]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.10
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.10:
   ; RV64I-NEXT:   [[LD10:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
   ; RV64I-NEXT:   [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY2]], 1
-  ; RV64I-NEXT:   BNE [[ANDI]], $x0, %bb.14
+  ; RV64I-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.11
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.11:
   ; RV64I-NEXT:   successors: %bb.14(0x50000000), %bb.12(0x30000000)
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT:   [[LD11:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BGE [[LD11]], $x0, %bb.14
+  ; RV64I-NEXT:   PseudoBGE [[LD11]], $x0, %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.12
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.12:
   ; RV64I-NEXT:   successors: %bb.14(0x30000000), %bb.13(0x50000000)
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT:   [[LD12:%[0-9]+]]:gpr = LD [[COPY1]], 0 :: (volatile load (s64))
-  ; RV64I-NEXT:   BGE $x0, [[LD12]], %bb.14
+  ; RV64I-NEXT:   PseudoBGE $x0, [[LD12]], %bb.14
   ; RV64I-NEXT:   PseudoBR %bb.13
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.13:
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-medium-rv64.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-medium-rv64.mir
index 5d980e7721458e..1df1578c8b4e41 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-medium-rv64.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-medium-rv64.mir
@@ -58,7 +58,7 @@ body:             |
   ; CHECK-NEXT:   [[ADDI:%[0-9]+]]:gpr = ADDI $x0, 5
   ; CHECK-NEXT:   [[ADDIW:%[0-9]+]]:gpr = ADDIW [[COPY]], 0
   ; CHECK-NEXT:   [[ADDI1:%[0-9]+]]:gpr = ADDI [[ADDIW]], -1
-  ; CHECK-NEXT:   BLTU [[ADDI]], [[ADDI1]], %bb.8
+  ; CHECK-NEXT:   PseudoBLTU [[ADDI]], [[ADDI1]], %bb.8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.entry:
   ; CHECK-NEXT:   successors: %bb.2(0x15555555), %bb.3(0x15555555), %bb.4(0x15555555), %bb.5(0x15555555), %bb.6(0x15555555), %bb.7(0x15555555)
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv32.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv32.mir
index 27fe465ccf696b..1b59f005405697 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv32.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv32.mir
@@ -63,7 +63,7 @@ body:             |
   ; CHECK-NEXT:   [[ADDI6:%[0-9]+]]:gpr = ADDI $x0, 4
   ; CHECK-NEXT:   [[ADDI7:%[0-9]+]]:gpr = ADDI $x0, 1000
   ; CHECK-NEXT:   [[ADDI8:%[0-9]+]]:gpr = ADDI [[COPY]], -1
-  ; CHECK-NEXT:   BLTU [[ADDI]], [[ADDI8]], %bb.8
+  ; CHECK-NEXT:   PseudoBLTU [[ADDI]], [[ADDI8]], %bb.8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.entry:
   ; CHECK-NEXT:   successors: %bb.2(0x15555555), %bb.3(0x15555555), %bb.4(0x15555555), %bb.5(0x15555555), %bb.6(0x15555555), %bb.7(0x15555555)
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv64.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv64.mir
index 77156b913c5e8b..cf38d7d9fb2e1b 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv64.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-pic-rv64.mir
@@ -58,7 +58,7 @@ body:             |
   ; CHECK-NEXT:   [[ADDI:%[0-9]+]]:gpr = ADDI $x0, 5
   ; CHECK-NEXT:   [[ADDIW:%[0-9]+]]:gpr = ADDIW [[COPY]], 0
   ; CHECK-NEXT:   [[ADDI1:%[0-9]+]]:gpr = ADDI [[ADDIW]], -1
-  ; CHECK-NEXT:   BLTU [[ADDI]], [[ADDI1]], %bb.8
+  ; CHECK-NEXT:   PseudoBLTU [[ADDI]], [[ADDI1]], %bb.8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.entry:
   ; CHECK-NEXT:   successors: %bb.2(0x15555555), %bb.3(0x15555555), %bb.4(0x15555555), %bb.5(0x15555555), %bb.6(0x15555555), %bb.7(0x15555555)
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-rv32.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-rv32.mir
index 388c238b86eb6f..0097263b658f98 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-rv32.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-rv32.mir
@@ -65,7 +65,7 @@ body:             |
   ; RV32-SMALL-NEXT:   [[ADDI6:%[0-9]+]]:gpr = ADDI $x0, 4
   ; RV32-SMALL-NEXT:   [[ADDI7:%[0-9]+]]:gpr = ADDI $x0, 1000
   ; RV32-SMALL-NEXT:   [[ADDI8:%[0-9]+]]:gpr = ADDI [[COPY]], -1
-  ; RV32-SMALL-NEXT:   BLTU [[ADDI]], [[ADDI8]], %bb.8
+  ; RV32-SMALL-NEXT:   PseudoBLTU [[ADDI]], [[ADDI8]], %bb.8
   ; RV32-SMALL-NEXT: {{  $}}
   ; RV32-SMALL-NEXT: bb.1.entry:
   ; RV32-SMALL-NEXT:   successors: %bb.2(0x15555555), %bb.3(0x15555555), %bb.4(0x15555555), %bb.5(0x15555555), %bb.6(0x15555555), %bb.7(0x15555555)
@@ -120,7 +120,7 @@ body:             |
   ; RV32-MEDIUM-NEXT:   [[ADDI6:%[0-9]+]]:gpr = ADDI $x0, 4
   ; RV32-MEDIUM-NEXT:   [[ADDI7:%[0-9]+]]:gpr = ADDI $x0, 1000
   ; RV32-MEDIUM-NEXT:   [[ADDI8:%[0-9]+]]:gpr = ADDI [[COPY]], -1
-  ; RV32-MEDIUM-NEXT:   BLTU [[ADDI]], [[ADDI8]], %bb.8
+  ; RV32-MEDIUM-NEXT:   PseudoBLTU [[ADDI]], [[ADDI8]], %bb.8
   ; RV32-MEDIUM-NEXT: {{  $}}
   ; RV32-MEDIUM-NEXT: bb.1.entry:
   ; RV32-MEDIUM-NEXT:   successors: %bb.2(0x15555555), %bb.3(0x15555555), %bb.4(0x15555555), %bb.5(0x15555555), %bb.6(0x15555555), %bb.7(0x15555555)
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-small-rv64.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-small-rv64.mir
index 09a855105c2627..07aeda55a127b3 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-small-rv64.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/jump-table-brjt-small-rv64.mir
@@ -58,7 +58,7 @@ body:             |
   ; CHECK-NEXT:   [[ADDI:%[0-9]+]]:gpr = ADDI $x0, 5
   ; CHECK-NEXT:   [[ADDIW:%[0-9]+]]:gpr = ADDIW [[COPY]], 0
   ; CHECK-NEXT:   [[ADDI1:%[0-9]+]]:gpr = ADDI [[ADDIW]], -1
-  ; CHECK-NEXT:   BLTU [[ADDI]], [[ADDI1]], %bb.8
+  ; CHECK-NEXT:   PseudoBLTU [[ADDI]], [[ADDI1]], %bb.8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.entry:
   ; CHECK-NEXT:   successors: %bb.2(0x15555555), %bb.3(0x15555555), %bb.4(0x15555555), %bb.5(0x15555555), %bb.6(0x15555555), %bb.7(0x15555555)
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv32.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv32.mir
index 459511eb94358c..fc994e0f426d62 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv32.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv32.mir
@@ -16,7 +16,7 @@ body:             |
   ; RV32I-NEXT:   [[COPY1:%[0-9]+]]:gpr = COPY $x11
   ; RV32I-NEXT:   [[COPY2:%[0-9]+]]:gpr = COPY $x12
   ; RV32I-NEXT:   [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY]], 1
-  ; RV32I-NEXT:   BNE [[ANDI]], $x0, %bb.2
+  ; RV32I-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.2
   ; RV32I-NEXT:   PseudoBR %bb.1
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.1:
@@ -58,7 +58,7 @@ body:             |
   ; RV32I-NEXT:   [[COPY1:%[0-9]+]]:gpr = COPY $x11
   ; RV32I-NEXT:   [[COPY2:%[0-9]+]]:gpr = COPY $x12
   ; RV32I-NEXT:   [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY]], 1
-  ; RV32I-NEXT:   BNE [[ANDI]], $x0, %bb.2
+  ; RV32I-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.2
   ; RV32I-NEXT:   PseudoBR %bb.1
   ; RV32I-NEXT: {{  $}}
   ; RV32I-NEXT: bb.1:
diff --git a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv64.mir b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv64.mir
index 8512af316f5410..af58856d14520b 100644
--- a/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv64.mir
+++ b/llvm/test/CodeGen/RISCV/GlobalISel/instruction-select/phi-rv64.mir
@@ -16,7 +16,7 @@ body:             |
   ; RV64I-NEXT:   [[COPY1:%[0-9]+]]:gpr = COPY $x11
   ; RV64I-NEXT:   [[COPY2:%[0-9]+]]:gpr = COPY $x12
   ; RV64I-NEXT:   [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY]], 1
-  ; RV64I-NEXT:   BNE [[ANDI]], $x0, %bb.2
+  ; RV64I-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.2
   ; RV64I-NEXT:   PseudoBR %bb.1
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.1:
@@ -58,7 +58,7 @@ body:             |
   ; RV64I-NEXT:   [[COPY1:%[0-9]+]]:gpr = COPY $x11
   ; RV64I-NEXT:   [[COPY2:%[0-9]+]]:gpr = COPY $x12
   ; RV64I-NEXT:   [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY]], 1
-  ; RV64I-NEXT:   BNE [[ANDI]], $x0, %bb.2
+  ; RV64I-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.2
   ; RV64I-NEXT:   PseudoBR %bb.1
   ; RV64I-NEXT: {{  $}}
   ; RV64I-NEXT: bb.1:
diff --git a/llvm/test/CodeGen/RISCV/MachineSink-implicit-x0.mir b/llvm/test/CodeGen/RISCV/MachineSink-implicit-x0.mir
index ef006fca60a401..85103ffc9429d8 100644
--- a/llvm/test/CodeGen/RISCV/MachineSink-implicit-x0.mir
+++ b/llvm/test/CodeGen/RISCV/MachineSink-implicit-x0.mir
@@ -20,7 +20,7 @@ body:             |
     liveins: $x10
     %10:gpr = COPY $x10
     %20:gpr = LUI 1, implicit $x0
-    BEQ %10, %10, %bb.2
+    PseudoBEQ %10, %10, %bb.2
     PseudoBR %bb.1
   bb.1:
     liveins: $x0
diff --git a/llvm/test/CodeGen/RISCV/branch-opt.mir b/llvm/test/CodeGen/RISCV/branch-opt.mir
index ba3a20f2fbfcd3..5e6d200a6f48e2 100644
--- a/llvm/test/CodeGen/RISCV/branch-opt.mir
+++ b/llvm/test/CodeGen/RISCV/branch-opt.mir
@@ -36,7 +36,7 @@ body:             |
   ; CHECK-NEXT:   PseudoCALL target-flags(riscv-call) @bar, csr_ilp32_lp64, implicit-def dead $x1, implicit $x10, implicit-def $x2
   ; CHECK-NEXT:   ADJCALLSTACKUP 0, 0, implicit-def dead $x2, implicit $x2
   ; CHECK-NEXT:   [[ADDI:%[0-9]+]]:gpr = ADDI $x0, 2
-  ; CHECK-NEXT:   BLTU [[COPY]], killed [[ADDI]], %bb.2
+  ; CHECK-NEXT:   PseudoBLTU [[COPY]], killed [[ADDI]], %bb.2
   ; CHECK-NEXT:   PseudoBR %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1 (%ir-block.3):
@@ -55,7 +55,7 @@ body:             |
     PseudoCALL target-flags(riscv-call) @bar, csr_ilp32_lp64, implicit-def dead $x1, implicit $x10, implicit-def $x2
     ADJCALLSTACKUP 0, 0, implicit-def dead $x2, implicit $x2
     %2:gpr = ADDI $x0, 2
-    BLTU %0, killed %2, %bb.2
+    PseudoBLTU %0, killed %2, %bb.2
     PseudoBR %bb.1
 
   bb.1 (%ir-block.3):
diff --git a/llvm/test/CodeGen/RISCV/copy-frameindex.mir b/llvm/test/CodeGen/RISCV/copy-frameindex.mir
index 31ffc3f0f83c64..634957826bc67d 100644
--- a/llvm/test/CodeGen/RISCV/copy-frameindex.mir
+++ b/llvm/test/CodeGen/RISCV/copy-frameindex.mir
@@ -43,7 +43,7 @@ body: |
   ; CHECK-NEXT:   liveins: $x10
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[COPY:%[0-9]+]]:gpr = COPY $x10
-  ; CHECK-NEXT:   BEQ killed [[COPY]], $x0, %bb.2
+  ; CHECK-NEXT:   PseudoBEQ killed [[COPY]], $x0, %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1:
   ; CHECK-NEXT:   successors: %bb.2(0x80000000)
@@ -59,7 +59,7 @@ body: |
     DBG_VALUE %stack.0, $noreg, !1, !DIExpression(), debug-location !3
     %1:gpr = ADDI %stack.0, 0
     DBG_VALUE %1, $noreg, !1, !DIExpression(DW_OP_plus_uconst, 0, DW_OP_stack_value), debug-location !3
-    BEQ killed %0:gpr, $x0, %bb.2
+    PseudoBEQ killed %0:gpr, $x0, %bb.2
   bb.1:
     SW $x0, killed %1:gpr, 0 :: (volatile store (s32) into %stack.0, align 4)
   bb.2:
diff --git a/llvm/test/CodeGen/RISCV/float-select-verify.ll b/llvm/test/CodeGen/RISCV/float-select-verify.ll
index 2d5d6d7cb48255..c20a457b84108d 100644
--- a/llvm/test/CodeGen/RISCV/float-select-verify.ll
+++ b/llvm/test/CodeGen/RISCV/float-select-verify.ll
@@ -17,7 +17,7 @@ define dso_local void @buz(i1 %pred, float %a, float %b) {
   ; CHECK-NEXT:   [[FMV_W_X2:%[0-9]+]]:fpr32 = FMV_W_X killed [[LUI]]
   ; CHECK-NEXT:   [[FSGNJX_S:%[0-9]+]]:fpr32 = FSGNJX_S [[FMV_W_X1]], [[FMV_W_X1]]
   ; CHECK-NEXT:   [[FLT_S:%[0-9]+]]:gpr = nofpexcept FLT_S [[FSGNJX_S]], [[FMV_W_X2]]
-  ; CHECK-NEXT:   BEQ [[FLT_S]], $x0, %bb.2
+  ; CHECK-NEXT:   PseudoBEQ [[FLT_S]], $x0, %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.entry:
   ; CHECK-NEXT:   successors: %bb.2(0x80000000)
@@ -30,7 +30,7 @@ define dso_local void @buz(i1 %pred, float %a, float %b) {
   ; CHECK-NEXT:   successors: %bb.3(0x40000000), %bb.4(0x40000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[PHI:%[0-9]+]]:fpr32 = PHI [[FMV_W_X1]], %bb.0, [[FSGNJ_S]], %bb.1
-  ; CHECK-NEXT:   BNE [[ANDI]], $x0, %bb.4
+  ; CHECK-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3.entry:
   ; CHECK-NEXT:   successors: %bb.4(0x80000000)
@@ -41,7 +41,7 @@ define dso_local void @buz(i1 %pred, float %a, float %b) {
   ; CHECK-NEXT:   [[PHI1:%[0-9]+]]:fpr32 = PHI [[PHI]], %bb.2, [[FMV_W_X1]], %bb.3
   ; CHECK-NEXT:   [[FSGNJX_S1:%[0-9]+]]:fpr32 = FSGNJX_S [[FMV_W_X]], [[FMV_W_X]]
   ; CHECK-NEXT:   [[FLT_S1:%[0-9]+]]:gpr = nofpexcept FLT_S [[FSGNJX_S1]], [[FMV_W_X2]]
-  ; CHECK-NEXT:   BEQ [[FLT_S1]], $x0, %bb.6
+  ; CHECK-NEXT:   PseudoBEQ [[FLT_S1]], $x0, %bb.6
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.5.entry:
   ; CHECK-NEXT:   successors: %bb.6(0x80000000)
@@ -54,7 +54,7 @@ define dso_local void @buz(i1 %pred, float %a, float %b) {
   ; CHECK-NEXT:   successors: %bb.7(0x40000000), %bb.8(0x40000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[PHI2:%[0-9]+]]:fpr32 = PHI [[FMV_W_X]], %bb.4, [[FSGNJ_S1]], %bb.5
-  ; CHECK-NEXT:   BNE [[ANDI]], $x0, %bb.8
+  ; CHECK-NEXT:   PseudoBNE [[ANDI]], $x0, %bb.8
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.7.entry:
   ; CHECK-NEXT:   successors: %bb.8(0x80000000)
diff --git a/llvm/test/CodeGen/RISCV/rvv/copyprop.mir b/llvm/test/CodeGen/RISCV/rvv/copyprop.mir
index eb4c8bfdd67f9a..aee3f3dabd288e 100644
--- a/llvm/test/CodeGen/RISCV/rvv/copyprop.mir
+++ b/llvm/test/CodeGen/RISCV/rvv/copyprop.mir
@@ -54,7 +54,7 @@ body:             |
     %29:vr = PseudoVSSRA_VI_M1 %pt8, %3, 2, 0, 1, 6 /* e64 */, 0
     %pt9:vr = IMPLICIT_DEF
     %30:vr = PseudoVMV_V_I_M1 %pt9, 0, 1, 6 /* e64 */, 0
-    BGEU %1, $x0, %bb.2
+    PseudoBGEU %1, $x0, %bb.2
 
   bb.1.entry:
 
@@ -63,7 +63,7 @@ body:             |
 
     %31:vr = PHI %30, %bb.0, %29, %bb.1
     %32:gpr = ADDI $x0, 128
-    BGEU %1, %32, %bb.4
+    PseudoBGEU %1, %32, %bb.4
 
   bb.3.entry:
 
diff --git a/llvm/test/CodeGen/RISCV/rvv/emergency-slot.mir b/llvm/test/CodeGen/RISCV/rvv/emergency-slot.mir
index 600084632ce68a..7e83e1119b445e 100644
--- a/llvm/test/CodeGen/RISCV/rvv/emergency-slot.mir
+++ b/llvm/test/CodeGen/RISCV/rvv/emergency-slot.mir
@@ -140,7 +140,7 @@ body:             |
   ; CHECK-NEXT:   $x10 = LD $x2, 8 :: (load (s64) from %stack.15)
   ; CHECK-NEXT:   renamable $v0 = PseudoVSLIDEDOWN_VX_M1 undef renamable $v0, killed renamable $v0, killed renamable $x13, $noreg, 3 /* e8 */, 1 /* ta, mu */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   renamable $x13 = PseudoVMV_X_S killed renamable $v0, 3 /* e8 */, implicit $vl, implicit $vtype
-  ; CHECK-NEXT:   BLT killed renamable $x16, renamable $x27, %bb.2
+  ; CHECK-NEXT:   PseudoBLT killed renamable $x16, renamable $x27, %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1:
   ; CHECK-NEXT:   successors: %bb.2(0x80000000)
@@ -207,7 +207,7 @@ body:             |
     renamable $v0 = VL1RE8_V %stack.1 :: (load unknown-size from %stack.1, align 8)
     renamable $v0 = PseudoVSLIDEDOWN_VX_M1 undef renamable $v0, killed renamable $v0, killed renamable $x13, $noreg, 3, 1, implicit $vl, implicit $vtype
     renamable $x13 = PseudoVMV_X_S killed renamable $v0, 3, implicit $vl, implicit $vtype
-    BLT killed renamable $x16, renamable $x27, %bb.2
+    PseudoBLT killed renamable $x16, renamable $x27, %bb.2
 
   bb.1:
     successors: %bb.2
diff --git a/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert-crossbb.mir b/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert-crossbb.mir
index fdcce72a01eb3d..83d63aa644f159 100644
--- a/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert-crossbb.mir
+++ b/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert-crossbb.mir
@@ -189,7 +189,7 @@ body:             |
   ; CHECK-NEXT:   dead $x0 = PseudoVSETVLI [[COPY]], 216 /* e64, m1, ta, ma */, implicit-def $vl, implicit-def $vtype
   ; CHECK-NEXT:   [[PseudoVLE64_V_M1_:%[0-9]+]]:vr = PseudoVLE64_V_M1 %pt, [[COPY2]], $noreg, 6 /* e64 */, 0 /* tu, mu */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[COPY4:%[0-9]+]]:gpr = COPY $x0
-  ; CHECK-NEXT:   BEQ [[COPY3]], [[COPY4]], %bb.2
+  ; CHECK-NEXT:   PseudoBEQ [[COPY3]], [[COPY4]], %bb.2
   ; CHECK-NEXT:   PseudoBR %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.if.then:
@@ -220,7 +220,7 @@ body:             |
     %pt:vr = IMPLICIT_DEF
     %0:vr = PseudoVLE64_V_M1 %pt, %5, %7, 6, 0
     %8:gpr = COPY $x0
-    BEQ %4, %8, %bb.2
+    PseudoBEQ %4, %8, %bb.2
     PseudoBR %bb.1
 
   bb.1.if.then:
@@ -274,7 +274,7 @@ body:             |
   ; CHECK-NEXT:   dead $x0 = PseudoVSETVLI [[COPY]], 215 /* e32, mf2, ta, ma */, implicit-def $vl, implicit-def $vtype
   ; CHECK-NEXT:   [[PseudoVLE32_V_MF2_:%[0-9]+]]:vr = PseudoVLE32_V_MF2 %pt, [[COPY2]], $noreg, 5 /* e32 */, 0 /* tu, mu */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[COPY4:%[0-9]+]]:gpr = COPY $x0
-  ; CHECK-NEXT:   BEQ [[COPY3]], [[COPY4]], %bb.2
+  ; CHECK-NEXT:   PseudoBEQ [[COPY3]], [[COPY4]], %bb.2
   ; CHECK-NEXT:   PseudoBR %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.if.then:
@@ -307,7 +307,7 @@ body:             |
     %pt:vr = IMPLICIT_DEF
     %0:vr = PseudoVLE32_V_MF2 %pt, %5, %7, 5, 0
     %8:gpr = COPY $x0
-    BEQ %4, %8, %bb.2
+    PseudoBEQ %4, %8, %bb.2
     PseudoBR %bb.1
 
   bb.1.if.then:
@@ -358,7 +358,7 @@ body:             |
   ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:vr = COPY $v8
   ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:gpr = COPY $x10
   ; CHECK-NEXT:   [[COPY4:%[0-9]+]]:gpr = COPY $x0
-  ; CHECK-NEXT:   BEQ [[COPY3]], [[COPY4]], %bb.2
+  ; CHECK-NEXT:   PseudoBEQ [[COPY3]], [[COPY4]], %bb.2
   ; CHECK-NEXT:   PseudoBR %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.if.then:
@@ -390,7 +390,7 @@ body:             |
     %4:vr = COPY $v8
     %3:gpr = COPY $x10
     %7:gpr = COPY $x0
-    BEQ %3, %7, %bb.2
+    PseudoBEQ %3, %7, %bb.2
     PseudoBR %bb.1
 
   bb.1.if.then:
@@ -443,7 +443,7 @@ body:             |
   ; CHECK-NEXT:   [[COPY3:%[0-9]+]]:gpr = COPY $x10
   ; CHECK-NEXT:   [[PseudoVSETVLI:%[0-9]+]]:gprnox0 = PseudoVSETVLI [[COPY]], 88 /* e64, m1, ta, mu */, implicit-def $vl, implicit-def $vtype
   ; CHECK-NEXT:   [[COPY4:%[0-9]+]]:gpr = COPY $x0
-  ; CHECK-NEXT:   BEQ [[COPY3]], [[COPY4]], %bb.2
+  ; CHECK-NEXT:   PseudoBEQ [[COPY3]], [[COPY4]], %bb.2
   ; CHECK-NEXT:   PseudoBR %bb.1
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1.if.then:
@@ -473,7 +473,7 @@ body:             |
     %4:gpr = COPY $x10
     %0:gprnox0 = PseudoVSETVLI %7, 88, implicit-def dead $vl, implicit-def dead $vtype
     %8:gpr = COPY $x0
-    BEQ %4, %8, %bb.2
+    PseudoBEQ %4, %8, %bb.2
     PseudoBR %bb.1
 
   bb.1.if.then:
@@ -533,7 +533,7 @@ body:             |
   ; CHECK-NEXT:   dead $x0 = PseudoVSETVLIX0 killed $x0, 197 /* e8, mf8, ta, ma */, implicit-def $vl, implicit-def $vtype, implicit $vl
   ; CHECK-NEXT:   [[PseudoVCPOP_M_B1_:%[0-9]+]]:gpr = PseudoVCPOP_M_B1 [[PseudoVMSEQ_VI_MF2_]], -1, 0 /* e8 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[COPY2:%[0-9]+]]:gpr = COPY $x0
-  ; CHECK-NEXT:   BEQ killed [[PseudoVCPOP_M_B1_]], [[COPY2]], %bb.3
+  ; CHECK-NEXT:   PseudoBEQ killed [[PseudoVCPOP_M_B1_]], [[COPY2]], %bb.3
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2:
@@ -568,7 +568,7 @@ body:             |
     %6:vrnov0 = PseudoVLE32_V_MF2_MASK %4, killed %0, $v0, -1, 5, 0
     %7:gpr = PseudoVCPOP_M_B1 %5, -1, 0
     %8:gpr = COPY $x0
-    BEQ killed %7, %8, %bb.3
+    PseudoBEQ killed %7, %8, %bb.3
     PseudoBR %bb.2
 
   bb.2:
@@ -623,7 +623,7 @@ body:             |
   ; CHECK-NEXT:   [[ADD:%[0-9]+]]:gpr = ADD [[COPY]], [[MUL]]
   ; CHECK-NEXT:   PseudoVSE32_V_MF2 killed [[PseudoVADD_VX_M1_]], killed [[ADD]], -1, 5 /* e32 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[ADDI:%[0-9]+]]:gpr = ADDI [[PHI]], 1
-  ; CHECK-NEXT:   BLTU [[ADDI]], [[COPY1]], %bb.1
+  ; CHECK-NEXT:   PseudoBLTU [[ADDI]], [[COPY1]], %bb.1
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2:
@@ -648,7 +648,7 @@ body:             |
     %9:gpr = ADD %0:gpr, %8:gpr
     PseudoVSE32_V_MF2 killed %7:vr, killed %9:gpr, -1, 5
     %10:gpr = ADDI %6:gpr, 1
-    BLTU %10:gpr, %3:gpr, %bb.1
+    PseudoBLTU %10:gpr, %3:gpr, %bb.1
     PseudoBR %bb.2
 
   bb.2:
@@ -699,7 +699,7 @@ body:             |
   ; CHECK-NEXT: bb.2:
   ; CHECK-NEXT:   successors: %bb.1(0x40000000), %bb.3(0x40000000)
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT:   BLTU [[ADDI]], [[COPY1]], %bb.1
+  ; CHECK-NEXT:   PseudoBLTU [[ADDI]], [[COPY1]], %bb.1
   ; CHECK-NEXT:   PseudoBR %bb.3
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3:
@@ -727,7 +727,7 @@ body:             |
 
   bb.3:
     successors: %bb.1, %bb.2
-    BLTU %10:gpr, %3:gpr, %bb.1
+    PseudoBLTU %10:gpr, %3:gpr, %bb.1
     PseudoBR %bb.2
 
   bb.2:
@@ -799,7 +799,7 @@ body:             |
   ; CHECK-NEXT:   [[ADDI:%[0-9]+]]:gpr = nsw ADDI [[PHI1]], -4
   ; CHECK-NEXT:   [[ADDI1:%[0-9]+]]:gpr = ADDI [[PHI]], 16
   ; CHECK-NEXT:   [[COPY4:%[0-9]+]]:gpr = COPY $x0
-  ; CHECK-NEXT:   BNE [[ADDI]], [[COPY4]], %bb.1
+  ; CHECK-NEXT:   PseudoBNE [[ADDI]], [[COPY4]], %bb.1
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2.middle.block:
@@ -836,7 +836,7 @@ body:             |
     %4:gpr = nsw ADDI %1, -4
     %5:gpr = ADDI %0, 16
     %18:gpr = COPY $x0
-    BNE %4, %18, %bb.1
+    PseudoBNE %4, %18, %bb.1
     PseudoBR %bb.2
 
   bb.2.middle.block:
@@ -875,7 +875,7 @@ body:             |
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   %mask:vr = PseudoVMANDN_MM_MF8 %t6, %t3, -1, 0 /* e8 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   %t2:gpr = COPY $x0
-  ; CHECK-NEXT:   BEQ %a, %t2, %bb.3
+  ; CHECK-NEXT:   PseudoBEQ %a, %t2, %bb.3
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2:
@@ -913,7 +913,7 @@ body:             |
 
     %mask:vr = PseudoVMANDN_MM_MF8 %t6, %t3, -1, 0
     %t2:gpr = COPY $x0
-    BEQ %a, %t2, %bb.3
+    PseudoBEQ %a, %t2, %bb.3
     PseudoBR %bb.2
 
   bb.2:
@@ -965,7 +965,7 @@ body:             |
   ; CHECK-NEXT:   [[PseudoVMSLTU_VX_M1_:%[0-9]+]]:vr = PseudoVMSLTU_VX_M1 [[PseudoVADD_VX_M1_]], [[COPY1]], -1, 6 /* e64 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[PseudoVCPOP_M_B1_:%[0-9]+]]:gpr = PseudoVCPOP_M_B1 [[PseudoVMSLTU_VX_M1_]], -1, 0 /* e8 */, implicit $vl, implicit $vtype
   ; CHECK-NEXT:   [[COPY4:%[0-9]+]]:gpr = COPY $x0
-  ; CHECK-NEXT:   BEQ killed [[PseudoVCPOP_M_B1_]], [[COPY4]], %bb.3
+  ; CHECK-NEXT:   PseudoBEQ killed [[PseudoVCPOP_M_B1_]], [[COPY4]], %bb.3
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2:
@@ -984,7 +984,7 @@ body:             |
   ; CHECK-NEXT:   successors: %bb.1(0x7c000000), %bb.4(0x04000000)
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT:   [[ADD3:%[0-9]+]]:gpr = ADD [[PHI]], %inc
-  ; CHECK-NEXT:   BLTU [[ADD3]], %tc, %bb.1
+  ; CHECK-NEXT:   PseudoBLTU [[ADD3]], %tc, %bb.1
   ; CHECK-NEXT:   PseudoBR %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.4:
@@ -1016,7 +1016,7 @@ body:             |
     %62:vr = PseudoVMSLTU_VX_M1 %27, %11, -1, 6
     %63:gpr = PseudoVCPOP_M_B1 %62, -1, 0
     %64:gpr = COPY $x0
-    BEQ killed %63, %64, %bb.3
+    PseudoBEQ killed %63, %64, %bb.3
     PseudoBR %bb.2
 
   bb.2:
@@ -1034,7 +1034,7 @@ body:             |
     successors: %bb.1(0x7c000000), %bb.4(0x04000000)
 
     %28:gpr = ADD %26, %inc
-    BLTU %28, %tc, %bb.1
+    PseudoBLTU %28, %tc, %bb.1
     PseudoBR %bb.4
 
   bb.4:
diff --git a/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert.mir b/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert.mir
index 1850abe6363bc9..a6e0c5f77003bf 100644
--- a/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert.mir
+++ b/llvm/test/CodeGen/RISCV/rvv/vsetvli-insert.mir
@@ -464,7 +464,7 @@ body:             |
   ; CHECK-NEXT:   %cond:gpr = COPY $x10
   ; CHECK-NEXT:   dead $x0 = PseudoVSETIVLI 2, 215 /* e32, mf2, ta, ma */, implicit-def $vl, implicit-def $vtype
   ; CHECK-NEXT:   [[PseudoVMV_V_I_MF2_:%[0-9]+]]:vr = PseudoVMV_V_I_MF2 $noreg, 1, 2, 5 /* e32 */, 0 /* tu, mu */, implicit $vl, implicit $vtype
-  ; CHECK-NEXT:   BEQ %cond, $x0, %bb.2
+  ; CHECK-NEXT:   PseudoBEQ %cond, $x0, %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.1:
   ; CHECK-NEXT:   successors: %bb.2(0x80000000)
@@ -475,7 +475,7 @@ body:             |
   ; CHECK-NEXT: bb.2:
   ; CHECK-NEXT:   successors: %bb.4(0x40000000), %bb.3(0x40000000)
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT:   BEQ %cond, $x0, %bb.4
+  ; CHECK-NEXT:   PseudoBEQ %cond, $x0, %bb.4
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.3:
   ; CHECK-NEXT:   successors: %bb.4(0x80000000)
@@ -491,11 +491,11 @@ body:             |
     liveins: $x10
     %cond:gpr = COPY $x10
     %1:vr = PseudoVMV_V_I_MF2 $noreg, 1, 2, 5, 0
-    BEQ %cond, $x0, %bb.2
+    PseudoBEQ %cond, $x0, %bb.2
   bb.1:
     %2:vr = PseudoVMV_V_I_M1 $noreg, 1, 2, 6, 0
   bb.2: ; the exit info here should have sew/lmul ratio only
-    BEQ %cond, $x0, %bb.4
+    PseudoBEQ %cond, $x0, %bb.4
   bb.3:
     PseudoCALL $noreg, csr_ilp32_lp64
   bb.4: ; this block will have PRE attempted on it
diff --git a/llvm/test/CodeGen/RISCV/rvv/wrong-stack-offset-for-rvv-object.mir b/llvm/test/CodeGen/RISCV/rvv/wrong-stack-offset-for-rvv-object.mir
index 8248c26636793e..a2c8572b0cbbdd 100644
--- a/llvm/test/CodeGen/RISCV/rvv/wrong-stack-offset-for-rvv-object.mir
+++ b/llvm/test/CodeGen/RISCV/rvv/wrong-stack-offset-for-rvv-object.mir
@@ -1,3 +1,4 @@
+# NOTE: Assertions have been autogenerated by utils/update_mir_test_checks.py UTC_ARGS: --version 4
 # RUN: llc -mtriple riscv64 -mattr=+m,+v -run-pass=prologepilog \
 # RUN:     -riscv-v-vector-bits-min=512 -o - %s | FileCheck %s
 #
@@ -25,7 +26,6 @@
 # |--------------------------| -- <-- New SP
 
 --- |
-  ; ModuleID = 'wrong-stack-offset-for-rvv-object.ll'
   source_filename = "wrong-stack-offset-for-rvv-object.ll"
   target datalayout = "e-m:e-p:64:64-i64:64-i128:128-n64-S128"
   target triple = "riscv64"
@@ -120,28 +120,6 @@ machineFunctionInfo:
   varArgsSaveSize: 16
 body:             |
   ; CHECK-LABEL: name: asm_fprintf
-  ; CHECK: stack:
-  ; CHECK-NEXT:  - { id: 0, name: buf1, type: default, offset: -48, size: 1, alignment: 8,
-  ; CHECK-NEXT:      stack-id: default, callee-saved-register: '', callee-saved-restored: true,
-  ; CHECK-NEXT:      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
-  ; CHECK-NEXT:  - { id: 1, name: '', type: spill-slot, offset: -16, size: 8, alignment: 8,
-  ; CHECK-NEXT:      stack-id: scalable-vector, callee-saved-register: '', callee-saved-restored: true,
-  ; CHECK-NEXT:      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
-  ; CHECK-NEXT:  - { id: 2, name: '', type: spill-slot, offset: -24, size: 8, alignment: 8,
-  ; CHECK-NEXT:      stack-id: default, callee-saved-register: '$x1', callee-saved-restored: true,
-  ; CHECK-NEXT:      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
-  ; CHECK-NEXT:  - { id: 3, name: '', type: spill-slot, offset: -32, size: 8, alignment: 8,
-  ; CHECK-NEXT:      stack-id: default, callee-saved-register: '$x8', callee-saved-restored: true,
-  ; CHECK-NEXT:      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
-  ; CHECK-NEXT:  - { id: 4, name: '', type: spill-slot, offset: -40, size: 8, alignment: 8,
-  ; CHECK-NEXT:      stack-id: default, callee-saved-register: '$x9', callee-saved-restored: true,
-  ; CHECK-NEXT:      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
-  ; CHECK-NEXT:  - { id: 5, name: '', type: default, offset: -56, size: 8, alignment: 8,
-  ; CHECK-NEXT:      stack-id: default, callee-saved-register: '', callee-saved-restored: true,
-  ; CHECK-NEXT:      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
-  ; CHECK-NEXT:  - { id: 6, name: '', type: default, offset: -64, size: 8, alignment: 8,
-  ; CHECK-NEXT:      stack-id: default, callee-saved-register: '', callee-saved-restored: true,
-  ; CHECK-NEXT:      debug-info-variable: '', debug-info-expression: '', debug-info-location: '' }
   ; CHECK: bb.0.entry:
   ; CHECK-NEXT:   successors: %bb.1(0x80000000)
   ; CHECK-NEXT:   liveins: $x11, $x14, $x16, $x17, $x1, $x8, $x9
@@ -177,7 +155,7 @@ body:             |
   ; CHECK-NEXT:   successors: %bb.2(0x30000000), %bb.1(0x50000000)
   ; CHECK-NEXT:   liveins: $x8, $x9
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT:   BNE $x0, $x0, %bb.1
+  ; CHECK-NEXT:   PseudoBNE $x0, $x0, %bb.1
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2.sw.bb:
@@ -207,7 +185,7 @@ body:             |
     successors: %bb.2(0x30000000), %bb.1(0x50000000)
     liveins: $x8, $x9
 
-    BNE $x0, $x0, %bb.1
+    PseudoBNE $x0, $x0, %bb.1
     PseudoBR %bb.2
 
   bb.2.sw.bb:
diff --git a/llvm/test/CodeGen/RISCV/select-optimize-multiple.mir b/llvm/test/CodeGen/RISCV/select-optimize-multiple.mir
index e536a993a0967d..85efc55c18315a 100644
--- a/llvm/test/CodeGen/RISCV/select-optimize-multiple.mir
+++ b/llvm/test/CodeGen/RISCV/select-optimize-multiple.mir
@@ -52,7 +52,7 @@ body:             |
     ; RV32I-NEXT: [[COPY3:%[0-9]+]]:gpr = COPY $x10
     ; RV32I-NEXT: [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY3]], 1
     ; RV32I-NEXT: [[COPY4:%[0-9]+]]:gpr = COPY $x0
-    ; RV32I-NEXT: BNE [[ANDI]], [[COPY4]], %bb.2
+    ; RV32I-NEXT: PseudoBNE [[ANDI]], [[COPY4]], %bb.2
     ; RV32I-NEXT: {{  $}}
     ; RV32I-NEXT: .1:
     ; RV32I-NEXT: {{  $}}
@@ -61,7 +61,7 @@ body:             |
     ; RV32I-NEXT: {{  $}}
     ; RV32I-NEXT: [[PHI:%[0-9]+]]:gpr = PHI [[COPY2]], %bb.0, [[COPY1]], %bb.1
     ; RV32I-NEXT: [[ADDI:%[0-9]+]]:gpr = ADDI [[PHI]], 1
-    ; RV32I-NEXT: BNE [[ANDI]], [[COPY4]], %bb.4
+    ; RV32I-NEXT: PseudoBNE [[ANDI]], [[COPY4]], %bb.4
     ; RV32I-NEXT: {{  $}}
     ; RV32I-NEXT: .3:
     ; RV32I-NEXT: {{  $}}
@@ -70,6 +70,7 @@ body:             |
     ; RV32I-NEXT: [[ADD:%[0-9]+]]:gpr = ADD [[PHI]], killed [[PHI1]]
     ; RV32I-NEXT: $x10 = COPY [[ADD]]
     ; RV32I-NEXT: PseudoRET implicit $x10
+    ;
     ; RV64I-LABEL: name: cmov_interleaved_bad
     ; RV64I: successors: %bb.1, %bb.2
     ; RV64I-NEXT: liveins: $x10, $x11, $x12, $x13
@@ -80,7 +81,7 @@ body:             |
     ; RV64I-NEXT: [[COPY3:%[0-9]+]]:gpr = COPY $x10
     ; RV64I-NEXT: [[ANDI:%[0-9]+]]:gpr = ANDI [[COPY3]], 1
     ; RV64I-NEXT: [[COPY4:%[0-9]+]]:gpr = COPY $x0
-    ; RV64I-NEXT: BNE [[ANDI]], [[COPY4]], %bb.2
+    ; RV64I-NEXT: PseudoBNE [[ANDI]], [[COPY4]], %bb.2
     ; RV64I-NEXT: {{  $}}
     ; RV64I-NEXT: .1:
     ; RV64I-NEXT: {{  $}}
@@ -89,7 +90,7 @@ body:             |
     ; RV64I-NEXT: {{  $}}
     ; RV64I-NEXT: [[PHI:%[0-9]+]]:gpr = PHI [[COPY2]], %bb.0, [[COPY1]], %bb.1
     ; RV64I-NEXT: [[ADDI:%[0-9]+]]:gpr = ADDI [[PHI]], 1
-    ; RV64I-NEXT: BNE [[ANDI]], [[COPY4]], %bb.4
+    ; RV64I-NEXT: PseudoBNE [[ANDI]], [[COPY4]], %bb.4
     ; RV64I-NEXT: {{  $}}
     ; RV64I-NEXT: .3:
     ; RV64I-NEXT: {{  $}}
@@ -152,7 +153,7 @@ body:             |
     ; RV32I-NEXT: [[COPY4:%[0-9]+]]:gpr = COPY $x0
     ; RV32I-NEXT: [[ADDI:%[0-9]+]]:gpr = ADDI [[COPY3]], 1
     ; RV32I-NEXT: DBG_VALUE [[ADDI]], $noreg
-    ; RV32I-NEXT: BNE [[ANDI]], [[COPY4]], %bb.2
+    ; RV32I-NEXT: PseudoBNE [[ANDI]], [[COPY4]], %bb.2
     ; RV32I-NEXT: {{  $}}
     ; RV32I-NEXT: .1:
     ; RV32I-NEXT: {{  $}}
@@ -164,6 +165,7 @@ body:             |
     ; RV32I-NEXT: [[ADD:%[0-9]+]]:gpr = ADD [[PHI]], killed [[PHI1]]
     ; RV32I-NEXT: $x10 = COPY [[ADD]]
     ; RV32I-NEXT: PseudoRET implicit $x10
+    ;
     ; RV64I-LABEL: name: cmov_interleaved_debug_value
     ; RV64I: successors: %bb.1, %bb.2
     ; RV64I-NEXT: liveins: $x10, $x11, $x12, $x13
@@ -176,7 +178,7 @@ body:             |
     ; RV64I-NEXT: [[COPY4:%[0-9]+]]:gpr = COPY $x0
     ; RV64I-NEXT: [[ADDI:%[0-9]+]]:gpr = ADDI [[COPY3]], 1
     ; RV64I-NEXT: DBG_VALUE [[ADDI]], $noreg
-    ; RV64I-NEXT: BNE [[ANDI]], [[COPY4]], %bb.2
+    ; RV64I-NEXT: PseudoBNE [[ANDI]], [[COPY4]], %bb.2
     ; RV64I-NEXT: {{  $}}
     ; RV64I-NEXT: .1:
     ; RV64I-NEXT: {{  $}}
diff --git a/llvm/test/CodeGen/RISCV/zcmp-prolog-epilog-crash.mir b/llvm/test/CodeGen/RISCV/zcmp-prolog-epilog-crash.mir
index 64556ec0b343af..ddf6e601cfec7b 100644
--- a/llvm/test/CodeGen/RISCV/zcmp-prolog-epilog-crash.mir
+++ b/llvm/test/CodeGen/RISCV/zcmp-prolog-epilog-crash.mir
@@ -55,7 +55,7 @@ body:             |
   ; CHECK-NEXT: bb.1.if.end.i:
   ; CHECK-NEXT:   liveins: $x10
   ; CHECK-NEXT: {{  $}}
-  ; CHECK-NEXT:   BNE $x0, $x0, %bb.3
+  ; CHECK-NEXT:   PseudoBNE $x0, $x0, %bb.3
   ; CHECK-NEXT:   PseudoBR %bb.2
   ; CHECK-NEXT: {{  $}}
   ; CHECK-NEXT: bb.2.if.then12.i:
@@ -110,7 +110,7 @@ body:             |
   bb.1.if.end.i:
     liveins: $x10
 
-    BNE $x0, $x0, %bb.3
+    PseudoBNE $x0, $x0, %bb.3
     PseudoBR %bb.2
 
   bb.2.if.then12.i:



More information about the llvm-commits mailing list