[llvm-branch-commits] [lld] c161775 - [FastISel] Flush local value map on every instruction

Paul Robinson via llvm-branch-commits llvm-branch-commits at lists.llvm.org
Mon Jan 11 08:37:50 PST 2021


Author: Paul Robinson
Date: 2021-01-11T08:32:36-08:00
New Revision: c161775decddcc86fbbfefd7485a5d0ef5842aec

URL: https://github.com/llvm/llvm-project/commit/c161775decddcc86fbbfefd7485a5d0ef5842aec
DIFF: https://github.com/llvm/llvm-project/commit/c161775decddcc86fbbfefd7485a5d0ef5842aec.diff

LOG: [FastISel] Flush local value map on every instruction

Local values are constants or addresses that can't be folded into
the instruction that uses them. FastISel materializes these in a
"local value" area that always dominates the current insertion
point, to try to avoid materializing these values more than once
(per block).

https://reviews.llvm.org/D43093 added code to sink these local
value instructions to their first use, which has two beneficial
effects. One, it is likely to avoid some unnecessary spills and
reloads; two, it allows us to attach the debug location of the
user to the local value instruction. The latter effect can
improve the debugging experience for debuggers with a "set next
statement" feature, such as the Visual Studio debugger and PS4
debugger, because instructions to set up constants for a given
statement will be associated with the appropriate source line.

There are also some constants (primarily addresses) that could be
produced by no-op casts or GEP instructions; the main difference
from "local value" instructions is that these are values from
separate IR instructions, and therefore could have multiple users
across multiple basic blocks. D43093 avoided sinking these, even
though they were emitted to the same "local value" area as the
other instructions. The patch comment for D43093 states:

  Local values may also be used by no-op casts, which adds the
  register to the RegFixups table. Without reversing the RegFixups
  map direction, we don't have enough information to sink these
  instructions.

This patch undoes most of D43093, and instead flushes the local
value map after(*) every IR instruction, using that instruction's
debug location. This avoids sometimes incorrect locations used
previously, and emits instructions in a more natural order.

In addition, constants materialized due to PHI instructions are
not assigned a debug location immediately; instead, when the
local value map is flushed, if the first local value instruction
has no debug location, it is given the same location as the
first non-local-value-map instruction.  This prevents PHIs
from introducing unattributed instructions, which would either
be implicitly attributed to the location for the preceding IR
instruction, or given line 0 if they are at the beginning of
a machine basic block.  Neither of those consequences is good
for debugging.

This does mean materialized values are not re-used across IR
instruction boundaries; however, only about 5% of those values
were reused in an experimental self-build of clang.

(*) Actually, just prior to the next instruction. It seems like
it would be cleaner the other way, but I was having trouble
getting that to work.

This reapplies commits cf1c774d and dc35368c, and adds the
modification to PHI handling, which should avoid problems
with debugging under gdb.

Differential Revision: https://reviews.llvm.org/D91734

Added: 
    llvm/test/CodeGen/X86/fast-isel-prolog-dbgloc.ll

Modified: 
    lld/test/wasm/debug-removed-fn.ll
    lldb/test/Shell/SymbolFile/NativePDB/disassembly.cpp
    lldb/test/Shell/SymbolFile/NativePDB/load-pdb.cpp
    llvm/include/llvm/CodeGen/FastISel.h
    llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
    llvm/test/CodeGen/AArch64/arm64-abi_align.ll
    llvm/test/CodeGen/AArch64/arm64-fast-isel-call.ll
    llvm/test/CodeGen/AArch64/arm64-fast-isel-gv.ll
    llvm/test/CodeGen/AArch64/arm64-fast-isel-intrinsic.ll
    llvm/test/CodeGen/AArch64/arm64-fast-isel.ll
    llvm/test/CodeGen/AArch64/arm64-patchpoint-webkit_jscc.ll
    llvm/test/CodeGen/AArch64/cfguard-checks.ll
    llvm/test/CodeGen/AArch64/elf-globals-static.ll
    llvm/test/CodeGen/AArch64/large-stack.ll
    llvm/test/CodeGen/ARM/fast-isel-call.ll
    llvm/test/CodeGen/ARM/fast-isel-intrinsic.ll
    llvm/test/CodeGen/ARM/fast-isel-ldr-str-thumb-neg-index.ll
    llvm/test/CodeGen/ARM/fast-isel-ldrh-strh-arm.ll
    llvm/test/CodeGen/ARM/fast-isel-select.ll
    llvm/test/CodeGen/ARM/fast-isel.ll
    llvm/test/CodeGen/Mips/Fast-ISel/callabi.ll
    llvm/test/CodeGen/Mips/Fast-ISel/fastalloca.ll
    llvm/test/CodeGen/Mips/Fast-ISel/fpcmpa.ll
    llvm/test/CodeGen/Mips/Fast-ISel/icmpa.ll
    llvm/test/CodeGen/Mips/Fast-ISel/logopm.ll
    llvm/test/CodeGen/Mips/Fast-ISel/overflt.ll
    llvm/test/CodeGen/Mips/Fast-ISel/shftopm.ll
    llvm/test/CodeGen/Mips/Fast-ISel/simplestore.ll
    llvm/test/CodeGen/Mips/Fast-ISel/simplestorei.ll
    llvm/test/CodeGen/Mips/emergency-spill-slot-near-fp.ll
    llvm/test/CodeGen/PowerPC/elf-common.ll
    llvm/test/CodeGen/PowerPC/fast-isel-load-store.ll
    llvm/test/CodeGen/PowerPC/mcm-1.ll
    llvm/test/CodeGen/PowerPC/mcm-13.ll
    llvm/test/CodeGen/PowerPC/mcm-2.ll
    llvm/test/CodeGen/PowerPC/mcm-3.ll
    llvm/test/CodeGen/PowerPC/mcm-6.ll
    llvm/test/CodeGen/PowerPC/mcm-9.ll
    llvm/test/CodeGen/PowerPC/mcm-default.ll
    llvm/test/CodeGen/X86/atomic-unordered.ll
    llvm/test/CodeGen/X86/atomic64.ll
    llvm/test/CodeGen/X86/crash-O0.ll
    llvm/test/CodeGen/X86/fast-isel-mem.ll
    llvm/test/CodeGen/X86/fast-isel-select.ll
    llvm/test/CodeGen/X86/lvi-hardening-loads.ll
    llvm/test/CodeGen/X86/membarrier.ll
    llvm/test/CodeGen/X86/pr32241.ll
    llvm/test/CodeGen/X86/pr32256.ll
    llvm/test/CodeGen/X86/pr32284.ll
    llvm/test/CodeGen/X86/pr32340.ll
    llvm/test/CodeGen/X86/pr44749.ll
    llvm/test/CodeGen/X86/volatile.ll
    llvm/test/DebugInfo/COFF/lines-bb-start.ll
    llvm/test/DebugInfo/Mips/delay-slot.ll
    llvm/test/DebugInfo/X86/fission-ranges.ll

Removed: 
    llvm/test/CodeGen/X86/fast-isel-constant.ll


################################################################################
diff  --git a/lld/test/wasm/debug-removed-fn.ll b/lld/test/wasm/debug-removed-fn.ll
index 9d2c5dc107de..8dae48a136dc 100644
--- a/lld/test/wasm/debug-removed-fn.ll
+++ b/lld/test/wasm/debug-removed-fn.ll
@@ -3,7 +3,7 @@
 ; RUN: llvm-dwarfdump -debug-line -debug-ranges %t.wasm | FileCheck %s
 
 ; CHECK: Address
-; CHECK: 0x0000000000000005
+; CHECK: 0x0000000000000003
 ; CHECK-NEXT: 0x0000000000000006
 ; CHECK-EMPTY:
 

diff  --git a/lldb/test/Shell/SymbolFile/NativePDB/disassembly.cpp b/lldb/test/Shell/SymbolFile/NativePDB/disassembly.cpp
index 8d101ba280e8..d5365f1832fb 100644
--- a/lldb/test/Shell/SymbolFile/NativePDB/disassembly.cpp
+++ b/lldb/test/Shell/SymbolFile/NativePDB/disassembly.cpp
@@ -28,9 +28,9 @@ int main(int argc, char **argv) {
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+17>: mov    dword ptr [rsp + 0x24], ecx
 // CHECK:      ** 15     foo();
 // CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+21>: call   {{.*}}               ; foo at disassembly.cpp:12
-// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+26>: xor    eax, eax
 // CHECK:      ** 16     return 0;
 // CHECK-NEXT:    17   }
 // CHECK-NEXT:    18
-// CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+28>: add    rsp, 0x38
+// CHECK:      disassembly.cpp.tmp.exe[{{.*}}] <+26>: xor    eax, eax
+// CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+28>: add    rsp, 0x38
 // CHECK-NEXT: disassembly.cpp.tmp.exe[{{.*}}] <+32>: ret

diff  --git a/lldb/test/Shell/SymbolFile/NativePDB/load-pdb.cpp b/lldb/test/Shell/SymbolFile/NativePDB/load-pdb.cpp
index 2ffcece075b0..8840a1242a04 100644
--- a/lldb/test/Shell/SymbolFile/NativePDB/load-pdb.cpp
+++ b/lldb/test/Shell/SymbolFile/NativePDB/load-pdb.cpp
@@ -22,7 +22,7 @@ int main(int argc, char** argv) {
 // CHECK: (lldb) target symbols add {{.*}}bar.pdb
 // CHECK: symbol file '{{.*}}bar.pdb' has been added to '{{.*}}foo.exe'
 // CHECK: (lldb) b main
-// CHECK: Breakpoint 1: where = foo.exe`main + 23 at load-pdb.cpp:19, address = 0x0000000140001017
+// CHECK: Breakpoint 1: where = foo.exe`main + 21 at load-pdb.cpp:19, address = 0x0000000140001015
 // CHECK: (lldb) image dump symfile
 // CHECK: Types:
 // CHECK: {{.*}}: Type{0x00010024} , size = 0, compiler_type = {{.*}} int (int, char **)

diff  --git a/llvm/include/llvm/CodeGen/FastISel.h b/llvm/include/llvm/CodeGen/FastISel.h
index 0463272ea0a8..d20f443e3013 100644
--- a/llvm/include/llvm/CodeGen/FastISel.h
+++ b/llvm/include/llvm/CodeGen/FastISel.h
@@ -246,7 +246,7 @@ class FastISel {
   /// be appended.
   void startNewBlock();
 
-  /// Flush the local value map and sink local values if possible.
+  /// Flush the local value map.
   void finishBasicBlock();
 
   /// Return current debug location information.
@@ -313,10 +313,7 @@ class FastISel {
   void removeDeadCode(MachineBasicBlock::iterator I,
                       MachineBasicBlock::iterator E);
 
-  struct SavePoint {
-    MachineBasicBlock::iterator InsertPt;
-    DebugLoc DL;
-  };
+  using SavePoint = MachineBasicBlock::iterator;
 
   /// Prepare InsertPt to begin inserting instructions into the local
   /// value area and return the old insert position.
@@ -559,20 +556,6 @@ class FastISel {
   /// Removes dead local value instructions after SavedLastLocalvalue.
   void removeDeadLocalValueCode(MachineInstr *SavedLastLocalValue);
 
-  struct InstOrderMap {
-    DenseMap<MachineInstr *, unsigned> Orders;
-    MachineInstr *FirstTerminator = nullptr;
-    unsigned FirstTerminatorOrder = std::numeric_limits<unsigned>::max();
-
-    void initialize(MachineBasicBlock *MBB,
-                    MachineBasicBlock::iterator LastFlushPoint);
-  };
-
-  /// Sinks the local value materialization instruction LocalMI to its first use
-  /// in the basic block, or deletes it if it is not used.
-  void sinkLocalValueMaterialization(MachineInstr &LocalMI, Register DefReg,
-                                     InstOrderMap &OrderMap);
-
   /// Insertion point before trying to select the current instruction.
   MachineBasicBlock::iterator SavedInsertPt;
 

diff  --git a/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp b/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
index 6cf0be5c6533..27e359a5c696 100644
--- a/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/FastISel.cpp
@@ -139,7 +139,6 @@ void FastISel::startNewBlock() {
   LastLocalValue = EmitStartPt;
 }
 
-/// Flush the local CSE map and sink anything we can.
 void FastISel::finishBasicBlock() { flushLocalValueMap(); }
 
 bool FastISel::lowerArguments() {
@@ -164,48 +163,77 @@ bool FastISel::lowerArguments() {
 
 /// Return the defined register if this instruction defines exactly one
 /// virtual register and uses no other virtual registers. Otherwise return 0.
-static Register findSinkableLocalRegDef(MachineInstr &MI) {
+static Register findLocalRegDef(MachineInstr &MI) {
   Register RegDef;
   for (const MachineOperand &MO : MI.operands()) {
     if (!MO.isReg())
       continue;
     if (MO.isDef()) {
       if (RegDef)
-        return 0;
+        return Register();
       RegDef = MO.getReg();
     } else if (MO.getReg().isVirtual()) {
-      // This is another use of a vreg. Don't try to sink it.
+      // This is another use of a vreg. Don't delete it.
       return Register();
     }
   }
   return RegDef;
 }
 
+static bool isRegUsedByPhiNodes(Register DefReg,
+                                FunctionLoweringInfo &FuncInfo) {
+  for (auto &P : FuncInfo.PHINodesToUpdate)
+    if (P.second == DefReg)
+      return true;
+  return false;
+}
+
 void FastISel::flushLocalValueMap() {
-  // Try to sink local values down to their first use so that we can give them a
-  // better debug location. This has the side effect of shrinking local value
-  // live ranges, which helps out fast regalloc.
-  if (SinkLocalValues && LastLocalValue != EmitStartPt) {
-    // Sink local value materialization instructions between EmitStartPt and
-    // LastLocalValue. Visit them bottom-up, starting from LastLocalValue, to
-    // avoid inserting into the range that we're iterating over.
+  // If FastISel bails out, it could leave local value instructions behind
+  // that aren't used for anything.  Detect and erase those.
+  if (LastLocalValue != EmitStartPt) {
+    // Save the first instruction after local values, for later.
+    MachineBasicBlock::iterator FirstNonValue(LastLocalValue);
+    ++FirstNonValue;
+
     MachineBasicBlock::reverse_iterator RE =
         EmitStartPt ? MachineBasicBlock::reverse_iterator(EmitStartPt)
                     : FuncInfo.MBB->rend();
     MachineBasicBlock::reverse_iterator RI(LastLocalValue);
-
-    InstOrderMap OrderMap;
     for (; RI != RE;) {
       MachineInstr &LocalMI = *RI;
+      // Increment before erasing what it points to.
       ++RI;
-      bool Store = true;
-      if (!LocalMI.isSafeToMove(nullptr, Store))
+      Register DefReg = findLocalRegDef(LocalMI);
+      if (!DefReg)
         continue;
-      Register DefReg = findSinkableLocalRegDef(LocalMI);
-      if (DefReg == 0)
+      if (FuncInfo.RegsWithFixups.count(DefReg))
         continue;
+      bool UsedByPHI = isRegUsedByPhiNodes(DefReg, FuncInfo);
+      if (!UsedByPHI && MRI.use_nodbg_empty(DefReg)) {
+        if (EmitStartPt == &LocalMI)
+          EmitStartPt = EmitStartPt->getPrevNode();
+        LLVM_DEBUG(dbgs() << "removing dead local value materialization"
+                          << LocalMI);
+        LocalMI.eraseFromParent();
+      }
+    }
 
-      sinkLocalValueMaterialization(LocalMI, DefReg, OrderMap);
+    if (FirstNonValue != FuncInfo.MBB->end()) {
+      // See if there are any local value instructions left.  If so, we want to
+      // make sure the first one has a debug location; if it doesn't, use the
+      // first non-value instruction's debug location.
+
+      // If EmitStartPt is non-null, this block had copies at the top before
+      // FastISel started doing anything; it points to the last one, so the
+      // first local value instruction is the one after EmitStartPt.
+      // If EmitStartPt is null, the first local value instruction is at the
+      // top of the block.
+      MachineBasicBlock::iterator FirstLocalValue =
+          EmitStartPt ? ++MachineBasicBlock::iterator(EmitStartPt)
+                      : FuncInfo.MBB->begin();
+      if (FirstLocalValue != FirstNonValue && !FirstLocalValue->getDebugLoc())
+        FirstLocalValue->setDebugLoc(FirstNonValue->getDebugLoc());
     }
   }
 
@@ -216,131 +244,6 @@ void FastISel::flushLocalValueMap() {
   LastFlushPoint = FuncInfo.InsertPt;
 }
 
-static bool isRegUsedByPhiNodes(Register DefReg,
-                                FunctionLoweringInfo &FuncInfo) {
-  for (auto &P : FuncInfo.PHINodesToUpdate)
-    if (P.second == DefReg)
-      return true;
-  return false;
-}
-
-static bool isTerminatingEHLabel(MachineBasicBlock *MBB, MachineInstr &MI) {
-  // Ignore non-EH labels.
-  if (!MI.isEHLabel())
-    return false;
-
-  // Any EH label outside a landing pad must be for an invoke. Consider it a
-  // terminator.
-  if (!MBB->isEHPad())
-    return true;
-
-  // If this is a landingpad, the first non-phi instruction will be an EH_LABEL.
-  // Don't consider that label to be a terminator.
-  return MI.getIterator() != MBB->getFirstNonPHI();
-}
-
-/// Build a map of instruction orders. Return the first terminator and its
-/// order. Consider EH_LABEL instructions to be terminators as well, since local
-/// values for phis after invokes must be materialized before the call.
-void FastISel::InstOrderMap::initialize(
-    MachineBasicBlock *MBB, MachineBasicBlock::iterator LastFlushPoint) {
-  unsigned Order = 0;
-  for (MachineInstr &I : *MBB) {
-    if (!FirstTerminator &&
-        (I.isTerminator() || isTerminatingEHLabel(MBB, I))) {
-      FirstTerminator = &I;
-      FirstTerminatorOrder = Order;
-    }
-    Orders[&I] = Order++;
-
-    // We don't need to order instructions past the last flush point.
-    if (I.getIterator() == LastFlushPoint)
-      break;
-  }
-}
-
-void FastISel::sinkLocalValueMaterialization(MachineInstr &LocalMI,
-                                             Register DefReg,
-                                             InstOrderMap &OrderMap) {
-  // If this register is used by a register fixup, MRI will not contain all
-  // the uses until after register fixups, so don't attempt to sink or DCE
-  // this instruction. Register fixups typically come from no-op cast
-  // instructions, which replace the cast instruction vreg with the local
-  // value vreg.
-  if (FuncInfo.RegsWithFixups.count(DefReg))
-    return;
-
-  // We can DCE this instruction if there are no uses and it wasn't a
-  // materialized for a successor PHI node.
-  bool UsedByPHI = isRegUsedByPhiNodes(DefReg, FuncInfo);
-  if (!UsedByPHI && MRI.use_nodbg_empty(DefReg)) {
-    if (EmitStartPt == &LocalMI)
-      EmitStartPt = EmitStartPt->getPrevNode();
-    LLVM_DEBUG(dbgs() << "removing dead local value materialization "
-                      << LocalMI);
-    OrderMap.Orders.erase(&LocalMI);
-    LocalMI.eraseFromParent();
-    return;
-  }
-
-  // Number the instructions if we haven't yet so we can efficiently find the
-  // earliest use.
-  if (OrderMap.Orders.empty())
-    OrderMap.initialize(FuncInfo.MBB, LastFlushPoint);
-
-  // Find the first user in the BB.
-  MachineInstr *FirstUser = nullptr;
-  unsigned FirstOrder = std::numeric_limits<unsigned>::max();
-  for (MachineInstr &UseInst : MRI.use_nodbg_instructions(DefReg)) {
-    auto I = OrderMap.Orders.find(&UseInst);
-    assert(I != OrderMap.Orders.end() &&
-           "local value used by instruction outside local region");
-    unsigned UseOrder = I->second;
-    if (UseOrder < FirstOrder) {
-      FirstOrder = UseOrder;
-      FirstUser = &UseInst;
-    }
-  }
-
-  // The insertion point will be the first terminator or the first user,
-  // whichever came first. If there was no terminator, this must be a
-  // fallthrough block and the insertion point is the end of the block.
-  MachineBasicBlock::instr_iterator SinkPos;
-  if (UsedByPHI && OrderMap.FirstTerminatorOrder < FirstOrder) {
-    FirstOrder = OrderMap.FirstTerminatorOrder;
-    SinkPos = OrderMap.FirstTerminator->getIterator();
-  } else if (FirstUser) {
-    SinkPos = FirstUser->getIterator();
-  } else {
-    assert(UsedByPHI && "must be users if not used by a phi");
-    SinkPos = FuncInfo.MBB->instr_end();
-  }
-
-  // Collect all DBG_VALUEs before the new insertion position so that we can
-  // sink them.
-  SmallVector<MachineInstr *, 1> DbgValues;
-  for (MachineInstr &DbgVal : MRI.use_instructions(DefReg)) {
-    if (!DbgVal.isDebugValue())
-      continue;
-    unsigned UseOrder = OrderMap.Orders[&DbgVal];
-    if (UseOrder < FirstOrder)
-      DbgValues.push_back(&DbgVal);
-  }
-
-  // Sink LocalMI before SinkPos and assign it the same DebugLoc.
-  LLVM_DEBUG(dbgs() << "sinking local value to first use " << LocalMI);
-  FuncInfo.MBB->remove(&LocalMI);
-  FuncInfo.MBB->insert(SinkPos, &LocalMI);
-  if (SinkPos != FuncInfo.MBB->end())
-    LocalMI.setDebugLoc(SinkPos->getDebugLoc());
-
-  // Sink any debug values that we've collected.
-  for (MachineInstr *DI : DbgValues) {
-    FuncInfo.MBB->remove(DI);
-    FuncInfo.MBB->insert(SinkPos, DI);
-  }
-}
-
 bool FastISel::hasTrivialKill(const Value *V) {
   // Don't consider constants or arguments to have trivial kills.
   const Instruction *I = dyn_cast<Instruction>(V);
@@ -578,12 +481,9 @@ void FastISel::removeDeadCode(MachineBasicBlock::iterator I,
 }
 
 FastISel::SavePoint FastISel::enterLocalValueArea() {
-  MachineBasicBlock::iterator OldInsertPt = FuncInfo.InsertPt;
-  DebugLoc OldDL = DbgLoc;
+  SavePoint OldInsertPt = FuncInfo.InsertPt;
   recomputeInsertPt();
-  DbgLoc = DebugLoc();
-  SavePoint SP = {OldInsertPt, OldDL};
-  return SP;
+  return OldInsertPt;
 }
 
 void FastISel::leaveLocalValueArea(SavePoint OldInsertPt) {
@@ -591,8 +491,7 @@ void FastISel::leaveLocalValueArea(SavePoint OldInsertPt) {
     LastLocalValue = &*std::prev(FuncInfo.InsertPt);
 
   // Restore the previous insert position.
-  FuncInfo.InsertPt = OldInsertPt.InsertPt;
-  DbgLoc = OldInsertPt.DL;
+  FuncInfo.InsertPt = OldInsertPt;
 }
 
 bool FastISel::selectBinaryOp(const User *I, unsigned ISDOpcode) {
@@ -1657,6 +1556,11 @@ void FastISel::removeDeadLocalValueCode(MachineInstr *SavedLastLocalValue)
 }
 
 bool FastISel::selectInstruction(const Instruction *I) {
+  // Flush the local value map before starting each instruction.
+  // This improves locality and debugging, and can reduce spills.
+  // Reuse of values across IR instructions is relatively uncommon.
+  flushLocalValueMap();
+
   MachineInstr *SavedLastLocalValue = getLastLocalValue();
   // Just before the terminator instruction, insert instructions to
   // feed PHI nodes in successor blocks.
@@ -2362,9 +2266,9 @@ bool FastISel::handlePHINodesInSuccessorBlocks(const BasicBlock *LLVMBB) {
 
       const Value *PHIOp = PN.getIncomingValueForBlock(LLVMBB);
 
-      // Set the DebugLoc for the copy. Prefer the location of the operand
-      // if there is one; use the location of the PHI otherwise.
-      DbgLoc = PN.getDebugLoc();
+      // Set the DebugLoc for the copy. Use the location of the operand if
+      // there is one; otherwise no location, flushLocalValueMap will fix it.
+      DbgLoc = DebugLoc();
       if (const auto *Inst = dyn_cast<Instruction>(PHIOp))
         DbgLoc = Inst->getDebugLoc();
 

diff  --git a/llvm/test/CodeGen/AArch64/arm64-abi_align.ll b/llvm/test/CodeGen/AArch64/arm64-abi_align.ll
index ba2995affe80..fe0b31fb2db9 100644
--- a/llvm/test/CodeGen/AArch64/arm64-abi_align.ll
+++ b/llvm/test/CodeGen/AArch64/arm64-abi_align.ll
@@ -456,11 +456,11 @@ entry:
 ; FAST: str {{x[0-9]+}}, [sp, #40]
 ; FAST: str {{x[0-9]+}}, [sp, #48]
 ; FAST: str {{x[0-9]+}}, [sp, #56]
-; FAST: str {{w[0-9]+}}, [sp]
 ; Address of s1 is passed on stack at sp+8
 ; FAST: sub x[[A:[0-9]+]], x29, #32
-; FAST: str x[[A]], [sp, #8]
 ; FAST: add x[[B:[0-9]+]], sp, #32
+; FAST: str {{w[0-9]+}}, [sp]
+; FAST: str x[[A]], [sp, #8]
 ; FAST: str x[[B]], [sp, #16]
   %tmp = alloca %struct.s43, align 16
   %tmp1 = alloca %struct.s43, align 16

diff  --git a/llvm/test/CodeGen/AArch64/arm64-fast-isel-call.ll b/llvm/test/CodeGen/AArch64/arm64-fast-isel-call.ll
index 00016a6a7fff..a677d4a14353 100644
--- a/llvm/test/CodeGen/AArch64/arm64-fast-isel-call.ll
+++ b/llvm/test/CodeGen/AArch64/arm64-fast-isel-call.ll
@@ -82,12 +82,12 @@ entry:
 ; CHECK:       mov x0, xzr
 ; CHECK:       mov w1, #-8
 ; CHECK:       mov [[REG2:w[0-9]+]], #1023
-; CHECK:       uxth w2, [[REG2]]
 ; CHECK:       mov [[REG3:w[0-9]+]], #2
-; CHECK:       sxtb w3, [[REG3]]
 ; CHECK:       mov [[REG4:w[0-9]+]], wzr
-; CHECK:       and w4, [[REG4]], #0x1
 ; CHECK:       mov [[REG5:w[0-9]+]], #1
+; CHECK:       uxth w2, [[REG2]]
+; CHECK:       sxtb w3, [[REG3]]
+; CHECK:       and w4, [[REG4]], #0x1
 ; CHECK:       and w5, [[REG5]], #0x1
 ; CHECK:       bl _func2
   %call = call i32 @func2(i64 zeroext 0, i32 signext -8, i16 zeroext 1023, i8 signext -254, i1 zeroext 0, i1 zeroext 1)

diff  --git a/llvm/test/CodeGen/AArch64/arm64-fast-isel-gv.ll b/llvm/test/CodeGen/AArch64/arm64-fast-isel-gv.ll
index 63854b97dea5..ef49962c380c 100644
--- a/llvm/test/CodeGen/AArch64/arm64-fast-isel-gv.ll
+++ b/llvm/test/CodeGen/AArch64/arm64-fast-isel-gv.ll
@@ -24,7 +24,11 @@ entry:
 ; CHECK: mov  [[REG3:x[0-9]+]], #13849
 ; CHECK: add  [[REG7:x[0-9]+]], [[REG6]], [[REG3]]
 ; CHECK: and  [[REG8:x[0-9]+]], [[REG7]], #0xffff
+; CHECK: adrp [[REG1:x[0-9]+]], _seed at GOTPAGE
+; CHECK: ldr  [[REG1]], {{\[}}[[REG1]], _seed at GOTPAGEOFF{{\]}}
 ; CHECK: str  [[REG8]], {{\[}}[[REG1]]{{\]}}
+; CHECK: adrp [[REG1:x[0-9]+]], _seed at GOTPAGE
+; CHECK: ldr  [[REG1]], {{\[}}[[REG1]], _seed at GOTPAGEOFF{{\]}}
 ; CHECK: ldr  {{x[0-9]+}}, {{\[}}[[REG1]]{{\]}}
   %0 = load i64, i64* @seed, align 8
   %mul = mul nsw i64 %0, 1309

diff  --git a/llvm/test/CodeGen/AArch64/arm64-fast-isel-intrinsic.ll b/llvm/test/CodeGen/AArch64/arm64-fast-isel-intrinsic.ll
index c1b7d790878d..e36f33c7f4b4 100644
--- a/llvm/test/CodeGen/AArch64/arm64-fast-isel-intrinsic.ll
+++ b/llvm/test/CodeGen/AArch64/arm64-fast-isel-intrinsic.ll
@@ -8,8 +8,8 @@ define void @t1() {
 ; ARM64: adrp x8, _message at PAGE
 ; ARM64: add x0, x8, _message at PAGEOFF
 ; ARM64: mov [[REG:w[0-9]+]], wzr
-; ARM64: uxtb w1, [[REG]]
 ; ARM64: mov x2, #80
+; ARM64: uxtb w1, [[REG]]
 ; ARM64: bl _memset
   call void @llvm.memset.p0i8.i64(i8* align 16 getelementptr inbounds ([80 x i8], [80 x i8]* @message, i32 0, i32 0), i8 0, i64 80, i1 false)
   ret void

diff  --git a/llvm/test/CodeGen/AArch64/arm64-fast-isel.ll b/llvm/test/CodeGen/AArch64/arm64-fast-isel.ll
index 6d402c95ee3f..3e10b3b6bf85 100644
--- a/llvm/test/CodeGen/AArch64/arm64-fast-isel.ll
+++ b/llvm/test/CodeGen/AArch64/arm64-fast-isel.ll
@@ -93,11 +93,12 @@ define void @t6() nounwind {
 declare void @llvm.trap() nounwind
 
 define void @ands(i32* %addr) {
-; CHECK-LABEL: ands:
-; CHECK: tst [[COND:w[0-9]+]], #0x1
-; CHECK-NEXT: mov w{{[0-9]+}}, #2
-; CHECK-NEXT: mov w{{[0-9]+}}, #1
-; CHECK-NEXT: csel [[COND]],
+; FIXME: 'select i1 undef' makes this unreliable (ub?).
+; COM: CHECK-LABEL: ands:
+; COM: CHECK: tst [[COND:w[0-9]+]], #0x1
+; COM: CHECK-NEXT: mov w{{[0-9]+}}, #2
+; COM: CHECK-NEXT: mov w{{[0-9]+}}, #1
+; COM: CHECK-NEXT: csel [[COND]],
 entry:
   %cond91 = select i1 undef, i32 1, i32 2
   store i32 %cond91, i32* %addr, align 4

diff  --git a/llvm/test/CodeGen/AArch64/arm64-patchpoint-webkit_jscc.ll b/llvm/test/CodeGen/AArch64/arm64-patchpoint-webkit_jscc.ll
index be52469ec55d..fa07beb2fb96 100644
--- a/llvm/test/CodeGen/AArch64/arm64-patchpoint-webkit_jscc.ll
+++ b/llvm/test/CodeGen/AArch64/arm64-patchpoint-webkit_jscc.ll
@@ -51,10 +51,10 @@ entry:
 ; CHECK-NEXT:  blr x16
 ; FAST-LABEL:  jscall_patchpoint_codegen2:
 ; FAST:        mov [[REG1:x[0-9]+]], #2
-; FAST-NEXT:   str [[REG1]], [sp]
 ; FAST-NEXT:   mov [[REG2:w[0-9]+]], #4
-; FAST-NEXT:   str [[REG2]], [sp, #16]
 ; FAST-NEXT:   mov [[REG3:x[0-9]+]], #6
+; FAST-NEXT:   str [[REG1]], [sp]
+; FAST-NEXT:   str [[REG2]], [sp, #16]
 ; FAST-NEXT:   str [[REG3]], [sp, #24]
 ; FAST:        Ltmp
 ; FAST-NEXT:   mov  x16, #281470681743360
@@ -87,14 +87,14 @@ entry:
 ; CHECK-NEXT:  blr x16
 ; FAST-LABEL:  jscall_patchpoint_codegen3:
 ; FAST:        mov [[REG1:x[0-9]+]], #2
-; FAST-NEXT:   str [[REG1]], [sp]
 ; FAST-NEXT:   mov [[REG2:w[0-9]+]], #4
-; FAST-NEXT:   str [[REG2]], [sp, #16]
 ; FAST-NEXT:   mov [[REG3:x[0-9]+]], #6
-; FAST-NEXT:   str [[REG3]], [sp, #24]
 ; FAST-NEXT:   mov [[REG4:w[0-9]+]], #8
-; FAST-NEXT:   str [[REG4]], [sp, #36]
 ; FAST-NEXT:   mov [[REG5:x[0-9]+]], #10
+; FAST-NEXT:   str [[REG1]], [sp]
+; FAST-NEXT:   str [[REG2]], [sp, #16]
+; FAST-NEXT:   str [[REG3]], [sp, #24]
+; FAST-NEXT:   str [[REG4]], [sp, #36]
 ; FAST-NEXT:   str [[REG5]], [sp, #48]
 ; FAST:        Ltmp
 ; FAST-NEXT:   mov   x16, #281470681743360

diff  --git a/llvm/test/CodeGen/AArch64/cfguard-checks.ll b/llvm/test/CodeGen/AArch64/cfguard-checks.ll
index 5ebe1dd13659..cb31decd57ba 100644
--- a/llvm/test/CodeGen/AArch64/cfguard-checks.ll
+++ b/llvm/test/CodeGen/AArch64/cfguard-checks.ll
@@ -36,10 +36,10 @@ entry:
 
   ; The call to __guard_check_icall_fptr should come immediately before the call to the target function.
   ; CHECK-LABEL: func_optnone_cf
-	; CHECK:        adrp x8, __guard_check_icall_fptr
-	; CHECK:        add x9, x8, __guard_check_icall_fptr
 	; CHECK:        adrp x8, target_func
 	; CHECK:        add x8, x8, target_func
+	; CHECK:        adrp x9, __guard_check_icall_fptr
+	; CHECK:        add x9, x9, __guard_check_icall_fptr
 	; CHECK:        ldr x9, [x9]
 	; CHECK:        mov x15, x8
 	; CHECK:        blr x9

diff  --git a/llvm/test/CodeGen/AArch64/elf-globals-static.ll b/llvm/test/CodeGen/AArch64/elf-globals-static.ll
index 393f42049011..86841a4178fb 100644
--- a/llvm/test/CodeGen/AArch64/elf-globals-static.ll
+++ b/llvm/test/CodeGen/AArch64/elf-globals-static.ll
@@ -15,6 +15,11 @@ define i8 @test_i8(i8 %new) {
 ; CHECK: ldrb {{w[0-9]+}}, [x[[HIREG]], :lo12:var8]
 ; CHECK: strb {{w[0-9]+}}, [x[[HIREG]], :lo12:var8]
 
+; CHECK-PIC-LABEL: test_i8:
+; CHECK-PIC: adrp x[[HIREG:[0-9]+]], :got:var8
+; CHECK-PIC: ldr x[[VAR_ADDR:[0-9]+]], [x[[HIREG]], :got_lo12:var8]
+; CHECK-PIC: ldrb {{w[0-9]+}}, [x[[VAR_ADDR]]]
+
 ; CHECK-FAST-LABEL: test_i8:
 ; CHECK-FAST: adrp x[[HIREG:[0-9]+]], var8
 ; CHECK-FAST: ldrb {{w[0-9]+}}, [x[[HIREG]], :lo12:var8]

diff  --git a/llvm/test/CodeGen/AArch64/large-stack.ll b/llvm/test/CodeGen/AArch64/large-stack.ll
index 11027652df92..60cac6302fd7 100644
--- a/llvm/test/CodeGen/AArch64/large-stack.ll
+++ b/llvm/test/CodeGen/AArch64/large-stack.ll
@@ -32,12 +32,13 @@ attributes #0 = { noinline optnone "frame-pointer"="all" }
 ; CHECK:                  sub	x[[INDEX:[0-9]+]], x[[FRAME]], #8
 ; CHECK-NEXT:             str	x0, [x[[INDEX]]]
 ; CHECK-NEXT:             ldr	x[[VAL1:[0-9]+]], [x[[INDEX]]]
-; CHECK-NEXT:             mov	x[[VAL2:[0-9]+]], #8
 ; CHECK-NEXT:             add	x[[VAL3:[0-9]+]], sp, #8
+; CHECK-NEXT:             mov	x[[VAL2:[0-9]+]], #8
 ; CHECK-NEXT:             madd	x[[VAL1]], x[[VAL1]], x[[VAL2]], x[[VAL3]]
 ; CHECK-NEXT:             mov	x[[TMP1:[0-9]+]], #1
 ; CHECK-NEXT:             str	x[[TMP1]], [x[[VAL1]]]
 ; CHECK-NEXT:             ldr	x[[INDEX]], [x[[INDEX]]]
+; CHECK-NEXT:             add   x[[VAL3:[0-9]+]], sp, #8
 ; CHECK-NEXT:             mov	x[[VAL4:[0-9]+]], #8
 ; CHECK-NEXT:             madd	x[[INDEX]], x[[INDEX]], x[[VAL4]], x[[VAL3]]
 ; CHECK-NEXT:             ldr	x1, [x[[INDEX]]

diff  --git a/llvm/test/CodeGen/ARM/fast-isel-call.ll b/llvm/test/CodeGen/ARM/fast-isel-call.ll
index c126343d8123..d08a3b41daba 100644
--- a/llvm/test/CodeGen/ARM/fast-isel-call.ll
+++ b/llvm/test/CodeGen/ARM/fast-isel-call.ll
@@ -100,8 +100,8 @@ entry:
 ; ARM-DAG: and [[R3]], [[R3]], #255
 ; ARM-DAG: and [[R4]], [[R4]], #255
 ; ARM-DAG: str [[R4]], [sp]
-; ARM-DAG: and [[R4]], [[R5]], #255
-; ARM-DAG: str [[R4]], [sp, #4]
+; ARM-DAG: and [[R5]], [[R5]], #255
+; ARM-DAG: str [[R5]], [sp, #4]
 ; ARM: bl {{_?}}bar
 ; ARM-LONG-LABEL: @t10
 
@@ -127,8 +127,8 @@ entry:
 ; THUMB-DAG: and [[R3]], [[R3]], #255
 ; THUMB-DAG: and [[R4]], [[R4]], #255
 ; THUMB-DAG: str.w [[R4]], [sp]
-; THUMB-DAG: and [[R4]], [[R5]], #255
-; THUMB-DAG: str.w [[R4]], [sp, #4]
+; THUMB-DAG: and [[R5]], [[R5]], #255
+; THUMB-DAG: str.w [[R5]], [sp, #4]
 ; THUMB: bl {{_?}}bar
 ; THUMB-LONG-LABEL: @t10
 ; THUMB-LONG: {{(movw)|(ldr.n)}} [[R1:l?r[0-9]*]], {{(:lower16:L_bar\$non_lazy_ptr)|(.LCPI)}}

diff  --git a/llvm/test/CodeGen/ARM/fast-isel-intrinsic.ll b/llvm/test/CodeGen/ARM/fast-isel-intrinsic.ll
index e739d94cd9ab..772ae0461157 100644
--- a/llvm/test/CodeGen/ARM/fast-isel-intrinsic.ll
+++ b/llvm/test/CodeGen/ARM/fast-isel-intrinsic.ll
@@ -36,8 +36,8 @@ define void @t1() nounwind ssp {
 ; THUMB: {{(movt r0, :upper16:_?message1)|(ldr r0, \[r0\])}}
 ; THUMB: adds r0, #5
 ; THUMB: movs r1, #64
-; THUMB: and r1, r1, #255
 ; THUMB: movs r2, #10
+; THUMB: and r1, r1, #255
 ; THUMB: bl {{_?}}memset
 ; THUMB-LONG-LABEL: t1:
 ; THUMB-LONG: movw r3, :lower16:L_memset$non_lazy_ptr

diff  --git a/llvm/test/CodeGen/ARM/fast-isel-ldr-str-thumb-neg-index.ll b/llvm/test/CodeGen/ARM/fast-isel-ldr-str-thumb-neg-index.ll
index 95942c271c9c..b692b4bd6ffc 100644
--- a/llvm/test/CodeGen/ARM/fast-isel-ldr-str-thumb-neg-index.ll
+++ b/llvm/test/CodeGen/ARM/fast-isel-ldr-str-thumb-neg-index.ll
@@ -106,10 +106,12 @@ entry:
 ; THUMB-LABEL: t12:
   %add.ptr = getelementptr inbounds i32, i32* %ptr, i32 -64
   store i32 0, i32* %add.ptr, align 4
+; THUMB: mov [[PTR:r[0-9]+]], r0
+; THUMB: movs [[VAL:r[0-9]+]], #0
 ; THUMB: movw [[REG:r[0-9]+]], #65280
 ; THUMB: movt [[REG]], #65535
-; THUMB: add [[REG]], r0
-; THUMB: str r{{[0-9]}}, {{\[}}[[REG]]]
+; THUMB: add [[PTR]], [[REG]]
+; THUMB: str [[VAL]], {{\[}}[[PTR]]]
   ret void
 }
 
@@ -138,10 +140,12 @@ entry:
 ; THUMB-LABEL: t15:
   %add.ptr = getelementptr inbounds i16, i16* %ptr, i32 -128
   store i16 0, i16* %add.ptr, align 2
+; THUMB: mov [[PTR:r[0-9]+]], r0
+; THUMB: movs [[VAL:r[0-9]+]], #0
 ; THUMB: movw [[REG:r[0-9]+]], #65280
 ; THUMB: movt [[REG]], #65535
-; THUMB: add [[REG]], r0
-; THUMB: strh r{{[0-9]}}, {{\[}}[[REG]]]
+; THUMB: add [[PTR]], [[REG]]
+; THUMB: strh [[VAL]], {{\[}}[[PTR]]]
   ret void
 }
 
@@ -170,9 +174,11 @@ entry:
 ; THUMB-LABEL: t18:
   %add.ptr = getelementptr inbounds i8, i8* %ptr, i32 -256
   store i8 0, i8* %add.ptr, align 1
+; THUMB: mov [[PTR:r[0-9]+]], r0
+; THUMB: movs [[VAL]], #0
 ; THUMB: movw [[REG:r[0-9]+]], #65280
 ; THUMB: movt [[REG]], #65535
-; THUMB: add [[REG]], r0
-; THUMB: strb r{{[0-9]}}, {{\[}}[[REG]]]
+; THUMB: add [[PTR]], [[REG]]
+; THUMB: strb [[VAL]], {{\[}}[[PTR]]]
   ret void
 }

diff  --git a/llvm/test/CodeGen/ARM/fast-isel-ldrh-strh-arm.ll b/llvm/test/CodeGen/ARM/fast-isel-ldrh-strh-arm.ll
index 1e0f144583bf..f2d728c8b361 100644
--- a/llvm/test/CodeGen/ARM/fast-isel-ldrh-strh-arm.ll
+++ b/llvm/test/CodeGen/ARM/fast-isel-ldrh-strh-arm.ll
@@ -94,9 +94,10 @@ entry:
 ; ARM-LABEL: t10:
   %add.ptr = getelementptr inbounds i16, i16* %a, i64 -128
   store i16 0, i16* %add.ptr, align 2
-; ARM: mvn r1, #255
-; ARM: add [[REG0:r[0-9]+]], r0, r1
+; ARM: mov r1, r0
 ; ARM: movw [[REG1:r[0-9]+]], #0
+; ARM: mvn [[REG2:r[0-9]+]], #255
+; ARM: add [[REG0:r[0-9]+]], r1, [[REG2]]
 ; ARM: strh [[REG1]], {{\[}}[[REG0]]]
   ret void
 }
@@ -118,8 +119,9 @@ entry:
 ; ARM-LABEL: t12:
   %add.ptr = getelementptr inbounds i16, i16* %a, i64 128
   store i16 0, i16* %add.ptr, align 2
-; ARM: add [[REG0:r[0-9]+]], r0, #256
+; ARM: mov r1, r0
 ; ARM: movw [[REG1:r[0-9]+]], #0
+; ARM: add [[REG0:r[0-9]+]], r1, #256
 ; ARM: strh [[REG1]], {{\[}}[[REG0]]]
   ret void
 }

diff  --git a/llvm/test/CodeGen/ARM/fast-isel-select.ll b/llvm/test/CodeGen/ARM/fast-isel-select.ll
index aec45c6b4e53..1b6973ddb411 100644
--- a/llvm/test/CodeGen/ARM/fast-isel-select.ll
+++ b/llvm/test/CodeGen/ARM/fast-isel-select.ll
@@ -6,12 +6,14 @@
 define i32 @t1(i1 %c) nounwind readnone {
 entry:
 ; ARM-LABEL: t1:
-; ARM: tst r0, #1
+; ARM: mov r1, r0
 ; ARM: movw r0, #10
+; ARM: tst r1, #1
 ; ARM: moveq r0, #20
 ; THUMB-LABEL: t1:
-; THUMB: tst.w r0, #1
-; THUMB: movw r0, #10
+; THUMB: mov r1, r0
+; THUMB: movs r0, #10
+; THUMB: tst.w r1, #1
 ; THUMB: it eq
 ; THUMB: moveq r0, #20
   %0 = select i1 %c, i32 10, i32 20
@@ -50,12 +52,14 @@ entry:
 define i32 @t4(i1 %c) nounwind readnone {
 entry:
 ; ARM-LABEL: t4:
-; ARM: tst r0, #1
+; ARM: mov r1, r0
 ; ARM: mvn r0, #9
+; ARM: tst r1, #1
 ; ARM: mvneq r0, #0
 ; THUMB-LABEL: t4:
-; THUMB: tst.w r0, #1
+; THUMB: mov r1, r0
 ; THUMB: mvn r0, #9
+; THUMB: tst.w r1, #1
 ; THUMB: it eq
 ; THUMB: mvneq r0, #0
   %0 = select i1 %c, i32 -10, i32 -1

diff  --git a/llvm/test/CodeGen/ARM/fast-isel.ll b/llvm/test/CodeGen/ARM/fast-isel.ll
index 272996893925..2437bda498ba 100644
--- a/llvm/test/CodeGen/ARM/fast-isel.ll
+++ b/llvm/test/CodeGen/ARM/fast-isel.ll
@@ -152,7 +152,10 @@ define void @test4() {
 ; THUMB: ldr [[REG:r[0-9]+]], [r0]
 ; THUMB: ldr [[REG1:r[0-9]+]], {{\[}}[[REG]]]
 ; THUMB: adds [[REG1]], #1
-; THUMB: str [[REG1]], {{\[}}[[REG]]]
+; THUMB: {{(movw r1, :lower16:L_test4g\$non_lazy_ptr)|(ldr.n r0, .LCPI)}}
+; THUMB: {{(movt r1, :upper16:L_test4g\$non_lazy_ptr)?}}
+; THUMB: ldr [[REG2:r[0-9]+]], [r1]
+; THUMB: str [[REG1]], {{\[}}[[REG2]]]
 
 ; ARM-MACHO: {{(movw r0, :lower16:L_test4g\$non_lazy_ptr)|(ldr r0, .LCPI)}}
 ; ARM-MACHO: {{(movt r0, :upper16:L_test4g\$non_lazy_ptr)?}}
@@ -161,9 +164,17 @@ define void @test4() {
 ; ARM-ELF: movw [[REG:r[0-9]+]], :lower16:test4g
 ; ARM-ELF: movt [[REG]], :upper16:test4g
 
-; ARM: ldr r0, [r1]
-; ARM: add r0, r0, #1
-; ARM: str r0, [r1]
+; ARM: ldr [[REG1:r[0-9]+]], {{\[}}[[REG]]]
+; ARM: add [[REG2:r[0-9]+]], [[REG1]], #1
+
+; ARM-MACHO: {{(movw r1, :lower16:L_test4g\$non_lazy_ptr)|(ldr r0, .LCPI)}}
+; ARM-MACHO: {{(movt r1, :upper16:L_test4g\$non_lazy_ptr)?}}
+; ARM-MACHO: ldr [[REG3:r[0-9]+]], [r1]
+
+; ARM-ELF: movw [[REG3:r[0-9]+]], :lower16:test4g
+; ARM-ELF: movt [[REG3]], :upper16:test4g
+
+; ARM: str [[REG2]], {{\[}}[[REG3]]]
 }
 
 ; ARM: @urem_fold

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/callabi.ll b/llvm/test/CodeGen/Mips/Fast-ISel/callabi.ll
index f22fbcc7b73e..a0f5d1e75dfb 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/callabi.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/callabi.ll
@@ -163,22 +163,22 @@ declare void @xcccc(i8, i8, i8, i8)
 define void @cxcccc() {
   ; ALL-LABEL:    cxcccc:
 
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 88
-  ; 32R1:       sll     $[[R:[0-9]+]], $[[R]], 24
-  ; 32R1:       sra     $4, $[[R]], 24
-  ; 32R2:       seb     $4, $[[R]]
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 44
-  ; 32R1:       sll     $[[R:[0-9]+]], $[[R]], 24
-  ; 32R1:       sra     $5, $[[R]], 24
-  ; 32R2:       seb     $5, $[[R]]
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 11
-  ; 32R1:       sll     $[[R:[0-9]+]], $[[R]], 24
-  ; 32R1:       sra     $6, $[[R]], 24
-  ; 32R2:       seb     $6, $[[R]]
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 33
-  ; 32R1:       sll     $[[R:[0-9]+]], $[[R]], 24
-  ; 32R1:       sra     $7, $[[R]], 24
-  ; 32R2:       seb     $7, $[[R]]
+  ; ALL:        addiu   $[[R1:[0-9]+]], $zero, 88
+  ; ALL:        addiu   $[[R2:[0-9]+]], $zero, 44
+  ; ALL:        addiu   $[[R3:[0-9]+]], $zero, 11
+  ; ALL:        addiu   $[[R4:[0-9]+]], $zero, 33
+  ; 32R1:       sll     $[[R1:[0-9]+]], $[[R1]], 24
+  ; 32R1:       sra     $4, $[[R1]], 24
+  ; 32R2:       seb     $4, $[[R1]]
+  ; 32R1:       sll     $[[R2]], $[[R2]], 24
+  ; 32R1:       sra     $5, $[[R2]], 24
+  ; 32R2:       seb     $5, $[[R2]]
+  ; 32R1:       sll     $[[R3]], $[[R3]], 24
+  ; 32R1:       sra     $6, $[[R3]], 24
+  ; 32R2:       seb     $6, $[[R3]]
+  ; 32R1:       sll     $[[R4]], $[[R4]], 24
+  ; 32R1:       sra     $7, $[[R4]], 24
+  ; 32R2:       seb     $7, $[[R4]]
 
   ; ALL:        lw      $25, %got(xcccc)(${{[0-9]+}})
   ; ALL:        jalr    $25
@@ -192,22 +192,22 @@ declare void @xhhhh(i16, i16, i16, i16)
 define void @cxhhhh() {
   ; ALL-LABEL:    cxhhhh:
 
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 88
-  ; 32R1:       sll     $[[R]], $[[R]], 16
-  ; 32R1:       sra     $4, $[[R]], 16
-  ; 32R2:       seh     $4, $[[R]]
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 44
-  ; 32R1:       sll     $[[R]], $[[R]], 16
-  ; 32R1:       sra     $5, $[[R]], 16
-  ; 32R2:       seh     $5, $[[R]]
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 11
-  ; 32R1:       sll     $[[R]], $[[R]], 16
-  ; 32R1:       sra     $6, $[[R]], 16
-  ; 32R2:       seh     $6, $[[R]]
-  ; ALL:        addiu   $[[R:[0-9]+]], $zero, 33
-  ; 32R1:       sll     $[[R]], $[[R]], 16
-  ; 32R1:       sra     $7, $[[R]], 16
-  ; 32R2:       seh     $7, $[[R]]
+  ; ALL:        addiu   $[[R1:[0-9]+]], $zero, 88
+  ; ALL:        addiu   $[[R2:[0-9]+]], $zero, 44
+  ; ALL:        addiu   $[[R3:[0-9]+]], $zero, 11
+  ; ALL:        addiu   $[[R4:[0-9]+]], $zero, 33
+  ; 32R1:       sll     $[[R1]], $[[R1]], 16
+  ; 32R1:       sra     $4, $[[R1]], 16
+  ; 32R2:       seh     $4, $[[R1]]
+  ; 32R1:       sll     $[[R2]], $[[R2]], 16
+  ; 32R1:       sra     $5, $[[R2]], 16
+  ; 32R2:       seh     $5, $[[R2]]
+  ; 32R1:       sll     $[[R3]], $[[R3]], 16
+  ; 32R1:       sra     $6, $[[R3]], 16
+  ; 32R2:       seh     $6, $[[R3]]
+  ; 32R1:       sll     $[[R4]], $[[R4]], 16
+  ; 32R1:       sra     $7, $[[R4]], 16
+  ; 32R2:       seh     $7, $[[R4]]
 
   ; ALL:        lw      $25, %got(xhhhh)(${{[0-9]+}})
   ; ALL:        jalr    $25
@@ -426,13 +426,13 @@ define void @cxiff() {
   ; ALL-LABEL:    cxiff:
 
   ; ALL:            addu    $[[REG_GP:[0-9]+]], ${{[0-9]+}}, ${{[0-9+]}}
+  ; ALL-DAG:        addiu   $4, $zero, 12239
   ; ALL-DAG:        lui     $[[REGF0_1:[0-9]+]], 17526
   ; ALL-DAG:        ori     $[[REGF0_2:[0-9]+]], $[[REGF0_1]], 55706
   ; ALL-DAG:        mtc1    $[[REGF0_2]], $f[[REGF0_3:[0-9]+]]
   ; ALL-DAG:        lui     $[[REGF1_1:[0-9]+]], 16543
   ; ALL-DAG:        ori     $[[REGF1_2:[0-9]+]], $[[REGF1_1]], 65326
   ; ALL:            mtc1    $[[REGF1_2]], $f[[REGF1_3:[0-9]+]]
-  ; ALL-DAG:        addiu   $4, $zero, 12239
   ; ALL-DAG:        mfc1    $5, $f[[REGF0_3]]
   ; ALL-DAG:        mfc1    $6, $f[[REGF1_3]]
   ; ALL-DAG:        lw      $25, %got(xiff)($[[REG_GP]])
@@ -470,11 +470,11 @@ define void @cxifif() {
   ; ALL-DAG:        lui     $[[REGF0_1:[0-9]+]], 17527
   ; ALL-DAG:        ori     $[[REGF0_2:[0-9]+]], $[[REGF0_1]], 2015
   ; ALL-DAG:        mtc1    $[[REGF0_2]], $f[[REGF0_3:[0-9]+]]
+  ; ALL-DAG:        addiu   $6, $zero, 9991
   ; ALL-DAG:        lui     $[[REGF1_1:[0-9]+]], 17802
   ; ALL-DAG:        ori     $[[REGF1_2:[0-9]+]], $[[REGF1_1]], 58470
   ; ALL:            mtc1    $[[REGF1_2]], $f[[REGF1_3:[0-9]+]]
   ; ALL-DAG:        mfc1    $5, $f[[REGF0_3]]
-  ; ALL-DAG:        addiu   $6, $zero, 9991
   ; ALL-DAG:        mfc1    $7, $f[[REGF1_3]]
   ; ALL-DAG:        lw      $25, %got(xifif)($[[REG_GP]])
   ; ALL:            jalr    $25
@@ -489,13 +489,13 @@ define void @cxiffi() {
   ; ALL-LABEL:    cxiffi:
 
   ; ALL:            addu    $[[REG_GP:[0-9]+]], ${{[0-9]+}}, ${{[0-9+]}}
+  ; ALL-DAG:        addiu   $4, $zero, 45
   ; ALL-DAG:        lui     $[[REGF0_1:[0-9]+]], 16307
   ; ALL-DAG:        ori     $[[REGF0_2:[0-9]+]], $[[REGF0_1]], 13107
   ; ALL-DAG:        mtc1    $[[REGF0_2]], $f[[REGF0_3:[0-9]+]]
   ; ALL-DAG:        lui     $[[REGF1_1:[0-9]+]], 17529
   ; ALL-DAG:        ori     $[[REGF1_2:[0-9]+]], $[[REGF1_1]], 39322
   ; ALL:            mtc1    $[[REGF1_2]], $f[[REGF1_3:[0-9]+]]
-  ; ALL-DAG:        addiu   $4, $zero, 45
   ; ALL-DAG:        mfc1    $5, $f[[REGF0_3]]
   ; ALL-DAG:        mfc1    $6, $f[[REGF1_3]]
   ; ALL-DAG:        addiu   $7, $zero, 234

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/fastalloca.ll b/llvm/test/CodeGen/Mips/Fast-ISel/fastalloca.ll
index ad2a0f8f2a84..a7a537e68a7e 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/fastalloca.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/fastalloca.ll
@@ -22,8 +22,8 @@ entry:
   %2 = load i32, i32* %x2, align 4
   store i32 %2, i32* @i, align 4
   %3 = load i32, i32* %retval
-; CHECK:        lw      $[[I_ADDR:[0-9]+]], %got(i)($[[REG_GP:[0-9]+]])
 ; CHECK:        addiu   $[[A_ADDR:[0-9]+]], $sp, 8
+; CHECK-DAG:    lw      $[[I_ADDR:[0-9]+]], %got(i)($[[REG_GP:[0-9]+]])
 ; CHECK-DAG:    sw      $[[A_ADDR]], [[A_ADDR_FI:[0-9]+]]($sp)
 ; CHECK-DAG:    lw      $[[A_ADDR2:[0-9]+]], [[A_ADDR_FI]]($sp)
 ; CHECK-DAG:    lw      $[[A_X:[0-9]+]], 0($[[A_ADDR2]])

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/fpcmpa.ll b/llvm/test/CodeGen/Mips/Fast-ISel/fpcmpa.ll
index b27b20c8f719..67b49c6a8ea2 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/fpcmpa.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/fpcmpa.ll
@@ -16,10 +16,10 @@ entry:
   %1 = load float, float* @f2, align 4
   %cmp = fcmp oeq float %0, %1
 ; CHECK-LABEL:  feq1:
-; CHECK-DAG:    lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
-; CHECK-DAG:    lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
-; CHECK-DAG:    lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.eq.s  $f[[REG_F1]], $f[[REG_F2]]
@@ -37,10 +37,10 @@ entry:
   %1 = load float, float* @f2, align 4
   %cmp = fcmp une float %0, %1
 ; CHECK-LABEL:  fne1:
-; CHECK-DAG:    lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
-; CHECK-DAG:    lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
-; CHECK-DAG:    lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.eq.s  $f[[REG_F1]], $f[[REG_F2]]
@@ -57,10 +57,10 @@ entry:
   %1 = load float, float* @f2, align 4
   %cmp = fcmp olt float %0, %1
 ; CHECK-LABEL:  flt1:
-; CHECK-DAG:    lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
-; CHECK-DAG:    lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
-; CHECK-DAG:    lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.olt.s  $f[[REG_F1]], $f[[REG_F2]]
@@ -78,10 +78,10 @@ entry:
   %1 = load float, float* @f2, align 4
   %cmp = fcmp ogt float %0, %1
 ; CHECK-LABEL: fgt1:
-; CHECK-DAG:    lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
-; CHECK-DAG:    lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
-; CHECK-DAG:    lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.ule.s  $f[[REG_F1]], $f[[REG_F2]]
@@ -98,10 +98,10 @@ entry:
   %1 = load float, float* @f2, align 4
   %cmp = fcmp ole float %0, %1
 ; CHECK-LABEL:  fle1:
-; CHECK-DAG:    lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
-; CHECK-DAG:    lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
-; CHECK-DAG:    lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.ole.s  $f[[REG_F1]], $f[[REG_F2]]
@@ -118,10 +118,10 @@ entry:
   %1 = load float, float* @f2, align 4
   %cmp = fcmp oge float %0, %1
 ; CHECK-LABEL:  fge1:
-; CHECK-DAG:    lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
-; CHECK-DAG:    lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
-; CHECK-DAG:    lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F1_GOT:[0-9]+]], %got(f1)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F1:[0-9]+]], 0($[[REG_F1_GOT]])
+; CHECK:        lw      $[[REG_F2_GOT:[0-9]+]], %got(f2)(${{[0-9]+}})
+; CHECK:        lwc1    $f[[REG_F2:[0-9]+]], 0($[[REG_F2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.ult.s  $f[[REG_F1]], $f[[REG_F2]]
@@ -138,10 +138,10 @@ entry:
   %1 = load double, double* @d2, align 8
   %cmp = fcmp oeq double %0, %1
 ; CHECK-LABEL:  deq1:
-; CHECK-DAG:    lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
-; CHECK-DAG:    ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
-; CHECK-DAG:    ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.eq.d  $f[[REG_D1]], $f[[REG_D2]]
@@ -158,10 +158,10 @@ entry:
   %1 = load double, double* @d2, align 8
   %cmp = fcmp une double %0, %1
 ; CHECK-LABEL:  dne1:
-; CHECK-DAG:    lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
-; CHECK-DAG:    ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
-; CHECK-DAG:    ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.eq.d  $f[[REG_D1]], $f[[REG_D2]]
@@ -178,12 +178,12 @@ entry:
   %1 = load double, double* @d2, align 8
   %cmp = fcmp olt double %0, %1
 ; CHECK-LABEL:  dlt1:
-; CHECK-DAG:    lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
-; CHECK-DAG:    ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
-; CHECK-DAG:    ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
-; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
-; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
+; CHECK:        lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
+; CHECK:        addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
+; CHECK:        addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.olt.d  $f[[REG_D1]], $f[[REG_D2]]
 ; CHECK:        movt  $[[REG_ZERO]], $[[REG_ONE]], $fcc0
   %conv = zext i1 %cmp to i32
@@ -198,10 +198,10 @@ entry:
   %1 = load double, double* @d2, align 8
   %cmp = fcmp ogt double %0, %1
 ; CHECK-LABEL:  dgt1:
-; CHECK-DAG:    lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
-; CHECK-DAG:    ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
-; CHECK-DAG:    ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.ule.d  $f[[REG_D1]], $f[[REG_D2]]
@@ -218,10 +218,10 @@ entry:
   %1 = load double, double* @d2, align 8
   %cmp = fcmp ole double %0, %1
 ; CHECK-LABEL:  dle1:
-; CHECK-DAG:    lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
-; CHECK-DAG:    ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
-; CHECK-DAG:    ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.ole.d  $f[[REG_D1]], $f[[REG_D2]]
@@ -238,10 +238,10 @@ entry:
   %1 = load double, double* @d2, align 8
   %cmp = fcmp oge double %0, %1
 ; CHECK-LABEL:  dge1:
-; CHECK-DAG:    lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
-; CHECK-DAG:    lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
-; CHECK-DAG:    ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
-; CHECK-DAG:    ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D1_GOT:[0-9]+]], %got(d1)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D1:[0-9]+]], 0($[[REG_D1_GOT]])
+; CHECK:        lw      $[[REG_D2_GOT:[0-9]+]], %got(d2)(${{[0-9]+}})
+; CHECK:        ldc1    $f[[REG_D2:[0-9]+]], 0($[[REG_D2_GOT]])
 ; CHECK-DAG:    addiu   $[[REG_ZERO:[0-9]+]], $zero, 0
 ; CHECK-DAG:    addiu   $[[REG_ONE:[0-9]+]], $zero, 1
 ; CHECK:        c.ult.d  $f[[REG_D1]], $f[[REG_D2]]

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/icmpa.ll b/llvm/test/CodeGen/Mips/Fast-ISel/icmpa.ll
index cc19beafd38d..e9bf30ed7b1d 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/icmpa.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/icmpa.ll
@@ -60,10 +60,10 @@ entry:
   %1 = load i32, i32* @ud, align 4
   %cmp = icmp ugt i32 %0, %1
   %conv = zext i1 %cmp to i32
-; CHECK-DAG:  lw	$[[REG_UD_GOT:[0-9+]]], %got(ud)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_UC_GOT:[0-9+]]], %got(uc)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_UD:[0-9]+]], 0($[[REG_UD_GOT]])
-; CHECK-DAG:  lw	$[[REG_UC:[0-9]+]], 0($[[REG_UC_GOT]])
+; CHECK:  lw	$[[REG_UC_GOT:[0-9+]]], %got(uc)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_UC:[0-9]+]], 0($[[REG_UC_GOT]])
+; CHECK:  lw	$[[REG_UD_GOT:[0-9+]]], %got(ud)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_UD:[0-9]+]], 0($[[REG_UD_GOT]])
 ; CHECK:  sltu  $[[REG1:[0-9]+]], $[[REG_UD]], $[[REG_UC]]
 ; FIXME: This instruction is redundant. The sltu can only produce 0 and 1.
 ; CHECK:  andi  ${{[0-9]+}}, $[[REG1]], 1
@@ -119,10 +119,10 @@ entry:
   %1 = load i32, i32* @ud, align 4
   %cmp = icmp ule i32 %0, %1
   %conv = zext i1 %cmp to i32
-; CHECK-DAG:  lw	$[[REG_UD_GOT:[0-9+]]], %got(ud)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_UC_GOT:[0-9+]]], %got(uc)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_UD:[0-9]+]], 0($[[REG_UD_GOT]])
-; CHECK-DAG:  lw	$[[REG_UC:[0-9]+]], 0($[[REG_UC_GOT]])
+; CHECK:  lw	$[[REG_UC_GOT:[0-9+]]], %got(uc)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_UC:[0-9]+]], 0($[[REG_UC_GOT]])
+; CHECK:  lw	$[[REG_UD_GOT:[0-9+]]], %got(ud)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_UD:[0-9]+]], 0($[[REG_UD_GOT]])
 ; CHECK:  sltu  $[[REG1:[0-9]+]], $[[REG_UD]], $[[REG_UC]]
 ; CHECK:  xori  $[[REG2:[0-9]+]], $[[REG1]], 1
 ; FIXME: This instruction is redundant. The sltu can only produce 0 and 1.
@@ -139,10 +139,10 @@ entry:
   %1 = load i32, i32* @d, align 4
   %cmp = icmp sgt i32 %0, %1
   %conv = zext i1 %cmp to i32
-; CHECK-DAG:  lw	$[[REG_D_GOT:[0-9+]]], %got(d)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_C_GOT:[0-9+]]], %got(c)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_D:[0-9]+]], 0($[[REG_D_GOT]])
-; CHECK-DAG:  lw	$[[REG_C:[0-9]+]], 0($[[REG_C_GOT]])
+; CHECK:  lw	$[[REG_C_GOT:[0-9+]]], %got(c)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_C:[0-9]+]], 0($[[REG_C_GOT]])
+; CHECK:  lw	$[[REG_D_GOT:[0-9+]]], %got(d)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_D:[0-9]+]], 0($[[REG_D_GOT]])
 ; CHECK:  slt  $[[REG1:[0-9]+]], $[[REG_D]], $[[REG_C]]
 ; FIXME: This instruction is redundant. The slt can only produce 0 and 1.
 ; CHECK:  andi  ${{[0-9]+}}, $[[REG1]], 1
@@ -197,10 +197,10 @@ entry:
   %1 = load i32, i32* @d, align 4
   %cmp = icmp sle i32 %0, %1
   %conv = zext i1 %cmp to i32
-; CHECK-DAG:  lw	$[[REG_D_GOT:[0-9+]]], %got(d)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_C_GOT:[0-9+]]], %got(c)(${{[0-9]+}})
-; CHECK-DAG:  lw	$[[REG_D:[0-9]+]], 0($[[REG_D_GOT]])
-; CHECK-DAG:  lw	$[[REG_C:[0-9]+]], 0($[[REG_C_GOT]])
+; CHECK:  lw	$[[REG_C_GOT:[0-9+]]], %got(c)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_C:[0-9]+]], 0($[[REG_C_GOT]])
+; CHECK:  lw	$[[REG_D_GOT:[0-9+]]], %got(d)(${{[0-9]+}})
+; CHECK:  lw	$[[REG_D:[0-9]+]], 0($[[REG_D_GOT]])
 ; CHECK:        slt     $[[REG1:[0-9]+]], $[[REG_D]], $[[REG_C]]
 ; CHECK:        xori    $[[REG2:[0-9]+]], $[[REG1]], 1
 ; FIXME: This instruction is redundant. The slt can only produce 0 and 1.

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/logopm.ll b/llvm/test/CodeGen/Mips/Fast-ISel/logopm.ll
index bd35eeedaf9a..65ca213235fe 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/logopm.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/logopm.ll
@@ -33,13 +33,13 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB2_ADDR:[0-9]+]], %got(ub2)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    lbu     $[[UB2:[0-9]+]], 0($[[UB2_ADDR]])
 ; CHECK-DAG:    and     $[[RES1:[0-9]+]], $[[UB2]], $[[UB1]]
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
   ret void
 }
@@ -56,11 +56,11 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    and     $[[RES1:[0-9]+]], $[[UB1]], $zero
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
 ; CHECK:        .end    andUb0
   ret void
@@ -79,12 +79,12 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    addiu   $[[CONST:[0-9]+]], $zero, 1
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    and     $[[RES1:[0-9]+]], $[[UB1]], $[[CONST]]
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
 ; CHECK:        .end    andUb1
   ret void
@@ -104,13 +104,13 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB2_ADDR:[0-9]+]], %got(ub2)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    lbu     $[[UB2:[0-9]+]], 0($[[UB2_ADDR]])
 ; CHECK-DAG:    or      $[[RES1:[0-9]+]], $[[UB2]], $[[UB1]]
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
   ret void
 }
@@ -127,10 +127,10 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[UB1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
 ; CHECK:        .end    orUb0
   ret void
@@ -148,12 +148,12 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    addiu   $[[CONST:[0-9]+]], $zero, 1
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    or      $[[RES1:[0-9]+]], $[[UB1]], $[[CONST]]
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
 ; CHECK:        .end    orUb1
   ret void
@@ -173,13 +173,13 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB2_ADDR:[0-9]+]], %got(ub2)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    lbu     $[[UB2:[0-9]+]], 0($[[UB2_ADDR]])
 ; CHECK-DAG:    xor     $[[RES1:[0-9]+]], $[[UB2]], $[[UB1]]
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
   ret void
 }
@@ -196,11 +196,11 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    xor     $[[RES1:[0-9]+]], $[[UB1]], $zero
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
 ; CHECK:        .end    xorUb0
   ret void
@@ -218,12 +218,12 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK:        addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK:        addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK-DAG:    addiu   $[[CONST:[0-9]+]], $zero, 1
 ; CHECK-DAG:    lw      $[[UB1_ADDR:[0-9]+]], %got(ub1)($[[REG_GP]])
 ; CHECK-DAG:    lbu     $[[UB1:[0-9]+]], 0($[[UB1_ADDR]])
 ; CHECK-DAG:    xor     $[[RES1:[0-9]+]], $[[UB1]], $[[CONST]]
 ; CHECK:        andi    $[[RES:[0-9]+]], $[[RES1]], 1
+; CHECK:        lw      $[[UB_ADDR:[0-9]+]], %got(ub)($[[REG_GP]])
 ; CHECK:        sb      $[[RES]], 0($[[UB_ADDR]])
 ; CHECK:        .end    xorUb1
   ret void

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/overflt.ll b/llvm/test/CodeGen/Mips/Fast-ISel/overflt.ll
index ed1dc2157355..6ee50fe3fe99 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/overflt.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/overflt.ll
@@ -15,10 +15,10 @@ entry:
   %0 = load float*, float** @y, align 4
   %arrayidx = getelementptr inbounds float, float* %0, i32 64000
   store float 5.500000e+00, float* %arrayidx, align 4
-; CHECK:        lui     $[[REG_FPCONST_INT:[0-9]+]], 16560
-; CHECK:        mtc1    $[[REG_FPCONST_INT]], $f[[REG_FPCONST:[0-9]+]]
 ; CHECK:        lw      $[[REG_Y_GOT:[0-9]+]], %got(y)(${{[0-9]+}})
 ; CHECK:        lw      $[[REG_Y:[0-9]+]], 0($[[REG_Y_GOT]])
+; CHECK:        lui     $[[REG_FPCONST_INT:[0-9]+]], 16560
+; CHECK:        mtc1    $[[REG_FPCONST_INT]], $f[[REG_FPCONST:[0-9]+]]
 ; CHECK:        lui     $[[REG_IDX_UPPER:[0-9]+]], 3
 ; CHECK:        ori     $[[REG_IDX:[0-9]+]], $[[REG_IDX_UPPER]], 59392
 ; CHECK:        addu    $[[REG_Y_IDX:[0-9]+]], $[[REG_IDX]], $[[REG_Y]]

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/shftopm.ll b/llvm/test/CodeGen/Mips/Fast-ISel/shftopm.ll
index 184db73ceab0..871accfff2fe 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/shftopm.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/shftopm.ll
@@ -25,7 +25,7 @@ entry:
 ; CHECK-DAG:    lw      $[[S1_ADDR:[0-9]+]], %got(s1)($[[REG_GP]])
 ; CHECK-DAG:    lhu     $[[S1:[0-9]+]], 0($[[S1_ADDR]])
 ; CHECK-DAG:    lhu     $[[S2:[0-9]+]], 0($[[S2_ADDR]])
-; CHECK:        sllv    $[[RES:[0-9]+]], $[[S1]], $[[S2]]
+; CHECK-DAG:    sllv    $[[RES:[0-9]+]], $[[S1]], $[[S2]]
 ; CHECK:        sh      $[[RES]], 0($[[S3_ADDR]])
   ret void
 }
@@ -39,10 +39,10 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK-DAG:    addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK-DAG:    addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[S3_ADDR:[0-9]+]], %got(s3)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[S1_ADDR:[0-9]+]], %got(s1)($[[REG_GP]])
 ; CHECK-DAG:    lhu     $[[S1:[0-9]+]], 0($[[S1_ADDR]])
 ; CHECK:        sll     $[[RES:[0-9]+]], $[[S1]],  5
+; CHECK:        lw      $[[S3_ADDR:[0-9]+]], %got(s3)($[[REG_GP]])
 ; CHECK:        sh      $[[RES]], 0($[[S3_ADDR]])
   ret void
 }
@@ -58,12 +58,12 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK-DAG:    addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK-DAG:    addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[US3_ADDR:[0-9]+]], %got(us3)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[US2_ADDR:[0-9]+]], %got(us2)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[US1_ADDR:[0-9]+]], %got(us1)($[[REG_GP]])
 ; CHECK-DAG:    lhu     $[[US1:[0-9]+]], 0($[[US1_ADDR]])
 ; CHECK-DAG:    lhu     $[[US2:[0-9]+]], 0($[[US2_ADDR]])
 ; CHECK:        srlv    $[[RES:[0-9]+]], $[[US1]], $[[US2]]
+; CHECK:        lw      $[[US3_ADDR:[0-9]+]], %got(us3)($[[REG_GP]])
 ; CHECK:        sh      $[[RES]], 0($[[S3_ADDR]])
 }
 
@@ -76,10 +76,10 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK-DAG:    addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK-DAG:    addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[US3_ADDR:[0-9]+]], %got(us3)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[US1_ADDR:[0-9]+]], %got(us1)($[[REG_GP]])
 ; CHECK-DAG:    lhu     $[[US1:[0-9]+]], 0($[[US1_ADDR]])
 ; CHECK:        srl     $[[RES:[0-9]+]], $[[US1]], 4
+; CHECK:        lw      $[[US3_ADDR:[0-9]+]], %got(us3)($[[REG_GP]])
 ; CHECK:        sh      $[[RES]], 0($[[S3_ADDR]])
   ret void
 }
@@ -94,12 +94,12 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK-DAG:    addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK-DAG:    addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[S3_ADDR:[0-9]+]], %got(s3)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[S2_ADDR:[0-9]+]], %got(s2)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[S1_ADDR:[0-9]+]], %got(s1)($[[REG_GP]])
 ; CHECK-DAG:    lhu     $[[S1:[0-9]+]], 0($[[S1_ADDR]])
 ; CHECK-DAG:    lhu     $[[S2:[0-9]+]], 0($[[S2_ADDR]])
 ; CHECK:        srav    $[[RES:[0-9]+]], $[[S1]], $[[S2]]
+; CHECK:        lw      $[[S3_ADDR:[0-9]+]], %got(s3)($[[REG_GP]])
 ; CHECK:        sh      $[[RES]], 0($[[S3_ADDR]])
   ret void
 }
@@ -113,10 +113,10 @@ entry:
 ; CHECK:        lui     $[[REG_GPa:[0-9]+]], %hi(_gp_disp)
 ; CHECK-DAG:    addiu   $[[REG_GPb:[0-9]+]], $[[REG_GPa]], %lo(_gp_disp)
 ; CHECK-DAG:    addu    $[[REG_GP:[0-9]+]], $[[REG_GPb]], $25
-; CHECK-DAG:    lw      $[[S3_ADDR:[0-9]+]], %got(s3)($[[REG_GP]])
 ; CHECK-DAG:    lw      $[[S1_ADDR:[0-9]+]], %got(s1)($[[REG_GP]])
 ; CHECK-DAG:    lhu     $[[S1:[0-9]+]], 0($[[S1_ADDR]])
 ; CHECK:        sra     $[[RES:[0-9]+]], $[[S1]], 2
+; CHECK:        lw      $[[S3_ADDR:[0-9]+]], %got(s3)($[[REG_GP]])
 ; CHECK:        sh      $[[RES]], 0($[[S3_ADDR]])
   ret void
 }

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/simplestore.ll b/llvm/test/CodeGen/Mips/Fast-ISel/simplestore.ll
index 5cac57b798a8..49b9375f3fa7 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/simplestore.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/simplestore.ll
@@ -9,8 +9,8 @@
 define void @foo()  {
 entry:
   store i32 12345, i32* @abcd, align 4
-; CHECK: 	lw	$[[REG2:[0-9]+]], %got(abcd)(${{[0-9]+}})
 ; CHECK: 	addiu	$[[REG1:[0-9]+]], $zero, 12345
+; CHECK: 	lw	$[[REG2:[0-9]+]], %got(abcd)(${{[0-9]+}})
 ; CHECK: 	sw	$[[REG1]], 0($[[REG2]])
   ret void
 }

diff  --git a/llvm/test/CodeGen/Mips/Fast-ISel/simplestorei.ll b/llvm/test/CodeGen/Mips/Fast-ISel/simplestorei.ll
index 7fc0841e3e70..798c40b161d2 100644
--- a/llvm/test/CodeGen/Mips/Fast-ISel/simplestorei.ll
+++ b/llvm/test/CodeGen/Mips/Fast-ISel/simplestorei.ll
@@ -10,8 +10,8 @@ define void @si2_1() #0 {
 entry:
   store i32 32767, i32* @ijk, align 4
 ; CHECK:        .ent    si2_1
-; CHECK:        lw      $[[REG2:[0-9]+]], %got(ijk)(${{[0-9]+}})
 ; CHECK:        addiu   $[[REG1:[0-9]+]], $zero, 32767
+; CHECK:        lw      $[[REG2:[0-9]+]], %got(ijk)(${{[0-9]+}})
 ; CHECK:        sw      $[[REG1]], 0($[[REG2]])
 
   ret void
@@ -34,8 +34,8 @@ define void @ui2_1() #0 {
 entry:
   store i32 65535, i32* @ijk, align 4
 ; CHECK:        .ent    ui2_1
-; CHECK:        lw      $[[REG2:[0-9]+]], %got(ijk)(${{[0-9]+}})
 ; CHECK:        ori     $[[REG1:[0-9]+]], $zero, 65535
+; CHECK:        lw      $[[REG2:[0-9]+]], %got(ijk)(${{[0-9]+}})
 ; CHECK:        sw      $[[REG1]], 0($[[REG2]])
   ret void
 }
@@ -45,8 +45,8 @@ define void @ui4_1() #0 {
 entry:
   store i32 983040, i32* @ijk, align 4
 ; CHECK:        .ent    ui4_1
-; CHECK:        lw      $[[REG2:[0-9]+]], %got(ijk)(${{[0-9]+}})
 ; CHECK:        lui     $[[REG1:[0-9]+]], 15
+; CHECK:        lw      $[[REG2:[0-9]+]], %got(ijk)(${{[0-9]+}})
 ; CHECK:        sw      $[[REG1]], 0($[[REG2]])
   ret void
 }

diff  --git a/llvm/test/CodeGen/Mips/emergency-spill-slot-near-fp.ll b/llvm/test/CodeGen/Mips/emergency-spill-slot-near-fp.ll
index 625abc1a7e04..042d003a8224 100644
--- a/llvm/test/CodeGen/Mips/emergency-spill-slot-near-fp.ll
+++ b/llvm/test/CodeGen/Mips/emergency-spill-slot-near-fp.ll
@@ -2,6 +2,9 @@
 ; Check that register scavenging spill slot is close to $fp.
 target triple="mipsel--"
 
+; FIXME: After recent rework to FastISel, don't know how to trigger the
+; emergency spill slot.  Filed PR48301.
+; XFAIL: *
 @var = external global i32
 @ptrvar = external global i8*
 

diff  --git a/llvm/test/CodeGen/PowerPC/elf-common.ll b/llvm/test/CodeGen/PowerPC/elf-common.ll
index 3adeabbcf0a6..e1ce0aecebdd 100644
--- a/llvm/test/CodeGen/PowerPC/elf-common.ll
+++ b/llvm/test/CodeGen/PowerPC/elf-common.ll
@@ -12,9 +12,11 @@ define dso_local signext i32 @test_comm() nounwind {
 ; NOOPT-LABEL: test_comm:
 ; NOOPT:       # %bb.0: # %entry
 ; NOOPT-NEXT:    addis 3, 2, comm_glob at toc@ha
-; NOOPT-NEXT:    addi 5, 3, comm_glob at toc@l
-; NOOPT-NEXT:    lwz 3, 0(5)
+; NOOPT-NEXT:    addi 3, 3, comm_glob at toc@l
+; NOOPT-NEXT:    lwz 3, 0(3)
 ; NOOPT-NEXT:    addi 4, 3, 1
+; NOOPT-NEXT:    addis 5, 2, comm_glob at toc@ha
+; NOOPT-NEXT:    addi 5, 5, comm_glob at toc@l
 ; NOOPT-NEXT:    stw 4, 0(5)
 ; NOOPT-NEXT:    extsw 3, 3
 ; NOOPT-NEXT:    blr

diff  --git a/llvm/test/CodeGen/PowerPC/fast-isel-load-store.ll b/llvm/test/CodeGen/PowerPC/fast-isel-load-store.ll
index 5bd736b30842..edd56f31d3a1 100644
--- a/llvm/test/CodeGen/PowerPC/fast-isel-load-store.ll
+++ b/llvm/test/CodeGen/PowerPC/fast-isel-load-store.ll
@@ -90,8 +90,8 @@ define dso_local void @t7(i8 %v) nounwind {
 ; ELF64-LABEL: t7:
   %1 = add nsw i8 %v, 1
   store i8 %1, i8* @a, align 1
-; ELF64: addis
 ; ELF64: addi
+; ELF64: addis
 ; ELF64: addi
 ; ELF64: stb
   ret void
@@ -101,8 +101,8 @@ define dso_local void @t8(i16 %v) nounwind {
 ; ELF64-LABEL: t8:
   %1 = add nsw i16 %v, 1
   store i16 %1, i16* @b, align 2
-; ELF64: addis
 ; ELF64: addi
+; ELF64: addis
 ; ELF64: addi
 ; ELF64: sth
   ret void
@@ -112,8 +112,8 @@ define dso_local void @t9(i32 %v) nounwind {
 ; ELF64-LABEL: t9:
   %1 = add nsw i32 %v, 1
   store i32 %1, i32* @c, align 4
-; ELF64: addis
 ; ELF64: addi
+; ELF64: addis
 ; ELF64: addi
 ; ELF64: stw
   ret void
@@ -123,8 +123,8 @@ define dso_local void @t10(i64 %v) nounwind {
 ; ELF64-LABEL: t10:
   %1 = add nsw i64 %v, 1
   store i64 %1, i64* @d, align 4
-; ELF64: addis
 ; ELF64: addi
+; ELF64: addis
 ; ELF64: addi
 ; ELF64: std
   ret void
@@ -184,8 +184,8 @@ define dso_local void @t15(i64 %v) nounwind {
 ; ELF64-LABEL: t15:
   %1 = add nsw i64 %v, 1
   store i64 %1, i64* getelementptr inbounds (%struct.t, %struct.t* @h, i32 0, i32 1), align 1
-; ELF64: addis
 ; ELF64: addi
+; ELF64: addis
 ; ELF64: addi
 ; ELF64: li
 ; ELF64: stdx
@@ -209,8 +209,8 @@ define dso_local void @t17(i64 %v) nounwind {
 ; ELF64-LABEL: t17:
   %1 = add nsw i64 %v, 1
   store i64 %1, i64* getelementptr inbounds ([8192 x i64], [8192 x i64]* @i, i32 0, i64 5000), align 8
-; ELF64: addis
 ; ELF64: addi
+; ELF64: addis
 ; ELF64: addi
 ; ELF64: lis
 ; ELF64: ori

diff  --git a/llvm/test/CodeGen/PowerPC/mcm-1.ll b/llvm/test/CodeGen/PowerPC/mcm-1.ll
index 08bae365f18b..47266563edc5 100644
--- a/llvm/test/CodeGen/PowerPC/mcm-1.ll
+++ b/llvm/test/CodeGen/PowerPC/mcm-1.ll
@@ -33,13 +33,15 @@ entry:
 ; CHECK: addis [[REG1:[0-9]+]], 2, .LC[[TOCNUM:[0-9]+]]@toc at ha
 ; CHECK: ld [[REG2:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG1]])
 ; CHECK: lwz {{[0-9]+}}, 0([[REG2]])
-; CHECK: stw {{[0-9]+}}, 0([[REG2]])
+; CHECK: addis [[REG3:[0-9]+]], 2, .LC[[TOCNUM]]@toc at ha
+; CHECK: ld [[REG4:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG3]])
+; CHECK: stw {{[0-9]+}}, 0([[REG4]])
 
 
 ; CHECK:      .section .toc,"aw", at progbits
 ; CHECK-NEXT: .LC0:
 ; CHECK-NEXT:  .tc ei[TC],ei
 ; CHECK-NEXT: .LC1:
-; CHECK-NEXT:  .tc fi[TC],fi
-; CHECK-NEXT: .LC2:
 ; CHECK-NEXT:  .tc gi[TC],gi
+; CHECK-NEXT: .LC2:
+; CHECK-NEXT:  .tc fi[TC],fi

diff  --git a/llvm/test/CodeGen/PowerPC/mcm-13.ll b/llvm/test/CodeGen/PowerPC/mcm-13.ll
index d7c50efbbae5..86748e3f4119 100644
--- a/llvm/test/CodeGen/PowerPC/mcm-13.ll
+++ b/llvm/test/CodeGen/PowerPC/mcm-13.ll
@@ -21,7 +21,9 @@ entry:
 ; CHECK: addis [[REG1:[0-9]+]], 2, .LC[[TOCNUM:[0-9]+]]@toc at ha
 ; CHECK: ld [[REG2:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG1]])
 ; CHECK: lwz {{[0-9]+}}, 0([[REG2]])
-; CHECK: stw {{[0-9]+}}, 0([[REG2]])
+; CHECK: addis [[REG3:[0-9]+]], 2, .LC[[TOCNUM]]@toc at ha
+; CHECK: ld [[REG4:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG3]])
+; CHECK: stw {{[0-9]+}}, 0([[REG4]])
 ; CHECK: .section .toc
 ; CHECK: .LC[[TOCNUM]]:
 ; CHECK: .tc {{[a-z0-9A-Z_.]+}}[TC],{{[a-z0-9A-Z_.]+}}

diff  --git a/llvm/test/CodeGen/PowerPC/mcm-2.ll b/llvm/test/CodeGen/PowerPC/mcm-2.ll
index 980fcb7caa10..5fbc5ffa5b9c 100644
--- a/llvm/test/CodeGen/PowerPC/mcm-2.ll
+++ b/llvm/test/CodeGen/PowerPC/mcm-2.ll
@@ -21,7 +21,9 @@ entry:
 ; MEDIUM: addis [[REG1:[0-9]+]], 2, [[VAR:[a-z0-9A-Z_.]+]]@toc at ha
 ; MEDIUM: addi [[REG2:[0-9]+]], [[REG1]], [[VAR]]@toc at l
 ; MEDIUM: lwz {{[0-9]+}}, 0([[REG2]])
-; MEDIUM: stw {{[0-9]+}}, 0([[REG2]])
+; MEDIUM: addis [[REG3:[0-9]+]], 2, [[VAR]]@toc at ha
+; MEDIUM: addi [[REG4:[0-9]+]], [[REG3]], [[VAR]]@toc at l
+; MEDIUM: stw {{[0-9]+}}, 0([[REG4]])
 ; MEDIUM: .type [[VAR]], at object
 ; MEDIUM: .lcomm [[VAR]],4,4
 
@@ -29,7 +31,9 @@ entry:
 ; LARGE: addis [[REG1:[0-9]+]], 2, [[VAR:[a-z0-9A-Z_.]+]]@toc at ha
 ; LARGE: ld [[REG2:[0-9]+]], [[VAR]]@toc at l([[REG1]])
 ; LARGE: lwz {{[0-9]+}}, 0([[REG2]])
-; LARGE: stw {{[0-9]+}}, 0([[REG2]])
+; LARGE: addis [[REG3:[0-9]+]], 2, [[VAR]]@toc at ha
+; LARGE: ld [[REG4:[0-9]+]], [[VAR]]@toc at l([[REG3]])
+; LARGE: stw {{[0-9]+}}, 0([[REG4]])
 ; LARGE:      .type test_fn_static.si, at object
 ; LARGE-NEXT: .lcomm test_fn_static.si,4,4
 

diff  --git a/llvm/test/CodeGen/PowerPC/mcm-3.ll b/llvm/test/CodeGen/PowerPC/mcm-3.ll
index 27ea250bcfb5..b33bc2d2f12e 100644
--- a/llvm/test/CodeGen/PowerPC/mcm-3.ll
+++ b/llvm/test/CodeGen/PowerPC/mcm-3.ll
@@ -21,7 +21,9 @@ entry:
 ; MEDIUM: addis [[REG1:[0-9]+]], 2, [[VAR:[a-z0-9A-Z_.]+]]@toc at ha
 ; MEDIUM: addi [[REG2:[0-9]+]], [[REG1]], [[VAR]]@toc at l
 ; MEDIUM: lwz {{[0-9]+}}, 0([[REG2]])
-; MEDIUM: stw {{[0-9]+}}, 0([[REG2]])
+; MEDIUM: addis [[REG3:[0-9]+]], 2, [[VAR]]@toc at ha
+; MEDIUM: addi [[REG4:[0-9]+]], [[REG3]], [[VAR]]@toc at l
+; MEDIUM: stw {{[0-9]+}}, 0([[REG4]])
 ; MEDIUM: .type [[VAR]], at object
 ; MEDIUM: .data
 ; MEDIUM: .globl [[VAR]]
@@ -32,7 +34,9 @@ entry:
 ; LARGE: addis [[REG1:[0-9]+]], 2, [[VAR:[a-z0-9A-Z_.]+]]@toc at ha
 ; LARGE: ld [[REG2:[0-9]+]], [[VAR]]@toc at l([[REG1]])
 ; LARGE: lwz {{[0-9]+}}, 0([[REG2]])
-; LARGE: stw {{[0-9]+}}, 0([[REG2]])
+; LARGE: addis [[REG3:[0-9]+]], 2, [[VAR]]@toc at ha
+; LARGE: ld [[REG4:[0-9]+]], [[VAR]]@toc at l([[REG3]])
+; LARGE: stw {{[0-9]+}}, 0([[REG4]])
 ; LARGE:      .type gi, at object
 ; LARGE-NEXT: .data
 ; LARGE-NEXT: .globl gi

diff  --git a/llvm/test/CodeGen/PowerPC/mcm-6.ll b/llvm/test/CodeGen/PowerPC/mcm-6.ll
index 0572f107dff4..5a8eb7bcb0ab 100644
--- a/llvm/test/CodeGen/PowerPC/mcm-6.ll
+++ b/llvm/test/CodeGen/PowerPC/mcm-6.ll
@@ -21,7 +21,9 @@ entry:
 ; CHECK: addis [[REG1:[0-9]+]], 2, .LC[[TOCNUM:[0-9]+]]@toc at ha
 ; CHECK: ld [[REG2:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG1]])
 ; CHECK: lwz {{[0-9]+}}, 0([[REG2]])
-; CHECK: stw {{[0-9]+}}, 0([[REG2]])
+; CHECK: addis [[REG3:[0-9]+]], 2, .LC[[TOCNUM]]@toc at ha
+; CHECK: ld [[REG4:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG3]])
+; CHECK: stw {{[0-9]+}}, 0([[REG4]])
 ; CHECK: .comm ti,4,4
 ; CHECK:      .section .toc,"aw", at progbits
 ; CHECK-NEXT: .LC[[TOCNUM]]:

diff  --git a/llvm/test/CodeGen/PowerPC/mcm-9.ll b/llvm/test/CodeGen/PowerPC/mcm-9.ll
index c4950ad4a92e..9afd15a98fd7 100644
--- a/llvm/test/CodeGen/PowerPC/mcm-9.ll
+++ b/llvm/test/CodeGen/PowerPC/mcm-9.ll
@@ -21,7 +21,9 @@ entry:
 ; CHECK: addis [[REG1:[0-9]+]], 2, .LC[[TOCNUM:[0-9]+]]@toc at ha
 ; CHECK: ld [[REG2:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG1]])
 ; CHECK: lwz {{[0-9]+}}, 0([[REG2]])
-; CHECK: stw {{[0-9]+}}, 0([[REG2]])
+; CHECK: addis [[REG3:[0-9]+]], 2, .LC[[TOCNUM]]@toc at ha
+; CHECK: ld [[REG4:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG3]])
+; CHECK: stw {{[0-9]+}}, 0([[REG4]])
 ; CHECK: .section .toc
 ; CHECK: .LC[[TOCNUM]]:
 ; CHECK: .tc {{[a-z0-9A-Z_.]+}}[TC],{{[a-z0-9A-Z_.]+}}

diff  --git a/llvm/test/CodeGen/PowerPC/mcm-default.ll b/llvm/test/CodeGen/PowerPC/mcm-default.ll
index a7ac560971b9..69f74eeeeac4 100644
--- a/llvm/test/CodeGen/PowerPC/mcm-default.ll
+++ b/llvm/test/CodeGen/PowerPC/mcm-default.ll
@@ -20,7 +20,9 @@ entry:
 ; CHECK: addis [[REG1:[0-9]+]], 2, .LC[[TOCNUM:[0-9]+]]@toc at ha
 ; CHECK: ld [[REG2:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG1]])
 ; CHECK: lwz {{[0-9]+}}, 0([[REG2]])
-; CHECK: stw {{[0-9]+}}, 0([[REG2]])
+; CHECK: addis [[REG3:[0-9]+]], 2, .LC[[TOCNUM]]@toc at ha
+; CHECK: ld [[REG4:[0-9]+]], .LC[[TOCNUM]]@toc at l([[REG3]])
+; CHECK: stw {{[0-9]+}}, 0([[REG4]])
 ; CHECK: .section .toc
 ; CHECK: .LC[[TOCNUM]]:
 ; CHECK: .tc {{[a-z0-9A-Z_.]+}}[TC],{{[a-z0-9A-Z_.]+}}

diff  --git a/llvm/test/CodeGen/X86/atomic-unordered.ll b/llvm/test/CodeGen/X86/atomic-unordered.ll
index fe34cc3290f9..f5eaa6f2409a 100644
--- a/llvm/test/CodeGen/X86/atomic-unordered.ll
+++ b/llvm/test/CodeGen/X86/atomic-unordered.ll
@@ -366,18 +366,17 @@ define void @store_i256(i256* %ptr, i256 %v) {
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    subq $40, %rsp
 ; CHECK-O0-NEXT:    .cfi_def_cfa_offset 48
-; CHECK-O0-NEXT:    movq %rcx, %rax
-; CHECK-O0-NEXT:    movq %rdx, (%rsp) # 8-byte Spill
-; CHECK-O0-NEXT:    movq %rsi, %r9
+; CHECK-O0-NEXT:    movq %rdx, %rax
+; CHECK-O0-NEXT:    movq %rsi, (%rsp) # 8-byte Spill
 ; CHECK-O0-NEXT:    movq %rdi, %rsi
 ; CHECK-O0-NEXT:    movq (%rsp), %rdi # 8-byte Reload
-; CHECK-O0-NEXT:    xorl %ecx, %ecx
 ; CHECK-O0-NEXT:    leaq {{[0-9]+}}(%rsp), %rdx
-; CHECK-O0-NEXT:    movq %r9, {{[0-9]+}}(%rsp)
 ; CHECK-O0-NEXT:    movq %rdi, {{[0-9]+}}(%rsp)
 ; CHECK-O0-NEXT:    movq %rax, {{[0-9]+}}(%rsp)
+; CHECK-O0-NEXT:    movq %rcx, {{[0-9]+}}(%rsp)
 ; CHECK-O0-NEXT:    movq %r8, {{[0-9]+}}(%rsp)
 ; CHECK-O0-NEXT:    movl $32, %edi
+; CHECK-O0-NEXT:    xorl %ecx, %ecx
 ; CHECK-O0-NEXT:    callq __atomic_store
 ; CHECK-O0-NEXT:    addq $40, %rsp
 ; CHECK-O0-NEXT:    .cfi_def_cfa_offset 8
@@ -724,8 +723,8 @@ define i64 @load_fold_sdiv1(i64* %p) {
 ; CHECK-O0-LABEL: load_fold_sdiv1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
-; CHECK-O0-NEXT:    cqto
 ; CHECK-O0-NEXT:    movl $15, %ecx
+; CHECK-O0-NEXT:    cqto
 ; CHECK-O0-NEXT:    idivq %rcx
 ; CHECK-O0-NEXT:    retq
 ;
@@ -814,9 +813,9 @@ define i64 @load_fold_udiv1(i64* %p) {
 ; CHECK-O0-LABEL: load_fold_udiv1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
-; CHECK-O0-NEXT:    xorl %ecx, %ecx
-; CHECK-O0-NEXT:    movl %ecx, %edx
 ; CHECK-O0-NEXT:    movl $15, %ecx
+; CHECK-O0-NEXT:    xorl %edx, %edx
+; CHECK-O0-NEXT:    # kill: def $rdx killed $edx
 ; CHECK-O0-NEXT:    divq %rcx
 ; CHECK-O0-NEXT:    retq
 ;
@@ -908,8 +907,8 @@ define i64 @load_fold_srem1(i64* %p) {
 ; CHECK-O0-LABEL: load_fold_srem1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
-; CHECK-O0-NEXT:    cqto
 ; CHECK-O0-NEXT:    movl $15, %ecx
+; CHECK-O0-NEXT:    cqto
 ; CHECK-O0-NEXT:    idivq %rcx
 ; CHECK-O0-NEXT:    movq %rdx, %rax
 ; CHECK-O0-NEXT:    retq
@@ -1007,9 +1006,9 @@ define i64 @load_fold_urem1(i64* %p) {
 ; CHECK-O0-LABEL: load_fold_urem1:
 ; CHECK-O0:       # %bb.0:
 ; CHECK-O0-NEXT:    movq (%rdi), %rax
-; CHECK-O0-NEXT:    xorl %ecx, %ecx
-; CHECK-O0-NEXT:    movl %ecx, %edx
 ; CHECK-O0-NEXT:    movl $15, %ecx
+; CHECK-O0-NEXT:    xorl %edx, %edx
+; CHECK-O0-NEXT:    # kill: def $rdx killed $edx
 ; CHECK-O0-NEXT:    divq %rcx
 ; CHECK-O0-NEXT:    movq %rdx, %rax
 ; CHECK-O0-NEXT:    retq

diff  --git a/llvm/test/CodeGen/X86/atomic64.ll b/llvm/test/CodeGen/X86/atomic64.ll
index 0e81663020a7..42745baa554f 100644
--- a/llvm/test/CodeGen/X86/atomic64.ll
+++ b/llvm/test/CodeGen/X86/atomic64.ll
@@ -705,10 +705,10 @@ define void @atomic_fetch_cmpxchg64() nounwind {
 ; I486-NEXT:    movl %esp, %ebp
 ; I486-NEXT:    andl $-8, %esp
 ; I486-NEXT:    subl $32, %esp
-; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    leal {{[0-9]+}}(%esp), %ecx
 ; I486-NEXT:    movl $0, {{[0-9]+}}(%esp)
 ; I486-NEXT:    movl $0, {{[0-9]+}}(%esp)
+; I486-NEXT:    leal sc64, %eax
 ; I486-NEXT:    movl %esp, %eax
 ; I486-NEXT:    movl %ecx, 4(%eax)
 ; I486-NEXT:    movl $2, 20(%eax)
@@ -786,10 +786,10 @@ define void @atomic_fetch_swapf64(double %x) nounwind {
 ; I486-NEXT:    andl $-8, %esp
 ; I486-NEXT:    subl $24, %esp
 ; I486-NEXT:    fldl 8(%ebp)
-; I486-NEXT:    leal fsc64, %eax
 ; I486-NEXT:    fstpl {{[0-9]+}}(%esp)
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %ecx
 ; I486-NEXT:    movl {{[0-9]+}}(%esp), %edx
+; I486-NEXT:    leal fsc64, %eax
 ; I486-NEXT:    movl %esp, %eax
 ; I486-NEXT:    movl %edx, 8(%eax)
 ; I486-NEXT:    movl %ecx, 4(%eax)

diff  --git a/llvm/test/CodeGen/X86/crash-O0.ll b/llvm/test/CodeGen/X86/crash-O0.ll
index 54f7c7597e50..1b64247d170a 100644
--- a/llvm/test/CodeGen/X86/crash-O0.ll
+++ b/llvm/test/CodeGen/X86/crash-O0.ll
@@ -14,16 +14,15 @@ define i32 @div8() nounwind {
 ; CHECK:       ## %bb.0: ## %entry
 ; CHECK-NEXT:    pushq %rbp
 ; CHECK-NEXT:    movq %rsp, %rbp
+; CHECK-NEXT:    ## implicit-def: $rax
+; CHECK-NEXT:    movb %al, %cl
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    ## kill: def $al killed $al killed $eax
-; CHECK-NEXT:    movb %al, {{[-0-9]+}}(%r{{[sb]}}p) ## 1-byte Spill
-; CHECK-NEXT:    ## implicit-def: $rcx
-; CHECK-NEXT:    ## kill: def $cl killed $cl killed $rcx
 ; CHECK-NEXT:    movzbw %al, %ax
 ; CHECK-NEXT:    divb %cl
-; CHECK-NEXT:    movb %al, %dl
-; CHECK-NEXT:    movb {{[-0-9]+}}(%r{{[sb]}}p), %al ## 1-byte Reload
-; CHECK-NEXT:    movb %dl, {{[-0-9]+}}(%r{{[sb]}}p) ## 1-byte Spill
+; CHECK-NEXT:    movb %al, {{[-0-9]+}}(%r{{[sb]}}p) ## 1-byte Spill
+; CHECK-NEXT:    xorl %eax, %eax
+; CHECK-NEXT:    ## kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    movzbw %al, %ax
 ; CHECK-NEXT:    divb %cl
 ; CHECK-NEXT:    shrw $8, %ax
@@ -35,7 +34,9 @@ define i32 @div8() nounwind {
 ; CHECK-NEXT:    movzbl %al, %eax
 ; CHECK-NEXT:    ## implicit-def: $ecx
 ; CHECK-NEXT:    imull %ecx, %eax
+; CHECK-NEXT:    ## implicit-def: $ecx
 ; CHECK-NEXT:    addl %ecx, %eax
+; CHECK-NEXT:    ## implicit-def: $ecx
 ; CHECK-NEXT:    cmpl %ecx, %eax
 ; CHECK-NEXT:    je LBB0_3
 ; CHECK-NEXT:  LBB0_2: ## %"40"

diff  --git a/llvm/test/CodeGen/X86/fast-isel-constant.ll b/llvm/test/CodeGen/X86/fast-isel-constant.ll
deleted file mode 100644
index 2c7cbc64da01..000000000000
--- a/llvm/test/CodeGen/X86/fast-isel-constant.ll
+++ /dev/null
@@ -1,24 +0,0 @@
-; RUN: llc < %s -mtriple=x86_64-- -O0 | FileCheck %s
-; Make sure fast-isel doesn't reset the materialised constant map
-; across an intrinsic call.
-
-; CHECK: movl	$100000
-; CHECK-NOT: movl	$100000
-define i1 @test1(i32 %v1, i32 %v2, i32* %X) nounwind {
-entry:
-  %a = shl i32 100000, %v1
-  %t = call {i32, i1} @llvm.sadd.with.overflow.i32(i32 %a, i32 %v2)
-  %ext = extractvalue {i32, i1} %t, 0
-  %sum = shl i32 100000, %ext
-  %obit = extractvalue {i32, i1} %t, 1
-  br i1 %obit, label %overflow, label %normal
-
-normal:
-  store i32 %sum, i32* %X
-  br label %overflow
-
-overflow:
-  ret i1 false
-}
-
-declare {i32, i1} @llvm.sadd.with.overflow.i32(i32, i32)

diff  --git a/llvm/test/CodeGen/X86/fast-isel-mem.ll b/llvm/test/CodeGen/X86/fast-isel-mem.ll
index 42f210199e64..3564c5db3567 100644
--- a/llvm/test/CodeGen/X86/fast-isel-mem.ll
+++ b/llvm/test/CodeGen/X86/fast-isel-mem.ll
@@ -14,24 +14,30 @@ entry:
 	ret i32 %2
 ; This should fold one of the loads into the add.
 ; CHECK-LABEL: loadgv:
+; CHECK: 	movl	L_src$non_lazy_ptr, %eax
+; CHECK: 	movl	(%eax), %eax
 ; CHECK: 	movl	L_src$non_lazy_ptr, %ecx
-; CHECK: 	movl	(%ecx), %eax
 ; CHECK: 	addl	(%ecx), %eax
+; CHECK: 	movl	L_src$non_lazy_ptr, %ecx
 ; CHECK: 	movl	%eax, (%ecx)
 ; CHECK: 	ret
 
 ; ATOM:	loadgv:
-; ATOM:		movl    L_src$non_lazy_ptr, %ecx
-; ATOM:         movl    (%ecx), %eax
+; ATOM:	        movl    L_src$non_lazy_ptr, %eax
+; ATOM:         movl    (%eax), %eax
+; ATOM:	        movl    L_src$non_lazy_ptr, %ecx
 ; ATOM:         addl    (%ecx), %eax
+; ATOM:	        movl    L_src$non_lazy_ptr, %ecx
 ; ATOM:         movl    %eax, (%ecx)
 ; ATOM:         ret
 
 ;; dso_preemptable src is loaded via GOT indirection.
 ; ELF64-LABEL: loadgv:
-; ELF64:        movq    src at GOTPCREL(%rip), %rcx
-; ELF64-NEXT:   movl    (%rcx), %eax
+; ELF64:        movq    src at GOTPCREL(%rip), %rax
+; ELF64-NEXT:   movl    (%rax), %eax
+; ELF64-NEXT:   movq    src at GOTPCREL(%rip), %rcx
 ; ELF64-NEXT:   addl    (%rcx), %eax
+; ELF64-NEXT:   movq    src at GOTPCREL(%rip), %rcx
 ; ELF64-NEXT:   movl    %eax, (%rcx)
 ; ELF64-NEXT:   retq
 

diff  --git a/llvm/test/CodeGen/X86/fast-isel-prolog-dbgloc.ll b/llvm/test/CodeGen/X86/fast-isel-prolog-dbgloc.ll
new file mode 100644
index 000000000000..e9800ff1197e
--- /dev/null
+++ b/llvm/test/CodeGen/X86/fast-isel-prolog-dbgloc.ll
@@ -0,0 +1,74 @@
+;; A parameter list long enough to put one parameter on the stack, plus
+;; at least one float parameter, triggered a corner case that messed up
+;; setting prologue_end.
+;;
+;; Generated using -gmlt from this source:
+;; void call_7i_1f (char c1, float f1, char c2, char c3, char c4, char c5, char c6, short s)
+;; {
+;;   c1 = 'a'; f1 = 0.1; c2 = 5; c3 = 6; c4 = 7;
+;;   c5 = 's'; c6 = 'f'; s = 77;
+;; }
+
+; RUN: llc -mtriple x86_64-- -fast-isel < %s | FileCheck %s
+
+define dso_local void @call_7i_1f(i8 signext %c1, float %f1, i8 signext %c2, i8 signext %c3, i8 signext %c4, i8 signext %c5, i8 signext %c6, i16 signext %s) !dbg !7 {
+entry:
+  %c1.addr = alloca i8, align 1
+  %f1.addr = alloca float, align 4
+  %c2.addr = alloca i8, align 1
+  %c3.addr = alloca i8, align 1
+  %c4.addr = alloca i8, align 1
+  %c5.addr = alloca i8, align 1
+  %c6.addr = alloca i8, align 1
+  %s.addr = alloca i16, align 2
+  store i8 %c1, i8* %c1.addr, align 1
+  store float %f1, float* %f1.addr, align 4
+  store i8 %c2, i8* %c2.addr, align 1
+  store i8 %c3, i8* %c3.addr, align 1
+  store i8 %c4, i8* %c4.addr, align 1
+  store i8 %c5, i8* %c5.addr, align 1
+  store i8 %c6, i8* %c6.addr, align 1
+  store i16 %s, i16* %s.addr, align 2
+  store i8 97, i8* %c1.addr, align 1, !dbg !9
+  store float 0x3FB99999A0000000, float* %f1.addr, align 4, !dbg !10
+  store i8 5, i8* %c2.addr, align 1, !dbg !11
+  store i8 6, i8* %c3.addr, align 1, !dbg !12
+  store i8 7, i8* %c4.addr, align 1, !dbg !13
+  store i8 115, i8* %c5.addr, align 1, !dbg !14
+  store i8 102, i8* %c6.addr, align 1, !dbg !15
+  store i16 77, i16* %s.addr, align 2, !dbg !16
+  ret void, !dbg !17
+}
+
+;; All incoming parameter registers should be homed as part of the prologue.
+; CHECK-DAG: %dil
+; CHECK-DAG: %xmm0
+; CHECK-DAG: %sil
+; CHECK-DAG: %dl
+; CHECK-DAG: %cl
+; CHECK-DAG: %r8b
+; CHECK-DAG: %r9b
+; CHECK:     prologue_end
+
+!llvm.dbg.cu = !{!0}
+!llvm.module.flags = !{!3, !4, !5}
+!llvm.ident = !{!6}
+
+!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 12.0.0", isOptimized: false, runtimeVersion: 0, emissionKind: LineTablesOnly, enums: !2, splitDebugInlining: false, nameTableKind: None)
+!1 = !DIFile(filename: "fast-isel-prolog-dbgloc.c", directory: "/home/probinson/projects/scratch")
+!2 = !{}
+!3 = !{i32 7, !"Dwarf Version", i32 4}
+!4 = !{i32 2, !"Debug Info Version", i32 3}
+!5 = !{i32 1, !"wchar_size", i32 4}
+!6 = !{!"clang version 12.0.0"}
+!7 = distinct !DISubprogram(name: "call_7i_1f", scope: !1, file: !1, line: 3, type: !8, scopeLine: 4, flags: DIFlagPrototyped, spFlags: DISPFlagDefinition, unit: !0, retainedNodes: !2)
+!8 = !DISubroutineType(types: !2)
+!9 = !DILocation(line: 5, column: 6, scope: !7)
+!10 = !DILocation(line: 5, column: 16, scope: !7)
+!11 = !DILocation(line: 5, column: 26, scope: !7)
+!12 = !DILocation(line: 5, column: 34, scope: !7)
+!13 = !DILocation(line: 5, column: 42, scope: !7)
+!14 = !DILocation(line: 6, column: 6, scope: !7)
+!15 = !DILocation(line: 6, column: 16, scope: !7)
+!16 = !DILocation(line: 6, column: 25, scope: !7)
+!17 = !DILocation(line: 7, column: 1, scope: !7)

diff  --git a/llvm/test/CodeGen/X86/fast-isel-select.ll b/llvm/test/CodeGen/X86/fast-isel-select.ll
index 5f65dde68a4a..94477f3286e4 100644
--- a/llvm/test/CodeGen/X86/fast-isel-select.ll
+++ b/llvm/test/CodeGen/X86/fast-isel-select.ll
@@ -9,12 +9,12 @@
 define i32 @fastisel_select(i1 %exchSub2211_, i1 %trunc_8766) {
 ; CHECK-LABEL: fastisel_select:
 ; CHECK:       ## %bb.0:
-; CHECK-NEXT:    movb %sil, %dl
-; CHECK-NEXT:    movb %dil, %cl
+; CHECK-NEXT:    movb %sil, %al
+; CHECK-NEXT:    movb %dil, %dl
+; CHECK-NEXT:    subb %al, %dl
 ; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    subb %dl, %cl
-; CHECK-NEXT:    testb $1, %cl
 ; CHECK-NEXT:    movl $1204476887, %ecx ## imm = 0x47CADBD7
+; CHECK-NEXT:    testb $1, %dl
 ; CHECK-NEXT:    cmovnel %ecx, %eax
 ; CHECK-NEXT:    retq
   %shuffleInternal15257_8932 = sub i1 %exchSub2211_, %trunc_8766

diff  --git a/llvm/test/CodeGen/X86/lvi-hardening-loads.ll b/llvm/test/CodeGen/X86/lvi-hardening-loads.ll
index ff8276f6f1c2..365698925074 100644
--- a/llvm/test/CodeGen/X86/lvi-hardening-loads.ll
+++ b/llvm/test/CodeGen/X86/lvi-hardening-loads.ll
@@ -79,8 +79,8 @@ for.body:                                         ; preds = %for.cond
 ; X64-NOOPT-NEXT: # in Loop: Header=BB0_1 Depth=1
 ; X64-NOOPT-NEXT:  lfence
 ; X64-NOOPT-NEXT:      movl -{{[0-9]+}}(%rsp), %eax
-; X64-NOOPT-NEXT:      cltd
 ; X64-NOOPT-NEXT:      movl $2, %ecx
+; X64-NOOPT-NEXT:      cltd
 ; X64-NOOPT-NEXT:      idivl %ecx
 ; X64-NOOPT-NEXT:      cmpl $0, %edx
 ; X64-NOOPT-NEXT:  lfence

diff  --git a/llvm/test/CodeGen/X86/membarrier.ll b/llvm/test/CodeGen/X86/membarrier.ll
index 59b9365f294a..b70a0a03d609 100644
--- a/llvm/test/CodeGen/X86/membarrier.ll
+++ b/llvm/test/CodeGen/X86/membarrier.ll
@@ -5,11 +5,11 @@
 define i32 @t() {
 ; CHECK-LABEL: t:
 ; CHECK:       # %bb.0:
-; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    movl $1, -{{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    mfence
 ; CHECK-NEXT:    lock decl -{{[0-9]+}}(%rsp)
 ; CHECK-NEXT:    mfence
+; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    retq
   %i = alloca i32, align 4
   store i32 1, i32* %i, align 4

diff  --git a/llvm/test/CodeGen/X86/pr32241.ll b/llvm/test/CodeGen/X86/pr32241.ll
index 6fb770b4a75e..186839e65e07 100644
--- a/llvm/test/CodeGen/X86/pr32241.ll
+++ b/llvm/test/CodeGen/X86/pr32241.ll
@@ -11,8 +11,8 @@ define i32 @_Z3foov() {
 ; CHECK-NEXT:    movw $19417, {{[0-9]+}}(%esp) # imm = 0x4BD9
 ; CHECK-NEXT:    movzwl {{[0-9]+}}(%esp), %eax
 ; CHECK-NEXT:    movl %eax, {{[-0-9]+}}(%e{{[sb]}}p) # 4-byte Spill
-; CHECK-NEXT:    cmpw $0, {{[0-9]+}}(%esp)
 ; CHECK-NEXT:    movb $1, %al
+; CHECK-NEXT:    cmpw $0, {{[0-9]+}}(%esp)
 ; CHECK-NEXT:    movb %al, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
 ; CHECK-NEXT:    jne .LBB0_2
 ; CHECK-NEXT:  # %bb.1: # %lor.rhs
@@ -28,10 +28,10 @@ define i32 @_Z3foov() {
 ; CHECK-NEXT:    cmpl %ecx, %eax
 ; CHECK-NEXT:    setl %al
 ; CHECK-NEXT:    andb $1, %al
-; CHECK-NEXT:    movzbl %al, %eax
-; CHECK-NEXT:    xorl $-1, %eax
-; CHECK-NEXT:    cmpl $0, %eax
+; CHECK-NEXT:    movzbl %al, %ecx
+; CHECK-NEXT:    xorl $-1, %ecx
 ; CHECK-NEXT:    movb $1, %al
+; CHECK-NEXT:    cmpl $0, %ecx
 ; CHECK-NEXT:    movb %al, {{[-0-9]+}}(%e{{[sb]}}p) # 1-byte Spill
 ; CHECK-NEXT:    jne .LBB0_4
 ; CHECK-NEXT:  # %bb.3: # %lor.rhs4

diff  --git a/llvm/test/CodeGen/X86/pr32256.ll b/llvm/test/CodeGen/X86/pr32256.ll
index 8cfc9e0f1ca3..ab51eedff32b 100644
--- a/llvm/test/CodeGen/X86/pr32256.ll
+++ b/llvm/test/CodeGen/X86/pr32256.ll
@@ -9,10 +9,10 @@ define void @_Z1av() {
 ; CHECK:       # %bb.0: # %entry
 ; CHECK-NEXT:    subl $2, %esp
 ; CHECK-NEXT:    .cfi_def_cfa_offset 6
-; CHECK-NEXT:    xorl %eax, %eax
-; CHECK-NEXT:    # kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    movb c, %cl
 ; CHECK-NEXT:    xorb $-1, %cl
+; CHECK-NEXT:    xorl %eax, %eax
+; CHECK-NEXT:    # kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    testb $1, %cl
 ; CHECK-NEXT:    movb %al, (%esp) # 1-byte Spill
 ; CHECK-NEXT:    jne .LBB0_1

diff  --git a/llvm/test/CodeGen/X86/pr32284.ll b/llvm/test/CodeGen/X86/pr32284.ll
index ad3a3ffb19a8..b07322573ffb 100644
--- a/llvm/test/CodeGen/X86/pr32284.ll
+++ b/llvm/test/CodeGen/X86/pr32284.ll
@@ -9,12 +9,12 @@
 define void @foo() {
 ; X86-O0-LABEL: foo:
 ; X86-O0:       # %bb.0: # %entry
+; X86-O0-NEXT:    movzbl c, %ecx
+; X86-O0-NEXT:    xorl %eax, %eax
+; X86-O0-NEXT:    subl %ecx, %eax
+; X86-O0-NEXT:    movslq %eax, %rcx
 ; X86-O0-NEXT:    xorl %eax, %eax
 ; X86-O0-NEXT:    # kill: def $rax killed $eax
-; X86-O0-NEXT:    xorl %ecx, %ecx
-; X86-O0-NEXT:    movzbl c, %edx
-; X86-O0-NEXT:    subl %edx, %ecx
-; X86-O0-NEXT:    movslq %ecx, %rcx
 ; X86-O0-NEXT:    subq %rcx, %rax
 ; X86-O0-NEXT:    # kill: def $al killed $al killed $rax
 ; X86-O0-NEXT:    cmpb $0, %al

diff  --git a/llvm/test/CodeGen/X86/pr32340.ll b/llvm/test/CodeGen/X86/pr32340.ll
index ab48ad2d8e2e..5daec0f1e921 100644
--- a/llvm/test/CodeGen/X86/pr32340.ll
+++ b/llvm/test/CodeGen/X86/pr32340.ll
@@ -13,34 +13,34 @@
 define void @foo() {
 ; X64-LABEL: foo:
 ; X64:       # %bb.0: # %entry
-; X64-NEXT:    xorl %eax, %eax
-; X64-NEXT:    # kill: def $rax killed $eax
 ; X64-NEXT:    movw $0, var_825
-; X64-NEXT:    movzwl var_32, %edx
-; X64-NEXT:    movzwl var_901, %ecx
-; X64-NEXT:    movl %edx, %esi
-; X64-NEXT:    xorl %ecx, %esi
-; X64-NEXT:    movl %edx, %ecx
-; X64-NEXT:    xorl %esi, %ecx
-; X64-NEXT:    addl %edx, %ecx
-; X64-NEXT:    movslq %ecx, %rcx
-; X64-NEXT:    movq %rcx, var_826
 ; X64-NEXT:    movzwl var_32, %ecx
-; X64-NEXT:    # kill: def $rcx killed $ecx
-; X64-NEXT:    movzwl var_901, %edx
-; X64-NEXT:    xorl $51981, %edx # imm = 0xCB0D
-; X64-NEXT:    movslq %edx, %rsi
-; X64-NEXT:    movabsq $-1142377792914660288, %rdx # imm = 0xF02575732E06E440
-; X64-NEXT:    xorq %rdx, %rsi
-; X64-NEXT:    movq %rcx, %rdx
-; X64-NEXT:    xorq %rsi, %rdx
-; X64-NEXT:    xorq $-1, %rdx
+; X64-NEXT:    movzwl var_901, %eax
+; X64-NEXT:    movl %ecx, %edx
+; X64-NEXT:    xorl %eax, %edx
+; X64-NEXT:    movl %ecx, %eax
+; X64-NEXT:    xorl %edx, %eax
+; X64-NEXT:    addl %ecx, %eax
+; X64-NEXT:    cltq
+; X64-NEXT:    movq %rax, var_826
+; X64-NEXT:    movzwl var_32, %eax
+; X64-NEXT:    # kill: def $rax killed $eax
+; X64-NEXT:    movzwl var_901, %ecx
+; X64-NEXT:    xorl $51981, %ecx # imm = 0xCB0D
+; X64-NEXT:    movslq %ecx, %rdx
+; X64-NEXT:    movabsq $-1142377792914660288, %rcx # imm = 0xF02575732E06E440
+; X64-NEXT:    xorq %rcx, %rdx
+; X64-NEXT:    movq %rax, %rcx
 ; X64-NEXT:    xorq %rdx, %rcx
-; X64-NEXT:    movq %rcx, %rdx
-; X64-NEXT:    orq var_57, %rdx
-; X64-NEXT:    orq %rdx, %rcx
-; X64-NEXT:    # kill: def $cx killed $cx killed $rcx
-; X64-NEXT:    movw %cx, var_900
+; X64-NEXT:    xorq $-1, %rcx
+; X64-NEXT:    xorq %rcx, %rax
+; X64-NEXT:    movq %rax, %rcx
+; X64-NEXT:    orq var_57, %rcx
+; X64-NEXT:    orq %rcx, %rax
+; X64-NEXT:    # kill: def $ax killed $ax killed $rax
+; X64-NEXT:    movw %ax, var_900
+; X64-NEXT:    xorl %eax, %eax
+; X64-NEXT:    # kill: def $rax killed $eax
 ; X64-NEXT:    cmpq var_28, %rax
 ; X64-NEXT:    setne %al
 ; X64-NEXT:    andb $1, %al

diff  --git a/llvm/test/CodeGen/X86/pr44749.ll b/llvm/test/CodeGen/X86/pr44749.ll
index daf7e25884a4..7f3db698ed69 100644
--- a/llvm/test/CodeGen/X86/pr44749.ll
+++ b/llvm/test/CodeGen/X86/pr44749.ll
@@ -6,8 +6,6 @@ define i32 @a() {
 ; CHECK:       ## %bb.0: ## %entry
 ; CHECK-NEXT:    pushq %rax
 ; CHECK-NEXT:    .cfi_def_cfa_offset 16
-; CHECK-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; CHECK-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
 ; CHECK-NEXT:    xorl %eax, %eax
 ; CHECK-NEXT:    ## kill: def $al killed $al killed $eax
 ; CHECK-NEXT:    callq _b

diff  --git a/llvm/test/CodeGen/X86/volatile.ll b/llvm/test/CodeGen/X86/volatile.ll
index 089f72a31c62..50c023446e2e 100644
--- a/llvm/test/CodeGen/X86/volatile.ll
+++ b/llvm/test/CodeGen/X86/volatile.ll
@@ -1,18 +1,28 @@
 ; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py
-; RUN: llc < %s -mtriple=i686-- -mattr=sse2 | FileCheck %s
-; RUN: llc < %s -mtriple=i686-- -mattr=sse2 -O0 | FileCheck %s
+; RUN: llc < %s -mtriple=i686-- -mattr=sse2 | FileCheck %s --check-prefixes=ALL,OPT
+; RUN: llc < %s -mtriple=i686-- -mattr=sse2 -O0 | FileCheck %s --check-prefixes=ALL,NOOPT
 
 @x = external dso_local global double
 
 define void @foo() nounwind  {
-; CHECK-LABEL: foo:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; CHECK-NEXT:    xorps %xmm0, %xmm0
-; CHECK-NEXT:    movsd %xmm0, x
-; CHECK-NEXT:    movsd %xmm0, x
-; CHECK-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; CHECK-NEXT:    retl
+; OPT-LABEL: foo:
+; OPT:       # %bb.0:
+; OPT-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; OPT-NEXT:    xorps %xmm0, %xmm0
+; OPT-NEXT:    movsd %xmm0, x
+; OPT-NEXT:    movsd %xmm0, x
+; OPT-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; OPT-NEXT:    retl
+;
+; NOOPT-LABEL: foo:
+; NOOPT:       # %bb.0:
+; NOOPT-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; NOOPT-NEXT:    xorps %xmm0, %xmm0
+; NOOPT-NEXT:    movsd %xmm0, x
+; NOOPT-NEXT:    xorps %xmm0, %xmm0
+; NOOPT-NEXT:    movsd %xmm0, x
+; NOOPT-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; NOOPT-NEXT:    retl
   %a = load volatile double, double* @x
   store volatile double 0.0, double* @x
   store volatile double 0.0, double* @x
@@ -21,10 +31,10 @@ define void @foo() nounwind  {
 }
 
 define void @bar() nounwind  {
-; CHECK-LABEL: bar:
-; CHECK:       # %bb.0:
-; CHECK-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
-; CHECK-NEXT:    retl
+; ALL-LABEL: bar:
+; ALL:       # %bb.0:
+; ALL-NEXT:    movsd {{.*#+}} xmm0 = mem[0],zero
+; ALL-NEXT:    retl
   %c = load volatile double, double* @x
   ret void
 }

diff  --git a/llvm/test/DebugInfo/COFF/lines-bb-start.ll b/llvm/test/DebugInfo/COFF/lines-bb-start.ll
index 711ba704ffd3..4fb923424093 100644
--- a/llvm/test/DebugInfo/COFF/lines-bb-start.ll
+++ b/llvm/test/DebugInfo/COFF/lines-bb-start.ll
@@ -92,7 +92,6 @@ return:                                           ; preds = %if.end, %if.then
 ; CHECK: LBB2_{{.*}}:                                 # %if.end
 ; CHECK-NEXT:    .cv_loc {{.*}} # t.c:5:3
 ; CHECK:         leal 4(%esp), %[[reg:[^ ]*]]
-; CHECK:         #DEBUG_VALUE: lea_dbg_value:v <- [DW_OP_deref] $[[reg]]
 ; CHECK:         movl    %[[reg]], (%esp)
 ; CHECK:         calll   _use_i32
 

diff  --git a/llvm/test/DebugInfo/Mips/delay-slot.ll b/llvm/test/DebugInfo/Mips/delay-slot.ll
index 07c2caa8999c..cbd9358b86c6 100644
--- a/llvm/test/DebugInfo/Mips/delay-slot.ll
+++ b/llvm/test/DebugInfo/Mips/delay-slot.ll
@@ -16,6 +16,7 @@
 ; CHECK: 0x0000000000000004      2      0      1   0             0  is_stmt prologue_end
 ; CHECK: 0x0000000000000020      3      0      1   0             0  is_stmt
 ; CHECK: 0x0000000000000030      4      0      1   0             0  is_stmt
+; CHECK: 0x0000000000000040      0      0      1   0             0
 ; CHECK: 0x0000000000000048      5      0      1   0             0  is_stmt
 ; CHECK: 0x0000000000000050      5      0      1   0             0  is_stmt end_sequence
 

diff  --git a/llvm/test/DebugInfo/X86/fission-ranges.ll b/llvm/test/DebugInfo/X86/fission-ranges.ll
index 1ee80872fae1..0b691a702e2e 100644
--- a/llvm/test/DebugInfo/X86/fission-ranges.ll
+++ b/llvm/test/DebugInfo/X86/fission-ranges.ll
@@ -45,31 +45,31 @@
 ; if they've changed due to a bugfix, change in register allocation, etc.
 
 ; CHECK:      [[A]]:
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000002, 0x0000000f): DW_OP_consts +0, DW_OP_stack_value
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000003, 0x0000000b): DW_OP_reg0 RAX
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000004, 0x00000012): DW_OP_breg7 RSP-4
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000001, 0x00000011): DW_OP_consts +0, DW_OP_stack_value
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000002, 0x0000000b): DW_OP_reg0 RAX
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000003, 0x00000012): DW_OP_breg7 RSP-4
 ; CHECK-NEXT:   DW_LLE_end_of_list   ()
 ; CHECK:      [[E]]:
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000005, 0x0000000b): DW_OP_reg0 RAX
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000006, 0x0000005a): DW_OP_breg7 RSP-48
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000004, 0x0000000b): DW_OP_reg0 RAX
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000005, 0x0000005a): DW_OP_breg7 RSP-48
 ; CHECK-NEXT:   DW_LLE_end_of_list   ()
 ; CHECK:      [[B]]:
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000007, 0x0000000b): DW_OP_reg0 RAX
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000008, 0x00000042): DW_OP_breg7 RSP-24
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000006, 0x0000000b): DW_OP_reg0 RAX
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000007, 0x00000042): DW_OP_breg7 RSP-24
 ; CHECK-NEXT:   DW_LLE_end_of_list   ()
 ; CHECK:      [[D]]:
-; CHECK-NEXT:   DW_LLE_startx_length (0x00000009, 0x0000000b): DW_OP_reg0 RAX
-; CHECK-NEXT:   DW_LLE_startx_length (0x0000000a, 0x0000002a): DW_OP_breg7 RSP-12
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000008, 0x0000000b): DW_OP_reg0 RAX
+; CHECK-NEXT:   DW_LLE_startx_length (0x00000009, 0x0000002a): DW_OP_breg7 RSP-12
 ; CHECK-NEXT:   DW_LLE_end_of_list   ()
 
 ; Make sure we don't produce any relocations in any .dwo section (though in particular, debug_info.dwo)
 ; HDR-NOT: .rela.{{.*}}.dwo
 
 ; Make sure we have enough stuff in the debug_addr to cover the address indexes
-; (10 is the last index in debug_loc.dwo, making 11 entries of 8 bytes each,
-; 11 * 8 == 88 base 10 == 58 base 16)
+; (9 is the last index in debug_loc.dwo, making 10 entries of 8 bytes each,
+; 10 * 8 == 80 base 10 == 50 base 16)
 
-; HDR: .debug_addr 00000058
+; HDR: .debug_addr 00000050
 ; HDR-NOT: .rela.{{.*}}.dwo
 
 ; Check for the existence of a DWARF v5-style range list table in the .debug_rnglists


        


More information about the llvm-branch-commits mailing list