[PATCH] D96765: [ARM][AtomicExpand] Bundle exclusive loads and stores created by AtomicExpandPass

[Tomas Matheson via Phabricator via llvm-commits]

tmatheson created this revision.
Herald added subscribers: danielkiss, nikic, kerbowa, jfb, hiraditya, kristof.beyls, mgorny, nhaehnle, jvesely, qcolombet, MatzeB.
tmatheson requested review of this revision.
Herald added a project: LLVM.
Herald added a subscriber: llvm-commits.

AtomicExpandPass expands atomicrmw instructions to loop structures. On
ARM/AArch64, these make use of exclusive load/store instructions. Any additional
store that occurs between these instructions will invalidate the exclusive
access monitor, and potentially cause an infinite loop.  Therefore the register
allocator must be prevented from inserting spills between these two points.

The approach taken here is to create a bundle containing all the instructions
between the exclusive load and store. This prevents the register allocator from
inserting spills.

This exposed an issue with RegAllocFast, wherein a virtual register defined
inside the bundle might be assigned the same physical register as a virtual
register with a use that occurs after the def. For example:

  %0 = something global
  BUNDLE implicit-def %1, implicit %0 {
    %1 = MOVi 123
    store %0, ...
  }

In the above example was possible to allocate the same physical register to both
%0 and %1. RegAllocFast has been updated to avoid this. RegAllocGreedy does not
have a similar problem, since it uses liveness analysis.

Finally, UnpackMachineBundles is added after register allocation for ARM/AArch64
to remove the bundles.

Differential Review: https://reviews.llvm.org/D94949


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D96765

Files:
  llvm/include/llvm/CodeGen/AtomicLoopBundler.h
  llvm/lib/CodeGen/RegAllocFast.cpp
  llvm/lib/Target/AArch64/AArch64.h
  llvm/lib/Target/AArch64/AArch64AtomicLoopBundler.cpp
  llvm/lib/Target/AArch64/AArch64AtomicLoopBundler.h
  llvm/lib/Target/AArch64/AArch64TargetMachine.cpp
  llvm/lib/Target/AArch64/CMakeLists.txt
  llvm/lib/Target/ARM/ARM.h
  llvm/lib/Target/ARM/ARMAtomicLoopBundler.cpp
  llvm/lib/Target/ARM/ARMAtomicLoopBundler.h
  llvm/lib/Target/ARM/ARMTargetMachine.cpp
  llvm/lib/Target/ARM/CMakeLists.txt
  llvm/test/CodeGen/AArch64/O0-pipeline.ll
  llvm/test/CodeGen/AArch64/atomicrmw_exclusive_monitor.ll
  llvm/test/CodeGen/AMDGPU/fast-regalloc-bundles.mir
  llvm/test/CodeGen/ARM/atomicrmw_exclusive_monitor.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D96765.323934.patch
Type: text/x-patch
Size: 46191 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210216/9cb88a06/attachment.bin>