[PATCH] D97729: [ARM] Improve WLS lowering

[Dave Green via Phabricator via llvm-commits]

dmgreen created this revision.
dmgreen added reviewers: SjoerdMeijer, samtebbs, efriedma, ostannard, simon_tatham.
Herald added subscribers: danielkiss, jdoerfert, hiraditya, kristof.beyls.
dmgreen requested review of this revision.
Herald added a project: LLVM.

Recently we improved the lowering of low overhead loops and tail predicated loops, but concentrated first on the DLS do style loops. This extends those improvements over to the WLS while loops, improving the chance of lowering them successfully. To do this the lowering has to change a little as the instructions are terminators that produce a value - something that needs to be treated carefully.

Lowering starts at the Hardware Loop pass, inserting a new `llvm.test.start.loop.iterations` that produces both an i1 to control the loop entry and an i32 similar to the llvm.start.loop.iterations intrinsic added for do loops. This feeds into the loop phi, properly gluing the values together:

    %wls = call { i32, i1 } @llvm.test.start.loop.iterations.i32(i32 %div)
    %wls0 = extractvalue { i32, i1 } %wls, 0
    %wls1 = extractvalue { i32, i1 } %wls, 1
    br i1 %wls1, label %loop.ph, label %loop.exit
  ...
  loop:
    %lsr.iv = phi i32 [ %wls0, %loop.ph ], [ %iv.next, %loop ]
    ..
    %iv.next = call i32 @llvm.loop.decrement.reg.i32(i32 %lsr.iv, i32 1)
    %cmp = icmp ne i32 %iv.next, 0
    br i1 %cmp, label %loop, label %loop.exit

The `llvm.test.start.loop.iterations` need to be lowered through ISel lowering as a pair of WLS and WLSSETUP nodes, which each get converted to t2WhileLoopSetup and t2WhileLoopStart Pseudos. This helps prevent t2WhileLoopStart from being a terminator that produces a value, something difficult to control at that stage in the pipeline. Instead the t2WhileLoopSetup produces the value of LR (essentially acting as a `lr = subs rn, 0`), t2WhileLoopStart consumes that lr value (the Bcc).

These are then converted into a single t2WhileLoopStartLR at the same point as t2DoLoopStartTP and t2LoopEndDec. Otherwise we revert the loop to prevent them from progressing further in the pipeline. The t2WhileLoopStartLR is the a single instruction that takes a GPR and produces LR, similar to the WLS instruction.

    %1:gprlr = t2WhileLoopStartLR %0:rgpr, %bb.3
    t2B %bb.1
  ...
  bb.2.loop:
    %2:gprlr = PHI %1:gprlr, %bb.1, %3:gprlr, %bb.2
    ...
    %3:gprlr = t2LoopEndDec %2:gprlr, %bb.2
    t2B %bb.3

The t2WhileLoopStartLR can then be treated similar to the other low overhead loop pseudos, eventually being lowered to a WLS providing the branches are within range.


https://reviews.llvm.org/D97729

Files:
  llvm/docs/LangRef.rst
  llvm/include/llvm/IR/Intrinsics.td
  llvm/lib/CodeGen/HardwareLoops.cpp
  llvm/lib/Target/ARM/ARMBaseInstrInfo.cpp
  llvm/lib/Target/ARM/ARMBaseInstrInfo.h
  llvm/lib/Target/ARM/ARMBlockPlacement.cpp
  llvm/lib/Target/ARM/ARMISelDAGToDAG.cpp
  llvm/lib/Target/ARM/ARMISelLowering.cpp
  llvm/lib/Target/ARM/ARMISelLowering.h
  llvm/lib/Target/ARM/ARMInstrThumb2.td
  llvm/lib/Target/ARM/ARMLowOverheadLoops.cpp
  llvm/lib/Target/ARM/ARMTargetTransformInfo.cpp
  llvm/lib/Target/ARM/MVETailPredUtils.h
  llvm/lib/Target/ARM/MVETailPredication.cpp
  llvm/lib/Target/ARM/MVEVPTOptimisationsPass.cpp
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/add_reduce.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/biquad-cascade-default.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/biquad-cascade-optsize-strd-lr.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/biquad-cascade-optsize.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/branch-targets.ll
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/fast-fp-loops.ll
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/loop-guards.ll
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/mve-float-loops.ll
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/predicated-liveout.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/revert-non-loop.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/revert-while.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/vmaxmin_vpred_r.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/vmldava_in_vpt.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/while-loops.ll
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/while-negative-offset.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/while.mir
  llvm/test/CodeGen/Thumb2/LowOverheadLoops/wlstp.mir
  llvm/test/CodeGen/Thumb2/block-placement.mir
  llvm/test/CodeGen/Thumb2/mve-float16regloops.ll
  llvm/test/CodeGen/Thumb2/mve-float32regloops.ll
  llvm/test/CodeGen/Thumb2/mve-postinc-distribute.ll
  llvm/test/CodeGen/Thumb2/mve-postinc-lsr.ll
  llvm/test/CodeGen/Thumb2/mve-vmaxnma-commute.ll
  llvm/test/Transforms/HardwareLoops/ARM/do-rem.ll
  llvm/test/Transforms/HardwareLoops/ARM/simple-do.ll
  llvm/test/Transforms/HardwareLoops/ARM/structure.ll
  llvm/test/Transforms/HardwareLoops/loop-guards.ll
  llvm/test/Transforms/HardwareLoops/scalar-while.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D97729.327262.patch
Type: text/x-patch
Size: 158194 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20210301/bdfd9b1a/attachment-0001.bin>