<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hello,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We are working on predication for our vector extension (MVE). Since quite a few people are working on predication and different forms of it (e.g. SVE, RISC-V, NEC), I thought I would share what we would like to add to the loop vectoriser. Hopefully it's just
a minor one and not intrusive, but could be interesting and useful for others, and feedback on this is welcome of course.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
TL;DR:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We would like the loop vectoriser to emit a new IR intrinsic for certain loops:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
void @llvm.set.loop.elements.i32(i32 )<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
This represents the number of data elements processed by a vector loop, and will be emitted in the preheader block of the vector loop after querying TTI that the backend understands this intrinsic and that it should be emitted for that loop. The vectoriser
patch is available in D79100, and we pick this intrinsic up in the ARM backend here in D79175.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Context:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
We are working on predication form that we call tail-predication: a vector hardwareloop has an implicit form of predication that sets active/inactive lanes for the last iteration of the vector loop. Thus, the scalar epilogue loop (if there is one) is tail-folded
and tail-predicated in the main vector body. And to support this, we need to know the number of data elements processed by the loop, which is used in the set up of a tail-predicated vector loop. This new intrinsic communicates this information from the vectoriser
to the codegen passes where we further lower these loops. In our case, we essentially let @llvm.set.loop.elements.i32 emit the trip count of the scalar loop, which represents the number of data elements processed. Thus, we let the vectoriser emits both the
scalar and vector loop trip count.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Although in a different stage in the optimisation pipeline, this is exactly what the generic HardwareLoop pass is doing to communicate its information to target specific codegen passes; it emits a few intrinsics to mark a hardware loop. To illustrate this and
also the new intrinsic, this is the flow and life of a tail-predicated vector loop using some heavily edited/reduced examples. First, the vectoriser emits the number of elements processed, and the loads/stores are masked because tail-folding is applied:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span> vector.ph:<br>
</span>
<div> call void @llvm.set.loop.elements.i32(i32 %N)<br>
</div>
<div> br label %vector.body<br>
</div>
<div> vector.body:<br>
</div>
<div> call <4 x i32> @llvm.masked.load<br>
</div>
<div> call <4 x i32> @llvm.masked.load<br>
</div>
<span> call void @llvm.masked.store</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span> <span>br i1 %12, label %.*, label %vector.body</span></span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
After the HardwareLoop pass this is transformed into this, which adds the hardware loop intrinsics:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span> vector.ph:<br>
</span>
<div> call void @llvm.set.loop.elements.i32(i32 %N)</div>
<div> call void @llvm.set.loop.iterations.i32(i32 %5)<br>
</div>
<div> br label %vector.body<br>
</div>
<div> vector.body:<br>
</div>
<div> call <4 x i32> @llvm.masked.load<br>
</div>
<div> call <4 x i32> @llvm.masked.load<br>
</div>
<span> call void @llvm.masked.store</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span> call i32 @llvm.loop.decrement.reg<br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span> <span>br i1 %12, label %.*, label %vector.body</span></span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
We then pick this up in our tail-predication pass, remove @llvm.set.loop.elements intrinsic, and add @vctp which is our intrinsic that generates the mask of active/inactive lanes:<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span> vector.ph:</span><br>
<div> call void @llvm.set.loop.iterations.i32(i32 %5)<br>
</div>
<div> br label %vector.body<br>
</div>
<div> vector.body:</div>
<div> call <4 x i1> @llvm.arm.mve.vctp32<br>
</div>
<div> call <4 x i32> @llvm.masked.load<br>
</div>
<div> call <4 x i32> @llvm.masked.load<br>
</div>
<span> call void @llvm.masked.store</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span> call i32 @llvm.loop.decrement.reg<br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<span> <span>br i1 %12, label %.*, label %vector.body</span></span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
And this is then further lowered to a tail-predicted loop, or reverted to a 'normal' vector loop if some restrictions are not met.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
Cheers,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
Sjoerd.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt">
</div>
<br>
</div>
<br>
</div>
</body>
</html>