<html><body><p><font size="2" face="sans-serif">The resume value for the widened induction is the only problem I'm aware of. </font><br><br><font size="2" face="sans-serif">The issue is that normally scalar induction resume values are created/updated as part of skeleton creation. However for widened inductions in the epilogue loop, we have corresponding recipes in the vplan that haven't been executed at the time of skeleton creation. We either have to find the related phis after the fact and fix them up, or change the vplan to update the incoming values of the widened IVs before executing on it. Florian demonstrate the latter idea in </font><a href="https://reviews.llvm.org/D92132"><font size="2" face="sans-serif">https://reviews.llvm.org/D92132</font></a><font size="2" face="sans-serif">, so maybe he has more details to share.</font><br><br><font size="2" face="sans-serif">Bardia Mahjour<br>Compiler Optimizations<br>IBM Toronto Software Lab<br></font><br><br><img width="16" height="16" src="cid:1__=8FBB0DF5DFFC949F8f9e8a93df938690@ibm.com" border="0" alt="Inactive hide details for "Venkataramanan Kumar" ---2021/10/06 12:33:00 PM---Hi, I wrote a small test case and tried to force e"><font size="2" color="#424282" face="sans-serif">"Venkataramanan Kumar" ---2021/10/06 12:33:00 PM---Hi, I wrote a small test case and tried to force epilog vectorization for the</font><br><br><font size="1" color="#5F5F5F" face="sans-serif">From: </font><font size="1" face="sans-serif">"Venkataramanan Kumar" <venkataramanan.kumar.llvm@gmail.com></font><br><font size="1" color="#5F5F5F" face="sans-serif">To: </font><font size="1" face="sans-serif">"llvm-dev" <llvm-dev@lists.llvm.org></font><br><font size="1" color="#5F5F5F" face="sans-serif">Cc: </font><font size="1" face="sans-serif">bmahjour@ca.ibm.com, "Florian Hahn" <florian_hahn@apple.com></font><br><font size="1" color="#5F5F5F" face="sans-serif">Date: </font><font size="1" face="sans-serif">2021/10/06 12:33 PM</font><br><font size="1" color="#5F5F5F" face="sans-serif">Subject: </font><font size="1" face="sans-serif">[EXTERNAL] Help: Question on Epilog Vectorization</font><br><hr width="100%" size="2" align="left" noshade style="color:#8091A5; "><br><br><br><font size="1" color="#FFFFFF" face="serif">Hi, I wrote a small test case and tried to force epilog vectorization for the loop. void foo(double * restrict a, double * restrict b, int N) { for(int i = 0; i < N; ++i) a[i] = sin(i); } clang -O3 -mavx2 -fveclib=libmvec sin.c </font><br><font size="3" face="serif">Hi, <br><br>I wrote a small test case and tried to force epilog vectorization for the loop.<br><br>void foo(double * restrict a, double * restrict b, int N) {<br>for(int i = 0; i < N; ++i)<br> a[i] = sin(i);<br>}<br><br>clang -O3 -mavx2 -fveclib=libmvec sin.c -mllvm -epilogue-vectorization-minimum-VF=4 -S -emit-llvm -fno-unroll-loops<br><br>But I ended up with epilog vectorization failing at this check.<br>In the function "isCandidateForEpilogueVectorization", I find the below check.</font><br><br><font size="3" face="serif">-- Snip llvm/lib/Transforms/Vectorize/LoopVectorize.cpp --<br><br>// Induction variables that are widened require special handling that is<br>// currently not supported.<br>if (any_of(Legal->getInductionVars(), [&](auto &Entry) {<br> return !(this->isScalarAfterVectorization(Entry.first, VF) ||<br> this->isProfitableToScalarize(Entry.first, VF));<br>-- Snip -- </font><br><font size="3" face="serif"><br>I understand that when induction variables are widened as per the VPLAN , we don't support such loops <br>for epilog vectorization at the moment. <br><br>But can someone please explain the "special handling" we need to do here?<br><br>If I remove the check from the source, the epilog vectorization is happening, but the generated LLVM IR seems to be wrong. <br><br>---Snip--<br><br>12: ; preds = %12, %10<br> %13 = phi i64 [ 0, %10 ], [ %19, %12 ]<br> %14 = phi <4 x i32> [ <i32 0, i32 1, i32 2, i32 3>, %10 ], [ %20, %12 ]<br> %15 = sitofp <4 x i32> %14 to <4 x double><br> %16 = call <4 x double> @_ZGVdN4v_sin(<4 x double> %15)<br> %17 = getelementptr inbounds double, double* %0, i64 %13<br> %18 = bitcast double* %17 to <4 x double>*<br> store <4 x double> %16, <4 x double>* %18, align 8, !tbaa !3<br> %19 = add nuw i64 %13, 4<br> %20 = add <4 x i32> %14, <i32 4, i32 4, i32 4, i32 4><br> %21 = icmp eq i64 %19, %11<br> br i1 %21, label %22, label %12, !llvm.loop !7<br><br>22: ; preds = %12<br> %23 = icmp eq i64 %11, %6<br> br i1 %23, label %44, label %24<br><br>24: ; preds = %22<br> %25 = and i64 %6, 2<br> %26 = icmp eq i64 %25, 0<br> br i1 %26, label %42, label %27<br><br>27: ; preds = %8, %24<br> %28 = phi i64 [ %11, %24 ], [ 0, %8 ]<br> %29 = and i64 %6, 4294967294<br> br label %30<br><br>30: ; preds = %30, %27<br> %31 = phi i64 [ %28, %27 ], [ %37, %30 ]<br> %32 = phi <2 x i32> [ <i32 0, i32 1>, %27 ], [ %38, %30 ] <== Resume value seem to be wrong. <br> %33 = sitofp <2 x i32> %32 to <2 x double><br> %34 = call <2 x double> @_ZGVbN2v_sin(<2 x double> %33)<br> %35 = getelementptr inbounds double, double* %0, i64 %31<br> %36 = bitcast double* %35 to <2 x double>*<br> store <2 x double> %34, <2 x double>* %36, align 8, !tbaa !3<br> %37 = add nuw i64 %31, 2<br> %38 = add <2 x i32> %32, <i32 2, i32 2><br> %39 = icmp eq i64 %37, %29<br> br i1 %39, label %40, label %30, !llvm.loop !11<br>--- Snip--<br><br>I see the resume value for the widened phi node in the epilog loop is not updated correctly.<br>Are there any other issues here apart from handling the widened induction variable's resume value ?</font><br><br><font size="3" face="serif">Regards,</font><br><font size="3" face="serif">Venkat.</font><br><br><BR>
<BR>
</body></html>