<div dir="ltr"><div class="gmail_default" style="font-family:verdana,sans-serif">Thank you Reid. Have you reverted the change?</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">
I think the alternatives are to disable vectorization of bswap on 32-bit or teach llvm how to do the expansion.</div><div class="gmail_default" style="font-family:verdana,sans-serif"><br></div><div class="gmail_default" style="font-family:verdana,sans-serif">
I'm using the same code as the Loop vectorizer to decide what to vectorize, which makes me wonder if the same issue could be hit with loop vectorization. I'll investigate a bit further and figure out what to do next.</div>
<div class="gmail_default" style="font-family:verdana,sans-serif"><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jan 31, 2014 at 5:40 PM, Reid Kleckner <span dir="ltr"><<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hey Raul,<div><br></div><div>This patch broke the 32-bit self-host build. The x86 backend claims that the legalizer knows how to expand bswap intrinsics on vector types, but this is not the case. Running llc the test case I gave produces:</div>
<div><div>Unhandled Expand type in BSWAP!</div><div>UNREACHABLE executed at ..\lib\CodeGen\SelectionDAG\LegalizeDAG.cpp:2537!</div></div><div><br></div><div>I'm going to revert this for now, but what should happen is that we should learn how to expand bswap on vectors.</div>
<div><br></div><div>Reid</div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Jan 31, 2014 at 5:17 PM, Reid Kleckner <span dir="ltr"><<a href="mailto:rnk@google.com" target="_blank">rnk@google.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">This change caused us to vectorize bswap, which we then try to expand on i686, which hits an unreachable. Running llc on this repros:<div>
<br><div><div>declare <2 x i64> @llvm.bswap.v2i64(<2 x i64>) #8</div>
<div>define <2 x i64> @foo(<2 x i64> %v) {</div><div> %s = call <2 x i64> @llvm.bswap.v2i64(<2 x i64> %v)</div><div> ret <2 x i64> %s</div><div>}</div><div>attributes #8 = { nounwind readnone }</div>
</div><div><br></div><div>I don't have a reduced test case of input to the SLP vectorizer yet because I'm new to reducing LLVM IR.</div></div></div><div><div><div class="gmail_extra"><br>
<br><div class="gmail_quote"><div><div class="h5">On Fri, Jan 31, 2014 at 1:14 PM, Chandler Carruth <span dir="ltr"><<a href="mailto:chandlerc@gmail.com" target="_blank">chandlerc@gmail.com</a>></span> wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">Author: chandlerc<br>
Date: Fri Jan 31 15:14:40 2014<br>
New Revision: 200576<br>
<br>
URL: <a href="http://llvm.org/viewvc/llvm-project?rev=200576&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=200576&view=rev</a><br>
Log:<br>
[SLPV] Recognize vectorizable intrinsics during SLP vectorization and<br>
transform accordingly. Based on similar code from Loop vectorization.<br>
Subsequent commits will include vectorization of function calls to<br>
vector intrinsics and form function calls to vector library calls.<br>
<br>
Patch by Raul Silvera! (Much delayed due to my not running dcommit)<br>
<br></div></div>
Added:<br>
llvm/trunk/test/Transforms/SLPVectorizer/X86/intrinsic.ll<br>
Modified:<br>
llvm/trunk/lib/Transforms/Vectorize/SLPVectorizer.cpp<br>
<br>
Modified: llvm/trunk/lib/Transforms/Vectorize/SLPVectorizer.cpp<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/SLPVectorizer.cpp?rev=200576&r1=200575&r2=200576&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/SLPVectorizer.cpp?rev=200576&r1=200575&r2=200576&view=diff</a><br>
==============================================================================<br>
--- llvm/trunk/lib/Transforms/Vectorize/SLPVectorizer.cpp (original)<br>
+++ llvm/trunk/lib/Transforms/Vectorize/SLPVectorizer.cpp Fri Jan 31 15:14:40 2014<br>
@@ -947,6 +947,39 @@ void BoUpSLP::buildTree_rec(ArrayRef<Val<br>
buildTree_rec(Operands, Depth + 1);<br>
return;<br>
}<br>
+ case Instruction::Call: {<br>
+ // Check if the calls are all to the same vectorizable intrinsic.<br>
+ IntrinsicInst *II = dyn_cast<IntrinsicInst>(VL[0]);<br>
+ if (II==NULL) {<br>
+ newTreeEntry(VL, false);<br>
+ DEBUG(dbgs() << "SLP: Non-vectorizable call.\n");<br>
+ return;<br>
+ }<br>
+<br>
+ Intrinsic::ID ID = II->getIntrinsicID();<br>
+<br>
+ for (unsigned i = 1, e = VL.size(); i != e; ++i) {<br>
+ IntrinsicInst *II2 = dyn_cast<IntrinsicInst>(VL[i]);<br>
+ if (!II2 || II2->getIntrinsicID() != ID) {<br>
+ newTreeEntry(VL, false);<br>
+ DEBUG(dbgs() << "SLP: mismatched calls:" << *II << "!=" << *VL[i]<br>
+ << "\n");<br>
+ return;<br>
+ }<br>
+ }<br>
+<br>
+ newTreeEntry(VL, true);<br>
+ for (unsigned i = 0, e = II->getNumArgOperands(); i != e; ++i) {<br>
+ ValueList Operands;<br>
+ // Prepare the operand vector.<br>
+ for (unsigned j = 0; j < VL.size(); ++j) {<br>
+ IntrinsicInst *II2 = dyn_cast<IntrinsicInst>(VL[j]);<br>
+ Operands.push_back(II2->getArgOperand(i));<br>
+ }<br>
+ buildTree_rec(Operands, Depth + 1);<br>
+ }<br>
+ return;<br>
+ }<br>
default:<br>
newTreeEntry(VL, false);<br>
DEBUG(dbgs() << "SLP: Gathering unknown instruction.\n");<br>
@@ -1072,6 +1105,30 @@ int BoUpSLP::getEntryCost(TreeEntry *E)<br>
int VecStCost = TTI->getMemoryOpCost(Instruction::Store, VecTy, 1, 0);<br>
return VecStCost - ScalarStCost;<br>
}<br>
+ case Instruction::Call: {<br>
+ CallInst *CI = cast<CallInst>(VL0);<br>
+ IntrinsicInst *II = cast<IntrinsicInst>(CI);<br>
+ Intrinsic::ID ID = II->getIntrinsicID();<br>
+<br>
+ // Calculate the cost of the scalar and vector calls.<br>
+ SmallVector<Type*, 4> ScalarTys, VecTys;<br>
+ for (unsigned op = 0, opc = II->getNumArgOperands(); op!= opc; ++op) {<br>
+ ScalarTys.push_back(CI->getArgOperand(op)->getType());<br>
+ VecTys.push_back(VectorType::get(CI->getArgOperand(op)->getType(),<br>
+ VecTy->getNumElements()));<br>
+ }<br>
+<br>
+ int ScalarCallCost = VecTy->getNumElements() *<br>
+ TTI->getIntrinsicInstrCost(ID, ScalarTy, ScalarTys);<br>
+<br>
+ int VecCallCost = TTI->getIntrinsicInstrCost(ID, VecTy, VecTys);<br>
+<br>
+ DEBUG(dbgs() << "SLP: Call cost "<< VecCallCost - ScalarCallCost<br>
+ << " (" << VecCallCost << "-" << ScalarCallCost << ")"<br>
+ << " for " << *II << "\n");<br>
+<br>
+ return VecCallCost - ScalarCallCost;<br>
+ }<br>
default:<br>
llvm_unreachable("Unknown instruction");<br>
}<br>
@@ -1086,10 +1143,10 @@ bool BoUpSLP::isFullyVectorizableTinyTre<br>
return false;<br>
<br>
// Gathering cost would be too much for tiny trees.<br>
- if (VectorizableTree[0].NeedToGather || VectorizableTree[1].NeedToGather)<br>
- return false;<br>
+ if (VectorizableTree[0].NeedToGather || VectorizableTree[1].NeedToGather)<br>
+ return false;<br>
<br>
- return true;<br>
+ return true;<br>
}<br>
<br>
int BoUpSLP::getTreeCost() {<br>
@@ -1555,6 +1612,32 @@ Value *BoUpSLP::vectorizeTree(TreeEntry<br>
E->VectorizedValue = S;<br>
return propagateMetadata(S, E->Scalars);<br>
}<br>
+ case Instruction::Call: {<br>
+ CallInst *CI = cast<CallInst>(VL0);<br>
+<br>
+ setInsertPointAfterBundle(E->Scalars);<br>
+ std::vector<Value *> OpVecs;<br>
+ for (int j = 0, e = CI->getNumArgOperands(); j < e; ++j) {<br>
+ ValueList OpVL;<br>
+ for (int i = 0, e = E->Scalars.size(); i < e; ++i) {<br>
+ CallInst *CEI = cast<CallInst>(E->Scalars[i]);<br>
+ OpVL.push_back(CEI->getArgOperand(j));<br>
+ }<br>
+<br>
+ Value *OpVec = vectorizeTree(OpVL);<br>
+ DEBUG(dbgs() << "SLP: OpVec[" << j << "]: " << *OpVec << "\n");<br>
+ OpVecs.push_back(OpVec);<br>
+ }<br>
+<br>
+ Module *M = F->getParent();<br>
+ IntrinsicInst *II = cast<IntrinsicInst>(CI);<br>
+ Intrinsic::ID ID = II->getIntrinsicID();<br>
+ Type *Tys[] = { VectorType::get(CI->getType(), E->Scalars.size()) };<br>
+ Function *CF = Intrinsic::getDeclaration(M, ID, Tys);<br>
+ Value *V = Builder.CreateCall(CF, OpVecs);<br>
+ E->VectorizedValue = V;<br>
+ return V;<br>
+ }<br>
default:<br>
llvm_unreachable("unknown inst");<br>
}<br>
<br>
Added: llvm/trunk/test/Transforms/SLPVectorizer/X86/intrinsic.ll<br>
URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SLPVectorizer/X86/intrinsic.ll?rev=200576&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/SLPVectorizer/X86/intrinsic.ll?rev=200576&view=auto</a><br>
==============================================================================<br>
--- llvm/trunk/test/Transforms/SLPVectorizer/X86/intrinsic.ll (added)<br>
+++ llvm/trunk/test/Transforms/SLPVectorizer/X86/intrinsic.ll Fri Jan 31 15:14:40 2014<br>
@@ -0,0 +1,75 @@<br>
+; RUN: opt < %s -basicaa -slp-vectorizer -slp-threshold=-999 -dce -S -mtriple=x86_64-apple-macosx10.8.0 -mcpu=corei7-avx | FileCheck %s<br>
+<br>
+target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"<br>
+target triple = "x86_64-apple-macosx10.8.0"<br>
+<br>
+declare double @llvm.fabs.f64(double) nounwind readnone<br>
+<br>
+;CHECK-LABEL: @vec_fabs_f64(<br>
+;CHECK: load <2 x double><br>
+;CHECK: load <2 x double><br>
+;CHECK: call <2 x double> @llvm.fabs.v2f64<br>
+;CHECK: store <2 x double><br>
+;CHECK: ret<br>
+define void @vec_fabs_f64(double* %a, double* %b, double* %c) {<br>
+entry:<br>
+ %i0 = load double* %a, align 8<br>
+ %i1 = load double* %b, align 8<br>
+ %mul = fmul double %i0, %i1<br>
+ %call = tail call double @llvm.fabs.f64(double %mul) nounwind readnone<br>
+ %arrayidx3 = getelementptr inbounds double* %a, i64 1<br>
+ %i3 = load double* %arrayidx3, align 8<br>
+ %arrayidx4 = getelementptr inbounds double* %b, i64 1<br>
+ %i4 = load double* %arrayidx4, align 8<br>
+ %mul5 = fmul double %i3, %i4<br>
+ %call5 = tail call double @llvm.fabs.f64(double %mul5) nounwind readnone<br>
+ store double %call, double* %c, align 8<br>
+ %arrayidx5 = getelementptr inbounds double* %c, i64 1<br>
+ store double %call5, double* %arrayidx5, align 8<br>
+ ret void<br>
+}<br>
+<br>
+declare float @llvm.copysign.f32(float, float) nounwind readnone<br>
+<br>
+;CHECK-LABEL: @vec_copysign_f32(<br>
+;CHECK: load <4 x float><br>
+;CHECK: load <4 x float><br>
+;CHECK: call <4 x float> @llvm.copysign.v4f32<br>
+;CHECK: store <4 x float><br>
+;CHECK: ret<br>
+define void @vec_copysign_f32(float* %a, float* %b, float* noalias %c) {<br>
+entry:<br>
+ %0 = load float* %a, align 4<br>
+ %1 = load float* %b, align 4<br>
+ %call0 = tail call float @llvm.copysign.f32(float %0, float %1) nounwind readnone<br>
+ store float %call0, float* %c, align 4<br>
+<br>
+ %ix2 = getelementptr inbounds float* %a, i64 1<br>
+ %2 = load float* %ix2, align 4<br>
+ %ix3 = getelementptr inbounds float* %b, i64 1<br>
+ %3 = load float* %ix3, align 4<br>
+ %call1 = tail call float @llvm.copysign.f32(float %2, float %3) nounwind readnone<br>
+ %c1 = getelementptr inbounds float* %c, i64 1<br>
+ store float %call1, float* %c1, align 4<br>
+<br>
+ %ix4 = getelementptr inbounds float* %a, i64 2<br>
+ %4 = load float* %ix4, align 4<br>
+ %ix5 = getelementptr inbounds float* %b, i64 2<br>
+ %5 = load float* %ix5, align 4<br>
+ %call2 = tail call float @llvm.copysign.f32(float %4, float %5) nounwind readnone<br>
+ %c2 = getelementptr inbounds float* %c, i64 2<br>
+ store float %call2, float* %c2, align 4<br>
+<br>
+ %ix6 = getelementptr inbounds float* %a, i64 3<br>
+ %6 = load float* %ix6, align 4<br>
+ %ix7 = getelementptr inbounds float* %b, i64 3<br>
+ %7 = load float* %ix7, align 4<br>
+ %call3 = tail call float @llvm.copysign.f32(float %6, float %7) nounwind readnone<br>
+ %c3 = getelementptr inbounds float* %c, i64 3<br>
+ store float %call3, float* %c3, align 4<br>
+<br>
+ ret void<br>
+}<br>
+<br>
+<br>
+<br>
<br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@cs.uiuc.edu" target="_blank">llvm-commits@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</blockquote></div><br><br clear="all"><div><br></div>-- <br><div dir="ltr"><div><font size="4" face="arial black, sans-serif" style="background-color:rgb(0,0,0)" color="#b45f06"> Raúl E. Silvera </font></div><div><br></div>
</div>
</div>