[llvm-commits] [llvm] r171798 - in /llvm/trunk: lib/Transforms/Vectorize/LoopVectorize.cpp test/Transforms/LoopVectorize/X86/unroll-small-loops.ll
Chris Lattner
clattner at apple.com
Mon Jan 7 17:02:11 PST 2013
On Jan 7, 2013, at 1:54 PM, Nadav Rotem <nrotem at apple.com> wrote:
> Author: nadav
> Date: Mon Jan 7 15:54:51 2013
> New Revision: 171798
>
> URL: http://llvm.org/viewvc/llvm-project?rev=171798&view=rev
> Log:
> LoopVectorizer: When we vectorizer and widen loops we process many elements at once. This is a good thing, except for
> small loops. On small loops post-loop that handles scalars (and runs slower) can take more time to execute than the
> rest of the loop. This patch disables widening of loops with a small static trip count.
Isn't it still (extremely) valuable to vectorize loops that are a multiple of the vectorization threshold? Turning a loop that adds 4 element arrays into a single SIMD add is a pretty nice win and requires no cleanup loop.
-Chris
>
>
> Added:
> llvm/trunk/test/Transforms/LoopVectorize/X86/unroll-small-loops.ll
> Modified:
> llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp
>
> Modified: llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp?rev=171798&r1=171797&r2=171798&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp (original)
> +++ llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp Mon Jan 7 15:54:51 2013
> @@ -101,7 +101,13 @@
> cl::desc("Enable if-conversion during vectorization."));
>
> /// We don't vectorize loops with a known constant trip count below this number.
> -static const unsigned TinyTripCountThreshold = 16;
> +static const unsigned TinyTripCountVectorThreshold = 16;
> +
> +/// We don't unroll loops with a known constant trip count below this number.
> +static const unsigned TinyTripCountUnrollThreshold = 128;
> +
> +/// We don't unroll loops that are larget than this threshold.
> +static const unsigned MaxLoopSizeThreshold = 32;
>
> /// When performing a runtime memory check, do not check more than this
> /// number of pointers. Notice that the check is quadratic!
> @@ -2016,7 +2022,7 @@
>
> // Do not loop-vectorize loops with a tiny trip count.
> unsigned TC = SE->getSmallConstantTripCount(TheLoop, Latch);
> - if (TC > 0u && TC < TinyTripCountThreshold) {
> + if (TC > 0u && TC < TinyTripCountVectorThreshold) {
> DEBUG(dbgs() << "LV: Found a loop with a very small trip count. " <<
> "This loop is not worth vectorizing.\n");
> return false;
> @@ -2678,6 +2684,12 @@
> if (OptForSize)
> return 1;
>
> + // Do not unroll loops with a relatively small trip count.
> + unsigned TC = SE->getSmallConstantTripCount(TheLoop,
> + TheLoop->getLoopLatch());
> + if (TC > 1 && TC < TinyTripCountUnrollThreshold)
> + return 1;
> +
> unsigned TargetVectorRegisters = TTI.getNumberOfRegisters(true);
> DEBUG(dbgs() << "LV: The target has " << TargetVectorRegisters <<
> " vector registers\n");
> @@ -2698,7 +2710,7 @@
>
> // We don't want to unroll the loops to the point where they do not fit into
> // the decoded cache. Assume that we only allow 32 IR instructions.
> - UF = std::min(UF, (32 / R.NumInstructions));
> + UF = std::min(UF, (MaxLoopSizeThreshold / R.NumInstructions));
>
> // Clamp the unroll factor ranges to reasonable factors.
> if (UF > MaxUnrollSize)
>
> Added: llvm/trunk/test/Transforms/LoopVectorize/X86/unroll-small-loops.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/LoopVectorize/X86/unroll-small-loops.ll?rev=171798&view=auto
> ==============================================================================
> --- llvm/trunk/test/Transforms/LoopVectorize/X86/unroll-small-loops.ll (added)
> +++ llvm/trunk/test/Transforms/LoopVectorize/X86/unroll-small-loops.ll Mon Jan 7 15:54:51 2013
> @@ -0,0 +1,50 @@
> +; RUN: opt < %s -loop-vectorize -mtriple=x86_64-apple-macosx10.8.0 -mcpu=corei7-avx2 -force-vector-width=4 -force-vector-unroll=0 -dce -S | FileCheck %s
> +
> +target datalayout = "e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v64:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
> +target triple = "x86_64-apple-macosx10.8.0"
> +;CHECK: @foo
> +;CHECK: load <4 x i32>
> +;CHECK-NOT: load <4 x i32>
> +;CHECK: store <4 x i32>
> +;CHECK-NOT: store <4 x i32>
> +;CHECK: ret
> +define i32 @foo(i32* nocapture %A) nounwind uwtable ssp {
> + br label %1
> +
> +; <label>:1 ; preds = %1, %0
> + %indvars.iv = phi i64 [ 0, %0 ], [ %indvars.iv.next, %1 ]
> + %2 = getelementptr inbounds i32* %A, i64 %indvars.iv
> + %3 = load i32* %2, align 4
> + %4 = add nsw i32 %3, 6
> + store i32 %4, i32* %2, align 4
> + %indvars.iv.next = add i64 %indvars.iv, 1
> + %lftr.wideiv = trunc i64 %indvars.iv.next to i32
> + %exitcond = icmp eq i32 %lftr.wideiv, 100
> + br i1 %exitcond, label %5, label %1
> +
> +; <label>:5 ; preds = %1
> + ret i32 undef
> +}
> +
> +;CHECK: @bar
> +;CHECK: store <4 x i32>
> +;CHECK: store <4 x i32>
> +;CHECK: ret
> +define i32 @bar(i32* nocapture %A, i32 %n) nounwind uwtable ssp {
> + %1 = icmp sgt i32 %n, 0
> + br i1 %1, label %.lr.ph, label %._crit_edge
> +
> +.lr.ph: ; preds = %0, %.lr.ph
> + %indvars.iv = phi i64 [ %indvars.iv.next, %.lr.ph ], [ 0, %0 ]
> + %2 = getelementptr inbounds i32* %A, i64 %indvars.iv
> + %3 = load i32* %2, align 4
> + %4 = add nsw i32 %3, 6
> + store i32 %4, i32* %2, align 4
> + %indvars.iv.next = add i64 %indvars.iv, 1
> + %lftr.wideiv = trunc i64 %indvars.iv.next to i32
> + %exitcond = icmp eq i32 %lftr.wideiv, %n
> + br i1 %exitcond, label %._crit_edge, label %.lr.ph
> +
> +._crit_edge: ; preds = %.lr.ph, %0
> + ret i32 undef
> +}
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list