[llvm] r292254 - [LV] Mark non-consecutive-like pointers non-uniform

Thu Jan 19 10:46:41 PST 2017

Merged them r292503.

Thank you both,
Hans

On Wed, Jan 18, 2017 at 10:13 AM, Michael Kuperstein <mkuper at google.com> wrote:
> Sounds good to me - this looks like it should be safe, and the bug it fixes
> is probably fairly common if you enable scatter/gather (that is, if you're
> on AVX512).
> Hans?
>
> On Wed, Jan 18, 2017 at 8:33 AM, Matthew Simpson <mssimpso at codeaurora.org>
> wrote:
>>
>> Hans,
>>
>> I think r292254 and r292280 (test case fix-up) should probably be added to
>> the release branch. Michael, what do you think?
>>
>> -- Matt
>>
>> On Tue, Jan 17, 2017 at 3:51 PM, Matthew Simpson via llvm-commits
>> <llvm-commits at lists.llvm.org> wrote:
>>>
>>> Author: mssimpso
>>> Date: Tue Jan 17 14:51:39 2017
>>> New Revision: 292254
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=292254&view=rev
>>> Log:
>>> [LV] Mark non-consecutive-like pointers non-uniform
>>>
>>> If a memory instruction will be vectorized, but it's pointer operand is
>>> non-consecutive-like, the instruction is a gather or scatter operation.
>>> Its
>>> pointer operand will be non-uniform. This should fix PR31671.
>>>
>>> Reference: https://llvm.org/bugs/show_bug.cgi?id=31671
>>> Differential Revision: https://reviews.llvm.org/D28819
>>>
>>> Added:
>>>
>>> llvm/trunk/test/Transforms/LoopVectorize/X86/consecutive-ptr-uniforms.ll
>>> Modified:
>>>     llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp
>>>
>>> Modified: llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp?rev=292254&r1=292253&r2=292254&view=diff
>>>
>>> ==============================================================================
>>> --- llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp (original)
>>> +++ llvm/trunk/lib/Transforms/Vectorize/LoopVectorize.cpp Tue Jan 17
>>> 14:51:39 2017
>>> @@ -5602,6 +5602,13 @@ void LoopVectorizationLegality::collectL
>>>        // is consecutive-like, the pointer operand should remain uniform.
>>>        else if (hasConsecutiveLikePtrOperand(&I))
>>>          ConsecutiveLikePtrs.insert(Ptr);
>>> +
>>> +      // Otherwise, if the memory instruction will be vectorized and its
>>> +      // pointer operand is non-consecutive-like, the memory instruction
>>> should
>>> +      // be a gather or scatter operation. Its pointer operand will be
>>> +      // non-uniform.
>>> +      else
>>> +        PossibleNonUniformPtrs.insert(Ptr);
>>>      }
>>>
>>>    // Add to the Worklist all consecutive and consecutive-like pointers
>>> that
>>>
>>> Added:
>>> llvm/trunk/test/Transforms/LoopVectorize/X86/consecutive-ptr-uniforms.ll
>>> URL:
>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/LoopVectorize/X86/consecutive-ptr-uniforms.ll?rev=292254&view=auto
>>>
>>> ==============================================================================
>>> ---
>>> llvm/trunk/test/Transforms/LoopVectorize/X86/consecutive-ptr-uniforms.ll
>>> (added)
>>> +++
>>> llvm/trunk/test/Transforms/LoopVectorize/X86/consecutive-ptr-uniforms.ll Tue
>>> Jan 17 14:51:39 2017
>>> @@ -0,0 +1,55 @@
>>> +; RUN: opt < %s -loop-vectorize -instcombine -S
>>> -debug-only=loop-vectorize -disable-output -print-after=instcombine 2>&1 |
>>> FileCheck %s
>>> +
>>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>> +target triple = "x86_64-unknown-linux-gnu"
>>> +
>>> +; CHECK-LABEL: PR31671
>>> +;
>>> +; Check a pointer in which one of its uses is consecutive-like and
>>> another of
>>> +; its uses is non-consecutive-like. In the test case below, %tmp3 is the
>>> +; pointer operand of an interleaved load, making it consecutive-like.
>>> However,
>>> +; it is also the pointer operand of a non-interleaved store that will
>>> become a
>>> +; scatter operation. %tmp3 (and the induction variable) should not be
>>> marked
>>> +; uniform-after-vectorization.
>>> +;
>>> +; CHECK:     LV: Found uniform instruction: %tmp0 = getelementptr
>>> inbounds %data, %data* %d, i64 0, i32 3, i64 %i
>>> +; CHECK-NOT: LV: Found uniform instruction: %tmp3 = getelementptr
>>> inbounds %data, %data* %d, i64 0, i32 0, i64 %i
>>> +; CHECK-NOT: LV: Found uniform instruction: %i = phi i64 [ %i.next,
>>> %for.body ], [ 0, %entry ]
>>> +; CHECK-NOT: LV: Found uniform instruction: %i.next = add nuw nsw i64
>>> %i, 5
>>> +; CHECK:     vector.body:
>>> +; CHECK:       %vec.ind = phi <16 x i64>
>>> +; CHECK:       %[[T0:.+]] = extractelement <16 x i64> %vec.ind, i32 0
>>> +; CHECK:       %[[T1:.+]] = getelementptr inbounds %data, %data* %d, i64
>>> 0, i32 3, i64 %[[T0]]
>>> +; CHECK:       %[[T2:.+]] = bitcast float* %[[T1]] to <80 x float>*
>>> +; CHECK:       load <80 x float>, <80 x float>* %[[T2]], align 4
>>> +; CHECK:       %[[T3:.+]] = getelementptr inbounds %data, %data* %d, i64
>>> 0, i32 0, i64 %[[T0]]
>>> +; CHECK:       %[[T4:.+]] = bitcast float* %[[T3]] to <80 x float>*
>>> +; CHECK:       load <80 x float>, <80 x float>* %[[T4]], align 4
>>> +; CHECK:       %VectorGep = getelementptr inbounds %data, %data* %d, i64
>>> 0, i32 0, <16 x i64> %vec.ind
>>> +; CHECK:       call void @llvm.masked.scatter.v16f32({{.*}}, <16 x
>>> float*> %VectorGep, {{.*}})
>>> +; CHECK:       br i1 {{.*}}, label %middle.block, label %vector.body
>>> +
>>> +%data = type { [32000 x float], [3 x i32], [4 x i8], [32000 x float] }
>>> +
>>> +define void @PR31671(float %x, %data* %d) #0 {
>>> +entry:
>>> +  br label %for.body
>>> +
>>> +for.body:
>>> +  %i = phi i64 [ %i.next, %for.body ], [ 0, %entry ]
>>> +  %tmp0 = getelementptr inbounds %data, %data* %d, i64 0, i32 3, i64 %i
>>> +  %tmp1 = load float, float* %tmp0, align 4
>>> +  %tmp2 = fmul float %x, %tmp1
>>> +  %tmp3 = getelementptr inbounds %data, %data* %d, i64 0, i32 0, i64 %i
>>> +  %tmp4 = load float, float* %tmp3, align 4
>>> +  %tmp5 = fadd float %tmp4, %tmp2
>>> +  store float %tmp5, float* %tmp3, align 4
>>> +  %i.next = add nuw nsw i64 %i, 5
>>> +  %cond = icmp slt i64 %i.next, 32000
>>> +  br i1 %cond, label %for.body, label %for.end
>>> +
>>> +for.end:
>>> +  ret void
>>> +}
>>> +
>>> +attributes #0 = { "target-cpu"="knl" }
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>>
>