[PATCH] D12214: [LoopVectorize] Propagate 'nontemporal' attribute into vectorized instructions.

[Nadav Rotem via llvm-commits]

LGTM.  

Maybe in a future patch we should refactor this logic in the SLP vectorizer and LoopVectorizer into a single function that verifies the metadata kind? I assume that the list should be identical. 


> On Aug 20, 2015, at 1:50 PM, Michael Zolotukhin <mzolotukhin at apple.com> wrote:
> 
> mzolotukhin created this revision.
> mzolotukhin added reviewers: hfinkel, aschwaighofer, nadav.
> mzolotukhin added a subscriber: llvm-commits.
> 
> Currently we drop nontemporal attribute when vectorizing instructions, and this
> patch fixes it. It hasn't been a problem, because there weren't cases when we
> needed to vectorize scalar instructions with nontemporal attribute. But I'm
> working on adding language attribute 'attribute((nontemporal)' to front-end,
> after which we'll encounter such cases.
> 
> This is similar to D12213, but for loop-vectorizer.
> 
> https://urldefense.proofpoint.com/v2/url?u=http-3A__reviews.llvm.org_D12214&d=BQIFaQ&c=eEvniauFctOgLOKGJOplqw&r=byTd1qh_yjgCW4vvTtdnsg&m=CnA73ZIfYPjYc7HtBsNeEq7i_A1XFtnNcoG4ZdLJybw&s=vzaeGMCuo6aAmIr0WCH2Bx_fZp6-TwXkS-4BcD3Z7Hg&e= 
> 
> Files:
>  lib/Transforms/Vectorize/LoopVectorize.cpp
>  test/Transforms/LoopVectorize/nontemporal.ll
> 
> Index: test/Transforms/LoopVectorize/nontemporal.ll
> ===================================================================
> --- /dev/null
> +++ test/Transforms/LoopVectorize/nontemporal.ll
> @@ -0,0 +1,47 @@
> +; RUN: opt < %s  -loop-vectorize -force-vector-width=4 -force-vector-interleave=1 -instcombine -S | FileCheck %s
> +
> +target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
> +target triple = "arm64-apple-ios5.0.0"
> +
> +; CHECK-LABEL: @foo(
> +define void @foo(float* noalias %a, float* noalias %b, float* noalias %c, i32 %N) {
> +entry:
> +  %cmp.4 = icmp sgt i32 %N, 0
> +  br i1 %cmp.4, label %for.body.preheader, label %for.end
> +
> +for.body.preheader:                               ; preds = %entry
> +  br label %for.body
> +
> +for.body:                                         ; preds = %for.body.preheader, %for.body
> +  %indvars.iv = phi i64 [ %indvars.iv.next, %for.body ], [ 0, %for.body.preheader ]
> +
> +; Check that we don't lose !nontemporal hint when vectorizing loads.
> +; CHECK: %wide.load{{[0-9]*}} = load <4 x float>, <4 x float>* %{{[0-9]+}}, align 4, !nontemporal !0
> +  %arrayidx = getelementptr inbounds float, float* %b, i64 %indvars.iv
> +  %0 = load float, float* %arrayidx, align 4, !nontemporal !0
> +
> +; Check that we don't introduce !nontemporal hint when the original scalar loads didn't have it.
> +; CHECK: %wide.load{{[0-9]+}} = load <4 x float>, <4 x float>* %{{[0-9]+}}, align 4{{$}}
> +  %arrayidx2 = getelementptr inbounds float, float* %c, i64 %indvars.iv
> +  %1 = load float, float* %arrayidx2, align 4
> +  %add = fadd float %0, %1
> +
> +; Check that we don't lose !nontemporal hint when vectorizing stores.
> +; CHECK: store <4 x float> %{{[0-9]+}}, <4 x float>* %{{[0-9]+}}, align 4, !nontemporal !0
> +  %arrayidx4 = getelementptr inbounds float, float* %a, i64 %indvars.iv
> +  store float %add, float* %arrayidx4, align 4, !nontemporal !0
> +
> +  %indvars.iv.next = add nuw nsw i64 %indvars.iv, 1
> +  %lftr.wideiv = trunc i64 %indvars.iv.next to i32
> +  %exitcond = icmp eq i32 %lftr.wideiv, %N
> +  br i1 %exitcond, label %for.end.loopexit, label %for.body
> +
> +for.end.loopexit:                                 ; preds = %for.body
> +  br label %for.end
> +
> +for.end:                                          ; preds = %for.end.loopexit, %entry
> +; CHECK: ret void
> +  ret void
> +}
> +
> +!0 = !{i32 1}
> Index: lib/Transforms/Vectorize/LoopVectorize.cpp
> ===================================================================
> --- lib/Transforms/Vectorize/LoopVectorize.cpp
> +++ lib/Transforms/Vectorize/LoopVectorize.cpp
> @@ -552,7 +552,8 @@
>     if (Kind != LLVMContext::MD_tbaa &&
>         Kind != LLVMContext::MD_alias_scope &&
>         Kind != LLVMContext::MD_noalias &&
> -        Kind != LLVMContext::MD_fpmath)
> +        Kind != LLVMContext::MD_fpmath &&
> +        Kind != LLVMContext::MD_nontemporal)
>       continue;
> 
>     To->setMetadata(Kind, M.second);
> 
> 
> <D12214.32739.patch>