[llvm] r367898 - [AArch64] Expand bcmp() for small block lengths

Tue Aug 6 04:03:16 PDT 2019

Merged to release_90 in r368017.

On Mon, Aug 5, 2019 at 8:08 PM Evandro Menezes via llvm-commits
<llvm-commits at lists.llvm.org> wrote:
>
> Author: evandro
> Date: Mon Aug  5 11:09:14 2019
> New Revision: 367898
>
> URL: http://llvm.org/viewvc/llvm-project?rev=367898&view=rev
> Log:
> [AArch64] Expand bcmp() for small block lengths
>
> Patch D56593 by @courbet results in calls to `bcmp()` in some cases, should
> the target support the it.  Unless `TTI::MemCmpExpansionOptions()`
> is overridden by the target.
>
> In a proprietary benchmark we see a performance drop of about 12% on PNG
> compression before this patch, though it passes all tests.
>
> This patch mirrors X86 for AArch64 and initializes
> `TTI::MemCmpExpansionOptions()` to then expand calls to `bcmp()` when
> appropriate.  No tuning of the parameters was performed, but, at this point,
> it's enough to recover the performance drop above.
>
> This problem also exists on ARM.  Once a consensus is reached for AArch64, we
> can work to fix ARM as well.
>
> Authors:
> - Evandro Menezes (@evandro) <e.menezes at samsung.com>
> - Brian Rzycki (@brzycki) <b.rzycki at samsung.com>
>
> Differential revision: https://reviews.llvm.org/D64805
>
> Added:
>     llvm/trunk/test/CodeGen/AArch64/bcmp-inline-small.ll
> Modified:
>     llvm/trunk/lib/Target/AArch64/AArch64ISelLowering.cpp
>     llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
>     llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.h
>
> Modified: llvm/trunk/lib/Target/AArch64/AArch64ISelLowering.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AArch64/AArch64ISelLowering.cpp?rev=367898&r1=367897&r2=367898&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Target/AArch64/AArch64ISelLowering.cpp (original)
> +++ llvm/trunk/lib/Target/AArch64/AArch64ISelLowering.cpp Mon Aug  5 11:09:14 2019
> @@ -629,6 +629,10 @@ AArch64TargetLowering::AArch64TargetLowe
>
>    MaxStoresPerMemmoveOptSize = MaxStoresPerMemmove = 4;
>
> +  MaxLoadsPerMemcmpOptSize = 4;
> +  MaxLoadsPerMemcmp = Subtarget->requiresStrictAlign()
> +                      ? MaxLoadsPerMemcmpOptSize : 8;
> +
>    setStackPointerRegisterToSaveRestore(AArch64::SP);
>
>    setSchedulingPreference(Sched::Hybrid);
>
> Modified: llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.cpp?rev=367898&r1=367897&r2=367898&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.cpp (original)
> +++ llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.cpp Mon Aug  5 11:09:14 2019
> @@ -618,6 +618,19 @@ int AArch64TTIImpl::getCmpSelInstrCost(u
>    return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy, I);
>  }
>
> +AArch64TTIImpl::TTI::MemCmpExpansionOptions
> +AArch64TTIImpl::enableMemCmpExpansion(bool OptSize, bool IsZeroCmp) const {
> +  TTI::MemCmpExpansionOptions Options;
> +  Options.AllowOverlappingLoads = !ST->requiresStrictAlign();
> +  Options.MaxNumLoads = TLI->getMaxExpandSizeMemcmp(OptSize);
> +  Options.NumLoadsPerBlock = Options.MaxNumLoads;
> +  // TODO: Though vector loads usually perform well on AArch64, in some targets
> +  // they may wake up the FP unit, which raises the power consumption.  Perhaps
> +  // they could be used with no holds barred (-O3).
> +  Options.LoadSizes = {8, 4, 2, 1};
> +  return Options;
> +}
> +
>  int AArch64TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Ty,
>                                      unsigned Alignment, unsigned AddressSpace,
>                                      const Instruction *I) {
>
> Modified: llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.h?rev=367898&r1=367897&r2=367898&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.h (original)
> +++ llvm/trunk/lib/Target/AArch64/AArch64TargetTransformInfo.h Mon Aug  5 11:09:14 2019
> @@ -130,6 +130,9 @@ public:
>    int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy,
>                           const Instruction *I = nullptr);
>
> +  TTI::MemCmpExpansionOptions enableMemCmpExpansion(bool OptSize,
> +                                                    bool IsZeroCmp) const;
> +
>    int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
>                        unsigned AddressSpace, const Instruction *I = nullptr);
>
>
> Added: llvm/trunk/test/CodeGen/AArch64/bcmp-inline-small.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CodeGen/AArch64/bcmp-inline-small.ll?rev=367898&view=auto
> ==============================================================================
> --- llvm/trunk/test/CodeGen/AArch64/bcmp-inline-small.ll (added)
> +++ llvm/trunk/test/CodeGen/AArch64/bcmp-inline-small.ll Mon Aug  5 11:09:14 2019
> @@ -0,0 +1,44 @@
> +; RUN: llc -O2 < %s -mtriple=aarch64-linux-gnu                     | FileCheck %s --check-prefixes=CHECK,CHECKN
> +; RUN: llc -O2 < %s -mtriple=aarch64-linux-gnu -mattr=strict-align | FileCheck %s --check-prefixes=CHECK,CHECKS
> +
> +declare i32 @bcmp(i8*, i8*, i64) nounwind readonly
> +declare i32 @memcmp(i8*, i8*, i64) nounwind readonly
> +
> +define i1 @bcmp_b2(i8* %s1, i8* %s2) {
> +entry:
> +  %bcmp = call i32 @bcmp(i8* %s1, i8* %s2, i64 15)
> +  %ret = icmp eq i32 %bcmp, 0
> +  ret i1 %ret
> +
> +; CHECK-LABEL: bcmp_b2:
> +; CHECK-NOT:   bl bcmp
> +; CHECKN:      ldr  x
> +; CHECKN-NEXT: ldr  x
> +; CHECKN-NEXT: ldur x
> +; CHECKN-NEXT: ldur x
> +; CHECKS:      ldr  x
> +; CHECKS-NEXT: ldr  x
> +; CHECKS-NEXT: ldr  w
> +; CHECKS-NEXT: ldr  w
> +; CHECKS-NEXT: ldrh w
> +; CHECKS-NEXT: ldrh w
> +; CHECKS-NEXT: ldrb w
> +; CHECKS-NEXT: ldrb w
> +}
> +
> +define i1 @bcmp_bs(i8* %s1, i8* %s2) optsize {
> +entry:
> +  %memcmp = call i32 @memcmp(i8* %s1, i8* %s2, i64 31)
> +  %ret = icmp eq i32 %memcmp, 0
> +  ret i1 %ret
> +
> +; CHECK-LABEL: bcmp_bs:
> +; CHECKN-NOT:  bl memcmp
> +; CHECKN:      ldp  x
> +; CHECKN-NEXT: ldp  x
> +; CHECKN-NEXT: ldr  x
> +; CHECKN-NEXT: ldr  x
> +; CHECKN-NEXT: ldur x
> +; CHECKN-NEXT: ldur x
> +; CHECKS:      bl memcmp
> +}
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits