[llvm-branch-commits] [llvm-branch] r368017 - Merging r367898:

[Hans Wennborg via llvm-branch-commits]

Author: hans
Date: Tue Aug  6 04:04:11 2019
New Revision: 368017

URL: http://llvm.org/viewvc/llvm-project?rev=368017&view=rev
Log:
Merging r367898:
------------------------------------------------------------------------
r367898 | evandro | 2019-08-05 20:09:14 +0200 (Mon, 05 Aug 2019) | 22 lines

[AArch64] Expand bcmp() for small block lengths

Patch D56593 by @courbet results in calls to `bcmp()` in some cases, should
the target support the it.  Unless `TTI::MemCmpExpansionOptions()`
is overridden by the target.

In a proprietary benchmark we see a performance drop of about 12% on PNG
compression before this patch, though it passes all tests.

This patch mirrors X86 for AArch64 and initializes
`TTI::MemCmpExpansionOptions()` to then expand calls to `bcmp()` when
appropriate.  No tuning of the parameters was performed, but, at this point,
it's enough to recover the performance drop above.

This problem also exists on ARM.  Once a consensus is reached for AArch64, we
can work to fix ARM as well.

Authors:
- Evandro Menezes (@evandro) <e.menezes at samsung.com>
- Brian Rzycki (@brzycki) <b.rzycki at samsung.com>

Differential revision: https://reviews.llvm.org/D64805
------------------------------------------------------------------------

Added:
    llvm/branches/release_90/test/CodeGen/AArch64/bcmp-inline-small.ll
      - copied unchanged from r367898, llvm/trunk/test/CodeGen/AArch64/bcmp-inline-small.ll
Modified:
    llvm/branches/release_90/   (props changed)
    llvm/branches/release_90/lib/Target/AArch64/AArch64ISelLowering.cpp
    llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
    llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.h

Propchange: llvm/branches/release_90/
------------------------------------------------------------------------------
--- svn:mergeinfo (original)
+++ svn:mergeinfo Tue Aug  6 04:04:11 2019
@@ -1,3 +1,3 @@
 /llvm/branches/Apple/Pertwee:110850,110961
 /llvm/branches/type-system-rewrite:133420-134817
-/llvm/trunk:155241,366431,366481,366487,366527,366570,366660,366868,366925,367030,367062,367124,367215,367292,367304,367306,367314,367340-367341,367394,367396,367398,367417,367662,367753,367846-367847,367941,368004
+/llvm/trunk:155241,366431,366481,366487,366527,366570,366660,366868,366925,367030,367062,367124,367215,367292,367304,367306,367314,367340-367341,367394,367396,367398,367417,367662,367753,367846-367847,367898,367941,368004

Modified: llvm/branches/release_90/lib/Target/AArch64/AArch64ISelLowering.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/branches/release_90/lib/Target/AArch64/AArch64ISelLowering.cpp?rev=368017&r1=368016&r2=368017&view=diff
==============================================================================
--- llvm/branches/release_90/lib/Target/AArch64/AArch64ISelLowering.cpp (original)
+++ llvm/branches/release_90/lib/Target/AArch64/AArch64ISelLowering.cpp Tue Aug  6 04:04:11 2019
@@ -606,6 +606,10 @@ AArch64TargetLowering::AArch64TargetLowe
 
   MaxStoresPerMemmoveOptSize = MaxStoresPerMemmove = 4;
 
+  MaxLoadsPerMemcmpOptSize = 4;
+  MaxLoadsPerMemcmp = Subtarget->requiresStrictAlign()
+                      ? MaxLoadsPerMemcmpOptSize : 8;
+
   setStackPointerRegisterToSaveRestore(AArch64::SP);
 
   setSchedulingPreference(Sched::Hybrid);

Modified: llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.cpp?rev=368017&r1=368016&r2=368017&view=diff
==============================================================================
--- llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.cpp (original)
+++ llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.cpp Tue Aug  6 04:04:11 2019
@@ -618,6 +618,19 @@ int AArch64TTIImpl::getCmpSelInstrCost(u
   return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy, I);
 }
 
+AArch64TTIImpl::TTI::MemCmpExpansionOptions
+AArch64TTIImpl::enableMemCmpExpansion(bool OptSize, bool IsZeroCmp) const {
+  TTI::MemCmpExpansionOptions Options;
+  Options.AllowOverlappingLoads = !ST->requiresStrictAlign();
+  Options.MaxNumLoads = TLI->getMaxExpandSizeMemcmp(OptSize);
+  Options.NumLoadsPerBlock = Options.MaxNumLoads;
+  // TODO: Though vector loads usually perform well on AArch64, in some targets
+  // they may wake up the FP unit, which raises the power consumption.  Perhaps
+  // they could be used with no holds barred (-O3).
+  Options.LoadSizes = {8, 4, 2, 1};
+  return Options;
+}
+
 int AArch64TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Ty,
                                     unsigned Alignment, unsigned AddressSpace,
                                     const Instruction *I) {

Modified: llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.h
URL: http://llvm.org/viewvc/llvm-project/llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.h?rev=368017&r1=368016&r2=368017&view=diff
==============================================================================
--- llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.h (original)
+++ llvm/branches/release_90/lib/Target/AArch64/AArch64TargetTransformInfo.h Tue Aug  6 04:04:11 2019
@@ -130,6 +130,9 @@ public:
   int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy,
                          const Instruction *I = nullptr);
 
+  TTI::MemCmpExpansionOptions enableMemCmpExpansion(bool OptSize,
+                                                    bool IsZeroCmp) const;
+
   int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
                       unsigned AddressSpace, const Instruction *I = nullptr);