[PATCH] D64805: [AArch64] Don't call bcmp for small byte comparisons

[Brian Rzycki via Phabricator via llvm-commits]

brzycki created this revision.
brzycki added reviewers: evandro, rengolin, efriedma, SjoerdMeijer, t.p.northover, kristof.beyls.
Herald added subscribers: hiraditya, javed.absar.
Herald added a project: LLVM.

Patch D56593 <https://reviews.llvm.org/D56593> by @courbet causes `bcmp()` to be emitted in some cases should the target libc support the call. Unfortunately, the AArch64 target does not initialize `TTI::MemCmpExpansionOptions()` which causes emitted calls to `bcmp()` for small constant byte values. In a proprietary benchmark that relies on zlib-ng we see a performance drop of about 12% on PNG compression before this patch. Passes `ninja check` and `ninja check-all`.

We mimic how x86_64 initializes `TTI::MemCmpExpansionOptions()` to properly detect and replace these expensive external calls.

This problem also exists on ARM 32-bit (`arm-unknown-linux-gnueabi`). Once a consensus is reached for AArch64 we can work to fix ARM 32-bit as well.

Patch is originally authored by @evandro and he asked me to curate while is out of the office.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D64805

Files:
  llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
  llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
  llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
  llvm/test/CodeGen/AArch64/bcmp-inline-small.ll


Index: llvm/test/CodeGen/AArch64/bcmp-inline-small.ll
===================================================================
--- /dev/null
+++ llvm/test/CodeGen/AArch64/bcmp-inline-small.ll
@@ -0,0 +1,12 @@
+; RUN: llc < %s -mtriple=aarch64-linux-gnu | FileCheck %s
+
+; CHECK-LABEL: bcmp_b2:
+; CHECK-NOT: bl bcmp
+define i1 @bcmp_b2(i8* %s1, i8* %s2, i32 %ok) {
+bb:
+  %bcmp = tail call i32 @bcmp(i8* %s1, i8* %s2, i64 2)
+  %ret = icmp eq i32 %bcmp, %ok
+  ret i1 %ret
+}
+
+declare i32 @bcmp(i8*, i8*, i64)
Index: llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
===================================================================
--- llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
+++ llvm/lib/Target/AArch64/AArch64TargetTransformInfo.h
@@ -130,6 +130,9 @@
   int getCmpSelInstrCost(unsigned Opcode, Type *ValTy, Type *CondTy,
                          const Instruction *I = nullptr);
 
+  TTI::MemCmpExpansionOptions enableMemCmpExpansion(bool OptSize,
+                                                    bool IsZeroCmp) const;
+
   int getMemoryOpCost(unsigned Opcode, Type *Src, unsigned Alignment,
                       unsigned AddressSpace, const Instruction *I = nullptr);
 
Index: llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
===================================================================
--- llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
+++ llvm/lib/Target/AArch64/AArch64TargetTransformInfo.cpp
@@ -618,6 +618,19 @@
   return BaseT::getCmpSelInstrCost(Opcode, ValTy, CondTy, I);
 }
 
+AArch64TTIImpl::TTI::MemCmpExpansionOptions
+AArch64TTIImpl::enableMemCmpExpansion(bool OptSize, bool IsZeroCmp) const {
+  TTI::MemCmpExpansionOptions Options;
+  Options.MaxNumLoads = TLI->getMaxExpandSizeMemcmp(OptSize);
+  Options.NumLoadsPerBlock = 2;
+  Options.AllowOverlappingLoads = true;
+  Options.LoadSizes.push_back(8);
+  Options.LoadSizes.push_back(4);
+  Options.LoadSizes.push_back(2);
+  Options.LoadSizes.push_back(1);
+  return Options;
+}
+
 int AArch64TTIImpl::getMemoryOpCost(unsigned Opcode, Type *Ty,
                                     unsigned Alignment, unsigned AddressSpace,
                                     const Instruction *I) {
Index: llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
===================================================================
--- llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
+++ llvm/lib/Target/AArch64/AArch64ISelLowering.cpp
@@ -606,6 +606,8 @@
 
   MaxStoresPerMemmoveOptSize = MaxStoresPerMemmove = 4;
 
+  MaxLoadsPerMemcmpOptSize = MaxLoadsPerMemcmp = 2;
+
   setStackPointerRegisterToSaveRestore(AArch64::SP);
 
   setSchedulingPreference(Sched::Hybrid);


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D64805.210116.patch
Type: text/x-patch
Size: 2666 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20190716/4b647468/attachment.bin>