[llvm] r205495 - Account for scalarization costs in BasicTTI::getMemoryOpCost for extending vector loads

[Hal Finkel]

Author: hfinkel
Date: Wed Apr  2 19:53:59 2014
New Revision: 205495

URL: http://llvm.org/viewvc/llvm-project?rev=205495&view=rev
Log:
Account for scalarization costs in BasicTTI::getMemoryOpCost for extending vector loads

When a vector type legalizes to a larger vector type, and the target does not
support the associated extending load (or truncating store), then legalization
will scalarize the load (or store) resulting in an associated scalarization
cost.  BasicTTI::getMemoryOpCost needs to account for this.

Between this, and r205487, PowerPC on the P7 with VSX enabled shows:

MultiSource/Benchmarks/PAQ8p/paq8p: 43% speedup
SingleSource/Benchmarks/BenchmarkGame/puzzle: 51% speedup
SingleSource/UnitTests/Vectorizer/gcc-loops 28% speedup

(some of these are new; some of these, such as PAQ8p, just reverse regressions
that VSX support would trigger)

Modified:
    llvm/trunk/lib/CodeGen/BasicTargetTransformInfo.cpp
    llvm/trunk/test/Analysis/CostModel/PowerPC/load_store.ll

Modified: llvm/trunk/lib/CodeGen/BasicTargetTransformInfo.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/CodeGen/BasicTargetTransformInfo.cpp?rev=205495&r1=205494&r2=205495&view=diff
==============================================================================
--- llvm/trunk/lib/CodeGen/BasicTargetTransformInfo.cpp (original)
+++ llvm/trunk/lib/CodeGen/BasicTargetTransformInfo.cpp Wed Apr  2 19:53:59 2014
@@ -416,8 +416,30 @@ unsigned BasicTTI::getMemoryOpCost(unsig
   assert(!Src->isVoidTy() && "Invalid type");
   std::pair<unsigned, MVT> LT = getTLI()->getTypeLegalizationCost(Src);
 
-  // Assume that all loads of legal types cost 1.
-  return LT.first;
+  // Assuming that all loads of legal types cost 1.
+  unsigned Cost = LT.first;
+
+  if (Src->isVectorTy() &&
+      Src->getPrimitiveSizeInBits() < LT.second.getSizeInBits()) {
+    // This is a vector load that legalizes to a larger type than the vector
+    // itself. Unless the corresponding extending load or truncating store is
+    // legal, then this will scalarize.
+    TargetLowering::LegalizeAction LA;
+    MVT MemVT = getTLI()->getSimpleValueType(Src, true);
+    if (Opcode == Instruction::Store)
+      LA = getTLI()->getTruncStoreAction(LT.second, MemVT);
+    else
+      LA = getTLI()->getLoadExtAction(ISD::EXTLOAD, MemVT);
+
+    if (LA != TargetLowering::Legal && LA != TargetLowering::Custom) {
+      // This is a vector load/store for some illegal type that is scalarized.
+      // We must account for the cost of building or decomposing the vector.
+      Cost += getScalarizationOverhead(Src, Opcode != Instruction::Store,
+                                            Opcode == Instruction::Store);
+    }
+  }
+
+  return Cost;
 }
 
 unsigned BasicTTI::getIntrinsicInstrCost(Intrinsic::ID IID, Type *RetTy,

Modified: llvm/trunk/test/Analysis/CostModel/PowerPC/load_store.ll
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Analysis/CostModel/PowerPC/load_store.ll?rev=205495&r1=205494&r2=205495&view=diff
==============================================================================
--- llvm/trunk/test/Analysis/CostModel/PowerPC/load_store.ll (original)
+++ llvm/trunk/test/Analysis/CostModel/PowerPC/load_store.ll Wed Apr  2 19:53:59 2014
@@ -29,6 +29,11 @@ define i32 @loads(i32 %arg) {
   ; CHECK: cost of 4 {{.*}} load
   load i128* undef, align 4
 
+  ; FIXME: There actually are sub-vector Altivec loads, and so we could handle
+  ; this with a small expense, but we don't currently.
+  ; CHECK: cost of 60 {{.*}} load
+  load <4 x i16>* undef, align 2
+
   ret i32 undef
 }