[llvm] [LoadStoreVectorizer] Fill gaps in load/store chains to enable vectorization (PR #159388)

[Akshay Deodhar via llvm-commits]

================
@@ -623,6 +660,56 @@ std::vector<Chain> Vectorizer::splitChainByContiguity(Chain &C) {
     dumpChain(C);
   });
 
+  // If the chain is not contiguous, we try to fill the gap with "extra"
+  // elements to artificially make it contiguous, to try to enable
+  // vectorization. We only fill gaps if there is potential to end up with a
+  // legal masked load/store given the target, address space, and element type.
+  // At this point, when querying the TTI, optimistically assume max alignment
+  // and max vector size, as splitChainByAlignment will ensure the final vector
+  // shape passes the legalization check.
+  unsigned AS = getLoadStoreAddressSpace(C[0].Inst);
+  Type *ElementType = getLoadStoreType(C[0].Inst)->getScalarType();
+  unsigned MaxVecRegBits = TTI.getLoadStoreVecRegBitWidth(AS);
+  Align OptimisticAlign = Align(MaxVecRegBits / 8);
+  unsigned int MaxVectorNumElems =
+      MaxVecRegBits / DL.getTypeSizeInBits(ElementType);
+  // Note: This check decides whether to try to fill gaps based on the masked
+  // legality of the target's maximum vector size (getLoadStoreVecRegBitWidth).
+  // If a target *does not* support a masked load/store with this max vector
+  // size, but *does* support a masked load/store with a *smaller* vector size,
+  // that optimization will be missed. This does not occur in any of the targets
+  // that currently support this API.
+  FixedVectorType *OptimisticVectorType =
+      FixedVectorType::get(ElementType, MaxVectorNumElems);
+  bool TryFillGaps =
+      isa<LoadInst>(C[0].Inst)
+          ? TTI.isLegalMaskedLoad(OptimisticVectorType, OptimisticAlign, AS,
+                                  TTI::MaskKind::ConstantMask)
+          : TTI.isLegalMaskedStore(OptimisticVectorType, OptimisticAlign, AS,
+                                   TTI::MaskKind::ConstantMask);
+
+  // Cache the best aligned element in the chain for use when creating extra
----------------
akshayrdeodhar wrote:

Possibly extract this out into a function? Returning two things isn't the cleanest, I'm okay either way. 

https://github.com/llvm/llvm-project/pull/159388