[llvm] 585c594 - Move delinearization logic out of SCEV [NFC]
Philip Reames via llvm-commits
llvm-commits at lists.llvm.org
Wed Sep 8 14:50:00 PDT 2021
They weren't part of the tree rooted in delinearize. We could move them
or not, either seems reasonable.
Philip
On 9/8/21 1:05 PM, Michael Kruse wrote:
> You left getIndexExpressionsFromGEP inside ScalarEvolution. Don't you
> consider it part of Delinearization? getElementSize is also only used
> in delinearization use cases/was introduced for it.
>
> Michael
>
> Am Mi., 8. Sept. 2021 um 14:28 Uhr schrieb Philip Reames via
> llvm-commits <llvm-commits at lists.llvm.org>:
>>
>> Author: Philip Reames
>> Date: 2021-09-08T12:28:35-07:00
>> New Revision: 585c594d749a2a88150b63804587af85abdabeaa
>>
>> URL: https://github.com/llvm/llvm-project/commit/585c594d749a2a88150b63804587af85abdabeaa
>> DIFF: https://github.com/llvm/llvm-project/commit/585c594d749a2a88150b63804587af85abdabeaa.diff
>>
>> LOG: Move delinearization logic out of SCEV [NFC]
>>
>> None of this logic has anything to do with SCEV's internals, it just uses the existing public APIs. As a result, we can move the code from ScalarEvolution.cpp/hpp to Delinearization.cpp/hpp with only minor changes.
>>
>> This was discussed in advance on today's loop opt call. It turned out to be easy as hoped.
>>
>> Added:
>>
>>
>> Modified:
>> llvm/include/llvm/Analysis/Delinearization.h
>> llvm/include/llvm/Analysis/ScalarEvolution.h
>> llvm/lib/Analysis/Delinearization.cpp
>> llvm/lib/Analysis/DependenceAnalysis.cpp
>> llvm/lib/Analysis/LoopCacheAnalysis.cpp
>> llvm/lib/Analysis/ScalarEvolution.cpp
>>
>> Removed:
>>
>>
>>
>> ################################################################################
>> diff --git a/llvm/include/llvm/Analysis/Delinearization.h b/llvm/include/llvm/Analysis/Delinearization.h
>> index 2658b6bbc80c0..dec3fd9b08a9f 100644
>> --- a/llvm/include/llvm/Analysis/Delinearization.h
>> +++ b/llvm/include/llvm/Analysis/Delinearization.h
>> @@ -16,10 +16,100 @@
>> #ifndef LLVM_ANALYSIS_DELINEARIZATION_H
>> #define LLVM_ANALYSIS_DELINEARIZATION_H
>>
>> +#include "llvm/ADT/SmallVector.h"
>> #include "llvm/IR/PassManager.h"
>> #include "llvm/Support/raw_ostream.h"
>>
>> namespace llvm {
>> +class ScalarEvolution;
>> +class SCEV;
>> +
>> +/// Compute the array dimensions Sizes from the set of Terms extracted from
>> +/// the memory access function of this SCEVAddRecExpr (second step of
>> +/// delinearization).
>> +void findArrayDimensions(ScalarEvolution &SE,
>> + SmallVectorImpl<const SCEV *> &Terms,
>> + SmallVectorImpl<const SCEV *> &Sizes,
>> + const SCEV *ElementSize);
>> +
>> +/// Collect parametric terms occurring in step expressions (first step of
>> +/// delinearization).
>> +void collectParametricTerms(ScalarEvolution &SE, const SCEV *Expr,
>> + SmallVectorImpl<const SCEV *> &Terms);
>> +
>> +/// Return in Subscripts the access functions for each dimension in Sizes
>> +/// (third step of delinearization).
>> +void computeAccessFunctions(ScalarEvolution &SE, const SCEV *Expr,
>> + SmallVectorImpl<const SCEV *> &Subscripts,
>> + SmallVectorImpl<const SCEV *> &Sizes);
>> +/// Split this SCEVAddRecExpr into two vectors of SCEVs representing the
>> +/// subscripts and sizes of an array access.
>> +///
>> +/// The delinearization is a 3 step process: the first two steps compute the
>> +/// sizes of each subscript and the third step computes the access functions
>> +/// for the delinearized array:
>> +///
>> +/// 1. Find the terms in the step functions
>> +/// 2. Compute the array size
>> +/// 3. Compute the access function: divide the SCEV by the array size
>> +/// starting with the innermost dimensions found in step 2. The Quotient
>> +/// is the SCEV to be divided in the next step of the recursion. The
>> +/// Remainder is the subscript of the innermost dimension. Loop over all
>> +/// array dimensions computed in step 2.
>> +///
>> +/// To compute a uniform array size for several memory accesses to the same
>> +/// object, one can collect in step 1 all the step terms for all the memory
>> +/// accesses, and compute in step 2 a unique array shape. This guarantees
>> +/// that the array shape will be the same across all memory accesses.
>> +///
>> +/// FIXME: We could derive the result of steps 1 and 2 from a description of
>> +/// the array shape given in metadata.
>> +///
>> +/// Example:
>> +///
>> +/// A[][n][m]
>> +///
>> +/// for i
>> +/// for j
>> +/// for k
>> +/// A[j+k][2i][5i] =
>> +///
>> +/// The initial SCEV:
>> +///
>> +/// A[{{{0,+,2*m+5}_i, +, n*m}_j, +, n*m}_k]
>> +///
>> +/// 1. Find the
>> diff erent terms in the step functions:
>> +/// -> [2*m, 5, n*m, n*m]
>> +///
>> +/// 2. Compute the array size: sort and unique them
>> +/// -> [n*m, 2*m, 5]
>> +/// find the GCD of all the terms = 1
>> +/// divide by the GCD and erase constant terms
>> +/// -> [n*m, 2*m]
>> +/// GCD = m
>> +/// divide by GCD -> [n, 2]
>> +/// remove constant terms
>> +/// -> [n]
>> +/// size of the array is A[unknown][n][m]
>> +///
>> +/// 3. Compute the access function
>> +/// a. Divide {{{0,+,2*m+5}_i, +, n*m}_j, +, n*m}_k by the innermost size m
>> +/// Quotient: {{{0,+,2}_i, +, n}_j, +, n}_k
>> +/// Remainder: {{{0,+,5}_i, +, 0}_j, +, 0}_k
>> +/// The remainder is the subscript of the innermost array dimension: [5i].
>> +///
>> +/// b. Divide Quotient: {{{0,+,2}_i, +, n}_j, +, n}_k by next outer size n
>> +/// Quotient: {{{0,+,0}_i, +, 1}_j, +, 1}_k
>> +/// Remainder: {{{0,+,2}_i, +, 0}_j, +, 0}_k
>> +/// The Remainder is the subscript of the next array dimension: [2i].
>> +///
>> +/// The subscript of the outermost dimension is the Quotient: [j+k].
>> +///
>> +/// Overall, we have: A[][n][m], and the access function: A[j+k][2i][5i].
>> +void delinearize(ScalarEvolution &SE, const SCEV *Expr,
>> + SmallVectorImpl<const SCEV *> &Subscripts,
>> + SmallVectorImpl<const SCEV *> &Sizes, const SCEV *ElementSize);
>> +
>> struct DelinearizationPrinterPass
>> : public PassInfoMixin<DelinearizationPrinterPass> {
>> explicit DelinearizationPrinterPass(raw_ostream &OS);
>>
>> diff --git a/llvm/include/llvm/Analysis/ScalarEvolution.h b/llvm/include/llvm/Analysis/ScalarEvolution.h
>> index 6ae865e011cf0..445a925c70e83 100644
>> --- a/llvm/include/llvm/Analysis/ScalarEvolution.h
>> +++ b/llvm/include/llvm/Analysis/ScalarEvolution.h
>> @@ -1099,29 +1099,11 @@ class ScalarEvolution {
>> /// Return the size of an element read or written by Inst.
>> const SCEV *getElementSize(Instruction *Inst);
>>
>> - /// Compute the array dimensions Sizes from the set of Terms extracted from
>> - /// the memory access function of this SCEVAddRecExpr (second step of
>> - /// delinearization).
>> - void findArrayDimensions(SmallVectorImpl<const SCEV *> &Terms,
>> - SmallVectorImpl<const SCEV *> &Sizes,
>> - const SCEV *ElementSize);
>> -
>> void print(raw_ostream &OS) const;
>> void verify() const;
>> bool invalidate(Function &F, const PreservedAnalyses &PA,
>> FunctionAnalysisManager::Invalidator &Inv);
>>
>> - /// Collect parametric terms occurring in step expressions (first step of
>> - /// delinearization).
>> - void collectParametricTerms(const SCEV *Expr,
>> - SmallVectorImpl<const SCEV *> &Terms);
>> -
>> - /// Return in Subscripts the access functions for each dimension in Sizes
>> - /// (third step of delinearization).
>> - void computeAccessFunctions(const SCEV *Expr,
>> - SmallVectorImpl<const SCEV *> &Subscripts,
>> - SmallVectorImpl<const SCEV *> &Sizes);
>> -
>> /// Gathers the individual index expressions from a GEP instruction.
>> ///
>> /// This function optimistically assumes the GEP references into a fixed size
>> @@ -1135,74 +1117,6 @@ class ScalarEvolution {
>> SmallVectorImpl<const SCEV *> &Subscripts,
>> SmallVectorImpl<int> &Sizes);
>>
>> - /// Split this SCEVAddRecExpr into two vectors of SCEVs representing the
>> - /// subscripts and sizes of an array access.
>> - ///
>> - /// The delinearization is a 3 step process: the first two steps compute the
>> - /// sizes of each subscript and the third step computes the access functions
>> - /// for the delinearized array:
>> - ///
>> - /// 1. Find the terms in the step functions
>> - /// 2. Compute the array size
>> - /// 3. Compute the access function: divide the SCEV by the array size
>> - /// starting with the innermost dimensions found in step 2. The Quotient
>> - /// is the SCEV to be divided in the next step of the recursion. The
>> - /// Remainder is the subscript of the innermost dimension. Loop over all
>> - /// array dimensions computed in step 2.
>> - ///
>> - /// To compute a uniform array size for several memory accesses to the same
>> - /// object, one can collect in step 1 all the step terms for all the memory
>> - /// accesses, and compute in step 2 a unique array shape. This guarantees
>> - /// that the array shape will be the same across all memory accesses.
>> - ///
>> - /// FIXME: We could derive the result of steps 1 and 2 from a description of
>> - /// the array shape given in metadata.
>> - ///
>> - /// Example:
>> - ///
>> - /// A[][n][m]
>> - ///
>> - /// for i
>> - /// for j
>> - /// for k
>> - /// A[j+k][2i][5i] =
>> - ///
>> - /// The initial SCEV:
>> - ///
>> - /// A[{{{0,+,2*m+5}_i, +, n*m}_j, +, n*m}_k]
>> - ///
>> - /// 1. Find the
>> diff erent terms in the step functions:
>> - /// -> [2*m, 5, n*m, n*m]
>> - ///
>> - /// 2. Compute the array size: sort and unique them
>> - /// -> [n*m, 2*m, 5]
>> - /// find the GCD of all the terms = 1
>> - /// divide by the GCD and erase constant terms
>> - /// -> [n*m, 2*m]
>> - /// GCD = m
>> - /// divide by GCD -> [n, 2]
>> - /// remove constant terms
>> - /// -> [n]
>> - /// size of the array is A[unknown][n][m]
>> - ///
>> - /// 3. Compute the access function
>> - /// a. Divide {{{0,+,2*m+5}_i, +, n*m}_j, +, n*m}_k by the innermost size m
>> - /// Quotient: {{{0,+,2}_i, +, n}_j, +, n}_k
>> - /// Remainder: {{{0,+,5}_i, +, 0}_j, +, 0}_k
>> - /// The remainder is the subscript of the innermost array dimension: [5i].
>> - ///
>> - /// b. Divide Quotient: {{{0,+,2}_i, +, n}_j, +, n}_k by next outer size n
>> - /// Quotient: {{{0,+,0}_i, +, 1}_j, +, 1}_k
>> - /// Remainder: {{{0,+,2}_i, +, 0}_j, +, 0}_k
>> - /// The Remainder is the subscript of the next array dimension: [2i].
>> - ///
>> - /// The subscript of the outermost dimension is the Quotient: [j+k].
>> - ///
>> - /// Overall, we have: A[][n][m], and the access function: A[j+k][2i][5i].
>> - void delinearize(const SCEV *Expr, SmallVectorImpl<const SCEV *> &Subscripts,
>> - SmallVectorImpl<const SCEV *> &Sizes,
>> - const SCEV *ElementSize);
>> -
>> /// Return the DataLayout associated with the module this SCEV instance is
>> /// operating on.
>> const DataLayout &getDataLayout() const {
>>
>> diff --git a/llvm/lib/Analysis/Delinearization.cpp b/llvm/lib/Analysis/Delinearization.cpp
>> index 448e970e9bcc9..8219d59b8f40e 100644
>> --- a/llvm/lib/Analysis/Delinearization.cpp
>> +++ b/llvm/lib/Analysis/Delinearization.cpp
>> @@ -17,6 +17,7 @@
>> #include "llvm/Analysis/LoopInfo.h"
>> #include "llvm/Analysis/Passes.h"
>> #include "llvm/Analysis/ScalarEvolution.h"
>> +#include "llvm/Analysis/ScalarEvolutionDivision.h"
>> #include "llvm/Analysis/ScalarEvolutionExpressions.h"
>> #include "llvm/IR/Constants.h"
>> #include "llvm/IR/DerivedTypes.h"
>> @@ -36,6 +37,454 @@ using namespace llvm;
>> #define DL_NAME "delinearize"
>> #define DEBUG_TYPE DL_NAME
>>
>> +// Return true when S contains at least an undef value.
>> +static inline bool containsUndefs(const SCEV *S) {
>> + return SCEVExprContains(S, [](const SCEV *S) {
>> + if (const auto *SU = dyn_cast<SCEVUnknown>(S))
>> + return isa<UndefValue>(SU->getValue());
>> + return false;
>> + });
>> +}
>> +
>> +namespace {
>> +
>> +// Collect all steps of SCEV expressions.
>> +struct SCEVCollectStrides {
>> + ScalarEvolution &SE;
>> + SmallVectorImpl<const SCEV *> &Strides;
>> +
>> + SCEVCollectStrides(ScalarEvolution &SE, SmallVectorImpl<const SCEV *> &S)
>> + : SE(SE), Strides(S) {}
>> +
>> + bool follow(const SCEV *S) {
>> + if (const SCEVAddRecExpr *AR = dyn_cast<SCEVAddRecExpr>(S))
>> + Strides.push_back(AR->getStepRecurrence(SE));
>> + return true;
>> + }
>> +
>> + bool isDone() const { return false; }
>> +};
>> +
>> +// Collect all SCEVUnknown and SCEVMulExpr expressions.
>> +struct SCEVCollectTerms {
>> + SmallVectorImpl<const SCEV *> &Terms;
>> +
>> + SCEVCollectTerms(SmallVectorImpl<const SCEV *> &T) : Terms(T) {}
>> +
>> + bool follow(const SCEV *S) {
>> + if (isa<SCEVUnknown>(S) || isa<SCEVMulExpr>(S) ||
>> + isa<SCEVSignExtendExpr>(S)) {
>> + if (!containsUndefs(S))
>> + Terms.push_back(S);
>> +
>> + // Stop recursion: once we collected a term, do not walk its operands.
>> + return false;
>> + }
>> +
>> + // Keep looking.
>> + return true;
>> + }
>> +
>> + bool isDone() const { return false; }
>> +};
>> +
>> +// Check if a SCEV contains an AddRecExpr.
>> +struct SCEVHasAddRec {
>> + bool &ContainsAddRec;
>> +
>> + SCEVHasAddRec(bool &ContainsAddRec) : ContainsAddRec(ContainsAddRec) {
>> + ContainsAddRec = false;
>> + }
>> +
>> + bool follow(const SCEV *S) {
>> + if (isa<SCEVAddRecExpr>(S)) {
>> + ContainsAddRec = true;
>> +
>> + // Stop recursion: once we collected a term, do not walk its operands.
>> + return false;
>> + }
>> +
>> + // Keep looking.
>> + return true;
>> + }
>> +
>> + bool isDone() const { return false; }
>> +};
>> +
>> +// Find factors that are multiplied with an expression that (possibly as a
>> +// subexpression) contains an AddRecExpr. In the expression:
>> +//
>> +// 8 * (100 + %p * %q * (%a + {0, +, 1}_loop))
>> +//
>> +// "%p * %q" are factors multiplied by the expression "(%a + {0, +, 1}_loop)"
>> +// that contains the AddRec {0, +, 1}_loop. %p * %q are likely to be array size
>> +// parameters as they form a product with an induction variable.
>> +//
>> +// This collector expects all array size parameters to be in the same MulExpr.
>> +// It might be necessary to later add support for collecting parameters that are
>> +// spread over
>> diff erent nested MulExpr.
>> +struct SCEVCollectAddRecMultiplies {
>> + SmallVectorImpl<const SCEV *> &Terms;
>> + ScalarEvolution &SE;
>> +
>> + SCEVCollectAddRecMultiplies(SmallVectorImpl<const SCEV *> &T,
>> + ScalarEvolution &SE)
>> + : Terms(T), SE(SE) {}
>> +
>> + bool follow(const SCEV *S) {
>> + if (auto *Mul = dyn_cast<SCEVMulExpr>(S)) {
>> + bool HasAddRec = false;
>> + SmallVector<const SCEV *, 0> Operands;
>> + for (auto Op : Mul->operands()) {
>> + const SCEVUnknown *Unknown = dyn_cast<SCEVUnknown>(Op);
>> + if (Unknown && !isa<CallInst>(Unknown->getValue())) {
>> + Operands.push_back(Op);
>> + } else if (Unknown) {
>> + HasAddRec = true;
>> + } else {
>> + bool ContainsAddRec = false;
>> + SCEVHasAddRec ContiansAddRec(ContainsAddRec);
>> + visitAll(Op, ContiansAddRec);
>> + HasAddRec |= ContainsAddRec;
>> + }
>> + }
>> + if (Operands.size() == 0)
>> + return true;
>> +
>> + if (!HasAddRec)
>> + return false;
>> +
>> + Terms.push_back(SE.getMulExpr(Operands));
>> + // Stop recursion: once we collected a term, do not walk its operands.
>> + return false;
>> + }
>> +
>> + // Keep looking.
>> + return true;
>> + }
>> +
>> + bool isDone() const { return false; }
>> +};
>> +
>> +} // end anonymous namespace
>> +
>> +/// Find parametric terms in this SCEVAddRecExpr. We first for parameters in
>> +/// two places:
>> +/// 1) The strides of AddRec expressions.
>> +/// 2) Unknowns that are multiplied with AddRec expressions.
>> +void llvm::collectParametricTerms(ScalarEvolution &SE, const SCEV *Expr,
>> + SmallVectorImpl<const SCEV *> &Terms) {
>> + SmallVector<const SCEV *, 4> Strides;
>> + SCEVCollectStrides StrideCollector(SE, Strides);
>> + visitAll(Expr, StrideCollector);
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Strides:\n";
>> + for (const SCEV *S : Strides)
>> + dbgs() << *S << "\n";
>> + });
>> +
>> + for (const SCEV *S : Strides) {
>> + SCEVCollectTerms TermCollector(Terms);
>> + visitAll(S, TermCollector);
>> + }
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Terms:\n";
>> + for (const SCEV *T : Terms)
>> + dbgs() << *T << "\n";
>> + });
>> +
>> + SCEVCollectAddRecMultiplies MulCollector(Terms, SE);
>> + visitAll(Expr, MulCollector);
>> +}
>> +
>> +static bool findArrayDimensionsRec(ScalarEvolution &SE,
>> + SmallVectorImpl<const SCEV *> &Terms,
>> + SmallVectorImpl<const SCEV *> &Sizes) {
>> + int Last = Terms.size() - 1;
>> + const SCEV *Step = Terms[Last];
>> +
>> + // End of recursion.
>> + if (Last == 0) {
>> + if (const SCEVMulExpr *M = dyn_cast<SCEVMulExpr>(Step)) {
>> + SmallVector<const SCEV *, 2> Qs;
>> + for (const SCEV *Op : M->operands())
>> + if (!isa<SCEVConstant>(Op))
>> + Qs.push_back(Op);
>> +
>> + Step = SE.getMulExpr(Qs);
>> + }
>> +
>> + Sizes.push_back(Step);
>> + return true;
>> + }
>> +
>> + for (const SCEV *&Term : Terms) {
>> + // Normalize the terms before the next call to findArrayDimensionsRec.
>> + const SCEV *Q, *R;
>> + SCEVDivision::divide(SE, Term, Step, &Q, &R);
>> +
>> + // Bail out when GCD does not evenly divide one of the terms.
>> + if (!R->isZero())
>> + return false;
>> +
>> + Term = Q;
>> + }
>> +
>> + // Remove all SCEVConstants.
>> + erase_if(Terms, [](const SCEV *E) { return isa<SCEVConstant>(E); });
>> +
>> + if (Terms.size() > 0)
>> + if (!findArrayDimensionsRec(SE, Terms, Sizes))
>> + return false;
>> +
>> + Sizes.push_back(Step);
>> + return true;
>> +}
>> +
>> +// Returns true when one of the SCEVs of Terms contains a SCEVUnknown parameter.
>> +static inline bool containsParameters(SmallVectorImpl<const SCEV *> &Terms) {
>> + for (const SCEV *T : Terms)
>> + if (SCEVExprContains(T, [](const SCEV *S) { return isa<SCEVUnknown>(S); }))
>> + return true;
>> +
>> + return false;
>> +}
>> +
>> +// Return the number of product terms in S.
>> +static inline int numberOfTerms(const SCEV *S) {
>> + if (const SCEVMulExpr *Expr = dyn_cast<SCEVMulExpr>(S))
>> + return Expr->getNumOperands();
>> + return 1;
>> +}
>> +
>> +static const SCEV *removeConstantFactors(ScalarEvolution &SE, const SCEV *T) {
>> + if (isa<SCEVConstant>(T))
>> + return nullptr;
>> +
>> + if (isa<SCEVUnknown>(T))
>> + return T;
>> +
>> + if (const SCEVMulExpr *M = dyn_cast<SCEVMulExpr>(T)) {
>> + SmallVector<const SCEV *, 2> Factors;
>> + for (const SCEV *Op : M->operands())
>> + if (!isa<SCEVConstant>(Op))
>> + Factors.push_back(Op);
>> +
>> + return SE.getMulExpr(Factors);
>> + }
>> +
>> + return T;
>> +}
>> +
>> +void llvm::findArrayDimensions(ScalarEvolution &SE,
>> + SmallVectorImpl<const SCEV *> &Terms,
>> + SmallVectorImpl<const SCEV *> &Sizes,
>> + const SCEV *ElementSize) {
>> + if (Terms.size() < 1 || !ElementSize)
>> + return;
>> +
>> + // Early return when Terms do not contain parameters: we do not delinearize
>> + // non parametric SCEVs.
>> + if (!containsParameters(Terms))
>> + return;
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Terms:\n";
>> + for (const SCEV *T : Terms)
>> + dbgs() << *T << "\n";
>> + });
>> +
>> + // Remove duplicates.
>> + array_pod_sort(Terms.begin(), Terms.end());
>> + Terms.erase(std::unique(Terms.begin(), Terms.end()), Terms.end());
>> +
>> + // Put larger terms first.
>> + llvm::sort(Terms, [](const SCEV *LHS, const SCEV *RHS) {
>> + return numberOfTerms(LHS) > numberOfTerms(RHS);
>> + });
>> +
>> + // Try to divide all terms by the element size. If term is not divisible by
>> + // element size, proceed with the original term.
>> + for (const SCEV *&Term : Terms) {
>> + const SCEV *Q, *R;
>> + SCEVDivision::divide(SE, Term, ElementSize, &Q, &R);
>> + if (!Q->isZero())
>> + Term = Q;
>> + }
>> +
>> + SmallVector<const SCEV *, 4> NewTerms;
>> +
>> + // Remove constant factors.
>> + for (const SCEV *T : Terms)
>> + if (const SCEV *NewT = removeConstantFactors(SE, T))
>> + NewTerms.push_back(NewT);
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Terms after sorting:\n";
>> + for (const SCEV *T : NewTerms)
>> + dbgs() << *T << "\n";
>> + });
>> +
>> + if (NewTerms.empty() || !findArrayDimensionsRec(SE, NewTerms, Sizes)) {
>> + Sizes.clear();
>> + return;
>> + }
>> +
>> + // The last element to be pushed into Sizes is the size of an element.
>> + Sizes.push_back(ElementSize);
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Sizes:\n";
>> + for (const SCEV *S : Sizes)
>> + dbgs() << *S << "\n";
>> + });
>> +}
>> +
>> +void llvm::computeAccessFunctions(ScalarEvolution &SE, const SCEV *Expr,
>> + SmallVectorImpl<const SCEV *> &Subscripts,
>> + SmallVectorImpl<const SCEV *> &Sizes) {
>> + // Early exit in case this SCEV is not an affine multivariate function.
>> + if (Sizes.empty())
>> + return;
>> +
>> + if (auto *AR = dyn_cast<SCEVAddRecExpr>(Expr))
>> + if (!AR->isAffine())
>> + return;
>> +
>> + const SCEV *Res = Expr;
>> + int Last = Sizes.size() - 1;
>> + for (int i = Last; i >= 0; i--) {
>> + const SCEV *Q, *R;
>> + SCEVDivision::divide(SE, Res, Sizes[i], &Q, &R);
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Res: " << *Res << "\n";
>> + dbgs() << "Sizes[i]: " << *Sizes[i] << "\n";
>> + dbgs() << "Res divided by Sizes[i]:\n";
>> + dbgs() << "Quotient: " << *Q << "\n";
>> + dbgs() << "Remainder: " << *R << "\n";
>> + });
>> +
>> + Res = Q;
>> +
>> + // Do not record the last subscript corresponding to the size of elements in
>> + // the array.
>> + if (i == Last) {
>> +
>> + // Bail out if the remainder is too complex.
>> + if (isa<SCEVAddRecExpr>(R)) {
>> + Subscripts.clear();
>> + Sizes.clear();
>> + return;
>> + }
>> +
>> + continue;
>> + }
>> +
>> + // Record the access function for the current subscript.
>> + Subscripts.push_back(R);
>> + }
>> +
>> + // Also push in last position the remainder of the last division: it will be
>> + // the access function of the innermost dimension.
>> + Subscripts.push_back(Res);
>> +
>> + std::reverse(Subscripts.begin(), Subscripts.end());
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "Subscripts:\n";
>> + for (const SCEV *S : Subscripts)
>> + dbgs() << *S << "\n";
>> + });
>> +}
>> +
>> +/// Splits the SCEV into two vectors of SCEVs representing the subscripts and
>> +/// sizes of an array access. Returns the remainder of the delinearization that
>> +/// is the offset start of the array. The SCEV->delinearize algorithm computes
>> +/// the multiples of SCEV coefficients: that is a pattern matching of sub
>> +/// expressions in the stride and base of a SCEV corresponding to the
>> +/// computation of a GCD (greatest common divisor) of base and stride. When
>> +/// SCEV->delinearize fails, it returns the SCEV unchanged.
>> +///
>> +/// For example: when analyzing the memory access A[i][j][k] in this loop nest
>> +///
>> +/// void foo(long n, long m, long o, double A[n][m][o]) {
>> +///
>> +/// for (long i = 0; i < n; i++)
>> +/// for (long j = 0; j < m; j++)
>> +/// for (long k = 0; k < o; k++)
>> +/// A[i][j][k] = 1.0;
>> +/// }
>> +///
>> +/// the delinearization input is the following AddRec SCEV:
>> +///
>> +/// AddRec: {{{%A,+,(8 * %m * %o)}<%for.i>,+,(8 * %o)}<%for.j>,+,8}<%for.k>
>> +///
>> +/// From this SCEV, we are able to say that the base offset of the access is %A
>> +/// because it appears as an offset that does not divide any of the strides in
>> +/// the loops:
>> +///
>> +/// CHECK: Base offset: %A
>> +///
>> +/// and then SCEV->delinearize determines the size of some of the dimensions of
>> +/// the array as these are the multiples by which the strides are happening:
>> +///
>> +/// CHECK: ArrayDecl[UnknownSize][%m][%o] with elements of sizeof(double)
>> +/// bytes.
>> +///
>> +/// Note that the outermost dimension remains of UnknownSize because there are
>> +/// no strides that would help identifying the size of the last dimension: when
>> +/// the array has been statically allocated, one could compute the size of that
>> +/// dimension by dividing the overall size of the array by the size of the known
>> +/// dimensions: %m * %o * 8.
>> +///
>> +/// Finally delinearize provides the access functions for the array reference
>> +/// that does correspond to A[i][j][k] of the above C testcase:
>> +///
>> +/// CHECK: ArrayRef[{0,+,1}<%for.i>][{0,+,1}<%for.j>][{0,+,1}<%for.k>]
>> +///
>> +/// The testcases are checking the output of a function pass:
>> +/// DelinearizationPass that walks through all loads and stores of a function
>> +/// asking for the SCEV of the memory access with respect to all enclosing
>> +/// loops, calling SCEV->delinearize on that and printing the results.
>> +void llvm::delinearize(ScalarEvolution &SE, const SCEV *Expr,
>> + SmallVectorImpl<const SCEV *> &Subscripts,
>> + SmallVectorImpl<const SCEV *> &Sizes,
>> + const SCEV *ElementSize) {
>> + // First step: collect parametric terms.
>> + SmallVector<const SCEV *, 4> Terms;
>> + collectParametricTerms(SE, Expr, Terms);
>> +
>> + if (Terms.empty())
>> + return;
>> +
>> + // Second step: find subscript sizes.
>> + findArrayDimensions(SE, Terms, Sizes, ElementSize);
>> +
>> + if (Sizes.empty())
>> + return;
>> +
>> + // Third step: compute the access functions for each subscript.
>> + computeAccessFunctions(SE, Expr, Subscripts, Sizes);
>> +
>> + if (Subscripts.empty())
>> + return;
>> +
>> + LLVM_DEBUG({
>> + dbgs() << "succeeded to delinearize " << *Expr << "\n";
>> + dbgs() << "ArrayDecl[UnknownSize]";
>> + for (const SCEV *S : Sizes)
>> + dbgs() << "[" << *S << "]";
>> +
>> + dbgs() << "\nArrayRef";
>> + for (const SCEV *S : Subscripts)
>> + dbgs() << "[" << *S << "]";
>> + dbgs() << "\n";
>> + });
>> +}
>> +
>> namespace {
>>
>> class Delinearization : public FunctionPass {
>> @@ -84,7 +533,7 @@ void printDelinearization(raw_ostream &O, Function *F, LoopInfo *LI,
>> O << "AccessFunction: " << *AccessFn << "\n";
>>
>> SmallVector<const SCEV *, 3> Subscripts, Sizes;
>> - SE->delinearize(AccessFn, Subscripts, Sizes, SE->getElementSize(&Inst));
>> + delinearize(*SE, AccessFn, Subscripts, Sizes, SE->getElementSize(&Inst));
>> if (Subscripts.size() == 0 || Sizes.size() == 0 ||
>> Subscripts.size() != Sizes.size()) {
>> O << "failed to delinearize\n";
>>
>> diff --git a/llvm/lib/Analysis/DependenceAnalysis.cpp b/llvm/lib/Analysis/DependenceAnalysis.cpp
>> index 9c09894f3a451..fa9c672119455 100644
>> --- a/llvm/lib/Analysis/DependenceAnalysis.cpp
>> +++ b/llvm/lib/Analysis/DependenceAnalysis.cpp
>> @@ -53,6 +53,7 @@
>> #include "llvm/ADT/STLExtras.h"
>> #include "llvm/ADT/Statistic.h"
>> #include "llvm/Analysis/AliasAnalysis.h"
>> +#include "llvm/Analysis/Delinearization.h"
>> #include "llvm/Analysis/LoopInfo.h"
>> #include "llvm/Analysis/ScalarEvolution.h"
>> #include "llvm/Analysis/ScalarEvolutionExpressions.h"
>> @@ -3439,16 +3440,16 @@ bool DependenceInfo::tryDelinearizeParametricSize(
>>
>> // First step: collect parametric terms in both array references.
>> SmallVector<const SCEV *, 4> Terms;
>> - SE->collectParametricTerms(SrcAR, Terms);
>> - SE->collectParametricTerms(DstAR, Terms);
>> + collectParametricTerms(*SE, SrcAR, Terms);
>> + collectParametricTerms(*SE, DstAR, Terms);
>>
>> // Second step: find subscript sizes.
>> SmallVector<const SCEV *, 4> Sizes;
>> - SE->findArrayDimensions(Terms, Sizes, ElementSize);
>> + findArrayDimensions(*SE, Terms, Sizes, ElementSize);
>>
>> // Third step: compute the access functions for each subscript.
>> - SE->computeAccessFunctions(SrcAR, SrcSubscripts, Sizes);
>> - SE->computeAccessFunctions(DstAR, DstSubscripts, Sizes);
>> + computeAccessFunctions(*SE, SrcAR, SrcSubscripts, Sizes);
>> + computeAccessFunctions(*SE, DstAR, DstSubscripts, Sizes);
>>
>> // Fail when there is only a subscript: that's a linearized access function.
>> if (SrcSubscripts.size() < 2 || DstSubscripts.size() < 2 ||
>>
>> diff --git a/llvm/lib/Analysis/LoopCacheAnalysis.cpp b/llvm/lib/Analysis/LoopCacheAnalysis.cpp
>> index 772e4eb239a9c..75bb8b02286c1 100644
>> --- a/llvm/lib/Analysis/LoopCacheAnalysis.cpp
>> +++ b/llvm/lib/Analysis/LoopCacheAnalysis.cpp
>> @@ -30,6 +30,7 @@
>> #include "llvm/ADT/Sequence.h"
>> #include "llvm/ADT/SmallVector.h"
>> #include "llvm/Analysis/AliasAnalysis.h"
>> +#include "llvm/Analysis/Delinearization.h"
>> #include "llvm/Analysis/DependenceAnalysis.h"
>> #include "llvm/Analysis/LoopInfo.h"
>> #include "llvm/Analysis/ScalarEvolutionExpressions.h"
>> @@ -344,8 +345,8 @@ bool IndexedReference::delinearize(const LoopInfo &LI) {
>> LLVM_DEBUG(dbgs().indent(2) << "In Loop '" << L->getName()
>> << "', AccessFn: " << *AccessFn << "\n");
>>
>> - SE.delinearize(AccessFn, Subscripts, Sizes,
>> - SE.getElementSize(&StoreOrLoadInst));
>> + llvm::delinearize(SE, AccessFn, Subscripts, Sizes,
>> + SE.getElementSize(&StoreOrLoadInst));
>>
>> if (Subscripts.empty() || Sizes.empty() ||
>> Subscripts.size() != Sizes.size()) {
>>
>> diff --git a/llvm/lib/Analysis/ScalarEvolution.cpp b/llvm/lib/Analysis/ScalarEvolution.cpp
>> index 50901f3524d1b..151883ab67367 100644
>> --- a/llvm/lib/Analysis/ScalarEvolution.cpp
>> +++ b/llvm/lib/Analysis/ScalarEvolution.cpp
>> @@ -12187,237 +12187,6 @@ static inline bool containsUndefs(const SCEV *S) {
>> });
>> }
>>
>> -namespace {
>> -
>> -// Collect all steps of SCEV expressions.
>> -struct SCEVCollectStrides {
>> - ScalarEvolution &SE;
>> - SmallVectorImpl<const SCEV *> &Strides;
>> -
>> - SCEVCollectStrides(ScalarEvolution &SE, SmallVectorImpl<const SCEV *> &S)
>> - : SE(SE), Strides(S) {}
>> -
>> - bool follow(const SCEV *S) {
>> - if (const SCEVAddRecExpr *AR = dyn_cast<SCEVAddRecExpr>(S))
>> - Strides.push_back(AR->getStepRecurrence(SE));
>> - return true;
>> - }
>> -
>> - bool isDone() const { return false; }
>> -};
>> -
>> -// Collect all SCEVUnknown and SCEVMulExpr expressions.
>> -struct SCEVCollectTerms {
>> - SmallVectorImpl<const SCEV *> &Terms;
>> -
>> - SCEVCollectTerms(SmallVectorImpl<const SCEV *> &T) : Terms(T) {}
>> -
>> - bool follow(const SCEV *S) {
>> - if (isa<SCEVUnknown>(S) || isa<SCEVMulExpr>(S) ||
>> - isa<SCEVSignExtendExpr>(S)) {
>> - if (!containsUndefs(S))
>> - Terms.push_back(S);
>> -
>> - // Stop recursion: once we collected a term, do not walk its operands.
>> - return false;
>> - }
>> -
>> - // Keep looking.
>> - return true;
>> - }
>> -
>> - bool isDone() const { return false; }
>> -};
>> -
>> -// Check if a SCEV contains an AddRecExpr.
>> -struct SCEVHasAddRec {
>> - bool &ContainsAddRec;
>> -
>> - SCEVHasAddRec(bool &ContainsAddRec) : ContainsAddRec(ContainsAddRec) {
>> - ContainsAddRec = false;
>> - }
>> -
>> - bool follow(const SCEV *S) {
>> - if (isa<SCEVAddRecExpr>(S)) {
>> - ContainsAddRec = true;
>> -
>> - // Stop recursion: once we collected a term, do not walk its operands.
>> - return false;
>> - }
>> -
>> - // Keep looking.
>> - return true;
>> - }
>> -
>> - bool isDone() const { return false; }
>> -};
>> -
>> -// Find factors that are multiplied with an expression that (possibly as a
>> -// subexpression) contains an AddRecExpr. In the expression:
>> -//
>> -// 8 * (100 + %p * %q * (%a + {0, +, 1}_loop))
>> -//
>> -// "%p * %q" are factors multiplied by the expression "(%a + {0, +, 1}_loop)"
>> -// that contains the AddRec {0, +, 1}_loop. %p * %q are likely to be array size
>> -// parameters as they form a product with an induction variable.
>> -//
>> -// This collector expects all array size parameters to be in the same MulExpr.
>> -// It might be necessary to later add support for collecting parameters that are
>> -// spread over
>> diff erent nested MulExpr.
>> -struct SCEVCollectAddRecMultiplies {
>> - SmallVectorImpl<const SCEV *> &Terms;
>> - ScalarEvolution &SE;
>> -
>> - SCEVCollectAddRecMultiplies(SmallVectorImpl<const SCEV *> &T, ScalarEvolution &SE)
>> - : Terms(T), SE(SE) {}
>> -
>> - bool follow(const SCEV *S) {
>> - if (auto *Mul = dyn_cast<SCEVMulExpr>(S)) {
>> - bool HasAddRec = false;
>> - SmallVector<const SCEV *, 0> Operands;
>> - for (auto Op : Mul->operands()) {
>> - const SCEVUnknown *Unknown = dyn_cast<SCEVUnknown>(Op);
>> - if (Unknown && !isa<CallInst>(Unknown->getValue())) {
>> - Operands.push_back(Op);
>> - } else if (Unknown) {
>> - HasAddRec = true;
>> - } else {
>> - bool ContainsAddRec = false;
>> - SCEVHasAddRec ContiansAddRec(ContainsAddRec);
>> - visitAll(Op, ContiansAddRec);
>> - HasAddRec |= ContainsAddRec;
>> - }
>> - }
>> - if (Operands.size() == 0)
>> - return true;
>> -
>> - if (!HasAddRec)
>> - return false;
>> -
>> - Terms.push_back(SE.getMulExpr(Operands));
>> - // Stop recursion: once we collected a term, do not walk its operands.
>> - return false;
>> - }
>> -
>> - // Keep looking.
>> - return true;
>> - }
>> -
>> - bool isDone() const { return false; }
>> -};
>> -
>> -} // end anonymous namespace
>> -
>> -/// Find parametric terms in this SCEVAddRecExpr. We first for parameters in
>> -/// two places:
>> -/// 1) The strides of AddRec expressions.
>> -/// 2) Unknowns that are multiplied with AddRec expressions.
>> -void ScalarEvolution::collectParametricTerms(const SCEV *Expr,
>> - SmallVectorImpl<const SCEV *> &Terms) {
>> - SmallVector<const SCEV *, 4> Strides;
>> - SCEVCollectStrides StrideCollector(*this, Strides);
>> - visitAll(Expr, StrideCollector);
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "Strides:\n";
>> - for (const SCEV *S : Strides)
>> - dbgs() << *S << "\n";
>> - });
>> -
>> - for (const SCEV *S : Strides) {
>> - SCEVCollectTerms TermCollector(Terms);
>> - visitAll(S, TermCollector);
>> - }
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "Terms:\n";
>> - for (const SCEV *T : Terms)
>> - dbgs() << *T << "\n";
>> - });
>> -
>> - SCEVCollectAddRecMultiplies MulCollector(Terms, *this);
>> - visitAll(Expr, MulCollector);
>> -}
>> -
>> -static bool findArrayDimensionsRec(ScalarEvolution &SE,
>> - SmallVectorImpl<const SCEV *> &Terms,
>> - SmallVectorImpl<const SCEV *> &Sizes) {
>> - int Last = Terms.size() - 1;
>> - const SCEV *Step = Terms[Last];
>> -
>> - // End of recursion.
>> - if (Last == 0) {
>> - if (const SCEVMulExpr *M = dyn_cast<SCEVMulExpr>(Step)) {
>> - SmallVector<const SCEV *, 2> Qs;
>> - for (const SCEV *Op : M->operands())
>> - if (!isa<SCEVConstant>(Op))
>> - Qs.push_back(Op);
>> -
>> - Step = SE.getMulExpr(Qs);
>> - }
>> -
>> - Sizes.push_back(Step);
>> - return true;
>> - }
>> -
>> - for (const SCEV *&Term : Terms) {
>> - // Normalize the terms before the next call to findArrayDimensionsRec.
>> - const SCEV *Q, *R;
>> - SCEVDivision::divide(SE, Term, Step, &Q, &R);
>> -
>> - // Bail out when GCD does not evenly divide one of the terms.
>> - if (!R->isZero())
>> - return false;
>> -
>> - Term = Q;
>> - }
>> -
>> - // Remove all SCEVConstants.
>> - erase_if(Terms, [](const SCEV *E) { return isa<SCEVConstant>(E); });
>> -
>> - if (Terms.size() > 0)
>> - if (!findArrayDimensionsRec(SE, Terms, Sizes))
>> - return false;
>> -
>> - Sizes.push_back(Step);
>> - return true;
>> -}
>> -
>> -// Returns true when one of the SCEVs of Terms contains a SCEVUnknown parameter.
>> -static inline bool containsParameters(SmallVectorImpl<const SCEV *> &Terms) {
>> - for (const SCEV *T : Terms)
>> - if (SCEVExprContains(T, [](const SCEV *S) { return isa<SCEVUnknown>(S); }))
>> - return true;
>> -
>> - return false;
>> -}
>> -
>> -// Return the number of product terms in S.
>> -static inline int numberOfTerms(const SCEV *S) {
>> - if (const SCEVMulExpr *Expr = dyn_cast<SCEVMulExpr>(S))
>> - return Expr->getNumOperands();
>> - return 1;
>> -}
>> -
>> -static const SCEV *removeConstantFactors(ScalarEvolution &SE, const SCEV *T) {
>> - if (isa<SCEVConstant>(T))
>> - return nullptr;
>> -
>> - if (isa<SCEVUnknown>(T))
>> - return T;
>> -
>> - if (const SCEVMulExpr *M = dyn_cast<SCEVMulExpr>(T)) {
>> - SmallVector<const SCEV *, 2> Factors;
>> - for (const SCEV *Op : M->operands())
>> - if (!isa<SCEVConstant>(Op))
>> - Factors.push_back(Op);
>> -
>> - return SE.getMulExpr(Factors);
>> - }
>> -
>> - return T;
>> -}
>> -
>> /// Return the size of an element read or written by Inst.
>> const SCEV *ScalarEvolution::getElementSize(Instruction *Inst) {
>> Type *Ty;
>> @@ -12432,211 +12201,6 @@ const SCEV *ScalarEvolution::getElementSize(Instruction *Inst) {
>> return getSizeOfExpr(ETy, Ty);
>> }
>>
>> -void ScalarEvolution::findArrayDimensions(SmallVectorImpl<const SCEV *> &Terms,
>> - SmallVectorImpl<const SCEV *> &Sizes,
>> - const SCEV *ElementSize) {
>> - if (Terms.size() < 1 || !ElementSize)
>> - return;
>> -
>> - // Early return when Terms do not contain parameters: we do not delinearize
>> - // non parametric SCEVs.
>> - if (!containsParameters(Terms))
>> - return;
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "Terms:\n";
>> - for (const SCEV *T : Terms)
>> - dbgs() << *T << "\n";
>> - });
>> -
>> - // Remove duplicates.
>> - array_pod_sort(Terms.begin(), Terms.end());
>> - Terms.erase(std::unique(Terms.begin(), Terms.end()), Terms.end());
>> -
>> - // Put larger terms first.
>> - llvm::sort(Terms, [](const SCEV *LHS, const SCEV *RHS) {
>> - return numberOfTerms(LHS) > numberOfTerms(RHS);
>> - });
>> -
>> - // Try to divide all terms by the element size. If term is not divisible by
>> - // element size, proceed with the original term.
>> - for (const SCEV *&Term : Terms) {
>> - const SCEV *Q, *R;
>> - SCEVDivision::divide(*this, Term, ElementSize, &Q, &R);
>> - if (!Q->isZero())
>> - Term = Q;
>> - }
>> -
>> - SmallVector<const SCEV *, 4> NewTerms;
>> -
>> - // Remove constant factors.
>> - for (const SCEV *T : Terms)
>> - if (const SCEV *NewT = removeConstantFactors(*this, T))
>> - NewTerms.push_back(NewT);
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "Terms after sorting:\n";
>> - for (const SCEV *T : NewTerms)
>> - dbgs() << *T << "\n";
>> - });
>> -
>> - if (NewTerms.empty() || !findArrayDimensionsRec(*this, NewTerms, Sizes)) {
>> - Sizes.clear();
>> - return;
>> - }
>> -
>> - // The last element to be pushed into Sizes is the size of an element.
>> - Sizes.push_back(ElementSize);
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "Sizes:\n";
>> - for (const SCEV *S : Sizes)
>> - dbgs() << *S << "\n";
>> - });
>> -}
>> -
>> -void ScalarEvolution::computeAccessFunctions(
>> - const SCEV *Expr, SmallVectorImpl<const SCEV *> &Subscripts,
>> - SmallVectorImpl<const SCEV *> &Sizes) {
>> - // Early exit in case this SCEV is not an affine multivariate function.
>> - if (Sizes.empty())
>> - return;
>> -
>> - if (auto *AR = dyn_cast<SCEVAddRecExpr>(Expr))
>> - if (!AR->isAffine())
>> - return;
>> -
>> - const SCEV *Res = Expr;
>> - int Last = Sizes.size() - 1;
>> - for (int i = Last; i >= 0; i--) {
>> - const SCEV *Q, *R;
>> - SCEVDivision::divide(*this, Res, Sizes[i], &Q, &R);
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "Res: " << *Res << "\n";
>> - dbgs() << "Sizes[i]: " << *Sizes[i] << "\n";
>> - dbgs() << "Res divided by Sizes[i]:\n";
>> - dbgs() << "Quotient: " << *Q << "\n";
>> - dbgs() << "Remainder: " << *R << "\n";
>> - });
>> -
>> - Res = Q;
>> -
>> - // Do not record the last subscript corresponding to the size of elements in
>> - // the array.
>> - if (i == Last) {
>> -
>> - // Bail out if the remainder is too complex.
>> - if (isa<SCEVAddRecExpr>(R)) {
>> - Subscripts.clear();
>> - Sizes.clear();
>> - return;
>> - }
>> -
>> - continue;
>> - }
>> -
>> - // Record the access function for the current subscript.
>> - Subscripts.push_back(R);
>> - }
>> -
>> - // Also push in last position the remainder of the last division: it will be
>> - // the access function of the innermost dimension.
>> - Subscripts.push_back(Res);
>> -
>> - std::reverse(Subscripts.begin(), Subscripts.end());
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "Subscripts:\n";
>> - for (const SCEV *S : Subscripts)
>> - dbgs() << *S << "\n";
>> - });
>> -}
>> -
>> -/// Splits the SCEV into two vectors of SCEVs representing the subscripts and
>> -/// sizes of an array access. Returns the remainder of the delinearization that
>> -/// is the offset start of the array. The SCEV->delinearize algorithm computes
>> -/// the multiples of SCEV coefficients: that is a pattern matching of sub
>> -/// expressions in the stride and base of a SCEV corresponding to the
>> -/// computation of a GCD (greatest common divisor) of base and stride. When
>> -/// SCEV->delinearize fails, it returns the SCEV unchanged.
>> -///
>> -/// For example: when analyzing the memory access A[i][j][k] in this loop nest
>> -///
>> -/// void foo(long n, long m, long o, double A[n][m][o]) {
>> -///
>> -/// for (long i = 0; i < n; i++)
>> -/// for (long j = 0; j < m; j++)
>> -/// for (long k = 0; k < o; k++)
>> -/// A[i][j][k] = 1.0;
>> -/// }
>> -///
>> -/// the delinearization input is the following AddRec SCEV:
>> -///
>> -/// AddRec: {{{%A,+,(8 * %m * %o)}<%for.i>,+,(8 * %o)}<%for.j>,+,8}<%for.k>
>> -///
>> -/// From this SCEV, we are able to say that the base offset of the access is %A
>> -/// because it appears as an offset that does not divide any of the strides in
>> -/// the loops:
>> -///
>> -/// CHECK: Base offset: %A
>> -///
>> -/// and then SCEV->delinearize determines the size of some of the dimensions of
>> -/// the array as these are the multiples by which the strides are happening:
>> -///
>> -/// CHECK: ArrayDecl[UnknownSize][%m][%o] with elements of sizeof(double) bytes.
>> -///
>> -/// Note that the outermost dimension remains of UnknownSize because there are
>> -/// no strides that would help identifying the size of the last dimension: when
>> -/// the array has been statically allocated, one could compute the size of that
>> -/// dimension by dividing the overall size of the array by the size of the known
>> -/// dimensions: %m * %o * 8.
>> -///
>> -/// Finally delinearize provides the access functions for the array reference
>> -/// that does correspond to A[i][j][k] of the above C testcase:
>> -///
>> -/// CHECK: ArrayRef[{0,+,1}<%for.i>][{0,+,1}<%for.j>][{0,+,1}<%for.k>]
>> -///
>> -/// The testcases are checking the output of a function pass:
>> -/// DelinearizationPass that walks through all loads and stores of a function
>> -/// asking for the SCEV of the memory access with respect to all enclosing
>> -/// loops, calling SCEV->delinearize on that and printing the results.
>> -void ScalarEvolution::delinearize(const SCEV *Expr,
>> - SmallVectorImpl<const SCEV *> &Subscripts,
>> - SmallVectorImpl<const SCEV *> &Sizes,
>> - const SCEV *ElementSize) {
>> - // First step: collect parametric terms.
>> - SmallVector<const SCEV *, 4> Terms;
>> - collectParametricTerms(Expr, Terms);
>> -
>> - if (Terms.empty())
>> - return;
>> -
>> - // Second step: find subscript sizes.
>> - findArrayDimensions(Terms, Sizes, ElementSize);
>> -
>> - if (Sizes.empty())
>> - return;
>> -
>> - // Third step: compute the access functions for each subscript.
>> - computeAccessFunctions(Expr, Subscripts, Sizes);
>> -
>> - if (Subscripts.empty())
>> - return;
>> -
>> - LLVM_DEBUG({
>> - dbgs() << "succeeded to delinearize " << *Expr << "\n";
>> - dbgs() << "ArrayDecl[UnknownSize]";
>> - for (const SCEV *S : Sizes)
>> - dbgs() << "[" << *S << "]";
>> -
>> - dbgs() << "\nArrayRef";
>> - for (const SCEV *S : Subscripts)
>> - dbgs() << "[" << *S << "]";
>> - dbgs() << "\n";
>> - });
>> -}
>> -
>> bool ScalarEvolution::getIndexExpressionsFromGEP(
>> const GetElementPtrInst *GEP, SmallVectorImpl<const SCEV *> &Subscripts,
>> SmallVectorImpl<int> &Sizes) {
>>
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list