[polly] r280948 - Add -polly-flatten-schedule pass.

Michael Kruse via llvm-commits llvm-commits at lists.llvm.org
Fri Sep 9 07:51:13 PDT 2016


Hi Tobias,

thanks for the notification. However, I cannot reproduce this with
r281039 on either Windows or Linux:

$ ninja check-polly-unittests
[217/217] Running polly unit tests only
Testing Time: 0.02s
  Expected Passes    : 5

$ tools/polly/unittests/Flatten/FlattenTests
[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from Flatten
[ RUN      ] Flatten.FlattenTrivial
[       OK ] Flatten.FlattenTrivial (1 ms)
[ RUN      ] Flatten.FlattenSequence
[       OK ] Flatten.FlattenSequence (2 ms)
[ RUN      ] Flatten.FlattenLoop
[       OK ] Flatten.FlattenLoop (1 ms)
[----------] 3 tests from Flatten (6 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (10 ms total)
[  PASSED  ] 3 tests.

Can you re-try and maybe find out which call to ISL causes this?

Michael



2016-09-09 14:52 GMT+02:00 Tobias Grosser <tobias at grosser.es>:
> Hi Michael,
>
> I see the following error in my unittests:
>
> Value of: checkFlatten( "[n] -> { A[i] -> [0, i] : 0 <= i < n; B[i] ->
> [1, i] : 0 <= i < n }", "[n] -> { A[i] -> [i] : 0 <= i < n; B[i] -> [n +
> i] : 0 <= i < n }")
>   Actual: false
> Expected: true
> [  FAILED  ] Flatten.FlattenSequence (5 ms)
> [----------] 1 test from Flatten (5 ms total)
>
> [----------] Global test environment tear-down
> [==========] 1 test from 1 test case ran. (5 ms total)
> [  PASSED  ] 0 tests.
> [  FAILED  ] 1 test, listed below:
> [  FAILED  ] Flatten.FlattenSequence
>
>  1 FAILED TEST
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
> /home/grosser/Projects/polly/git/tools/polly/lib/External/isl/isl_space.c:1775:
> not a parameter space
>
> Best,
> Tobias
>
> On Thu, Sep 8, 2016, at 05:02 PM, Michael Kruse via llvm-commits wrote:
>> Author: meinersbur
>> Date: Thu Sep  8 10:02:36 2016
>> New Revision: 280948
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=280948&view=rev
>> Log:
>> Add -polly-flatten-schedule pass.
>>
>> The -polly-flatten-schedule pass reduces the number of scattering
>> dimensions in its isl_union_map form to make them easier to understand.
>> It is not meant to be used in production, only for debugging and
>> regression tests.
>>
>> To illustrate, how it can make sets simpler, here is a lifetime set
>> used computed by the porposed DeLICM pass without flattening:
>>
>>     { Stmt_reduction_for[0, 4] -> [0, 2, o2, o3] : o2 < 0;
>>       Stmt_reduction_for[0, 4] -> [0, 1, o2, o3] : o2 >= 5;
>>       Stmt_reduction_for[0, 4] -> [0, 1, 4, o3] : o3 > 0;
>>       Stmt_reduction_for[0, i1] -> [0, 1, i1, 1] : 0 <= i1 <= 3;
>>       Stmt_reduction_for[0, 4] -> [0, 2, 0, o3] : o3 <= 0 }
>>
>> And here the same lifetime for a semantically identical one-dimensional
>> schedule:
>>
>>     { Stmt_reduction_for[0, i1] -> [2 + 3i1] : 0 <= i1 <= 4 }
>>
>> Differential Revision: https://reviews.llvm.org/D24310
>>
>> Added:
>>     polly/trunk/include/polly/FlattenAlgo.h
>>     polly/trunk/include/polly/FlattenSchedule.h
>>     polly/trunk/lib/Transform/FlattenAlgo.cpp
>>     polly/trunk/lib/Transform/FlattenSchedule.cpp
>>     polly/trunk/test/FlattenSchedule/
>>     polly/trunk/test/FlattenSchedule/gemm.ll
>>     polly/trunk/unittests/Flatten/
>>     polly/trunk/unittests/Flatten/CMakeLists.txt
>>     polly/trunk/unittests/Flatten/FlattenTest.cpp
>> Modified:
>>     polly/trunk/include/polly/LinkAllPasses.h
>>     polly/trunk/include/polly/Support/GICHelper.h
>>     polly/trunk/lib/CMakeLists.txt
>>     polly/trunk/lib/Support/GICHelper.cpp
>>     polly/trunk/lib/Support/RegisterPasses.cpp
>>     polly/trunk/test/update_check.py
>>     polly/trunk/unittests/CMakeLists.txt
>>
>> Added: polly/trunk/include/polly/FlattenAlgo.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/include/polly/FlattenAlgo.h?rev=280948&view=auto
>> ==============================================================================
>> --- polly/trunk/include/polly/FlattenAlgo.h (added)
>> +++ polly/trunk/include/polly/FlattenAlgo.h Thu Sep  8 10:02:36 2016
>> @@ -0,0 +1,38 @@
>> +//===------ FlattenAlgo.h --------------------------------------*- C++
>> -*-===//
>> +//
>> +//                     The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// Main algorithm of the FlattenSchedulePass. This is a separate file to
>> avoid
>> +// the unittest for this requiring linking against LLVM.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef POLLY_FLATTENALGO_H
>> +#define POLLY_FLATTENALGO_H
>> +
>> +#include "polly/Support/GICHelper.h"
>> +
>> +namespace polly {
>> +/// Recursively flatten a schedule.
>> +///
>> +/// Reduce the number of scatter dimensions as much as possible without
>> changing
>> +/// the relative order of instances in a schedule. Ideally, this results
>> in a
>> +/// single scatter dimension, but it may not always be possible to
>> combine
>> +/// dimensions, eg. if a dimension is unbounded. In worst case, the
>> original
>> +/// schedule is returned.
>> +///
>> +/// Schedules with fewer dimensions may be easier to understand for
>> humans, but
>> +/// it should make no difference to the computer.
>> +///
>> +/// @param Schedule The input schedule.
>> +///
>> +/// @return The flattened schedule.
>> +IslPtr<isl_union_map> flattenSchedule(IslPtr<isl_union_map> Schedule);
>> +} // namespace polly
>> +
>> +#endif /* POLLY_FLATTENALGO_H */
>>
>> Added: polly/trunk/include/polly/FlattenSchedule.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/include/polly/FlattenSchedule.h?rev=280948&view=auto
>> ==============================================================================
>> --- polly/trunk/include/polly/FlattenSchedule.h (added)
>> +++ polly/trunk/include/polly/FlattenSchedule.h Thu Sep  8 10:02:36 2016
>> @@ -0,0 +1,32 @@
>> +//===------ FlattenSchedule.h ----------------------------------*- C++
>> -*-===//
>> +//
>> +//                     The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// Try to reduce the number of scatter dimension. Useful to make
>> isl_union_map
>> +// schedules more understandable. This is only intended for debugging
>> and
>> +// unittests, not for optimizations themselves.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef POLLY_FLATTENSCHEDULE_H
>> +#define POLLY_FLATTENSCHEDULE_H
>> +
>> +namespace llvm {
>> +class PassRegistry;
>> +class Pass;
>> +} // anonymous namespace
>> +
>> +namespace polly {
>> +llvm::Pass *createFlattenSchedulePass();
>> +} // namespace polly
>> +
>> +namespace llvm {
>> +void initializeFlattenSchedulePass(llvm::PassRegistry &);
>> +} // namespace llvm
>> +
>> +#endif /* POLLY_FLATTENSCHEDULE_H */
>>
>> Modified: polly/trunk/include/polly/LinkAllPasses.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/include/polly/LinkAllPasses.h?rev=280948&r1=280947&r2=280948&view=diff
>> ==============================================================================
>> --- polly/trunk/include/polly/LinkAllPasses.h (original)
>> +++ polly/trunk/include/polly/LinkAllPasses.h Thu Sep  8 10:02:36 2016
>> @@ -47,6 +47,7 @@ llvm::Pass *createCodeGenerationPass();
>>  llvm::Pass *createPPCGCodeGenerationPass();
>>  #endif
>>  llvm::Pass *createIslScheduleOptimizerPass();
>> +llvm::Pass *createFlattenSchedulePass();
>>
>>  extern char &CodePreparationID;
>>  } // namespace polly
>> @@ -80,6 +81,7 @@ struct PollyForcePassLinking {
>>      polly::createPPCGCodeGenerationPass();
>>  #endif
>>      polly::createIslScheduleOptimizerPass();
>> +    polly::createFlattenSchedulePass();
>>    }
>>  } PollyForcePassLinking; // Force link by creating a global definition.
>>  } // namespace
>> @@ -97,6 +99,7 @@ void initializePPCGCodeGenerationPass(ll
>>  #endif
>>  void initializeIslScheduleOptimizerPass(llvm::PassRegistry &);
>>  void initializePollyCanonicalizePass(llvm::PassRegistry &);
>> +void initializeFlattenSchedulePass(llvm::PassRegistry &);
>>  } // namespace llvm
>>
>>  #endif
>>
>> Modified: polly/trunk/include/polly/Support/GICHelper.h
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/include/polly/Support/GICHelper.h?rev=280948&r1=280947&r2=280948&view=diff
>> ==============================================================================
>> --- polly/trunk/include/polly/Support/GICHelper.h (original)
>> +++ polly/trunk/include/polly/Support/GICHelper.h Thu Sep  8 10:02:36
>> 2016
>> @@ -16,21 +16,17 @@
>>
>>  #include "llvm/ADT/APInt.h"
>>  #include "llvm/Support/raw_ostream.h"
>> +#include "isl/aff.h"
>>  #include "isl/ctx.h"
>> +#include "isl/map.h"
>> +#include "isl/set.h"
>> +#include "isl/union_map.h"
>> +#include "isl/union_set.h"
>> +#include <functional>
>>  #include <string>
>>
>> -struct isl_map;
>> -struct isl_union_map;
>> -struct isl_set;
>> -struct isl_union_set;
>>  struct isl_schedule;
>>  struct isl_multi_aff;
>> -struct isl_pw_multi_aff;
>> -struct isl_union_pw_multi_aff;
>> -struct isl_aff;
>> -struct isl_pw_aff;
>> -struct isl_val;
>> -struct isl_space;
>>
>>  namespace llvm {
>>  class Value;
>> @@ -177,6 +173,239 @@ std::string getIslCompatibleName(const s
>>                                   const std::string &Middle,
>>                                   const std::string &Suffix);
>>
>> +/// IslObjTraits<isl_*> is a static class to invoke common functions
>> that all
>> +/// ISL objects have: isl_*_copy, isl_*_free, isl_*_get_ctx and
>> isl_*_to_str.
>> +/// These functions follow a common naming scheme, but not a base class
>> +/// hierarchy (as ISL is written in C). As such, the functions are
>> accessible
>> +/// only by constructing the function name using the preprocessor. This
>> class
>> +/// serves to make these names accessible to a C++ template scheme.
>> +///
>> +/// There is an isl_obj polymorphism layer, but its implementation is
>> +/// incomplete.
>> +template <typename T> class IslObjTraits;
>> +
>> +#define DECLARE_TRAITS(TYPE)
>>       \
>> +  template <> class IslObjTraits<isl_##TYPE> {
>>       \
>> +  public:
>>       \
>> +    static __isl_give isl_##TYPE *copy(__isl_keep isl_##TYPE *Obj) {
>>       \
>> +      return isl_##TYPE##_copy(Obj);
>>       \
>> +    }
>>       \
>> +    static void free(__isl_take isl_##TYPE *Obj) {
>> isl_##TYPE##_free(Obj); }   \
>> +    static isl_ctx *get_ctx(__isl_keep isl_##TYPE *Obj) {
>>       \
>> +      return isl_##TYPE##_get_ctx(Obj);
>>       \
>> +    }
>>       \
>> +    static std::string to_str(__isl_keep isl_##TYPE *Obj) {
>>       \
>> +      if (!Obj)
>>       \
>> +        return "null";
>>       \
>> +      char *cstr = isl_##TYPE##_to_str(Obj);
>>       \
>> +      if (!cstr)
>>       \
>> +        return "null";
>>       \
>> +      std::string Result{cstr};
>>       \
>> +      ::free(cstr);
>>       \
>> +      return Result;
>>       \
>> +    }
>>       \
>> +  };
>> +
>> +DECLARE_TRAITS(val)
>> +DECLARE_TRAITS(space)
>> +DECLARE_TRAITS(basic_map)
>> +DECLARE_TRAITS(map)
>> +DECLARE_TRAITS(union_map)
>> +DECLARE_TRAITS(basic_set)
>> +DECLARE_TRAITS(set)
>> +DECLARE_TRAITS(union_set)
>> +DECLARE_TRAITS(aff)
>> +DECLARE_TRAITS(pw_aff)
>> +DECLARE_TRAITS(union_pw_aff)
>> +DECLARE_TRAITS(multi_union_pw_aff)
>> +DECLARE_TRAITS(union_pw_multi_aff)
>> +
>> +template <typename T> class NonowningIslPtr;
>> +
>> +/// Smart pointer to an ISL object.
>> +///
>> +/// An object of this class owns an reference of an ISL object, meaning
>> if will
>> +/// free it when destroyed. Most ISL objects are reference counted such
>> that we
>> +/// gain an automatic memory management.
>> +///
>> +/// Function parameters in the ISL API are annotated using either
>> __isl_keep
>> +/// __isl_take. Return values that are objects are annotated using
>> __is_give,
>> +/// meaning the caller is responsible for releasing the object. When
>> annotated
>> +/// with __isl_keep, use the keep() function to pass a plain pointer to
>> the ISL
>> +/// object. For __isl_take-annotated parameters, use either copy() to
>> increase
>> +/// the reference counter by one, or take() to pass the ownership to the
>> called
>> +/// function. When IslPtr loses ownership, it cannot be used anymore and
>> won't
>> +/// free the object when destroyed. Use the give() function to wrap the
>> +/// ownership of a returned isl_* object into an IstPtr<isl_*>.
>> +///
>> +/// There is purposefully no implicit conversion from/to plain isl_*
>> pointers to
>> +/// avoid difficult to find bugs because keep/copy/take would have been
>> +/// required.
>> +template <typename T> class IslPtr {
>> +  typedef IslPtr<T> ThisTy;
>> +  typedef IslObjTraits<T> Traits;
>> +
>> +private:
>> +  T *Obj;
>> +
>> +  explicit IslPtr(__isl_take T *Obj, bool TakeOwnership)
>> +      : Obj(TakeOwnership ? Obj : Traits::copy(Obj)) {}
>> +
>> +public:
>> +  IslPtr() : Obj(nullptr) {}
>> +  /* implicit */ IslPtr(nullptr_t That) : IslPtr() {}
>> +
>> +  /* implicit */ IslPtr(const ThisTy &That) : IslPtr(That.Obj, false) {}
>> +  /* implicit */ IslPtr(ThisTy &&That) : IslPtr(That.Obj, true) {
>> +    That.Obj = nullptr;
>> +  }
>> +  /* implicit */ IslPtr(NonowningIslPtr<T> That) : IslPtr(That.copy(),
>> true) {}
>> +  ~IslPtr() { Traits::free(Obj); }
>> +
>> +  ThisTy &operator=(const ThisTy &That) {
>> +    Traits::free(this->Obj);
>> +    this->Obj = Traits::copy(That.Obj);
>> +    return *this;
>> +  }
>> +  ThisTy &operator=(ThisTy &&That) {
>> +    swap(*this, That);
>> +    return *this;
>> +  }
>> +
>> +  explicit operator bool() const { return Obj; }
>> +
>> +  static void swap(ThisTy &LHS, ThisTy &RHS) { std::swap(LHS.Obj,
>> RHS.Obj); }
>> +
>> +  static ThisTy give(T *Obj) { return ThisTy(Obj, true); }
>> +  T *keep() const { return Obj; }
>> +  T *take() {
>> +    auto *Result = Obj;
>> +    Obj = nullptr;
>> +    return Result;
>> +  }
>> +  T *copy() const { return Traits::copy(Obj); }
>> +
>> +  isl_ctx *getCtx() const { return Traits::get_ctx(Obj); }
>> +  std::string toStr() const { return Traits::to_str(Obj); }
>> +};
>> +
>> +template <typename T> static IslPtr<T> give(__isl_take T *Obj) {
>> +  return IslPtr<T>::give(Obj);
>> +}
>> +
>> +template <typename T>
>> +llvm::raw_ostream &operator<<(llvm::raw_ostream &OS, const IslPtr<T>
>> &Obj) {
>> +  OS << IslObjTraits<T>::to_str(Obj.keep());
>> +  return OS;
>> +}
>> +
>> +/// Smart pointer to an ISL object, but does not release it when
>> destroyed.
>> +///
>> +/// This is meant to be used as function parameter type. The caller
>> guarantees
>> +/// that the reference is alive during the function's execution and
>> hence
>> +/// doesn't need to add a reference. Therefore, it is equivalent to the
>> +/// __isl_keep annotation (IslPtr being equivalent to __isl_take which
>> can be
>> +/// either copied or moved).
>> +///
>> +/// Just as IslPtr, it has keep() and copy() methods. The take() method
>> is
>> +/// missing as this would steal the reference from the owner (the
>> caller).
>> +template <typename T> class NonowningIslPtr {
>> +  typedef NonowningIslPtr<T> ThisTy;
>> +  typedef IslObjTraits<T> Traits;
>> +
>> +private:
>> +  T *Obj;
>> +
>> +  /* implicit */ NonowningIslPtr(__isl_keep T *Obj) : Obj(Obj) {}
>> +
>> +public:
>> +  NonowningIslPtr() : Obj(nullptr) {}
>> +  /* implicit */ NonowningIslPtr(nullptr_t That) : NonowningIslPtr() {}
>> +
>> +  /* implicit */ NonowningIslPtr(const IslPtr<T> &That)
>> +      : NonowningIslPtr(That.keep()) {}
>> +
>> +  explicit operator bool() const { return Obj; }
>> +
>> +  static void swap(ThisTy &LHS, ThisTy &RHS) { std::swap(LHS.Obj,
>> RHS.Obj); }
>> +
>> +  T *keep() const { return Obj; }
>> +  T *copy() const { return Traits::copy(Obj); }
>> +
>> +  isl_ctx *getCtx() const { return Traits::get_ctx(Obj); }
>> +  std::string toStr() const { return Traits::to_str(Obj); }
>> +};
>> +
>> +template <typename T>
>> +static llvm::raw_ostream &operator<<(llvm::raw_ostream &OS,
>> +                                     NonowningIslPtr<T> Obj) {
>> +  OS << IslObjTraits<T>::to_str(Obj.keep());
>> +  return OS;
>> +}
>> +
>> +/// Enumerate all isl_maps of an isl_union_map.
>> +///
>> +/// This basically wraps isl_union_map_foreach_map() and allows to call
>> back
>> +/// C++11 closures.
>> +void foreachElt(NonowningIslPtr<isl_union_map> UMap,
>> +                const std::function<void(IslPtr<isl_map> Map)> &F);
>> +
>> +/// Enumerate all isl_pw_aff of an isl_union_pw_aff.
>> +///
>> +/// This basically wraps isl_union_pw_aff(), but also allows to call
>> back C++11
>> +/// closures.
>> +void foreachElt(NonowningIslPtr<isl_union_pw_aff> UPwAff,
>> +                const std::function<void(IslPtr<isl_pw_aff>)> &F);
>> +
>> +/// Enumerate all polyhedra of an isl_map.
>> +///
>> +/// This is a wrapper for isl_map_foreach_basic_map() that allows to
>> call back
>> +/// C++ closures. The callback has the possibility to interrupt (break)
>> the
>> +/// enumeration by returning isl_stat_error. A return value of
>> isl_stat_ok will
>> +/// continue enumerations, if any more elements are left.
>> +///
>> +/// @param UMap Collection to enumerate.
>> +/// @param F    The callback function, lambda or closure.
>> +///
>> +/// @return The isl_stat returned by the last callback invocation;
>> isl_stat_ok
>> +///         if the collection was empty.
>> +isl_stat
>> +foreachEltWithBreak(NonowningIslPtr<isl_map> Map,
>> +                    const std::function<isl_stat(IslPtr<isl_basic_map>)>
>> &F);
>> +
>> +/// Enumerate all isl_maps of an isl_union_map.
>> +///
>> +/// This is a wrapper for isl_union_map_foreach_map() that allows to
>> call back
>> +/// C++ closures. In contrast to the variant without "_with_break", the
>> callback
>> +/// has the possibility to interrupt (break) the enumeration by
>> returning
>> +/// isl_stat_error. A return value of isl_stat_ok will continue
>> enumerations, if
>> +/// any more elements are left.
>> +///
>> +/// @param UMap Collection to enumerate.
>> +/// @param F    The callback function, lambda or closure.
>> +///
>> +/// @return The isl_stat returned by the last callback invocation;
>> isl_stat_ok
>> +///         if the collection was initially empty.
>> +isl_stat
>> +foreachEltWithBreak(NonowningIslPtr<isl_union_map> UMap,
>> +                    const std::function<isl_stat(IslPtr<isl_map> Map)>
>> &F);
>> +
>> +/// Enumerate all pieces of an isl_pw_aff.
>> +///
>> +/// This is a wrapper around isl_pw_aff_foreach_piece() that allows to
>> call back
>> +/// C++11 closures. The callback has the possibility to interrupt
>> (break) the
>> +/// enumeration by returning isl_stat_error. A return value of
>> isl_stat_ok will
>> +/// continue enumerations, if any more elements are left.
>> +///
>> +/// @param UMap Collection to enumerate.
>> +/// @param F    The callback function, lambda or closure.
>> +///
>> +/// @return The isl_stat returned by the last callback invocation;
>> isl_stat_ok
>> +///         if the collection was initially empty.
>> +isl_stat foreachPieceWithBreak(
>> +    NonowningIslPtr<isl_pw_aff> PwAff,
>> +    const std::function<isl_stat(IslPtr<isl_set>, IslPtr<isl_aff>)> &F);
>> +
>>  } // end namespace polly
>>
>>  #endif
>>
>> Modified: polly/trunk/lib/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/lib/CMakeLists.txt?rev=280948&r1=280947&r2=280948&view=diff
>> ==============================================================================
>> --- polly/trunk/lib/CMakeLists.txt (original)
>> +++ polly/trunk/lib/CMakeLists.txt Thu Sep  8 10:02:36 2016
>> @@ -55,6 +55,8 @@ add_polly_library(Polly
>>    Transform/CodePreparation.cpp
>>    Transform/DeadCodeElimination.cpp
>>    Transform/ScheduleOptimizer.cpp
>> +  Transform/FlattenSchedule.cpp
>> +  Transform/FlattenAlgo.cpp
>>    ${POLLY_HEADER_FILES}
>>    )
>>
>>
>> Modified: polly/trunk/lib/Support/GICHelper.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/lib/Support/GICHelper.cpp?rev=280948&r1=280947&r2=280948&view=diff
>> ==============================================================================
>> --- polly/trunk/lib/Support/GICHelper.cpp (original)
>> +++ polly/trunk/lib/Support/GICHelper.cpp Thu Sep  8 10:02:36 2016
>> @@ -198,3 +198,71 @@ std::string polly::getIslCompatibleName(
>>    ValStr.erase(0, 1);
>>    return getIslCompatibleName(Prefix, ValStr, Suffix);
>>  }
>> +
>> +void polly::foreachElt(NonowningIslPtr<isl_union_map> UMap,
>> +                       const std::function<void(IslPtr<isl_map> Map)>
>> &F) {
>> +  isl_union_map_foreach_map(
>> +      UMap.keep(),
>> +      [](__isl_take isl_map *Map, void *User) -> isl_stat {
>> +        auto &F =
>> +            *static_cast<const std::function<void(IslPtr<isl_map>)>
>> *>(User);
>> +        F(give(Map));
>> +        return isl_stat_ok;
>> +      },
>> +      const_cast<void *>(static_cast<const void *>(&F)));
>> +}
>> +
>> +void polly::foreachElt(NonowningIslPtr<isl_union_pw_aff> UPwAff,
>> +                       const std::function<void(IslPtr<isl_pw_aff>)> &F)
>> {
>> +  isl_union_pw_aff_foreach_pw_aff(
>> +      UPwAff.keep(),
>> +      [](__isl_take isl_pw_aff *PwAff, void *User) -> isl_stat {
>> +        auto &F =
>> +            *static_cast<const std::function<void(IslPtr<isl_pw_aff>)>
>> *>(User);
>> +        F(give(PwAff));
>> +        return isl_stat_ok;
>> +      },
>> +      const_cast<void *>(static_cast<const void *>(&F)));
>> +}
>> +
>> +isl_stat polly::foreachEltWithBreak(
>> +    NonowningIslPtr<isl_map> Map,
>> +    const std::function<isl_stat(IslPtr<isl_basic_map>)> &F) {
>> +  return isl_map_foreach_basic_map(
>> +      Map.keep(),
>> +      [](__isl_take isl_basic_map *BMap, void *User) -> isl_stat {
>> +        auto &F = *static_cast<
>> +            const std::function<isl_stat(IslPtr<isl_basic_map>)>
>> *>(User);
>> +        return F(give(BMap));
>> +      },
>> +      const_cast<void *>(static_cast<const void *>(&F)));
>> +}
>> +
>> +isl_stat polly::foreachEltWithBreak(
>> +    NonowningIslPtr<isl_union_map> UMap,
>> +    const std::function<isl_stat(IslPtr<isl_map> Map)> &F) {
>> +  return isl_union_map_foreach_map(
>> +      UMap.keep(),
>> +      [](__isl_take isl_map *Map, void *User) -> isl_stat {
>> +        auto &F =
>> +            *static_cast<const std::function<isl_stat(IslPtr<isl_map>
>> Map)> *>(
>> +                User);
>> +        return F(give(Map));
>> +      },
>> +      const_cast<void *>(static_cast<const void *>(&F)));
>> +}
>> +
>> +isl_stat polly::foreachPieceWithBreak(
>> +    NonowningIslPtr<isl_pw_aff> PwAff,
>> +    const std::function<isl_stat(IslPtr<isl_set>, IslPtr<isl_aff>)> &F)
>> {
>> +  return isl_pw_aff_foreach_piece(
>> +      PwAff.keep(),
>> +      [](__isl_take isl_set *Domain, __isl_take isl_aff *Aff,
>> +         void *User) -> isl_stat {
>> +        auto &F = *static_cast<
>> +            const std::function<isl_stat(IslPtr<isl_set>,
>> IslPtr<isl_aff>)> *>(
>> +            User);
>> +        return F(give(Domain), give(Aff));
>> +      },
>> +      const_cast<void *>(static_cast<const void *>(&F)));
>> +}
>>
>> Modified: polly/trunk/lib/Support/RegisterPasses.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/lib/Support/RegisterPasses.cpp?rev=280948&r1=280947&r2=280948&view=diff
>> ==============================================================================
>> --- polly/trunk/lib/Support/RegisterPasses.cpp (original)
>> +++ polly/trunk/lib/Support/RegisterPasses.cpp Thu Sep  8 10:02:36 2016
>> @@ -24,6 +24,7 @@
>>  #include "polly/CodeGen/CodeGeneration.h"
>>  #include "polly/CodeGen/CodegenCleanup.h"
>>  #include "polly/DependenceInfo.h"
>> +#include "polly/FlattenSchedule.h"
>>  #include "polly/LinkAllPasses.h"
>>  #include "polly/Options.h"
>>  #include "polly/PolyhedralInfo.h"
>> @@ -180,6 +181,7 @@ void initializePollyPasses(PassRegistry
>>    initializeScopInfoRegionPassPass(Registry);
>>    initializeScopInfoWrapperPassPass(Registry);
>>    initializeCodegenCleanupPass(Registry);
>> +  initializeFlattenSchedulePass(Registry);
>>  }
>>
>>  /// Register Polly passes such that they form a polyhedral optimizer.
>>
>> Added: polly/trunk/lib/Transform/FlattenAlgo.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/lib/Transform/FlattenAlgo.cpp?rev=280948&view=auto
>> ==============================================================================
>> --- polly/trunk/lib/Transform/FlattenAlgo.cpp (added)
>> +++ polly/trunk/lib/Transform/FlattenAlgo.cpp Thu Sep  8 10:02:36 2016
>> @@ -0,0 +1,417 @@
>> +//===------ FlattenAlgo.cpp ------------------------------------*- C++
>> -*-===//
>> +//
>> +//                     The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// Main algorithm of the FlattenSchedulePass. This is a separate file to
>> avoid
>> +// the unittest for this requiring linking against LLVM.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "polly/FlattenAlgo.h"
>> +#include "llvm/Support/Debug.h"
>> +#define DEBUG_TYPE "polly-flatten-algo"
>> +
>> +using namespace polly;
>> +using namespace llvm;
>> +
>> +namespace {
>> +
>> +/// Whether a dimension of a set is bounded (lower and upper) by a
>> constant,
>> +/// i.e. there are two constants Min and Max, such that every value x of
>> the
>> +/// chosen dimensions is Min <= x <= Max.
>> +bool isDimBoundedByConstant(IslPtr<isl_set> Set, unsigned dim) {
>> +  auto ParamDims = isl_set_dim(Set.keep(), isl_dim_param);
>> +  Set = give(isl_set_project_out(Set.take(), isl_dim_param, 0,
>> ParamDims));
>> +  Set = give(isl_set_project_out(Set.take(), isl_dim_set, 0, dim));
>> +  auto SetDims = isl_set_dim(Set.keep(), isl_dim_set);
>> +  Set = give(isl_set_project_out(Set.take(), isl_dim_set, 1, SetDims -
>> 1));
>> +  return isl_set_is_bounded(Set.keep());
>> +}
>> +
>> +/// Whether a dimension of a set is (lower and upper) bounded by a
>> constant or
>> +/// parameters, i.e. there are two expressions Min_p and Max_p of the
>> parameters
>> +/// p, such that every value x of the chosen dimensions is
>> +/// Min_p <= x <= Max_p.
>> +bool isDimBoundedByParameter(IslPtr<isl_set> Set, unsigned dim) {
>> +  Set = give(isl_set_project_out(Set.take(), isl_dim_set, 0, dim));
>> +  auto SetDims = isl_set_dim(Set.keep(), isl_dim_set);
>> +  Set = give(isl_set_project_out(Set.take(), isl_dim_set, 1, SetDims -
>> 1));
>> +  return isl_set_is_bounded(Set.keep());
>> +}
>> +
>> +/// Whether BMap's first out-dimension is not a constant.
>> +bool isVariableDim(NonowningIslPtr<isl_basic_map> BMap) {
>> +  auto FixedVal =
>> +      give(isl_basic_map_plain_get_val_if_fixed(BMap.keep(),
>> isl_dim_out, 0));
>> +  return !FixedVal || isl_val_is_nan(FixedVal.keep());
>> +}
>> +
>> +/// Whether Map's first out dimension is no constant nor piecewise
>> constant.
>> +bool isVariableDim(NonowningIslPtr<isl_map> Map) {
>> +  return foreachEltWithBreak(Map, [](IslPtr<isl_basic_map> BMap) ->
>> isl_stat {
>> +    if (isVariableDim(BMap))
>> +      return isl_stat_error;
>> +    return isl_stat_ok;
>> +  });
>> +}
>> +
>> +/// Whether UMap's first out dimension is no (piecewise) constant.
>> +bool isVariableDim(NonowningIslPtr<isl_union_map> UMap) {
>> +  return foreachEltWithBreak(UMap, [](IslPtr<isl_map> Map) -> isl_stat {
>> +    if (isVariableDim(Map))
>> +      return isl_stat_error;
>> +    return isl_stat_ok;
>> +  });
>> +}
>> +
>> +/// If @p PwAff maps to a constant, return said constant. If @p Max/@p
>> Min, it
>> +/// can also be a piecewise constant and it would return the
>> minimum/maximum
>> +/// value. Otherwise, return NaN.
>> +IslPtr<isl_val> getConstant(IslPtr<isl_pw_aff> PwAff, bool Max, bool
>> Min) {
>> +  assert(!Max || !Min);
>> +  IslPtr<isl_val> Result;
>> +  foreachPieceWithBreak(
>> +      PwAff, [=, &Result](IslPtr<isl_set> Set, IslPtr<isl_aff> Aff) {
>> +        if (Result && isl_val_is_nan(Result.keep()))
>> +          return isl_stat_ok;
>> +
>> +        // TODO: If Min/Max, we can also determine a minimum/maximum
>> value if
>> +        // Set is constant-bounded.
>> +        if (!isl_aff_is_cst(Aff.keep())) {
>> +          Result = give(isl_val_nan(Aff.getCtx()));
>> +          return isl_stat_error;
>> +        }
>> +
>> +        auto ThisVal = give(isl_aff_get_constant_val(Aff.keep()));
>> +        if (!Result) {
>> +          Result = ThisVal;
>> +          return isl_stat_ok;
>> +        }
>> +
>> +        if (isl_val_eq(Result.keep(), ThisVal.keep()))
>> +          return isl_stat_ok;
>> +
>> +        if (Max && isl_val_gt(ThisVal.keep(), Result.keep())) {
>> +          Result = ThisVal;
>> +          return isl_stat_ok;
>> +        }
>> +
>> +        if (Min && isl_val_lt(ThisVal.keep(), Result.keep())) {
>> +          Result = ThisVal;
>> +          return isl_stat_ok;
>> +        }
>> +
>> +        // Not compatible
>> +        Result = give(isl_val_nan(Aff.getCtx()));
>> +        return isl_stat_error;
>> +      });
>> +  return Result;
>> +}
>> +
>> +/// Compute @p UPwAff - @p Val.
>> +IslPtr<isl_union_pw_aff> subtract(IslPtr<isl_union_pw_aff> UPwAff,
>> +                                  IslPtr<isl_val> Val) {
>> +  if (isl_val_is_zero(Val.keep()))
>> +    return UPwAff;
>> +
>> +  auto Result =
>> +
>> give(isl_union_pw_aff_empty(isl_union_pw_aff_get_space(UPwAff.keep())));
>> +  foreachElt(UPwAff, [=, &Result](IslPtr<isl_pw_aff> PwAff) {
>> +    auto ValAff = give(isl_pw_aff_val_on_domain(
>> +
>> isl_set_universe(isl_space_domain(isl_pw_aff_get_space(PwAff.keep()))),
>> +        Val.copy()));
>> +    auto Subtracted = give(isl_pw_aff_sub(PwAff.copy(), ValAff.take()));
>> +    Result = give(isl_union_pw_aff_union_add(
>> +        Result.take(),
>> isl_union_pw_aff_from_pw_aff(Subtracted.take())));
>> +  });
>> +  return Result;
>> +}
>> +
>> +/// Compute @UPwAff * @p Val.
>> +IslPtr<isl_union_pw_aff> multiply(IslPtr<isl_union_pw_aff> UPwAff,
>> +                                  IslPtr<isl_val> Val) {
>> +  if (isl_val_is_one(Val.keep()))
>> +    return UPwAff;
>> +
>> +  auto Result =
>> +
>> give(isl_union_pw_aff_empty(isl_union_pw_aff_get_space(UPwAff.keep())));
>> +  foreachElt(UPwAff, [=, &Result](IslPtr<isl_pw_aff> PwAff) {
>> +    auto ValAff = give(isl_pw_aff_val_on_domain(
>> +
>> isl_set_universe(isl_space_domain(isl_pw_aff_get_space(PwAff.keep()))),
>> +        Val.copy()));
>> +    auto Multiplied = give(isl_pw_aff_mul(PwAff.copy(), ValAff.take()));
>> +    Result = give(isl_union_pw_aff_union_add(
>> +        Result.take(),
>> isl_union_pw_aff_from_pw_aff(Multiplied.take())));
>> +  });
>> +  return Result;
>> +}
>> +
>> +/// Remove @p n dimensions from @p UMap's range, starting at @p first.
>> +///
>> +/// It is assumed that all maps in the maps have at least the necessary
>> number
>> +/// of out dimensions.
>> +IslPtr<isl_union_map> scheduleProjectOut(NonowningIslPtr<isl_union_map>
>> UMap,
>> +                                         unsigned first, unsigned n) {
>> +  if (n == 0)
>> +    return UMap; /* isl_map_project_out would also reset the tuple,
>> which should
>> +                    have no effect on schedule ranges */
>> +
>> +  auto Result =
>> give(isl_union_map_empty(isl_union_map_get_space(UMap.keep())));
>> +  foreachElt(UMap, [=, &Result](IslPtr<isl_map> Map) {
>> +    auto Outprojected =
>> +        give(isl_map_project_out(Map.take(), isl_dim_out, first, n));
>> +    Result = give(isl_union_map_add_map(Result.take(),
>> Outprojected.take()));
>> +  });
>> +  return Result;
>> +}
>> +
>> +/// Return the number of dimensions in the input map's range.
>> +///
>> +/// Because this function takes an isl_union_map, the out dimensions
>> could be
>> +/// different. We return the maximum number in this case. However, a
>> different
>> +/// number of dimensions is not supported by the other code in this
>> file.
>> +size_t scheduleScatterDims(NonowningIslPtr<isl_union_map> Schedule) {
>> +  unsigned Dims = 0;
>> +  foreachElt(Schedule, [&Dims](IslPtr<isl_map> Map) {
>> +    Dims = std::max(Dims, isl_map_dim(Map.keep(), isl_dim_out));
>> +  });
>> +  return Dims;
>> +}
>> +
>> +/// Return the @p pos' range dimension, converted to an
>> isl_union_pw_aff.
>> +IslPtr<isl_union_pw_aff> scheduleExtractDimAff(IslPtr<isl_union_map>
>> UMap,
>> +                                               unsigned pos) {
>> +  auto SingleUMap =
>> +      give(isl_union_map_empty(isl_union_map_get_space(UMap.keep())));
>> +  foreachElt(UMap, [=, &SingleUMap](IslPtr<isl_map> Map) {
>> +    auto MapDims = isl_map_dim(Map.keep(), isl_dim_out);
>> +    auto SingleMap = give(isl_map_project_out(Map.take(), isl_dim_out,
>> 0, pos));
>> +    SingleMap = give(isl_map_project_out(SingleMap.take(), isl_dim_out,
>> 1,
>> +                                         MapDims - pos - 1));
>> +    SingleUMap =
>> +        give(isl_union_map_add_map(SingleUMap.take(),
>> SingleMap.take()));
>> +  });
>> +
>> +  auto UAff =
>> give(isl_union_pw_multi_aff_from_union_map(SingleUMap.take()));
>> +  auto FirstMAff =
>> +      give(isl_multi_union_pw_aff_from_union_pw_multi_aff(UAff.take()));
>> +  return give(isl_multi_union_pw_aff_get_union_pw_aff(FirstMAff.keep(),
>> 0));
>> +}
>> +
>> +/// Flatten a sequence-like first dimension.
>> +///
>> +/// A sequence-like scatter dimension is constant, or at least only
>> small
>> +/// variation, typically the result of ordering a sequence of different
>> +/// statements. An example would be:
>> +///   { Stmt_A[] -> [0, X, ...]; Stmt_B[] -> [1, Y, ...] }
>> +/// to schedule all instances of Stmt_A before any instance of Stmt_B.
>> +///
>> +/// To flatten, first begin with an offset of zero. Then determine the
>> lowest
>> +/// possible value of the dimension, call it "i" [In the example we
>> start at 0].
>> +/// Considering only schedules with that value, consider only instances
>> with
>> +/// that value and determine the extent of the next dimension. Let
>> l_X(i) and
>> +/// u_X(i) its minimum (lower bound) and maximum (upper bound) value.
>> Add them
>> +/// as "Offset + X - l_X(i)" to the new schedule, then add "u_X(i) -
>> l_X(i) + 1"
>> +/// to Offset and remove all i-instances from the old schedule. Repeat
>> with the
>> +/// remaining lowest value i' until there are no instances in the old
>> schedule
>> +/// left.
>> +/// The example schedule would be transformed to:
>> +///   { Stmt_X[] -> [X - l_X, ...]; Stmt_B -> [l_X - u_X + 1 + Y - l_Y,
>> ...] }
>> +IslPtr<isl_union_map> tryFlattenSequence(IslPtr<isl_union_map> Schedule)
>> {
>> +  auto IslCtx = Schedule.getCtx();
>> +  auto ScatterSet =
>> +
>> give(isl_set_from_union_set(isl_union_map_range(Schedule.copy())));
>> +
>> +  auto ParamSpace = give(isl_union_map_get_space(Schedule.keep()));
>> +  auto Dims = isl_set_dim(ScatterSet.keep(), isl_dim_set);
>> +  assert(Dims >= 2);
>> +
>> +  // Would cause an infinite loop.
>> +  if (!isDimBoundedByConstant(ScatterSet, 0)) {
>> +    DEBUG(dbgs() << "Abort; dimension is not of fixed size\n");
>> +    return nullptr;
>> +  }
>> +
>> +  auto AllDomains = give(isl_union_map_domain(Schedule.copy()));
>> +  auto AllDomainsToNull =
>> +      give(isl_union_pw_multi_aff_from_domain(AllDomains.take()));
>> +
>> +  auto NewSchedule = give(isl_union_map_empty(ParamSpace.copy()));
>> +  auto Counter =
>> give(isl_pw_aff_zero_on_domain(isl_local_space_from_space(
>> +      isl_space_set_from_params(ParamSpace.copy()))));
>> +
>> +  while (!isl_set_is_empty(ScatterSet.keep())) {
>> +    DEBUG(dbgs() << "Next counter:\n  " << Counter << "\n");
>> +    DEBUG(dbgs() << "Remaining scatter set:\n  " << ScatterSet << "\n");
>> +    auto ThisSet =
>> +        give(isl_set_project_out(ScatterSet.copy(), isl_dim_set, 1, Dims
>> - 1));
>> +    auto ThisFirst = give(isl_set_lexmin(ThisSet.take()));
>> +    auto ScatterFirst =
>> +        give(isl_set_add_dims(ThisFirst.take(), isl_dim_set, Dims - 1));
>> +
>> +    auto SubSchedule = give(isl_union_map_intersect_range(
>> +        Schedule.copy(), isl_union_set_from_set(ScatterFirst.copy())));
>> +    SubSchedule = scheduleProjectOut(std::move(SubSchedule), 0, 1);
>> +    SubSchedule = flattenSchedule(std::move(SubSchedule));
>> +
>> +    auto SubDims = scheduleScatterDims(SubSchedule);
>> +    auto FirstSubSchedule = scheduleProjectOut(SubSchedule, 1, SubDims -
>> 1);
>> +    auto FirstScheduleAff = scheduleExtractDimAff(FirstSubSchedule, 0);
>> +    auto RemainingSubSchedule =
>> +        scheduleProjectOut(std::move(SubSchedule), 0, 1);
>> +
>> +    auto FirstSubScatter = give(
>> +
>> isl_set_from_union_set(isl_union_map_range(FirstSubSchedule.take())));
>> +    DEBUG(dbgs() << "Next step in sequence is:\n  " << FirstSubScatter
>> << "\n");
>> +
>> +    if (!isDimBoundedByParameter(FirstSubScatter, 0)) {
>> +      DEBUG(dbgs() << "Abort; sequence step is not bounded\n");
>> +      return nullptr;
>> +    }
>> +
>> +    auto FirstSubScatterMap =
>> give(isl_map_from_range(FirstSubScatter.take()));
>> +
>> +    // isl_set_dim_max returns a strange isl_pw_aff with domain tuple_id
>> of
>> +    // 'none'. It doesn't match with any space including a 0-dimensional
>> +    // anonymous tuple.
>> +    // Interesting, one can create such a set using
>> +    // isl_set_universe(ParamSpace). Bug?
>> +    auto PartMin = give(isl_map_dim_min(FirstSubScatterMap.copy(), 0));
>> +    auto PartMax = give(isl_map_dim_max(FirstSubScatterMap.take(), 0));
>> +    auto One = give(isl_pw_aff_val_on_domain(
>> +        isl_set_universe(isl_space_set_from_params(ParamSpace.copy())),
>> +        isl_val_one(IslCtx)));
>> +    auto PartLen = give(isl_pw_aff_add(
>> +        isl_pw_aff_add(PartMax.take(), isl_pw_aff_neg(PartMin.copy())),
>> +        One.take()));
>> +
>> +    auto AllPartMin = give(isl_union_pw_aff_pullback_union_pw_multi_aff(
>> +        isl_union_pw_aff_from_pw_aff(PartMin.take()),
>> AllDomainsToNull.copy()));
>> +    auto FirstScheduleAffNormalized =
>> +        give(isl_union_pw_aff_sub(FirstScheduleAff.take(),
>> AllPartMin.take()));
>> +    auto AllCounter = give(isl_union_pw_aff_pullback_union_pw_multi_aff(
>> +        isl_union_pw_aff_from_pw_aff(Counter.copy()),
>> AllDomainsToNull.copy()));
>> +    auto FirstScheduleAffWithOffset = give(isl_union_pw_aff_add(
>> +        FirstScheduleAffNormalized.take(), AllCounter.take()));
>> +
>> +    auto ScheduleWithOffset = give(isl_union_map_flat_range_product(
>> +
>> isl_union_map_from_union_pw_aff(FirstScheduleAffWithOffset.take()),
>> +        RemainingSubSchedule.take()));
>> +    NewSchedule = give(
>> +        isl_union_map_union(NewSchedule.take(),
>> ScheduleWithOffset.take()));
>> +
>> +    ScatterSet = give(isl_set_subtract(ScatterSet.take(),
>> ScatterFirst.take()));
>> +    Counter = give(isl_pw_aff_add(Counter.take(), PartLen.take()));
>> +  }
>> +
>> +  DEBUG(dbgs() << "Sequence-flatten result is:\n  " << NewSchedule <<
>> "\n");
>> +  return NewSchedule;
>> +}
>> +
>> +/// Flatten a loop-like first dimension.
>> +///
>> +/// A loop-like dimension is one that depends on a variable (usually a
>> loop's
>> +/// induction variable). Let the input schedule look like this:
>> +///   { Stmt[i] -> [i, X, ...] }
>> +///
>> +/// To flatten, we determine the largest extent of X which may not
>> depend on the
>> +/// actual value of i. Let l_X() the smallest possible value of X and
>> u_X() its
>> +/// largest value. Then, construct a new schedule
>> +///   { Stmt[i] -> [i * (u_X() - l_X() + 1), ...] }
>> +IslPtr<isl_union_map> tryFlattenLoop(IslPtr<isl_union_map> Schedule) {
>> +  assert(scheduleScatterDims(Schedule) >= 2);
>> +
>> +  auto Remaining = scheduleProjectOut(Schedule, 0, 1);
>> +  auto SubSchedule = flattenSchedule(Remaining);
>> +  auto SubDims = scheduleScatterDims(SubSchedule);
>> +
>> +  auto SubExtent =
>> +
>> give(isl_set_from_union_set(isl_union_map_range(SubSchedule.copy())));
>> +  auto SubExtentDims = isl_set_dim(SubExtent.keep(), isl_dim_param);
>> +  SubExtent = give(
>> +      isl_set_project_out(SubExtent.take(), isl_dim_param, 0,
>> SubExtentDims));
>> +  SubExtent =
>> +      give(isl_set_project_out(SubExtent.take(), isl_dim_set, 1, SubDims
>> - 1));
>> +
>> +  if (!isDimBoundedByConstant(SubExtent, 0)) {
>> +    DEBUG(dbgs() << "Abort; dimension not bounded by constant\n");
>> +    return nullptr;
>> +  }
>> +
>> +  auto Min = give(isl_set_dim_min(SubExtent.copy(), 0));
>> +  DEBUG(dbgs() << "Min bound:\n  " << Min << "\n");
>> +  auto MinVal = getConstant(Min, false, true);
>> +  auto Max = give(isl_set_dim_max(SubExtent.take(), 0));
>> +  DEBUG(dbgs() << "Max bound:\n  " << Max << "\n");
>> +  auto MaxVal = getConstant(Max, true, false);
>> +
>> +  if (!MinVal || !MaxVal || isl_val_is_nan(MinVal.keep()) ||
>> +      isl_val_is_nan(MaxVal.keep())) {
>> +    DEBUG(dbgs() << "Abort; dimension bounds could not be
>> determined\n");
>> +    return nullptr;
>> +  }
>> +
>> +  auto FirstSubScheduleAff = scheduleExtractDimAff(SubSchedule, 0);
>> +  auto RemainingSubSchedule = scheduleProjectOut(std::move(SubSchedule),
>> 0, 1);
>> +
>> +  auto LenVal =
>> +      give(isl_val_add_ui(isl_val_sub(MaxVal.take(), MinVal.copy()),
>> 1));
>> +  auto FirstSubScheduleNormalized = subtract(FirstSubScheduleAff,
>> MinVal);
>> +
>> +  // TODO: Normalize FirstAff to zero (convert to isl_map, determine
>> minimum,
>> +  // subtract it)
>> +  auto FirstAff = scheduleExtractDimAff(Schedule, 0);
>> +  auto Offset = multiply(FirstAff, LenVal);
>> +  auto Index = give(
>> +      isl_union_pw_aff_add(FirstSubScheduleNormalized.take(),
>> Offset.take()));
>> +  auto IndexMap = give(isl_union_map_from_union_pw_aff(Index.take()));
>> +
>> +  auto Result = give(isl_union_map_flat_range_product(
>> +      IndexMap.take(), RemainingSubSchedule.take()));
>> +  DEBUG(dbgs() << "Loop-flatten result is:\n  " << Result << "\n");
>> +  return Result;
>> +}
>> +} // anonymous namespace
>> +
>> +IslPtr<isl_union_map> polly::flattenSchedule(IslPtr<isl_union_map>
>> Schedule) {
>> +  auto Dims = scheduleScatterDims(Schedule);
>> +  DEBUG(dbgs() << "Recursive schedule to process:\n  " << Schedule <<
>> "\n");
>> +
>> +  // Base case; no dimensions left
>> +  if (Dims == 0) {
>> +    // TODO: Add one dimension?
>> +    return Schedule;
>> +  }
>> +
>> +  // Base case; already one-dimensional
>> +  if (Dims == 1)
>> +    return Schedule;
>> +
>> +  // Fixed dimension; no need to preserve variabledness.
>> +  if (!isVariableDim(Schedule)) {
>> +    DEBUG(dbgs() << "Fixed dimension; try sequence flattening\n");
>> +    auto NewScheduleSequence = tryFlattenSequence(Schedule);
>> +    if (NewScheduleSequence)
>> +      return NewScheduleSequence;
>> +  }
>> +
>> +  // Constant stride
>> +  DEBUG(dbgs() << "Try loop flattening\n");
>> +  auto NewScheduleLoop = tryFlattenLoop(Schedule);
>> +  if (NewScheduleLoop)
>> +    return NewScheduleLoop;
>> +
>> +  // Try again without loop condition (may blow up the number of
>> pieces!!)
>> +  DEBUG(dbgs() << "Try sequence flattening again\n");
>> +  auto NewScheduleSequence = tryFlattenSequence(Schedule);
>> +  if (NewScheduleSequence)
>> +    return NewScheduleSequence;
>> +
>> +  // Cannot flatten
>> +  return Schedule;
>> +}
>>
>> Added: polly/trunk/lib/Transform/FlattenSchedule.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/lib/Transform/FlattenSchedule.cpp?rev=280948&view=auto
>> ==============================================================================
>> --- polly/trunk/lib/Transform/FlattenSchedule.cpp (added)
>> +++ polly/trunk/lib/Transform/FlattenSchedule.cpp Thu Sep  8 10:02:36
>> 2016
>> @@ -0,0 +1,108 @@
>> +//===------ FlattenSchedule.cpp --------------------------------*- C++
>> -*-===//
>> +//
>> +//                     The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// Try to reduce the number of scatter dimension. Useful to make
>> isl_union_map
>> +// schedules more understandable. This is only intended for debugging
>> and
>> +// unittests, not for production use.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "polly/FlattenSchedule.h"
>> +#include "polly/FlattenAlgo.h"
>> +#include "polly/ScopInfo.h"
>> +#include "polly/ScopPass.h"
>> +#define DEBUG_TYPE "polly-flatten-schedule"
>> +
>> +using namespace polly;
>> +using namespace llvm;
>> +
>> +namespace {
>> +
>> +/// Print a schedule to @p OS.
>> +///
>> +/// Prints the schedule for each statements on a new line.
>> +void printSchedule(raw_ostream &OS, NonowningIslPtr<isl_union_map>
>> Schedule,
>> +                   int indent) {
>> +  foreachElt(Schedule, [&OS, indent](IslPtr<isl_map> Map) {
>> +    OS.indent(indent) << Map << "\n";
>> +  });
>> +}
>> +
>> +/// Flatten the schedule stored in an polly::Scop.
>> +class FlattenSchedule : public ScopPass {
>> +private:
>> +  FlattenSchedule(const FlattenSchedule &) = delete;
>> +  const FlattenSchedule &operator=(const FlattenSchedule &) = delete;
>> +
>> +  std::shared_ptr<isl_ctx> IslCtx;
>> +  IslPtr<isl_union_map> OldSchedule;
>> +
>> +public:
>> +  static char ID;
>> +  explicit FlattenSchedule() : ScopPass(ID) {}
>> +
>> +  virtual void getAnalysisUsage(AnalysisUsage &AU) const override {
>> +    AU.addRequiredTransitive<ScopInfoRegionPass>();
>> +    AU.setPreservesAll();
>> +  }
>> +
>> +  virtual bool runOnScop(Scop &S) override {
>> +    // Keep a reference to isl_ctx to ensure that it is not freed before
>> we free
>> +    // OldSchedule.
>> +    IslCtx = S.getSharedIslCtx();
>> +
>> +    DEBUG(dbgs() << "Going to flatten old schedule:\n");
>> +    OldSchedule = give(S.getSchedule());
>> +    DEBUG(printSchedule(dbgs(), OldSchedule, 2));
>> +
>> +    auto Domains = give(S.getDomains());
>> +    auto RestrictedOldSchedule = give(
>> +        isl_union_map_intersect_domain(OldSchedule.copy(),
>> Domains.copy()));
>> +    DEBUG(dbgs() << "Old schedule with domains:\n");
>> +    DEBUG(printSchedule(dbgs(), RestrictedOldSchedule, 2));
>> +
>> +    auto NewSchedule = flattenSchedule(RestrictedOldSchedule);
>> +
>> +    DEBUG(dbgs() << "Flattened new schedule:\n");
>> +    DEBUG(printSchedule(dbgs(), NewSchedule, 2));
>> +
>> +    NewSchedule =
>> +        give(isl_union_map_gist_domain(NewSchedule.take(),
>> Domains.take()));
>> +    DEBUG(dbgs() << "Gisted, flattened new schedule:\n");
>> +    DEBUG(printSchedule(dbgs(), NewSchedule, 2));
>> +
>> +    S.setSchedule(NewSchedule.take());
>> +    return false;
>> +  }
>> +
>> +  virtual void printScop(raw_ostream &OS, Scop &S) const override {
>> +    OS << "Schedule before flattening {\n";
>> +    printSchedule(OS, OldSchedule, 4);
>> +    OS << "}\n\n";
>> +
>> +    OS << "Schedule after flattening {\n";
>> +    printSchedule(OS, give(S.getSchedule()), 4);
>> +    OS << "}\n";
>> +  }
>> +
>> +  virtual void releaseMemory() override {
>> +    OldSchedule = nullptr;
>> +    IslCtx.reset();
>> +  }
>> +};
>> +
>> +char FlattenSchedule::ID;
>> +} // anonymous namespace
>> +
>> +Pass *polly::createFlattenSchedulePass() { return new FlattenSchedule();
>> }
>> +
>> +INITIALIZE_PASS_BEGIN(FlattenSchedule, "polly-flatten-schedule",
>> +                      "Polly - Flatten schedule", false, false)
>> +INITIALIZE_PASS_END(FlattenSchedule, "polly-flatten-schedule",
>> +                    "Polly - Flatten schedule", false, false)
>>
>> Added: polly/trunk/test/FlattenSchedule/gemm.ll
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/test/FlattenSchedule/gemm.ll?rev=280948&view=auto
>> ==============================================================================
>> --- polly/trunk/test/FlattenSchedule/gemm.ll (added)
>> +++ polly/trunk/test/FlattenSchedule/gemm.ll Thu Sep  8 10:02:36 2016
>> @@ -0,0 +1,98 @@
>> +; RUN: opt %loadPolly -polly-flatten-schedule -analyze < %s | FileCheck
>> %s
>> +;
>> +; dgemm kernel
>> +; C := alpha*A*B + beta*C
>> +; C[ni][nj]
>> +; A[ni][nk]
>> +; B[nk][nj]
>> +
>> +target datalayout = "e-m:x-p:32:32-i64:64-f80:32-n8:16:32-a:0:32-S32"
>> +
>> +define void @gemm(i32 %ni, i32 %nj, i32 %nk, double %alpha, double
>> %beta, double* noalias nonnull %C, double* noalias nonnull %A, double*
>> noalias nonnull %B) {
>> +entry:
>> +  br label %ni.for
>> +
>> +ni.for:
>> +  %i = phi i32 [0, %entry], [%i.inc, %ni.inc]
>> +  %i.cmp = icmp slt i32 %i, 3
>> +  br i1 %i.cmp, label %nj.for, label %ni.exit
>> +
>> +  nj.for:
>> +    %j = phi i32 [0, %ni.for], [%j.inc, %nj.inc]
>> +    %j.cmp = icmp slt i32 %j, 7
>> +    br i1 %j.cmp, label %nj_beta, label %nj.exit
>> +
>> +    nj_beta:
>> +     %c_stride = mul nsw i32 %i, 3; %nj
>> +     %c_idx_i = getelementptr inbounds double, double* %C, i32 %c_stride
>> +     %c_idx_ij = getelementptr inbounds double, double* %c_idx_i, i32 %j
>> +
>> +     ; C[i][j] *= beta
>> +     %c = load double, double* %c_idx_ij
>> +     %c_beta = fmul double %c, %beta
>> +     store double %c_beta, double* %c_idx_ij
>> +
>> +     br label %nk.for
>> +
>> +    nk.for:
>> +      %k = phi i32 [0, %nj_beta], [%k.inc, %nk.inc]
>> +      %k.cmp = icmp slt i32 %k, 3 ; %nk
>> +      br i1 %k.cmp, label %nk_alpha, label %nk.exit
>> +
>> +      nk_alpha:
>> +        %a_stride = mul nsw i32 %i, 3; %nk
>> +        %a_idx_i = getelementptr inbounds double, double* %A, i32
>> %a_stride
>> +        %a_idx_ik = getelementptr inbounds double, double* %a_idx_i, i32
>> %k
>> +
>> +        %b_stride = mul nsw i32 %k, 3; %nj
>> +        %b_idx_k = getelementptr inbounds double, double* %B, i32
>> %b_stride
>> +        %b_idx_kj = getelementptr inbounds double, double* %b_idx_k, i32
>> %j
>> +
>> +        ; C[i][j] += alpha * A[i][k] * B[k][j]
>> +        %a = load double, double* %a_idx_ik
>> +        %b = load double, double* %b_idx_kj
>> +        %beta_c = load double, double* %c_idx_ij
>> +
>> +        %alpha_a = fmul double %a, %alpha
>> +        %alpha_a_b = fmul double %alpha_a, %b
>> +        %beta_c_alpha_a_b = fadd double %beta_c, %alpha_a_b
>> +
>> +        store double %beta_c_alpha_a_b, double* %c_idx_ij
>> +
>> +        br label %nk.inc
>> +
>> +    nk.inc:
>> +      %k.inc = add nuw nsw i32 %k, 1
>> +      br label %nk.for
>> +
>> +    nk.exit:
>> +      ; store double %c, double* %c_idx_ij
>> +      br label %nj.inc
>> +
>> +  nj.inc:
>> +    %j.inc = add nuw nsw i32 %j, 1
>> +    br label %nj.for
>> +
>> +  nj.exit:
>> +    br label %ni.inc
>> +
>> +ni.inc:
>> +  %i.inc = add nuw nsw i32 %i, 1
>> +  br label %ni.for
>> +
>> +ni.exit:
>> +  br label %return
>> +
>> +return:
>> +  ret void
>> +}
>> +
>> +
>> +; CHECK:      Schedule before flattening {
>> +; CHECK-NEXT:     { Stmt_nk_alpha[i0, i1, i2] -> [i0, i1, 1, i2] }
>> +; CHECK-NEXT:     { Stmt_nj_beta[i0, i1] -> [i0, i1, 0, 0] }
>> +; CHECK-NEXT: }
>> +; CHECK:      Schedule after flattening {
>> +; CHECK-NEXT:     { Stmt_nj_beta[i0, i1] -> [28i0 + 4i1] }
>> +; CHECK-NEXT:     { Stmt_nk_alpha[i0, i1, i2] -> [1 + 28i0 + 4i1 + i2] }
>> +; CHECK-NEXT: }
>>
>> Modified: polly/trunk/test/update_check.py
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/test/update_check.py?rev=280948&r1=280947&r2=280948&view=diff
>> ==============================================================================
>> --- polly/trunk/test/update_check.py (original)
>> +++ polly/trunk/test/update_check.py Thu Sep  8 10:02:36 2016
>> @@ -170,6 +170,18 @@ def classyfier1(lines):
>>              continue
>>          elif line.startswith("New access function '"):
>>              yield {'NewAccessFunction'}
>> +        elif line == 'Schedule before flattening {':
>> +            while True:
>> +                yield  {'ScheduleBeforeFlattening'}
>> +                if line == '}':
>> +                    break
>> +                line = i.__next__()
>> +        elif line == 'Schedule after flattening {':
>> +            while True:
>> +                yield  {'ScheduleAfterFlattening'}
>> +                if line == '}':
>> +                    break
>> +                line = i.__next__()
>>          else:
>>              yield set()
>>          line = i.__next__()
>>
>> Modified: polly/trunk/unittests/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/unittests/CMakeLists.txt?rev=280948&r1=280947&r2=280948&view=diff
>> ==============================================================================
>> --- polly/trunk/unittests/CMakeLists.txt (original)
>> +++ polly/trunk/unittests/CMakeLists.txt Thu Sep  8 10:02:36 2016
>> @@ -20,3 +20,4 @@ function(add_polly_unittest test_name)
>>  endfunction()
>>
>>  add_subdirectory(Isl)
>> +add_subdirectory(Flatten)
>>
>> Added: polly/trunk/unittests/Flatten/CMakeLists.txt
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/unittests/Flatten/CMakeLists.txt?rev=280948&view=auto
>> ==============================================================================
>> --- polly/trunk/unittests/Flatten/CMakeLists.txt (added)
>> +++ polly/trunk/unittests/Flatten/CMakeLists.txt Thu Sep  8 10:02:36 2016
>> @@ -0,0 +1,3 @@
>> +add_polly_unittest(FlattenTests
>> +  FlattenTest.cpp
>> +  )
>>
>> Added: polly/trunk/unittests/Flatten/FlattenTest.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/polly/trunk/unittests/Flatten/FlattenTest.cpp?rev=280948&view=auto
>> ==============================================================================
>> --- polly/trunk/unittests/Flatten/FlattenTest.cpp (added)
>> +++ polly/trunk/unittests/Flatten/FlattenTest.cpp Thu Sep  8 10:02:36
>> 2016
>> @@ -0,0 +1,70 @@
>> +//===- FlattenTest.cpp
>> ----------------------------------------------------===//
>> +//
>> +//                     The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "polly/FlattenAlgo.h"
>> +#include "polly/Support/GICHelper.h"
>> +#include "gtest/gtest.h"
>> +#include "isl/union_map.h"
>> +
>> +using namespace llvm;
>> +using namespace polly;
>> +
>> +namespace {
>> +
>> +/// Flatten a schedule and compare to the expected result.
>> +///
>> +/// @param ScheduleStr The schedule to flatten as string.
>> +/// @param ExpectedStr The expected result as string.
>> +///
>> +/// @result Whether the flattened schedule is the same as the expected
>> schedule.
>> +bool checkFlatten(const char *ScheduleStr, const char *ExpectedStr) {
>> +  auto *Ctx = isl_ctx_alloc();
>> +  isl_bool Success;
>> +
>> +  {
>> +    auto Schedule = give(isl_union_map_read_from_str(Ctx, ScheduleStr));
>> +    auto Expected = give(isl_union_map_read_from_str(Ctx, ExpectedStr));
>> +
>> +    auto Result = flattenSchedule(std::move(Schedule));
>> +    Success = isl_union_map_is_equal(Result.keep(), Expected.keep());
>> +  }
>> +
>> +  isl_ctx_free(Ctx);
>> +  return Success == isl_bool_true;
>> +}
>> +
>> +TEST(Flatten, FlattenTrivial) {
>> +  EXPECT_TRUE(checkFlatten("{ A[] -> [0] }", "{ A[] -> [0] }"));
>> +  EXPECT_TRUE(checkFlatten("{ A[i] -> [i, 0] : 0 <= i < 10 }",
>> +                           "{ A[i] -> [i] : 0 <= i < 10 }"));
>> +  EXPECT_TRUE(checkFlatten("{ A[i] -> [0, i] : 0 <= i < 10 }",
>> +                           "{ A[i] -> [i] : 0 <= i < 10 }"));
>> +}
>> +
>> +TEST(Flatten, FlattenSequence) {
>> +  EXPECT_TRUE(checkFlatten(
>> +      "[n] -> { A[i] -> [0, i] : 0 <= i < n; B[i] -> [1, i] : 0 <= i < n
>> }",
>> +      "[n] -> { A[i] -> [i] : 0 <= i < n; B[i] -> [n + i] : 0 <= i < n
>> }"));
>> +
>> +  EXPECT_TRUE(checkFlatten(
>> +      "{ A[i] -> [0, i] : 0 <= i < 10; B[i] -> [1, i] : 0 <= i < 10 }",
>> +      "{ A[i] -> [i] : 0 <= i < 10; B[i] -> [10 + i] : 0 <= i < 10 }"));
>> +}
>> +
>> +TEST(Flatten, FlattenLoop) {
>> +  EXPECT_TRUE(checkFlatten(
>> +      "[n] -> { A[i] -> [i, 0] : 0 <= i < n; B[i] -> [i, 1] : 0 <= i < n
>> }",
>> +      "[n] -> { A[i] -> [2i] : 0 <= i < n; B[i] -> [2i + 1] : 0 <= i < n
>> }"));
>> +
>> +  EXPECT_TRUE(checkFlatten(
>> +      "{ A[i] -> [i, 0] : 0 <= i < 10; B[i] -> [i, 1] : 0 <= i < 10 }",
>> +      "{ A[i] -> [2i] : 0 <= i < 10; B[i] -> [2i + 1] : 0 <= i < 10
>> }"));
>> +}
>> +
>> +} // anonymous namespace
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits


More information about the llvm-commits mailing list