[all-commits] [llvm/llvm-project] e408d1: [SLP]Improve graph reordering.

[cilkplus via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: e408d1dfab42b27d0aa51b221e50fa6390fb5ed1
      https://github.com/llvm/llvm-project/commit/e408d1dfab42b27d0aa51b221e50fa6390fb5ed1
  Author: Alexey Bataev <a.bataev at outlook.com>
  Date:   2021-07-28 (Wed, 28 Jul 2021)

  Changed paths:
    M llvm/lib/Transforms/Vectorize/SLPVectorizer.cpp
    M llvm/test/Transforms/SLPVectorizer/AArch64/transpose-inseltpoison.ll
    M llvm/test/Transforms/SLPVectorizer/AArch64/transpose.ll
    M llvm/test/Transforms/SLPVectorizer/X86/addsub.ll
    M llvm/test/Transforms/SLPVectorizer/X86/crash_cmpop.ll
    M llvm/test/Transforms/SLPVectorizer/X86/extract.ll
    M llvm/test/Transforms/SLPVectorizer/X86/jumbled-load-multiuse.ll
    M llvm/test/Transforms/SLPVectorizer/X86/jumbled-load.ll
    M llvm/test/Transforms/SLPVectorizer/X86/jumbled_store_crash.ll
    M llvm/test/Transforms/SLPVectorizer/X86/reorder_repeated_ops.ll
    M llvm/test/Transforms/SLPVectorizer/X86/vectorize-reorder-reuse.ll

  Log Message:
  -----------
  [SLP]Improve graph reordering.

Reworked reordering algorithm. Originally, the compiler just tried to
detect the most common order in the reordarable nodes (loads, stores,
extractelements,extractvalues) and then fully rebuilding the graph in
the best order. This was not effecient, since it required an extra
memory and time for building/rebuilding tree, double the use of the
scheduling budget, which could lead to missing vectorization due to
exausted scheduling resources.

Patch provide 2-way approach for graph reodering problem. At first, all
reordering is done in-place, it doe not required tree
deleting/rebuilding, it just rotates the scalars/orders/reuses masks in
the graph node.

The first step (top-to bottom) rotates the whole graph, similarly to the previous
implementation. Compiler counts the number of the most used orders of
the graph nodes with the same vectorization factor and then rotates the
subgraph with the given vectorization factor to the most used order, if
it is not empty. Then repeats the same procedure for the subgraphs with
the smaller vectorization factor. We can do this because we still need
to reshuffle smaller subgraph when buildiong operands for the graph
nodes with lasrger vectorization factor, we can rotate just subgraph,
not the whole graph.

The second step (bottom-to-top) scans through the leaves and tries to
detect the users of the leaves which can be reordered. If the leaves can
be reorder in the best fashion, they are reordered and their user too.
It allows to remove double shuffles to the same ordering of the operands in
many cases and just reorder the user operations instead. Plus, it moves
the final shuffles closer to the top of the graph and in many cases
allows to remove extra shuffle because the same procedure is repeated
again and we can again merge some reordering masks and reorder user nodes
instead of the operands.

Also, patch improves cost model for gathering of loads, which improves
x264 benchmark in some cases.

Gives about +2% on AVX512 + LTO (more expected for AVX/AVX2) for {625,525}x264,
+3% for 508.namd, improves most of other benchmarks.
The compile and link time are almost the same, though in some cases it
should be better (we're not doing an extra instruction scheduling
anymore) + we may vectorize more code for the large basic blocks again
because of saving scheduling budget.

Differential Revision: https://reviews.llvm.org/D105020