[all-commits] [llvm/llvm-project] 4e1ba0: [RISCV] Don't accidentally match deinterleave mask...

[Luke Lau via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: 4e1ba0c518687b44ea76eed65581c9e3dbacbb40
      https://github.com/llvm/llvm-project/commit/4e1ba0c518687b44ea76eed65581c9e3dbacbb40
  Author: Luke Lau <luke at igalia.com>
  Date:   2023-03-16 (Thu, 16 Mar 2023)

  Changed paths:
    M llvm/include/llvm/IR/Instructions.h
    M llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp

  Log Message:
  -----------
  [RISCV] Don't accidentally match deinterleave masks as interleaves

Consider a shuffle mask of <0, 2>:
This is one of two deinterleave masks to deinterleave a vector of 4
elements with factor 2.
Unfortunately, this is also technically an interleave mask, where
two subvectors of length 1 at indexes 0 and 2 will be interleaved.
This is because a mask can interleave non-contiguous subvectors:
e.g. <0, 6, 4, 1, 7, 5> on a vector of size 8:

```
<0 1 2 3 4 5 6 7> indices
 ^ ^     ^ ^ ^ ^
 0 0     2 2 1 1  deinterleaved subvector
```

This means that deinterleaving shuffles can accidentally be costed as
interleaves.
And it's incorrect in the context of interleaves, because the
only interleave shuffles we model at the moment are single permutation
shuffles, i.e. we are interleaving the first vector below and ignoring
the second:

shufflevector <2 x i32> %v0, <2 x i32> poison, <2 x i32> <i32 0, i32 2>

A mask of <0, 2> interleaves across both vectors.

The fix here is to set NumInputElts correctly: We were setting it to
twice the mask length, i.e. using both input vectors. But in fact we're
actually only using the first vector here, and isInterleaveMask actually
already has logic to ensure that the mask indices stay within the bounds
of the input vectors.

This lacks a test case due to how we're unable to test deinterleave
shuffles (because they are length changing), but is covered in the tests
in D145155

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D146176


  Commit: b9238abe05372e09c0ee552002cba0e62c707f78
      https://github.com/llvm/llvm-project/commit/b9238abe05372e09c0ee552002cba0e62c707f78
  Author: Luke Lau <luke at igalia.com>
  Date:   2023-03-16 (Thu, 16 Mar 2023)

  Changed paths:
    M llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
    M llvm/lib/Target/RISCV/RISCVTargetTransformInfo.h
    M llvm/test/Transforms/LoopVectorize/RISCV/interleaved-accesses.ll
    A llvm/test/Transforms/LoopVectorize/RISCV/interleaved-cost.ll
    M llvm/test/Transforms/LoopVectorize/RISCV/zvl32b.ll

  Log Message:
  -----------
  [RISCV] Enable interleaved access vectorization

The loop vectorizer supports generating interleaved loads and stores via
shuffle patterns for fixed length vectors.
This enables it for RISC-V, since interleaved shuffle patterns can be
lowered to vlseg/vsseg in https://reviews.llvm.org/D145022

Reviewed By: reames

Differential Revision: https://reviews.llvm.org/D145155


Compare: https://github.com/llvm/llvm-project/compare/7d10b4745054...b9238abe0537