[Mlir-commits] [mlir] [mlir][Vector] Add `vector.shuffle` tree transformation (PR #145740)

[James Newling]

https://github.com/newling commented:

Thanks for the updates! Just a few more minor comments from me in this review. 

Regarding the top-level motivation:

> This is mostly implementing "2. Simplified Pattern Recognition and Optimization" in https://discourse.llvm.org/t/rfc-adding-vector-to-elements-op-to-the-vector-dialect/86779. It's not about making the lowering easier but far more efficient both in terms of performance and compile time.

I agree with this, but can we please get a few more details? Here are my initial thoughts along this line. Suppose the total number of elements in the final from_elements vector is `N` and the total number of to_elements ops is `T`. So `T` < `N`, and the ratio `N/T` is the mean number of extracted elements that go into the final vector. 

My understanding is that the total number of shuffles will still be `O(T)`, so the number of ops isn't reduced. But the mean size of the shuffles (mask size) will now hopefully be `O(1)` because of the leaves being merged with small masks. This is better than the `O(N)` with the current (naive) lowering. Is that correct, and is that where the efficiency gain will come from? 

Worst case scenario for this pattern? I'm wondering what happens with something that looks like a transpose where the elements are 'scattered'. Like 

```
[[ 0#0 1#0 ... 7#0 ]
 [ 0#1 1#1 ... 7#1 ]]
 ```
If I understand correctly this case the mean size of the shuffle is still O(N) (with a better constant) because from level 1 onwards the intervals are large. This is no worse than the 'naive' case, so it seems like the pattern can only improve the situation. Wondering if anyone has an even worse case in mind.

Only partially related to this PR, but I also wonder if there is something like this is LLVMIR, where a sequence of shufflevector instructions is turned into a tree to reduce the average mask size. 
 

https://github.com/llvm/llvm-project/pull/145740