[llvm] [AArch64] Add SVE shuffle optimization pass (PR #193951)

[Graham Hunter via llvm-commits]

huntergr-arm wrote:

> Where is the code coming from? If is from the vectorizer then the direction of travel recently has been to do the deinterleaving in VPlan (which has the added benefit of computing costs more accurately, not relying on ld4 being profitable enough).
> 
> Otherwise - new passes do cost more than existing passes, both in terms of the amount of code to maintain and the compile time cost of scanning code again and again.

So the incoming IR will be from the Loop Vectorizer, at least after some changes to interleaving have been made (it currently costs lower VFs as if all the strided loads were gathers, at least for SVE). We expect it to use the vector_deinterleaveN intrinsics, which we can then match along with other operations (like zext or uitofp) and replace with tbl. We can't represent tbl in a target-independent manner for scalable vectors, so it's target-specific for now.

The NEON equivalent was performed at the CodeGenPrepare level, but may be moving to a DAGCombine. This initial transform could also be done at that level, but the intent is to allow for more aggressive optimization that requires information about uses in other blocks. For example, the loops in MorphologyPrimitive for imagick (in spec 2017) have this pattern, but also have a reverse intrinsic applied to another value in the loop which is multiplied by the deinterleaved and converted values before being added to an accumulator. We need to confirm that the order of elements within the accumulator does not matter, and if the eventual final reduction allows reassociation then we can move the reverse to the tbls as well, further reducing instruction count.

If we expressed the accumulation in the loop as a partial reduction (which are explicitly unordered) even without a change in VF, then we might have enough information to just do this at the DAGCombine level. I'm not sure; I haven't prototyped that.

We would also like to use the bottom/top SVE instructions in cases where ordering must be preserved, which means introducing extra shuffles that can be combined into other operations.

https://github.com/llvm/llvm-project/pull/193951