[PATCH] D103274: [X86] AMD Zen 3 has fast per-lane variable shuffles

[Roman Lebedev via Phabricator via llvm-commits]

lebedev.ri created this revision.
lebedev.ri added a reviewer: RKSimon.
lebedev.ri added a project: LLVM.
Herald added subscribers: pengfei, hiraditya.
lebedev.ri requested review of this revision.

Currently, X86 backend only has a global one-size-fits-all `FeatureFastVariableShuffle` feature,
which controls profitability of both the cross-lane and per-lane variable shuffles.
I guess, this has been fine so far.

But at least on AMD Zen 3, while per-line variable shuffles (e.g. `VPSHUFB`)
are as fast as as shuffles with fixed/immediate mask,
while lane-crossing shuffles, e.g. `VPERMPS` is performing worse.

So to get the benefits of variable-mask shuffles, but not the drawbacks of lane-crossing shuffles,
as suggested by @RKSimon, split the feature flag into two.


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D103274

Files:
  llvm/lib/Target/X86/X86.td
  llvm/lib/Target/X86/X86ISelLowering.cpp
  llvm/lib/Target/X86/X86Subtarget.h
  llvm/lib/Target/X86/X86TargetTransformInfo.h
  llvm/test/CodeGen/X86/avx2-conversions.ll
  llvm/test/CodeGen/X86/avx2-vector-shifts.ll
  llvm/test/CodeGen/X86/avx512-extract-subvector-load-store.ll
  llvm/test/CodeGen/X86/avx512-shuffles/broadcast-vector-int.ll
  llvm/test/CodeGen/X86/avx512-shuffles/partial_permute.ll
  llvm/test/CodeGen/X86/avx512-trunc.ll
  llvm/test/CodeGen/X86/bitcast-int-to-vector-bool-sext.ll
  llvm/test/CodeGen/X86/bitcast-int-to-vector-bool-zext.ll
  llvm/test/CodeGen/X86/broadcastm-lowering.ll
  llvm/test/CodeGen/X86/combine-shl.ll
  llvm/test/CodeGen/X86/combine-sra.ll
  llvm/test/CodeGen/X86/combine-srl.ll
  llvm/test/CodeGen/X86/insertelement-zero.ll
  llvm/test/CodeGen/X86/oddshuffles.ll
  llvm/test/CodeGen/X86/oddsubvector.ll
  llvm/test/CodeGen/X86/paddus.ll
  llvm/test/CodeGen/X86/phaddsub.ll
  llvm/test/CodeGen/X86/psubus.ll
  llvm/test/CodeGen/X86/sadd_sat_vec.ll
  llvm/test/CodeGen/X86/shuffle-of-splat-multiuses.ll
  llvm/test/CodeGen/X86/shuffle-strided-with-offset-128.ll
  llvm/test/CodeGen/X86/shuffle-strided-with-offset-256.ll
  llvm/test/CodeGen/X86/shuffle-strided-with-offset-512.ll
  llvm/test/CodeGen/X86/shuffle-vs-trunc-128.ll
  llvm/test/CodeGen/X86/shuffle-vs-trunc-256.ll
  llvm/test/CodeGen/X86/shuffle-vs-trunc-512.ll
  llvm/test/CodeGen/X86/ssub_sat_vec.ll
  llvm/test/CodeGen/X86/uadd_sat_vec.ll
  llvm/test/CodeGen/X86/usub_sat_vec.ll
  llvm/test/CodeGen/X86/vec_saddo.ll
  llvm/test/CodeGen/X86/vec_smulo.ll
  llvm/test/CodeGen/X86/vec_ssubo.ll
  llvm/test/CodeGen/X86/vec_uaddo.ll
  llvm/test/CodeGen/X86/vec_umulo.ll
  llvm/test/CodeGen/X86/vec_usubo.ll
  llvm/test/CodeGen/X86/vector-half-conversions.ll
  llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-2.ll
  llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-3.ll
  llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-4.ll
  llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-5.ll
  llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-6.ll
  llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-2.ll
  llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-3.ll
  llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-4.ll
  llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-5.ll
  llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-6.ll
  llvm/test/CodeGen/X86/vector-shuffle-128-unpck.ll
  llvm/test/CodeGen/X86/vector-shuffle-128-v16.ll
  llvm/test/CodeGen/X86/vector-shuffle-128-v4.ll
  llvm/test/CodeGen/X86/vector-shuffle-128-v8.ll
  llvm/test/CodeGen/X86/vector-shuffle-256-v16.ll
  llvm/test/CodeGen/X86/vector-shuffle-256-v32.ll
  llvm/test/CodeGen/X86/vector-shuffle-256-v4.ll
  llvm/test/CodeGen/X86/vector-shuffle-256-v8.ll
  llvm/test/CodeGen/X86/vector-shuffle-combining.ll
  llvm/test/CodeGen/X86/vector-shuffle-v1.ll
  llvm/test/CodeGen/X86/vector-trunc-math.ll
  llvm/test/CodeGen/X86/vector-trunc-packus.ll
  llvm/test/CodeGen/X86/vector-trunc-ssat.ll
  llvm/test/CodeGen/X86/vector-trunc-usat.ll
  llvm/test/CodeGen/X86/vector-trunc.ll
  llvm/test/CodeGen/X86/vector-zext.ll