[all-commits] [llvm/llvm-project] cf9b1f: [X86] Split FeatureFastVariableShuffle tuning into...

[Roman Lebedev via All-commits]

  Branch: refs/heads/main
  Home:   https://github.com/llvm/llvm-project
  Commit: cf9b1f7a0e9da5d019a8bea853f3cff85d808d18
      https://github.com/llvm/llvm-project/commit/cf9b1f7a0e9da5d019a8bea853f3cff85d808d18
  Author: Roman Lebedev <lebedev.ri at gmail.com>
  Date:   2021-06-01 (Tue, 01 Jun 2021)

  Changed paths:
    M llvm/lib/Target/X86/X86.td
    M llvm/lib/Target/X86/X86ISelLowering.cpp
    M llvm/lib/Target/X86/X86Subtarget.h
    M llvm/lib/Target/X86/X86TargetTransformInfo.h
    M llvm/test/CodeGen/X86/avx2-conversions.ll
    M llvm/test/CodeGen/X86/avx2-vector-shifts.ll
    M llvm/test/CodeGen/X86/avx512-extract-subvector-load-store.ll
    M llvm/test/CodeGen/X86/avx512-shuffles/broadcast-vector-int.ll
    M llvm/test/CodeGen/X86/avx512-shuffles/partial_permute.ll
    M llvm/test/CodeGen/X86/avx512-trunc.ll
    M llvm/test/CodeGen/X86/bitcast-int-to-vector-bool-sext.ll
    M llvm/test/CodeGen/X86/bitcast-int-to-vector-bool-zext.ll
    M llvm/test/CodeGen/X86/broadcastm-lowering.ll
    M llvm/test/CodeGen/X86/combine-shl.ll
    M llvm/test/CodeGen/X86/combine-sra.ll
    M llvm/test/CodeGen/X86/combine-srl.ll
    M llvm/test/CodeGen/X86/insertelement-zero.ll
    M llvm/test/CodeGen/X86/oddshuffles.ll
    M llvm/test/CodeGen/X86/oddsubvector.ll
    M llvm/test/CodeGen/X86/paddus.ll
    M llvm/test/CodeGen/X86/phaddsub.ll
    M llvm/test/CodeGen/X86/psubus.ll
    M llvm/test/CodeGen/X86/sadd_sat_vec.ll
    M llvm/test/CodeGen/X86/shuffle-of-splat-multiuses.ll
    M llvm/test/CodeGen/X86/shuffle-strided-with-offset-128.ll
    M llvm/test/CodeGen/X86/shuffle-strided-with-offset-256.ll
    M llvm/test/CodeGen/X86/shuffle-strided-with-offset-512.ll
    M llvm/test/CodeGen/X86/shuffle-vs-trunc-128.ll
    M llvm/test/CodeGen/X86/shuffle-vs-trunc-256.ll
    M llvm/test/CodeGen/X86/shuffle-vs-trunc-512.ll
    M llvm/test/CodeGen/X86/ssub_sat_vec.ll
    M llvm/test/CodeGen/X86/uadd_sat_vec.ll
    M llvm/test/CodeGen/X86/usub_sat_vec.ll
    M llvm/test/CodeGen/X86/vec_saddo.ll
    M llvm/test/CodeGen/X86/vec_smulo.ll
    M llvm/test/CodeGen/X86/vec_ssubo.ll
    M llvm/test/CodeGen/X86/vec_uaddo.ll
    M llvm/test/CodeGen/X86/vec_umulo.ll
    M llvm/test/CodeGen/X86/vec_usubo.ll
    M llvm/test/CodeGen/X86/vector-half-conversions.ll
    M llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-2.ll
    M llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-3.ll
    M llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-4.ll
    M llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-5.ll
    M llvm/test/CodeGen/X86/vector-interleaved-load-i16-stride-6.ll
    M llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-2.ll
    M llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-3.ll
    M llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-4.ll
    M llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-5.ll
    M llvm/test/CodeGen/X86/vector-interleaved-store-i16-stride-6.ll
    M llvm/test/CodeGen/X86/vector-shuffle-128-unpck.ll
    M llvm/test/CodeGen/X86/vector-shuffle-128-v16.ll
    M llvm/test/CodeGen/X86/vector-shuffle-128-v4.ll
    M llvm/test/CodeGen/X86/vector-shuffle-128-v8.ll
    M llvm/test/CodeGen/X86/vector-shuffle-256-v16.ll
    M llvm/test/CodeGen/X86/vector-shuffle-256-v32.ll
    M llvm/test/CodeGen/X86/vector-shuffle-256-v4.ll
    M llvm/test/CodeGen/X86/vector-shuffle-256-v8.ll
    M llvm/test/CodeGen/X86/vector-shuffle-combining.ll
    M llvm/test/CodeGen/X86/vector-shuffle-v1.ll
    M llvm/test/CodeGen/X86/vector-trunc-math.ll
    M llvm/test/CodeGen/X86/vector-trunc-packus.ll
    M llvm/test/CodeGen/X86/vector-trunc-ssat.ll
    M llvm/test/CodeGen/X86/vector-trunc-usat.ll
    M llvm/test/CodeGen/X86/vector-trunc.ll
    M llvm/test/CodeGen/X86/vector-zext.ll

  Log Message:
  -----------
  [X86] Split FeatureFastVariableShuffle tuning into Lane-Crossing and Per-Lane variants

Currently, X86 backend only has a global one-size-fits-all `FeatureFastVariableShuffle` feature,
which controls profitability of both the cross-lane and per-lane variable shuffles.
I guess, this has been fine so far.

But at least on AMD Zen 3, while per-line variable shuffles (e.g. `VPSHUFB`)
are as fast as as shuffles with fixed/immediate mask,
while lane-crossing shuffles, e.g. `VPERMPS` is performing worse.

So to get the benefits of variable-mask shuffles, but not the drawbacks of lane-crossing shuffles,
as suggested by @RKSimon, split the feature flag into two.

Differential Revision: https://reviews.llvm.org/D103274


  Commit: a3b8695bf5927f0a43d295dfdfeafeef4da022ea
      https://github.com/llvm/llvm-project/commit/a3b8695bf5927f0a43d295dfdfeafeef4da022ea
  Author: Roman Lebedev <lebedev.ri at gmail.com>
  Date:   2021-06-01 (Tue, 01 Jun 2021)

  Changed paths:
    M llvm/lib/Target/X86/X86.td

  Log Message:
  -----------
  [X86] AMD Zen 3 has fast variable per-lane shuffles

... but lane-crossing shuffles are slow.


Compare: https://github.com/llvm/llvm-project/compare/41d7909368be...a3b8695bf592