[PATCH] D45655: [InstCombine][RFC] Add tests for mask canonicalization in masked merge

[Roman Lebedev via Phabricator via llvm-commits]

lebedev.ri created this revision.
lebedev.ri added reviewers: spatel, craig.topper.

Masked merge has a pattern of: `((x ^ y) & M) ^ y`.
Once PR6773 <https://bugs.llvm.org/show_bug.cgi?id=6773> fold is done, that pattern will be produced.

But, if the mask is constant, there is no difference between `((x ^ y) & M) ^ y` and `((x ^ y) & ~M) ^ x` and `((x ^ y) & ~(~M)) ^ y`,
I think the constant mask should be canonicalized.

I propose the following:

- if this is scalar:
  - if inverted mask has **less** bits set (Population count) than the original mask, **DO** invert
  - if zero-extended value of inverted mask is **smaller** than of the original mask, **DO** invert
  - else **DON'T** invert.
- if it is vector: *. apply that ^ scalar predicate to each non-`undef` element.
    - count number of elements where the predicate is `true`.
    - for `undef` elements the predicate is `false`.
  - if there are **more** elements where the predicate is `true`, than elements where predicate is `false`, **DO** invert
  - else **DON'T** invert.

These are strict **less**/**more** comparisons, not **less or equal**/**more or equal**.
This ensures that the canonicalization is stable, and won't be undone / won't cause cycles.

https://rise4fun.com/Alive/eox

Thoughts?


Repository:
  rL LLVM

https://reviews.llvm.org/D45655

Files:
  lib/Transforms/InstCombine/InstCombineAndOrXor.cpp
  test/Transforms/InstCombine/invert-mask-in-masked-merge-scalar.ll
  test/Transforms/InstCombine/invert-mask-in-masked-merge-vector.ll

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D45655.142515.patch
Type: text/x-patch
Size: 20534 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180414/9d0ecaa3/attachment.bin>