[cfe-dev] [RFC] Ternary operator with vector type predicate

[Min-Yih Hsu via cfe-dev]

Hi folks,

In clang’s vector extension, the ternary operator (?:) is able to select from two vector type values based on the boolean type predicate. For example:
```
typedef float float4 __attribute__((ext_vector_type(4)));

float4 foo(bool cond, float4 a, float4 b) {
  return cond? a : b;
}
```
However, clang currently doesn’t support vector type values as the predicates. So code shown below won’t compile:
```
float4 foo(float4 a, float4 b) {
  return a > b? a : b;
}
```
Which is essentially just a max function that can be implemented by hardware instructions (vmaxps in Intel AVX for example) efficiently.

I’m proposing to support vector type values as predicates in the ternary operator: If the predicate value has vector type, each of its element would be converted to bool before being used to select corresponding elements from the second or third operands. Note that this is exactly what GCC is doing right now [1] in their vector extensions.

On the (IR) codegen side, since the IR `select`, `icmp`, and `fcmp` instruction all support vector type values as the operands, I don’t think there are much trouble in it.

Here are the reasons and some advantages:
1. It provides more flexibility for programmers to do conditional selections on vectors. Especially for cases that can be optimized by hardware instructions like the motivated example above.

2. It will not break the current usages of ternary operator: If the predicate is a scalar value, it’s still sticking to the current model.

3. If the goal of our extensions is aiming to “support a broad range of GCC extensions”. Then the behavior should also be as consistent as possible.

I’ll try to come up with a patch for preview as soon as possible.
Also a slightly off-topic thing: In our document [2], the ternary operator has always been mis-typed as ‘:?’ Maybe we should fix it some times.

Thanks for the feedbacks in advance.
-Min

[1]: https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html <https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html>
[2]: https://clang.llvm.org/docs/LanguageExtensions.html#vectors-and-extended-vectors <https://clang.llvm.org/docs/LanguageExtensions.html#vectors-and-extended-vectors>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20200508/1d4e14a6/attachment.html>