[llvm] [AMDGPU] Rework dot4 signedness checks (PR #68757)

[Jeffrey Byrnes via llvm-commits]

================
@@ -13056,32 +13023,74 @@ static bool isMul(const SDValue Op) {
           Opcode == AMDGPUISD::MUL_I24);
 }
 
-static std::optional<bool> checkSignedness(const SDValue &N,
-                                           ByteProvider<SDValue> &Src0,
-                                           ByteProvider<SDValue> &Src1) {
+static std::optional<bool>
+checkSignedness(const SDValue &N, ByteProvider<SDValue> &Src0,
----------------
jrbyrnes wrote:

computeKnownBits on the top multiply does not provide sufficient information to determine dot lowering behavior for combine for all permutations.

For example, existing tests produce the following data:

```
S0SignBit, 	S1SignBit, 	Multiply KnownSignBit 	 |           Should Use Dot, 	Signed Dot
0 				?				? 			 |			0 				X
?				0 				? 			 |			0 				X
?				? 				? 			 |			1 				1

```
The S0SignBit and S1SignBit are sufficient to determine the behavior of combine in all cases, and we see that different expected combine behaviors map to the same sign bit of the KnownBits from top multiply. The logic of checkSignedness deviates from computeKnownBits: in checkSignedness, we know certain properties of the operands (only but the least-significant 8 bits are extension bits) and are able to do more special case reasoning.

https://github.com/llvm/llvm-project/pull/68757