[llvm] [SelectionDAG] Do not build illegal nodes with users (PR #108573)

[via llvm-commits]

ErikHogeman wrote:

Hi, this code is triggering some asserts in our downstream target. I have tried to reproduce it with a test that can run on an upstream target, but it has been very difficult.

What happens is basically this:
1. The code in FoldConstantArithmetic creates a new SETCC node with an illegal i1 type.
2. It tries to legalize this by inserting an integer extension, I believe the intention being here that this will be constant folded whenever we keep this node.
3. The result is not constant, so the function returns, leaving the node with illegal type still existing in the DAG.
4. Another node is processed by the DAG combiner, and this node gets simplified to another node which is an operand to the new illegal node. This causes the illegal node to be added to the DAG combiner worklist.
5. When the illegal node gets processed, it's not removed because it has a user, which is the integer extension that was added to legalize it.
6. This triggers an assert in SelectionDAGLegalize::LegalizeOp, which expects types to be legalized (because this is running after the type legalizer).
7. This fix avoids the issue by not adding any user of the illegal node in case it wasn't folded to a constant, which makes sure it gets removed in case it somehow ends up again in the DAG combiner worklist.

The tricky part here is to get another node optimized after the i1 node is created, to trigger it to get inserted into the worklist again before it gets removed later. I have some IR where this happens on our downstream target, but I haven't managed to create IR that makes this happen on an upstream target.

So at this point, I only have a code change, no test. I realize that is not ideal, but I still thought I should open this request to see what opinion people have about it. I'm also happy to give it another try to write a test, if someone has a good idea of anything I can try.

https://github.com/llvm/llvm-project/pull/108573