[llvm-dev] Limited use types in the back end

[Matt Arsenault via llvm-dev]

> On Jan 27, 2020, at 11:59, Nemanja Ivanovic via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> I am hoping that someone can offer advice on a somewhat unusual issue that I am facing with the SDAG. Namely, I am trying to implement some custom operations that do very specific things on multiple registers at a time. The operations themselves will simply be intrinsics since there are no equivalent operations in IR/SDAG. However, handling the types seems rather tricky.
> 
> One approach I tried is to create a register class that has the wide registers with proper sub registers and then telling the SDAG that the correspondingly wide type can go into those registers. While this works, it has a very unfortunate side effect that the type legalizer leaves any node with such a type untouched and I have to mark all operations as non-legal (mostly Expand).
> For example, I could say that the type v8i64 can go into these registers and then I can use the type for my intrinsics. However, the type legalizer will leave all nodes with this result/operand type alone which is not at all what I want.
> 
> Then I tried the opposite approach - just custom lower only specific nodes that have this result type and let the type legalizer handle all the others normally. This works quite well except if I want to expose those custom instructions through inline asm. The DAG builder complains if I am trying to assign one of these wide registers to a value with the wide type because it assumes that the wide value will need to be broken up.
> 
> I suppose I could define a new type for the IR/SDAG and use it, but that seems like a super pervasive approach.
> 
> So either direction I go in seems to have a major drawback.
> 

I’m missing some details on what your constraints are. You have an operation on contiguous v8i64 registers, and not some number of multiple i64 registers? If you really have the vector operation on the vector width, adding the legal type is the most honest strategy and probably your best bet despite the pain induced by needing to expand all of the vector operations. If you really wanted to trick the legalizer and hack out the type with ReplaceNodeResults, I would expect it’s theoretically possible to hack up the inline asm handling to deal with this, but I probably wouldn’t recommend this. This is the kind fo problem that’s avoided in GlobalISel, since the concept of type legalization is gone.

-Matt