[llvm] [BasicTTI] Use getTypeLegalizationCost to generalize vector cast cost. (PR #107303)

[Philip Reames via llvm-commits]

https://github.com/preames commented:

Meta comment after banging my head against this same problem for a bit.

I don't think the approach taken here works.  The fundamental problem is that when widening one of the types involved in a cast, we have to *scale* the other type in the same manner.  Legalizing the two types independently does not give the right result pair.  (As pointed out by the inline comment.)

I think we need to implement the entire type legalization state machine here.  This is complicated, but it's the only way I can convince myself we get the right results.  Note this is a different state machine (because of the coupling mentioned above) than the normal getTypeLegalizationCost which (implicitly) assumes input and output types match.  

I implemented a subset of this (for fixed vectors only), and a couple of my test cases from https://github.com/llvm/llvm-project/pull/110164 no longer crashed and produced sanish results.  Extending the WIP I have to scalable vectors and the other legalization step kinds would be non-trivial.  

https://github.com/llvm/llvm-project/pull/107303