[PATCH] D38546: [ConstantFolding] Avoid assert when folding ptrtoint of vectorized GEP

[Bjorn Pettersson via Phabricator via llvm-commits]

bjope added inline comments.


================
Comment at: test/Analysis/ConstantFolding/cast-vector.ll:10
+; CHECK-NEXT:  entry:
+; CHECK-NEXT:    ret <2 x i16> ptrtoint (<2 x i32*> getelementptr ([10 x i32], [10 x i32]* null, <2 x i64> zeroinitializer, <2 x i64> <i64 5, i64 7>) to <2 x i16>)
+;
----------------
filcab wrote:
> This is invalid IR (per langref). the `ptrtoint` constant expression only takes a pointer type argument:
> 
> > ptrtoint (CST to TYPE)
> > Convert a pointer typed constant to the corresponding integer constant. TYPE must be an integer type. CST must be of pointer type. The CST value is zero extended, truncated, or unchanged to make it fit in TYPE.
> 
> It seems to me that one of the following is the best fix:
>   - Fix whatever is creating a `ptrtoint` constant expression with a vector argument (make sure to also add an assert to make sure it doesn't happen again)
>   - Make `ptrtoint` of vectors a valid constant expression, which involves changing the langref and making sure these kinds of folds are valid.
> 
> 
> 
@filcab: Are you sure?

AFAIK this is valid according to verifiers. And it seems to work.
The examples for both ptrtoint and inttoptr in the "Instruction Reference" part of the langref includes examples with vectors. For ptrtoint the version of the langref that I've been looking at says:

> The ‘ptrtoint‘ instruction takes a value to cast, which must be a value of type pointer or a vector of pointers, and a type to cast it to ty2, which must be an integer or a vector of integers type.

I assume that you quoted the description from the "Constant Expressions" section of the langref. 
Are you saying that there is a limitaiton for constant expressions, and that the Instruction Reference part of the langref only is valid for non-constant expressions?

Besides, you say that I should "fix whatever is creating a ptrtoint constant expression with a vector argument". My patch is about bailing out from ConstantFoldCastInstruction without doing any folding.  Basically leaving the ptrtoint as it was in the input (which follows the example from the Instruction Reference part of the langref.... as the arguments isn't constant folded I guess it can't be seen as a constant expression...).


https://reviews.llvm.org/D38546