[llvm-dev] how to discriminate pointers in calling conventions

[David Chisnall via llvm-dev]

On 8 Mar 2018, at 23:54, Davis, Alan via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> I have a target whose calling conventions specify that pointer-typed arguments are passed in different registers than same-sized integers. It appears that in the SelectionDAGBuilder, arguments/formals with pointer type are lowered to the corresponding integer MVT (via this path: 
> SelectionDAGISel::LowerArguments
> llvm::ComputeValueVTs
> TargetLoweringBase::getValueType
> TargetLoweringBase::getPointerType
> )
> and from then on there is no way to discriminate them, for example in the CallingConv.td file and so on.
> 
> None of the functions in the above list are virtual and anyway it seems like a Bad Idea to just have getPointerType return iPTR or something like that since it’s used in many other contexts.
> 
> Anyone dealt with something like this? Any ideas?

We’ve dealt with this in CHERI by adding explicit pointer types to MVT and lowering IR pointers to these.  This is essential for us, because integers and pointers have different representations and are operated on by different instructions.

If your pointers have the same representation and are operated on in the same way as integers then you have a somewhat different problem.  It’s perhaps closer to platforms where a struct of two floats is handled differently from a _Complex(float).  These differences are, unfortunately, handled by informal (and, usually, undocumented) ad-hoc contracts between the front end and back end and often impede optimisations.  It would be nice to have a notion of ABI type carried through the IR to the back end, but that’s a lot of work for little short-term gain.

David