[LLVMdev] Changing pointer representation?

[Chris Lattner]

On Fri, 1 Dec 2006, Jules wrote:

> Having finally found some time to work on this project, I'm currently
> looking at mechanisms of augmenting LLVM to catch out-of-bounds pointer
> references.

ok
> I'm currently leaning towards the latter, primarily because it seems
> more general; in the end, I'm going to want at least x86 and x86-64
> support, and the former approach will mean I'll need to do the work
> twice for two different platforms.

This could work, but the transformation is tricky, particularly in the 
face of recursive types.  I suggest:

#3: change the CFE so that it lowers C pointers to your pair.  This should 
be relatively straight-forward, and solves the recursive type issue.

> It occurs to me that some of the people here have surely worked on this
> kind of thing before, and perhaps can relate some experiences of things
> that have either worked or not worked.  Am I doing anything stupid here?

I think the easiest thing to do is to change how the CFE expands the 
operations you care about.  This keeps the LLVM-level semantics the same 
as they are now (so all optzns will work, etc) and the bounds checks are 
exposed to the llvm optimizers, so they can be eliminated.

An alternative approach would be to do this entirely in the code 
generator, hiding all the action from the optimizers.  This would also 
work, but would be target-specific and would not let you do aggressive
optimziations of the bounds check code.

-Chris

-- 
http://nondot.org/sabre/
http://llvm.org/