[cfe-dev] clang feature request: integer overflow check in operator new

[David Chisnall]

On 23 Mar 2010, at 15:17, Felix von Leitner wrote:

> I have a humble feature request for clang, now that it officially
> supports C++: integer overflow check in operator new.
> 
> Basically, in code like this:
> 
>   int* foo=new int[somevalue];
> 
> the compiler does an implicit somevalue*sizeof(int) and passes that
> value to operator new.  If that multiplication overflows, this is a
> security vulnerability.  The Microsoft solution is to use the overflow
> flag on x86 to set the resulting value to (size_t)-1, which will then
> make operator new fail.  There are caveats to this, obviously, but it is
> an important step to make and it cannot be done without compiler help.


This looks like it should be relatively easy to add.  Line 444 of CGExprCXX.cpp just needs changing to emit a call to the overflow-checking intrinsic (see EmitOverflowCheckedBinOp() in CGExprScalar.cpp), and then a little extra handling for the case where it overflows.  I'm not sure what the effect of passing (size_t)max to new in GNU libstdc++ is - perhaps the best thing would be to bypass the new entirely and have it return 0 in overflow?

David

-- Sent from my IBM 1620