[cfe-dev] clang feature request: integer overflow check in operator new

[Török Edwin]

On 03/23/2010 06:39 PM, David Chisnall wrote:
> On 23 Mar 2010, at 15:17, Felix von Leitner wrote:
> 
>> I have a humble feature request for clang, now that it officially
>> supports C++: integer overflow check in operator new.
>>
>> Basically, in code like this:
>>
>>   int* foo=new int[somevalue];
>>
>> the compiler does an implicit somevalue*sizeof(int) and passes that
>> value to operator new.  If that multiplication overflows, this is a
>> security vulnerability.  The Microsoft solution is to use the overflow
>> flag on x86 to set the resulting value to (size_t)-1, which will then
>> make operator new fail.  There are caveats to this, obviously, but it is
>> an important step to make and it cannot be done without compiler help.
> 
> 
> This looks like it should be relatively easy to add.  Line 444 of CGExprCXX.cpp just needs changing to emit a call to the overflow-checking intrinsic (see EmitOverflowCheckedBinOp() in CGExprScalar.cpp), and then a little extra handling for the case where it overflows.  I'm not sure what the effect of passing (size_t)max to new in GNU libstdc++ is - perhaps the best thing would be to bypass the new entirely and have it return 0 in overflow?
> 

new should throw an exception if allocation fails, unless you use the
nothrow variant that returns NULL.

if you return NULL you may introduce a new vulnerability (null dereference).

Best regards,
--Edwin