[cfe-dev] clang feature request: integer overflow check in operator new

[Howard Hinnant]

On Mar 23, 2010, at 2:12 PM, Török Edwin wrote:

> On 03/23/2010 06:39 PM, David Chisnall wrote:
>> On 23 Mar 2010, at 15:17, Felix von Leitner wrote:
>> 
>>> I have a humble feature request for clang, now that it officially
>>> supports C++: integer overflow check in operator new.
>>> 
>>> Basically, in code like this:
>>> 
>>>  int* foo=new int[somevalue];
>>> 
>>> the compiler does an implicit somevalue*sizeof(int) and passes that
>>> value to operator new.  If that multiplication overflows, this is a
>>> security vulnerability.  The Microsoft solution is to use the overflow
>>> flag on x86 to set the resulting value to (size_t)-1, which will then
>>> make operator new fail.  There are caveats to this, obviously, but it is
>>> an important step to make and it cannot be done without compiler help.
>> 
>> 
>> This looks like it should be relatively easy to add.  Line 444 of CGExprCXX.cpp just needs changing to emit a call to the overflow-checking intrinsic (see EmitOverflowCheckedBinOp() in CGExprScalar.cpp), and then a little extra handling for the case where it overflows.  I'm not sure what the effect of passing (size_t)max to new in GNU libstdc++ is - perhaps the best thing would be to bypass the new entirely and have it return 0 in overflow?
>> 
> 
> new should throw an exception if allocation fails, unless you use the
> nothrow variant that returns NULL.
> 
> if you return NULL you may introduce a new vulnerability (null dereference).

C++0x says that the type of that thrown expression is std::bad_array_new_length.

-Howard