[cfe-dev] [RFC] Extending and improving Clang's undefined behavior checking

[Chris Lattner]

On Aug 22, 2012, at 9:24 AM, John Criswell <criswell at illinois.edu> wrote:
> On 8/21/12 6:33 PM, Chris Lattner wrote:
>>> 1) Completeness of checks. There are integer undefined behaviors which -ftrapv and -fcatch-undefined-behavior don't catch, and there is almost no checking available for any other undefined behaviors outside of those and the ones caught by {Address,Thread,Memory} Sanitizer.
>> Yes.  Nuno's bounds checking work can also be pulled into this eventually, as could stack canaries and "fortify source".
> 
> IMHO, I don't think ad-hoc techniques like stack canaries are suitable for this particular application.  Stack canaries are better suited for run-time protection against stack buffer-overflow attacks (if they're suited for anything at all).  I don't think canaries really tell you where in the code the stack is being smashed.

The current implementation doesn't, but that's because its reporting mechanism is currently hard coded to "abort()".  It could (in principle, not saying this is important) be extended to report better source-level diagnostics.

> For memory-related undefined behaviors, I think it would make sense to have various "levels" of checks in which each level adds more overhead but checks more things accurately.  ASan would be a good first or second level; it finds invalid loads and stores and can catch some out-of-bounds array accesses and dangling pointers. Another level could be ASan with SAFECode's array checks and points-to set checking.  A final level could be something like SoftBound + CETS which provides real dangling pointer detection in addition to the previously mentioned checks.

Right.  Some would be on by default with -fcatch-undefined-behavior, some would be opt-inable with -fcatch-undefined-behavior=something-specific

-Chris