[PATCH] D38210: [ubsan] Port the function sanitizer to C

[Vedant Kumar via Phabricator via cfe-commits]

vsk created this revision.

The function sanitizer relies on RTTI to check callee types, but this
scheme doesn't work well in languages without the ODR.

This patch introduces a simple, best-effort function type encoding
which can be used when RTTI isn't available. In this scheme, function
types are encoded within 32 bits. The return type and all parameter
types are recorded using a 3-bit encoding. Zero is a special value in
the 3-bit encoding which means "there is either no type here OR any type
would be permissible here".

This scheme allows false negatives, but not false positives. It's simple
and does not require any changes to the instrumentation.

Testing: I've found some minor issues with the new check, and no FPs.

https://trac.ffmpeg.org/ticket/6685
https://github.com/openssl/openssl/issues/4413


https://reviews.llvm.org/D38210

Files:
  docs/UndefinedBehaviorSanitizer.rst
  lib/CodeGen/CGExpr.cpp
  lib/CodeGen/CodeGenFunction.cpp
  lib/CodeGen/CodeGenModule.cpp
  lib/CodeGen/CodeGenModule.h
  test/CodeGen/sanitize-function-calls.c

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D38210.116449.patch
Type: text/x-patch
Size: 9189 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170925/da6d7b92/attachment-0001.bin>