[PATCH] DataFlowSanitizer: Add a design doc paragraph on checking ABI consistency.

[Peter Collingbourne]

I think that would work in the majority of cases.  But consider the case where an instrumented-ABI function A calls a native-ABI function B which then calls an instrumented-ABI function C once.  C would then test and reset the flag making it appear from A's perspective that B was instrumented-ABI.

Perhaps it would be better to store the expected callee function pointer in the thread local variable instead of 1, and have the callee check for its own pointer.

Kostya Serebryany <kcc at google.com> wrote:

>
>  LGTM
>
>  An idea for run-time checks. There are two failure modes:
>  1. Calling a native-ABI function as if instrumented-ABI
>  2. Calling a instrumented-ABI function as if native-ABI
>
>We can have a thread-local integer is_instrumented_abi_call, which is
>zero almost all the time.
>  Before instrumented-ABI call we increment it.
>In instrumented-ABI function we check that it's 1 (otherwise trap) and
>set it to 0.
>  After instrumented-ABI call we check that it's 0 (otherwise trap).
>
>  This will catch both failure modes.
>
>http://llvm-reviews.chandlerc.com/D1443
>
>ARCANIST PROJECT
>  clang

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20130820/f41c1402/attachment.html>