[llvm-dev] Runtime interception: design problem

[Pierre Gagelin via llvm-dev]

On 26 May 2016 at 16:46, Mehdi Amini <mehdi.amini at apple.com> wrote:

>
> > On May 25, 2016, at 6:05 AM, Pierre Gagelin via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
> >
> > Hi everyone,
> >
> > I am having troubles but this shouldn't be hard to solve for many people
> here. I am beginning a runtime feature for the BoundsChecking pass and I
> want to replace the libc malloc&free. I followed the design of
> AddressSanitizer (Asan) and tried to use the INTERCEPTOR macro from the
> interception.h file of compiler-rt library.
> >
> > Here is the problem. The file I modify (BoundsCheking.cpp) is in
> lib/Transforms/Instrumentation/ and I can't include properly interception.h
> (which is in projects/compiler-rt/lib/interception/).
>
> Something does not make sense to me here: lib/Transforms/... is about
> stuff that will transform/generate code, it does not contain code that will
> be part of the final binary. So this transform may generates calls to your
> runtime, but should not need the runtime to operate.
>
>
Yes I know this does not make sense. That's why I asked for some help to
clear the design (in my head and also in the code).

As I said further in the message, when I analyzed ASan I found the runtime
could be called by instrumented code through the function __asan_init() but
I haven't managed to reproduce the mechanism. Though I haven't spend much
time on it as I thought maybe another solution existed or someone could
explain it to me easily.

But as you said it could be the regular way to do it I looked a bit deeper
and it appears to be this pattern:
  - the doInitialization of the FunctionPass (which as more access than the
runOnFunction) calls ModuleUtils createSanitizerCtorAndInitFunctions
function to get two pointers (constructor_func, init_func)
  - stores it in the FunctionPass structure
  - runOnFunction may then call init_func via the IRBuilder when needed.
Am I right? I'll try to use it the same way.


> --
> Mehdi
>

Thank you Mehdi!
Pierre

>
>
>
> > I looked at the CMakeLists.txt and how other files included
> interception.h but they are all from compiler-rt lib directory.
> >
> > I assume this is normal as runtime project should be developped under
> the correct directory but I don't see how this should be designed in order
> that the code optimized by BoundsCheking Pass uses my own malloc&free
> functions.
> >
> > I spent quite some time on Asan runtime code and found that runtime
> could be initialized with a call to __asan_init() (function defined in
> asan_rtl.cc) from the instrumented code directly but I don't know if it's
> the only way to do it or how to reproduce it...
> >
> > So to resume, I want to make my own malloc and free function to be
> called by the code I instrument with BoundsChecking. Any suggestion is
> welcome =)
> >
> > Thanks,
> > Pierre
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160527/d8a081ae/attachment.html>