[LLVMdev] LLVMdev Digest, Vol 89, Issue 60

Kostya Serebryany kcc at google.com
Thu Nov 24 22:37:52 PST 2011 

On Thu, Nov 24, 2011 at 7:45 PM, Vikram Adve <vadve at illinois.edu> wrote:

> Daniel, Kostya,
>
> We had a meeting with the Clang people (Chris, Doug, Ted) on Thursday
> before the llvmdev meeting about adding dynamic checking tools into Clang
> -- IOC for undefined integer behaviors, and SAFECode for memory safety.
>  SAFECode has similar goals to AddressSanitizer, though at least for now it
> has more checks,


[off topic, could you please educate me on the kinds of bugs that SAFECode
finds and asan does not? Maybe in a separate thread. ]




> but is slower.
>
> The main conclusion was that we should have a common run-time API for
> various tools to report errors.  This API can be shared by tools like IOC,
> SAFECode, and (we hope) Address Sanitizer.  The benefits are that users
> have to learn only one error reporting style; and the code base can use a
> common set of mechanisms to track and report execution information, so that
> improvements by any one project will benefit the other projects as well.
>
> We're happy to keep the API simple for now, and to wrap the SAFECode and
> IOC run-times to use it.  Let us know if you'd like to help define and use
> this API.
>

I would like to have some level of sharing between all these tools (add
ThreadSanitizer to the mix).
But as I just discussed with John Regehr in a separate thread,  the tools
(asan and IOC) are quite different and sharing is limited.

Let me describe what asan (AddressSanitizer) and tsan (ThreadSanitizer)
need and also my understanding of IOC's needs. Then we'll see how much
sharing is possible between these and SAFECode.

asan:
  -- unwind stack on malloc/free (should be fast, libunwind  is too slow,
currently uses custom unwinder based on frame pointers)
  -- symbolize stack. Currently uses external script and add2line, which
sucks. Maybe could reuse some of lldb code.
  -- intercept functions (malloc, pthread, etc)
  -- small set of utility functions (e.g. printf, mmap, strlen) that does
not use libc.
  -- the functions that are called on error take strictly one parameter and
never return (for speed and code size). Another option is to report an
error by causing SIGILL using a call-free instruction sequence (5% faster).
  -- dies on first error.
  -- reports to stderr. Someone may need to customize it, but we did not
have such requests yet.
  -- report style: see example at
http://code.google.com/p/address-sanitizer/wiki/AddressSanitizer#Introduction .
The format of the reports is similar to that of tsan, which in turn is
similar to that of valgrind. I am reluctant to change the format because
quite a few existing users of asan and tsan (and memcheck) rely on it.
  -- no suppressions
  -- supports blacklist file (and we also may need something like
__attribute__(no-address-sanitizer)
-- major user-visible controls: unwind depth, redzone size  (passed via
env. var ASAN_OPTIONS)

 tsan:
    --  keep shadow call stack (unwinding is too slow)
    -- symbolize stack (same problem as with asan). gcc-based variant uses
libbfd, but I don't like that code. valgrind-based variant uses valgrind's
symbolizer. (ditto for PIN-based)
   -- intercept functions (malloc, pthread, etc)
   -- small set of utility functions (e.g. printf, mmap, strlen) that does
not use libc.
   -- suppressions (syntax is similar to valgrind's)
    --  ignore file (aka blacklist of functions/files)
    -- reports to stderr, may change to another stream.
    --  report style: see above. Examples are here:
http://code.google.com/p/data-race-test/wiki/UnderstandingThreadSanitizerReports#The_report
    -- major user-visible controls: stack depth, sensitivity options,
resource usage options. compiler-based variant gets the options via an env.
var.

IOC:
  - does not need unwinding (does not show the stack)
  - does not need symbolization (adds all info at compile time to the error
reporting call. questionable from performance point of view)
  - may need suppression, removal of duplicates, etc
  - (John, what else is important here?)


Currently, I can see that SAFECode can share these items with asan/tsan:
- unwinding
- symbolizing
- function interception mechanism
- option parsing
- maybe style of error reports
- maybe some low-level code (e.g. asan's tiny libc replacement)

--kcc


> --Vikram
> Professor, Computer Science
> University of Illinois at Urbana-Champaign
> http://llvm.org/~vadve
>
>
>
>
>
> On Nov 24, 2011, at 12:00 PM, <llvmdev-request at cs.uiuc.edu>
>  wrote:
>
> > Date: Thu, 24 Nov 2011 10:27:35 -0500
> > From: Daniel Dunbar <daniel at zuster.org>
> > Subject: Re: [LLVMdev] AddressSanitizer run-time in
> >       tools/clang/runtime/compiler-rt
> > To: Kostya Serebryany <kcc at google.com>
> > Cc: llvmdev at cs.uiuc.edu
> > Message-ID:
> >       <
> CAEU8z697Ey0_mMdLt8swJVUR+x0K9Rvays8NW7a3f1R3RbxQKg at mail.gmail.com>
> > Content-Type: text/plain; charset=ISO-8859-1
> >
> > Quick answers, I'm on txgiving break this week and not doing any real
> > work, but I will be doing more compiler-rt work when I get back
> > (initially focused at getting profile libs to come from compiler-rt on
> > Linux et al).
> >
> > On Wed, Nov 16, 2011 at 9:24 PM, Kostya Serebryany <kcc at google.com>
> wrote:
> >> Hi Daniel,
> >> Chris suggested to talk to you about committing the AddressSanitizer
> (asan)
> >> run-time into the llvm tree (llvm-project/compiler-rt).
> >> Questions:
> >> - What is the preferred name for the directory? (asan? libasan?
> >> address_sanitizer? AdressSanitizer?)
> >
> > I don't care. lib/asan seems perfectly reasonable to me, with a README
> > explaining what the module is for.
> >
> >> - Should the asan run-time use cmake, or just make, or what? The build
> is a
> >> bit tricky, especially for tests. We currently use make.
> >
> > The only build system I care about for compiler-rt is the one in make.
> > It's quite a crazy set up, although there is a reason for it to be as
> > complicated as it is. I can help (and/or) do the asan integration into
> > the compiler-rt make build.
> >
> >> - How would you suggest to do the code review?
> >> ? ?The code of the run-time is ~5
> >> KLOC.?
> http://code.google.com/p/address-sanitizer/source/browse/#svn%2Ftrunk%2Fasan
> >> ? ?The Apple-specific part may make some Apple experts cry (or maybe
> not).
> >> ? ?The code uses google's coding style, which is similar, but not
> equivalent
> >> to the LLVM's one. We check it using cpplint before commits.
> >
> > I personally would like to see it be in LLVM style, but the rest of
> > compiler-rt isn't really that way, so I don't think this is a blocker.
> >
> > Unless anyone objects, I think we should just bring the code in as is
> > so that ASAN works, and if people want to do more thorough review that
> > can happen post commit.
> >
> >> ? ?LLVM license is used.
> >> ? ?The tests are ~2.5 KLOC; most of the tests require the clang compiler
> >> with -faddress-sanitizer switch.
> >> ? ?If possible, I'd prefer not to review the patch in a usual way (too
> big),
> >> but instead have the comments to the files
> >> in?
> http://code.google.com/p/address-sanitizer/source/browse/#svn%2Ftrunk%2Fasan
> >> ? ?Or, alternatively, commit everything as is and then iterate over
> >> individual files.
> >> - The run-time library uses a couple of third_party files (BSD license)
> to
> >> parse process mapping.
> >> ? W/o these files asan will work, but will not produce symbolizable
> traces.
> >> ? Can we include these files as as in a separate directory? (They may be
> >> useful for other tools as well)
> >
> > Let's try to include the bits with extra licenses in subdirectories of
> > lib/asan. If someone wants to reuse them later we can find a different
> > way to factor things, I don't see a reason to worry about it now.
> >
> >> ??
> http://code.google.com/p/address-sanitizer/source/browse/trunk/asan/sysinfo.cc
> >> - Yet another piece of third-party code (mit license) is used for
> >> Apple-specific work (function overriding). Same question as above apply.
> >>
> >> ?
> http://code.google.com/p/address-sanitizer/source/browse/trunk/asan/mach_override.c
> >> - test-specific: can I rely on gtest being installed? (fresh version is
> >> required).
> >
> > Compiler-rt doesn't currently have a very good test set up. You'll
> > probably have to find a way to shoehorn this in for your own testing
> > initially, but we can try and work out a way to integrate it more
> > properly...
> >
> > - Daniel
> >
> >> Thanks,
> >> --kcc
> >>
> >>
> >
>
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20111124/2005db9a/attachment.html>

More information about the llvm-dev mailing list