[LLVMdev] radr://12777299, "potential pthread/eh bug exposed by libsanitizer"

[Jack Howarth]

On Tue, Dec 04, 2012 at 10:36:18AM -0800, Alexander Potapenko wrote:
> Currently the replacement of allocation routines is based on creating
> a new malloc zone and a new CFAllocator (because the allocator
> replacement is done later than it could be, we must have both). This
> makes us depend on CoreFoundation to call CFAllocatorSetDefault.
> Because of some bugs in CF which start firing after
> CFAllocatorSetDefault, we have to add several hacks to circumvent the
> effects of those bugs.
> The current hypothesis is that using dylib interposition for memory
> allocation routines (instead of playing with malloc zones) will allow
> us to remove the CF dependency together with all the hacks. I'm trying
> to check if this is true.
> The current plan is to have dynamic runtime for both iOS (for which we
> don't have any other option) and Mac OS.

Alexander,
    Out of curiosity, how do these CF bugs show up in a backtrace? For instance,
the additional darwin-specific failures I see with -fsanitize=address in the FSF g++
testsuite typically appear as simple segfaults such as with...

FAIL: g++.dg/abi/covariant3.C -std=c++98 execution test
FAIL: g++.dg/abi/covariant3.C -std=c++11 execution test

that backtraces as...

Starting program: /sw/src/fink.build/gcc48-4.8.0-1000/darwin_objdir/gcc/testsuite/g++/covariant3.exe 
Reading symbols for shared libraries +++++................................. done

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0x000010002000068a
0x00000001000015d0 in c1::c1 (this=0x100003d08, __vtt_parm=0x100003450) at /sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20121204/gcc/testsuite/g++.dg/abi/covariant3.C:10
10	struct c1 : virtual c0 {
(gdb) bt
#0  0x00000001000015d0 in c1::c1 (this=0x100003d08, __vtt_parm=0x100003450) at /sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20121204/gcc/testsuite/g++.dg/abi/covariant3.C:10
#1  0x000000010000168c in c18::c18 (this=0x100003d00) at /sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20121204/gcc/testsuite/g++.dg/abi/covariant3.C:29
#2  0x000000010000144c in __static_initialization_and_destruction_0 (__initialize_p=1, __priority=65535) at /sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20121204/gcc/testsuite/g++.dg/abi/covariant3.C:42
#3  0x0000000100001461 in _GLOBAL__sub_I_covariant3.C () at /sw/src/fink.build/gcc48-4.8.0-1000/gcc-4.8-20121204/gcc/testsuite/g++.dg/abi/covariant3.C:85
#4  0x00007fff5fc13378 in __dyld__ZN16ImageLoaderMachO18doModInitFunctionsERKN11ImageLoader11LinkContextE ()
#5  0x00007fff5fc13762 in __dyld__ZN16ImageLoaderMachO16doInitializationERKN11ImageLoader11LinkContextE ()
#6  0x00007fff5fc1006e in __dyld__ZN11ImageLoader23recursiveInitializationERKNS_11LinkContextEjRNS_21InitializerTimingListE ()
#7  0x00007fff5fc0feba in __dyld__ZN11ImageLoader15runInitializersERKNS_11LinkContextERNS_21InitializerTimingListE ()
#8  0x00007fff5fc01fc0 in __dyld__ZN4dyld24initializeMainExecutableEv ()
#9  0x00007fff5fc05b04 in __dyld__ZN4dyld5_mainEPK12macho_headermiPPKcS5_S5_Pm ()
#10 0x00007fff5fc01397 in __dyld__ZN13dyldbootstrap5startEPK12macho_headeriPPKclS2_Pm ()
#11 0x00007fff5fc0105e in __dyld__dyld_start ()

Does this look like those same CF bugs?
            Jack
ps Hopefully Apple will levearage the current dynamic asan implementation to smoke out all the potential
CF bugs so they can be fixed for darwin13. Nick, I could open a radar with stand-alone testcases generated
with -static-libasan -fsanitize=address from the g++ testsuite for that purpose if it would help.

> 
> On Tue, Dec 4, 2012 at 10:17 AM, Jack Howarth <howarth at bromo.med.uc.edu> wrote:
> > On Tue, Dec 04, 2012 at 09:46:09AM -0800, Alexander Potapenko wrote:
> >> +kledzik at apple.com
> >> The dynamic runtime is using dylib interposition (google for
> >> "__DATA,__interpose).
> >> If I'm understanding correctly (Nick, can you please confirm this?)
> >> this allows to interpose the function regardless of the two-level
> >> namespace.
> >> The support for dynamic runtime in ASan is almost there. But the new
> >> interposition method has revealed some issues with the allocator which
> >> were corked here and there before. Most of those are caused by a
> >> CoreFoundation dependency, which I'm trying to eliminate now.
> >>
> >
> > Alexander,
> >    Are you trying to eliminate the CoreFoundation dependency or the
> > issues it exposed in the allocator? I am also curious if these
> > issues could be related to the observation that libasan with the
> > mac interpose function support still shows 323 FSF g++ testsuite
> > failures compared to only 107 on x86_64 Fedora 15 linux? I planned
> > on trying to find some of those test cases which fail on both
> > FSF g++ and clang++ with the dyanmic runtime under darwin but
> > not under linux so I could open an llvm bugzilla on those.
> >               Jack
> >
> >>
> >> On Mon, Dec 3, 2012 at 8:50 PM, Rafael Espíndola
> >> <rafael.espindola at gmail.com> wrote:
> >> > On 30 November 2012 13:32, Alexander Potapenko <glider at google.com> wrote:
> >> >> No, we are not going to use mach_inject. This isn't portable and may
> >> >> be even harder to set up than mach_override.
> >> >> The new ASan runtime will use the dylib interposition and will in fact
> >> >> require DYLD_INSERT_LIBRARIES to work. However ASan already handles it
> >> >> correctly itself: if the corresponding env var is missing the app is
> >> >> just re-execed.
> >> >> Dylib interposition is supported by Apple and should work on iOS as
> >> >> well as Mac OS. It will also probably simplify hooking the memory
> >> >> allocations in ASan, which is now very tricky.
> >> >
> >> > This is interesting! I had some difficulties with mach_override myself
> >> > in firefox. Don't you have to disable the two-level namespace to be
> >> > able to override the functions you want? What currently blocks using
> >> > DYLD_INSERT_LIBRARIES instead of mach_override?
> >> >
> >> > Cheers,
> >> > Rafael
> >>
> >>
> >>
> >> --
> >> Alexander Potapenko
> >> Software Engineer
> >> Google Moscow
> 
> 
> 
> -- 
> Alexander Potapenko
> Software Engineer
> Google Moscow