[LLVMdev] Porting ASAN to new platform

[Ryan Govostes]

+ llvmdev

On Feb 18, 2014, at 2:55 AM, Alexander Potapenko <glider at google.com> wrote:

> Hi Ryan,
> 
> (CCing Kostya and Evgeniy)
> 
> On Feb 15, 2014 10:49 AM, "Ryan Govostes" <rzg at apple.com> wrote:
>> 
>> Hi Alexander,
>> 
>> I spent some time trying to get ASAN running on ARM64, but I’m not sure what the necessary changes are. (Note I’m on r199351 which is a little old. Just trying to get the basic steps down.)
>> 
>> 
>> - I defined a Mapping.Offset in llvm/lib/Transforms/Instrumentation/AddressSanitizer.cpp’s getShadowMapping() —— but I don’t know what the value should be
> Mapping.Offset must be the same as SHADOW_OFFSET in asan_mapping.h.
> This is the address at which the shadow memory range starts (the
> shadow occupies 1/8 of address space, 16Tb by default on 64-bit
> systems). I'd try to attach vmmap (or gdb, or whatever) to an existing
> non-ASan process (if iOS allows that) and look at the memory layout on
> ARM64.

I still have ASAN_FLEXIBLE_MAPPING_AND_OFFSET defined so I did not need to specify SHADOW_OFFSET. This was removed somewhat recently on trunk, I think. I’ll try to update.

> 
>> - In compiler-rt/lib/asan/asan_mapping.h, I defined kLowMemBeg to 0x100000000ULL which I understand is the correct lower bound on 64-bit iOS.
>> 
>> - In compiler-rt/lib/sanitizer_common/sanitizer_posix.cc, I made GetMaxVirtualAddress() return (0x1a0000000ULL - 1) which I understand is the correct upper bound on 64-bit iOS.
> So you've only 2 gigabytes (0x100000000 to 0x1a0000000) of memory
> available to the user? That's not much, and ASan is gonna take 256Mb
> of that space for shadow, plus some for the allocator.

I believe that is accurate.

> 
>> - In compiler-rt/lib/asan/asan_mac.cc, I redefined GetPcSpBp to return the ARM pc, sp, and lr registers.
>> 
>> - I made numerous hacks to the build scripts to basically build the OS X libclang_rt.asan_osx_dynamic.dylib for iOS instead. Maybe there are ARM-specific build settings used for the existing Android support that I should also use?
> You should take a look at how libclang_rt.asan_iossim_dynamic.dylib is
> built, I think that's closer to your needs. There might be some issues
> related to Xcode sysroots which I've resolved when porting to iossim.
> Actually, the support for iOS is now limited to iossim, and I haven't
> even tried to build ASan for iOS, because we didn't need to yet.
> That's why I haven't looked into the possible issues on iOS yet. My
> biggest concern is that the function interceptors (based on dyld
> interposing) may not work correctly, because they require re-execing
> the process with DYLD_INSERT_LIBRARIES. Do you know if that's at all
> possible?

I believe iOS does support DYLD_INSERT_LIBRARIES in some cases, such as for debugging with libgmalloc.

>> 
>> 
>> Beyond setting [ memory low, shadow offset, memory high ], what other platform-specific changes might I need to make? I expect maybe to have to write some new interceptors, but so far I’m stuck on just initializing ASAN.
> Nothing comes to mind, let us try to figure out what's wrong with startup first.
>> 
>> Right now, with the shadow offset as 0x110000000ULL, dyld dies by jumping into shadow memory while trying to resolve pthread_create. This doesn’t make too much sense to me, as the dyld implementations are effectively the same on OS X and iOS.
> The implementations can be the same, but IIUC the address space
> available to the user is much smaller on iOS, so we may be hitting
> some addresses used by dyld.
> There's a check in __asan_init() for existing Mach-O sections mapping
> into the shadow range, but if dyld allocates some memory in the shadow
> space using mmap or malloc we won't notice that.
> Or perhaps someone is loading code after __asan_init() has mapped the shadow?
> Is it possible to step over the program using a debugger?

Looking at vmmap of a non-ASAN process as you suggest, it does look like /usr/lib/dyld is loaded at ~0x120000000 which is inside the shadow map I chose. I’ll look for a bigger unused gap.

It looks like I might also need to set kAllocatorSpace and kAllocatorSize in asan_allocator2.cc, as the offset and size are well outside the bounds of what I have available. Any guidance on choosing appropriate values?

Thanks for your help.

Ryan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4792 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20140218/4d5bdfe3/attachment.bin>