[cfe-dev] Question about Clang/LLVM addresssanitizer

Tue Sep 20 20:41:07 PDT 2016

On Tue, Sep 20, 2016 at 6:47 PM, ZhaoKang <zhaokang at mail.tsinghua.edu.cn>
wrote:

> You mentioned that 'Some global settings in the system may for some
> reason reject such huge mappings.'
> And I want to know which kinds of settings may have this effect?
>

Something related to memory overcommit.

> And did any other users encounter this kind of issue before?
>

Not on Linux.

> And the output error message said that '==41651==ERROR: AddressSanitizer
> failed to allocate 0xdfff0001000 (15392894357504) bytes at address
> 2008fff7000 (errno: 12)'
> Does this feature need allocate so large memory?
>

Yes.
You may read
https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm for
details.

My suggestion: build a single binary with asan and run it under strace on
both systems that you have,
then compare the output.

Can you give me access to a machine where this fails?
Is this OS variant available on GCE (https://cloud.google.com/compute/)?

> or this feature dump the error message?
>
> Thanks a lot!
>
>
> -----原始邮件-----
> *发件人:* "Kostya Serebryany" <kcc at google.com>
> *发送时间:* 2016-09-21 00:35:55 (星期三)
> *收件人:* ZhaoKang <zhaokang at mail.tsinghua.edu.cn>
> *抄送:* "Clang Dev" <cfe-dev at lists.llvm.org>
> *主题:* Re: Re: Re: [cfe-dev] Question about Clang/LLVM addresssanitizer
>
>
>
>
> On Mon, Sep 19, 2016 at 9:41 PM, ZhaoKang <zhaokang at mail.tsinghua.edu.cn>
> wrote:
>
>> Yes, if just a small case like following also failed:
>>
>> int main() {
>>
>>     int *mem = new int[100];
>>
>>     for (unsigned i = 0; i<=100; i++ ) {
>>
>>         mem[i] = i;
>>
>>     }
>>
>> return 0;
>>
>> }
>>
>>
>>
>> And I build like this:
>>
>> fangqing@[xcoapps57 small_case]$ clang++  -fsanitize=address
>> -fsanitize=undefined main.cpp
>>
>> fangqing@[xcoapps57 small_case]$ ll -h a.out
>>
>> -rwxr-x--- 1 fangqing hd 9.6M Sep 19 21:24 a.out
>>
>
> 9.6M is a bit unusual, on my box I get a 2.4M binary,
> but not entirely bad. asan links lots of stuff into a binary.
>
>
>>
>>
>> fangqing@[xcoapps57 small_case]$ ASAN_OPTIONS=verbosity=1 ./a.out
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_printf'
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_sprintf'
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_snprintf'
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_fprintf'
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_vprintf'
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
>>
>> ==41651==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
>>
>> ==41651==AddressSanitizer: failed to intercept 'process_vm_readv'
>>
>> ==41651==AddressSanitizer: failed to intercept 'process_vm_writev'
>>
>> ==41651==AddressSanitizer: libc interceptors initialized
>>
>> || `[0x10007fff8000, 0x7fffffffffff]` || HighMem    ||
>>
>> || `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
>>
>> || `[0x005000000000, 0x02008fff6fff]` || ShadowGap3 ||
>>
>> || `[0x003000000000, 0x004fffffffff]` || MidMem     ||
>>
>> || `[0x000a7fff8000, 0x002fffffffff]` || ShadowGap2 ||
>>
>> || `[0x00067fff8000, 0x000a7fff7fff]` || MidShadow  ||
>>
>> || `[0x00008fff7000, 0x00067fff7fff]` || ShadowGap  ||
>>
>> || `[0x00007fff8000, 0x00008fff6fff]` || LowShadow  ||
>>
>> || `[0x000000000000, 0x00007fff7fff]` || LowMem     ||
>>
>> MemToShadow(shadow): 0x00008fff7000 0x000091ff6dff 0x004091ff6e00
>> 0x02008fff6fff 0x00014fff7000 0x0001cfff6fff
>>
>> redzone=16
>>
>> max_redzone=2048
>>
>> quarantine_size_mb=256M
>>
>> malloc_context_size=30
>>
>> SHADOW_SCALE: 3
>>
>> SHADOW_GRANULARITY: 8
>>
>> SHADOW_OFFSET: 0x7fff8000
>>
>> ==41651==ERROR: AddressSanitizer failed to allocate 0xdfff0001000
>> (15392894357504) bytes at address 2008fff7000 (errno: 12)
>>
>> ==41651==ReserveShadowMemoryRange failed while trying to map
>> 0xdfff0001000 bytes. Perhaps you're using ulimit -v
>>
>> Aborted (core dumped)
>>
>>
>>
>> If compile this small case without address-sanitizer, the generated
>> executable file is just 6.8K, and if with this feature, the executable
>> file is 9.6M.
>>
>> fangqing@[xcoapps57 small_case]$ clang++  main.cpp
>>
>> fangqing@[xcoapps57 small_case]$ ll -h a.out
>>
>> -rwxr-x--- 1 fangqing hd 6.8K Sep 19 21:28 a.out
>>
>>
>>
>> *And the most important is that if we move this case to another Linux
>> server, the address-sanitizer feature works fine.*
>>
>> Now the failed case is built on following RedHat server:
>>
>> fangqing@[xcoapps57 small_case]$ uname -a
>>
>> Linux xcoapps57 2.6.32-504.el6.x86_64 #1 SMP Tue Sep 16 01:56:35 EDT 2014
>> x86_64 x86_64 x86_64 GNU/Linux
>>
>> fangqing@[xcoapps57 small_case]$ cat /etc/redhat-release
>>
>> Red Hat Enterprise Linux Workstation release 6.6 (Santiago)
>>
>>
>>
>> So is it related to different platform? And what does this feature depend
>> on?
>>
>
> asan depends on the ability to mmap a huge range of address space
> with MAP_NORESERVE.
> Some global settings in the system may for some reason reject such huge
> mappings.
> But I have not seen this before.
>
> --kcc
>
>
>>
>>
>> Thanks a lot!
>>
>>
>> ----------
>> *From:* "Kostya Serebryany" <kcc at google.com>
>> *Time:* 2016-09-20 10:36:16 (星期二)
>> *To:* ZhaoKang <zhaokang at mail.tsinghua.edu.cn>
>> *cc:* "Clang Dev" <cfe-dev at lists.llvm.org>
>> *Title:* Re: Re: [cfe-dev] Question about Clang/LLVM addresssanitizer
>>
>>
>> Does this happen with any small application on your system, or only with
>> this (presumably big) one?
>> You will need to figure out why the asan's mmap fails here.
>> Best is to run the process under strace and see
>>   * are there any mmaps that intersect with this one
>>   * are there any syscalls that limit the address space (setrlimit)
>>
>> --kcc
>>
>>
>> On Mon, Sep 19, 2016 at 7:18 PM, ZhaoKang <zhaokang at mail.tsinghua.edu.cn>
>> wrote:
>>
>>> Dear Kostya,
>>>
>>> Thanks a lot for your reply!
>>> For the first question, we have dump out the info, please see the
>>> following:
>>> =============================================
>>>
>>> ASAN_OPTIONS=verbosity=1 ./csim.exe
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_printf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_sprintf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_snprintf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_fprintf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_vprintf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_vsprintf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_vsnprintf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept '__isoc99_vfprintf'
>>>
>>> ==25048==AddressSanitizer: failed to intercept 'process_vm_readv'
>>>
>>> ==25048==AddressSanitizer: failed to intercept 'process_vm_writev'
>>>
>>> ==25048==AddressSanitizer: libc interceptors initialized
>>>
>>> || `[0x10007fff8000, 0x7fffffffffff]` || HighMem    ||
>>>
>>> || `[0x02008fff7000, 0x10007fff7fff]` || HighShadow ||
>>>
>>> || `[0x005000000000, 0x02008fff6fff]` || ShadowGap3 ||
>>>
>>> || `[0x003000000000, 0x004fffffffff]` || MidMem     ||
>>>
>>> || `[0x000a7fff8000, 0x002fffffffff]` || ShadowGap2 ||
>>>
>>> || `[0x00067fff8000, 0x000a7fff7fff]` || MidShadow  ||
>>>
>>> || `[0x00008fff7000, 0x00067fff7fff]` || ShadowGap  ||
>>>
>>> || `[0x00007fff8000, 0x00008fff6fff]` || LowShadow  ||
>>>
>>> || `[0x000000000000, 0x00007fff7fff]` || LowMem     ||
>>>
>>> MemToShadow(shadow): 0x00008fff7000 0x000091ff6dff 0x004091ff6e00
>>> 0x02008fff6fff 0x00014fff7000 0x0001cfff6fff
>>>
>>> redzone=16
>>>
>>> max_redzone=2048
>>>
>>> quarantine_size_mb=256M
>>>
>>> malloc_context_size=30
>>>
>>> SHADOW_SCALE: 3
>>>
>>> SHADOW_GRANULARITY: 8
>>>
>>> SHADOW_OFFSET: 0x7fff8000
>>>
>>> ==25048==ERROR: AddressSanitizer failed to allocate 0xdfff0001000
>>> (15392894357504) bytes at address 2008fff7000 (errno: 12)
>>>
>>> ==25048==ReserveShadowMemoryRange failed while trying to map
>>> 0xdfff0001000 bytes. Perhaps you're using ulimit -v
>>>
>>> Aborted (core dumped)
>>>
>>>  =============================================
>>>
>>> Thanks!
>>>
>>> Kang
>>>
>>> ----------
>>> *From:* "Kostya Serebryany" <kcc at google.com>
>>> *Time:* 2016-09-09 05:50:03
>>> *Time:* ZhaoKang <zhaokang at mail.tsinghua.edu.cn>
>>> *CC:* "Clang Dev" <cfe-dev at lists.llvm.org>
>>> *Subject:* Re: [cfe-dev] Question about Clang/LLVM addresssanitizer
>>>
>>>
>>>
>>>
>>> On Thu, Sep 8, 2016 at 1:09 AM, ZhaoKang via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> Hello,
>>>>
>>>>
>>>>
>>>> We have one question about the clang compiler option:
>>>> -fsanitize=address. (We want to use the feature to detect potential bug in
>>>> out c++ design.)
>>>>
>>>> However, when using clang to compile our two cases with this option,
>>>> one case error out with following message:
>>>>
>>>>
>>>>
>>>> ==31183==ERROR: AddressSanitizer *failed to allocate 0x400000000*
>>>> (17179869184) bytes at address 67fff8000 (errno: 12)
>>>>
>>>> ==31183==ReserveShadowMemoryRange failed while trying to map
>>>> 0x400000000 bytes. Perhaps you're using ulimit –v
>>>>
>>>
>>> This happens at startup right?
>>> something causes asan to fail to allocate the shadow.
>>>
>>> please send the output of
>>>    ASAN_OPTIONS=verbosity=1 ./your-binary
>>>
>>> or, better, send the reproducer.
>>>
>>>
>>>>
>>>>
>>>> The other case error out with following message:
>>>>
>>>> ==30711==ERROR: AddressSanitizer: *stack-buffer-overflow* on address
>>>> 0x7fff8a931dcd at pc 0x000000861eec bp 0x7fff8a9303f0 sp 0x7fff8a9303e8
>>>>
>>>
>>> that smells like a real bug in your code.
>>>
>>>
>>>> READ of size 1 at 0x7fff8a931dcd thread T0
>>>>
>>>>     #0 0x861eeb in ap_private<8, false, true>::RType<32, true>::mult
>>>> ap_private<8, false, true>::operator*<32, true>(ap_private<32, true,
>>>> (32)<=(64)> const&) const (/wrk/xbj_vdi/fangqing/work/sp
>>>> rite/hls/BugSpray/crs/810730/hscale/solution1/csim/build/csi
>>>> m.exe+0x861eeb)
>>>>
>>>>     #1 0x7d717f in ap_private<8, false, (8)<=(64)>::RType<32,
>>>> true>::mult operator*<8, false>(ap_private<8, false, (8)<=(64)> const&,
>>>> int) (/wrk/xbj_vdi/fangqing/work/sprite/hls/BugSpray/crs/810730/h
>>>> scale/solution1/csim/build/csim.exe+0x7d717f)
>>>>
>>>>     …
>>>>
>>>>     #6 0x3099a1d9c3 in __libc_start_main (/lib64/libc.so.6+0x3099a1d9c3
>>>> )
>>>>
>>>>     #7 0x4c3648 in _start (/wrk/xbj_vdi/fangqing/work/sp
>>>> rite/hls/BugSpray/crs/810730/hscale/solution1/csim/build/csi
>>>> m.exe+0x4c3648)
>>>>
>>>>
>>>>
>>>> Address 0x7fff8a931dcd is located in stack of thread T0 at offset 1549
>>>> in frame
>>>>
>>>>     #0 0x78628f in hscale_core(hls::stream<HSC_MPIX_STRUCT>&,
>>>> ap_uint<4>, ap_uint<16>, ap_uint<16>, ap_uint<16>, ap_uint<32>, ap_uint<2>,
>>>> ap_int<16> (*) [8], hls::stream<HSC_MPIX_STRUCT>&)
>>>> (/wrk/xbj_vdi/fangqing/work/sprite/hls/BugSpray/crs/810730/h
>>>> scale/solution1/csim/build/csim.exe+0x78628f)
>>>>
>>>>
>>>>
>>>>   This frame has 215 object(s):
>>>>
>>>> [32, 33) 'RegSmplsPerClk'
>>>>
>>>> [48, 49) 'RegBitsPerCol'
>>>>
>>>> [64, 66) 'TotalLines'
>>>>
>>>> …
>>>>
>>>> [1552, 1553) 'ref.tmp65' <== Memory access at offset 1549 underflows
>>>> this variable
>>>>
>>>> [1568, 1569) 'ref.tmp68'
>>>>
>>>> …
>>>>
>>>>
>>>>
>>>> Both of them can be compiled successfully and run correctly when
>>>> compiled with clang without this addrsanitizer option. However both of them
>>>> failed when add this option.
>>>>
>>>> From the error message we can see AddressSanitizer need to allocate a
>>>> very large memory (about 16G byte) from heap memory pool (1st case),
>>>> or occupy large stack memory and cause stack-buffer-overflow (2nd
>>>> case). (ulimit –v shows unlimited)
>>>>
>>>> So our question is it is this feature’s shortcoming or there is
>>>> something wrong with our development environment?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>>>
>>>
>>>
>>>
>>>
>>
>>
>>
>>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160920/172b9a9e/attachment.html>