[LLVMdev] Weird msan problem

[Evgeniy Stepanov]

How is ccall() implemented? If it manually sets up a stack frame, then
it also needs to store argument shadow values in paramtls.

I don't think there is an overflow, unless you have a _lot_ of
arguments in a function call.

On Sun, Feb 2, 2014 at 9:26 AM, Keno Fischer
<kfischer at college.harvard.edu> wrote:
> Also, I was looking at the instrumented LLVM code and I noticed that the
> LLVM global variable is generated with size 1000, while
> kMsanParamTlsSizeInWords == 100, could there be some sort of overflow
> happening?
>
>
> On Sat, Feb 1, 2014 at 6:44 PM, Keno Fischer <kfischer at college.harvard.edu>
> wrote:
>>
>> I have verified that both TLS implementations indeed find the same area of
>> memory. Anything else I could look for?
>>
>>
>> On Tue, Jan 28, 2014 at 4:28 PM, Keno Fischer
>> <kfischer at college.harvard.edu> wrote:
>>>
>>> Yes, both JIT code and the native runtime are instrumented. I am under
>>> the impressions that the the C library should guarantee that from the way
>>> the relocations are implemented as long as both native and JITed code are on
>>> the same thread (but I will verify this and report back).
>>>
>>>
>>> On Tue, Jan 28, 2014 at 2:41 AM, Evgeniy Stepanov
>>> <eugeni.stepanov at gmail.com> wrote:
>>>>
>>>> I assume there are transitions between JITted code and native helper
>>>> functions. How are you handling them? Are native functions
>>>> MSan-instrumented?
>>>> MSan is passing shadow across function calls in TLS slots. Does your
>>>> TLS implementation guarantee that accesses to __msan_param_tls from
>>>> JITted and from native code map to the same memory?
>>>>
>>>>
>>>> On Mon, Jan 27, 2014 at 11:36 PM, Evgeniy Stepanov
>>>> <eugeni.stepanov at gmail.com> wrote:
>>>> > This is really cool. I've not heard of anyone using MSan with MSJIT
>>>> > before.
>>>> >
>>>> >
>>>> > On Mon, Jan 27, 2014 at 7:44 PM, Keno Fischer
>>>> > <kfischer at college.harvard.edu> wrote:
>>>> >> Hello everybody,
>>>> >>
>>>> >> I've run into some strange behavior with memory sanitizer that I
>>>> >> can't
>>>> >> explain and hope somebody with more knowledge of the implementation
>>>> >> would be
>>>> >> able to help me out or at least point me into the right direction.
>>>> >>
>>>> >> For background, I'm using memory sanitizer to check Julia
>>>> >> (julialang.org),
>>>> >> which uses (or at least will once I track down a few bugs) MCJIT for
>>>> >> the
>>>> >> code compilation. So far I have rebuilt the runtime and all
>>>> >> dependencies
>>>> >> (including LLVM, libcxx, etc.) with memory sanitizer enabled and
>>>> >> added the
>>>> >> instrumentation pass in the appropriate place in the julia code
>>>> >> generator.
>>>> >>
>>>> >> I'm now going through the usual bootstrap which basically loads the
>>>> >> standard
>>>> >> library and compiles it, does inference, etc. This works fine for
>>>> >> several
>>>> >> hours (this is usually much faster - by which I mean several hundred
>>>> >> time -
>>>> >> I suspect the issue is with MCJIT having to process a ton more
>>>> >> relocations
>>>> >> and code and being inefficient at it, but I can't prove that). That's
>>>> >> not
>>>> >> the issue however. Eventually, I get
>>>> >>
>>>> >> ==17150== WARNING: MemorySanitizer: use-of-uninitialized-value
>>>> >>     #0 0x7f417cea3189 in bitvector_any1
>>>> >> /home/kfischer/julia-san/src/support/bitvector.c:177
>>>> >> [ snip ]
>>>> >>
>>>> >>   Uninitialized value was created by a heap allocation
>>>> >>     #0 0x7f41815de543 in __interceptor_malloc
>>>> >>
>>>> >> /home/kfischer/julia-san/deps/llvm-svn/projects/compiler-rt/lib/msan/msan_interceptors.cc:854
>>>> >>     #1 0x7f417cc7d7f1 in alloc_big
>>>> >> /home/kfischer/julia-san/src/gc.c:355
>>>> >> [snip]
>>>> >>
>>>> >> Now, by going through it in the debugger, I see
>>>> >>
>>>> >> (gdb) f 3
>>>> >> #3  0x00007f417cea318a in bitvector_any1 (b=0x60c000607240,
>>>> >> b at entry=<optimized out>, offs=0, offs at entry=<optimized out>,
>>>> >> nbits=256,
>>>> >> nbits at entry=<optimized out>)
>>>> >>     at bitvector.c:177
>>>> >> 177         if ((b[0] & mask) != 0) return 1;
>>>> >> (gdb) p __msan_print_shadow(&b,8)
>>>> >> ff ff ff ff ff ff ff ff
>>>> >>  o: 3f0010a6  o: 80007666
>>>> >>
>>>> >> which seems to indicate that the local variable b has uninitialized
>>>> >> data.
>>>> >> I'm having a hard time believing that though, since if I look at the
>>>> >> functions before it, the place where it's coming from is initialized:
>>>> >>
>>>> >> #4  0x00007f41755208a8 in julia_isempty248 ()
>>>> >> #5  0x00007f417c163e3d in jl_apply (f=0x606000984d60,
>>>> >> f at entry=<optimized
>>>> >> out>, args=0x7fff9132da20, args at entry=<optimized out>, nargs=1,
>>>> >>     nargs at entry=<optimized out>) at ./julia.h:1043
>>>> >>
>>>> >> (here's the code of that julia function for reference)
>>>> >>
>>>> >> isempty(s::IntSet) =
>>>> >>     !s.fill1s && ccall(:bitvector_any1, Uint32, (Ptr{Uint32}, Uint64,
>>>> >> Uint64), s.bits, 0, s.limit)==0
>>>> >>
>>>> >> Looking at where that value is coming from:
>>>> >>
>>>> >> (gdb) f 5
>>>> >> #5  0x00007f417c163e3d in jl_apply (f=0x606000984d60,
>>>> >> f at entry=<optimized
>>>> >> out>, args=0x7fff9132da20, args at entry=<optimized out>, nargs=1,
>>>> >>     nargs at entry=<optimized out>) at ./julia.h:1043
>>>> >> 1043        return f->fptr((jl_value_t*)f, args, nargs);
>>>> >> (gdb) p ((jl_array_t*)((void**)args[0])[1])->data
>>>> >> $43 = (void *) 0x60c000607240
>>>> >> (gdb) p __msan_print_shadow(((jl_array_t*)((void**)args[0])[1]),0x30)
>>>> >> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>> >> 00 00
>>>> >> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
>>>> >>  o: d800496  o: d800496  o: d800496  o: d800496  o: d800496  o:
>>>> >> d800496  o:
>>>> >> d800496  o: d800496  o: d800496  o: d800496  o: d800496  o: d800496
>>>> >>
>>>> >> There are no uninitialized values to be seen anywhere and the `b`
>>>> >> value
>>>> >> isn't touched before that line, so I'm a little stumped.
>>>> >>
>>>> >> One note I should make is that I did have to implement TLS support
>>>> >> myself in
>>>> >> MCJIT for this to work (I'll upstream the patch soon), so I may have
>>>> >> made a
>>>> >> mistake, but I haven't found anything wrong yet. If nothing looks
>>>> >> unusual,
>>>> >> I'd also appreciate pointers on what to look for in the TLS
>>>> >> variables.
>>>> >>
>>>> >> Thank you for your help,
>>>> >> Keno
>>>> >>
>>>> >>
>>>> >> _______________________________________________
>>>> >> LLVM Developers mailing list
>>>> >> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>> >> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>> >>
>>>
>>>
>>
>