<div dir="ltr">Looks like when you materialize the stores, you should check the size of the the store and emit an appropriate amount of stores to the origin shadow (or just a memset intrinsic?). </div><div class="gmail_extra">
<br><br><div class="gmail_quote">On Wed, Feb 5, 2014 at 2:13 PM, Keno Fischer <span dir="ltr"><<a href="mailto:kfischer@college.harvard.edu" target="_blank">kfischer@college.harvard.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">The @entry stuff is just a gdb artifact. I've been tracking this back a little further, and it seems there's at least an origin propagation problem:<div><br></div><div><div>(gdb) list</div><div>281 goto value_to_pointer_error;</div>
<div>282 }</div><div>283 else {</div><div>284 if (jl_is_cpointer_type(jvt) && jl_tparam0(jvt) == jt) {</div><div>285 void *ptr = jl_unbox_voidpointer(v);</div><div>286 assert(__msan_test_shadow(&ptr,sizeof(void**)) == -1);</div>
<div>287 return (void*)ptr;</div><div>288 }</div><div>289 }</div><div>290</div><div>(gdb) p __msan_print_shadow(v,16)</div><div>00 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff</div><div>
o: 40000e23 o: 40000e23 o: 40000e23 o: 40000e23</div><div>$21 = void</div><div>(gdb) p __msan_print_shadow(&ptr,8)</div><div class="im"><div>ff ff ff ff ff ff ff ff</div></div><div> o: 40000e23 o: 80007614</div>
<div>$22 = void</div></div>
<div><br></div><div>Notice the origin of the lower bits is off. Any idea as to why?</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">On Mon, Feb 3, 2014 at 2:52 AM, Evgeniy Stepanov <span dir="ltr"><<a href="mailto:eugeni.stepanov@gmail.com" target="_blank">eugeni.stepanov@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">The code for ccall looks right. Sounds like you have a very small<br>
range of instructions where an uninitialized value appear. You could<br>
try debugging at asm level. Shadow for b should be passed at offset 0<br>
in __msan_param_tls.<br>
<br>
MSan could propagate shadow through arithmetic and even some logic<br>
operations (like select). It could be that b is clean on function<br>
entry, but then something uninitialized gets mixed in.<br>
<br>
Also, what is this @entry stuff? It looks like bitvector_any1 has 6<br>
arguments, but your ccall snippet only sets up tls for 3.<br>
<br>
<br>
On Sun, Feb 2, 2014 at 10:39 PM, Keno Fischer<br>
<div><div><<a href="mailto:kfischer@college.harvard.edu" target="_blank">kfischer@college.harvard.edu</a>> wrote:<br>
> I don't think we're doing anything unusual. Here's the relevant snippet from<br>
> the generated IR for ccall (after instrumentation):<br>
><br>
> ccall: ; preds = %pass6, %113<br>
> %114 = phi i8* [ %110, %113 ], [ %108, %pass6 ]<br>
> %115 = inttoptr i64 %17 to i64*<br>
> %116 = bitcast i8* %69 to i32*, !dbg !7<br>
> %117 = bitcast i8* %114 to i32 (i32*, i64, i64)*, !dbg !7<br>
> store i64 %_msret13, i64* getelementptr inbounds ([1000 x i64]*<br>
> @__msan_param_tls, i64 0, i64 0), align 8, !dbg !7<br>
> store i32 %70, i32* getelementptr inbounds ([1000 x i32]*<br>
> @__msan_param_origin_tls, i64 0, i64 0), align 4, !dbg !7<br>
> store i64 0, i64* inttoptr (i64 add (i64 ptrtoint ([1000 x i64]*<br>
> @__msan_param_tls to i64), i64 8) to i64*), align 8, !dbg !7<br>
> store i32 0, i32* inttoptr (i64 add (i64 ptrtoint ([1000 x i32]*<br>
> @__msan_param_origin_tls to i64), i64 8) to i32*), align 4, !dbg !7<br>
> store i64 0, i64* inttoptr (i64 add (i64 ptrtoint ([1000 x i64]*<br>
> @__msan_param_tls to i64), i64 16) to i64*), align 8, !dbg !7<br>
> store i32 0, i32* inttoptr (i64 add (i64 ptrtoint ([1000 x i32]*<br>
> @__msan_param_origin_tls to i64), i64 16) to i32*), align 4, !dbg !7<br>
> store i32 0, i32* bitcast ([8 x i64]* @__msan_retval_tls to i32*), align<br>
> 8, !dbg !7<br>
> %118 = call i32 %117(i32* %116, i64 %87, i64 %107), !dbg !7, !julia_type<br>
> !9<br>
><br>
><br>
><br>
> On Sun, Feb 2, 2014 at 6:18 AM, Evgeniy Stepanov <<a href="mailto:eugeni.stepanov@gmail.com" target="_blank">eugeni.stepanov@gmail.com</a>><br>
> wrote:<br>
>><br>
>> How is ccall() implemented? If it manually sets up a stack frame, then<br>
>> it also needs to store argument shadow values in paramtls.<br>
>><br>
>> I don't think there is an overflow, unless you have a _lot_ of<br>
>> arguments in a function call.<br>
>><br>
>> On Sun, Feb 2, 2014 at 9:26 AM, Keno Fischer<br>
>> <<a href="mailto:kfischer@college.harvard.edu" target="_blank">kfischer@college.harvard.edu</a>> wrote:<br>
>> > Also, I was looking at the instrumented LLVM code and I noticed that the<br>
>> > LLVM global variable is generated with size 1000, while<br>
>> > kMsanParamTlsSizeInWords == 100, could there be some sort of overflow<br>
>> > happening?<br>
>> ><br>
>> ><br>
>> > On Sat, Feb 1, 2014 at 6:44 PM, Keno Fischer<br>
>> > <<a href="mailto:kfischer@college.harvard.edu" target="_blank">kfischer@college.harvard.edu</a>><br>
>> > wrote:<br>
>> >><br>
>> >> I have verified that both TLS implementations indeed find the same area<br>
>> >> of<br>
>> >> memory. Anything else I could look for?<br>
>> >><br>
>> >><br>
>> >> On Tue, Jan 28, 2014 at 4:28 PM, Keno Fischer<br>
>> >> <<a href="mailto:kfischer@college.harvard.edu" target="_blank">kfischer@college.harvard.edu</a>> wrote:<br>
>> >>><br>
>> >>> Yes, both JIT code and the native runtime are instrumented. I am under<br>
>> >>> the impressions that the the C library should guarantee that from the<br>
>> >>> way<br>
>> >>> the relocations are implemented as long as both native and JITed code<br>
>> >>> are on<br>
>> >>> the same thread (but I will verify this and report back).<br>
>> >>><br>
>> >>><br>
>> >>> On Tue, Jan 28, 2014 at 2:41 AM, Evgeniy Stepanov<br>
>> >>> <<a href="mailto:eugeni.stepanov@gmail.com" target="_blank">eugeni.stepanov@gmail.com</a>> wrote:<br>
>> >>>><br>
>> >>>> I assume there are transitions between JITted code and native helper<br>
>> >>>> functions. How are you handling them? Are native functions<br>
>> >>>> MSan-instrumented?<br>
>> >>>> MSan is passing shadow across function calls in TLS slots. Does your<br>
>> >>>> TLS implementation guarantee that accesses to __msan_param_tls from<br>
>> >>>> JITted and from native code map to the same memory?<br>
>> >>>><br>
>> >>>><br>
>> >>>> On Mon, Jan 27, 2014 at 11:36 PM, Evgeniy Stepanov<br>
>> >>>> <<a href="mailto:eugeni.stepanov@gmail.com" target="_blank">eugeni.stepanov@gmail.com</a>> wrote:<br>
>> >>>> > This is really cool. I've not heard of anyone using MSan with MSJIT<br>
>> >>>> > before.<br>
>> >>>> ><br>
>> >>>> ><br>
>> >>>> > On Mon, Jan 27, 2014 at 7:44 PM, Keno Fischer<br>
>> >>>> > <<a href="mailto:kfischer@college.harvard.edu" target="_blank">kfischer@college.harvard.edu</a>> wrote:<br>
>> >>>> >> Hello everybody,<br>
>> >>>> >><br>
>> >>>> >> I've run into some strange behavior with memory sanitizer that I<br>
>> >>>> >> can't<br>
>> >>>> >> explain and hope somebody with more knowledge of the<br>
>> >>>> >> implementation<br>
>> >>>> >> would be<br>
>> >>>> >> able to help me out or at least point me into the right direction.<br>
>> >>>> >><br>
>> >>>> >> For background, I'm using memory sanitizer to check Julia<br>
>> >>>> >> (<a href="http://julialang.org" target="_blank">julialang.org</a>),<br>
>> >>>> >> which uses (or at least will once I track down a few bugs) MCJIT<br>
>> >>>> >> for<br>
>> >>>> >> the<br>
>> >>>> >> code compilation. So far I have rebuilt the runtime and all<br>
>> >>>> >> dependencies<br>
>> >>>> >> (including LLVM, libcxx, etc.) with memory sanitizer enabled and<br>
>> >>>> >> added the<br>
>> >>>> >> instrumentation pass in the appropriate place in the julia code<br>
>> >>>> >> generator.<br>
>> >>>> >><br>
>> >>>> >> I'm now going through the usual bootstrap which basically loads<br>
>> >>>> >> the<br>
>> >>>> >> standard<br>
>> >>>> >> library and compiles it, does inference, etc. This works fine for<br>
>> >>>> >> several<br>
>> >>>> >> hours (this is usually much faster - by which I mean several<br>
>> >>>> >> hundred<br>
>> >>>> >> time -<br>
>> >>>> >> I suspect the issue is with MCJIT having to process a ton more<br>
>> >>>> >> relocations<br>
>> >>>> >> and code and being inefficient at it, but I can't prove that).<br>
>> >>>> >> That's<br>
>> >>>> >> not<br>
>> >>>> >> the issue however. Eventually, I get<br>
>> >>>> >><br>
>> >>>> >> ==17150== WARNING: MemorySanitizer: use-of-uninitialized-value<br>
>> >>>> >> #0 0x7f417cea3189 in bitvector_any1<br>
>> >>>> >> /home/kfischer/julia-san/src/support/bitvector.c:177<br>
>> >>>> >> [ snip ]<br>
>> >>>> >><br>
>> >>>> >> Uninitialized value was created by a heap allocation<br>
>> >>>> >> #0 0x7f41815de543 in __interceptor_malloc<br>
>> >>>> >><br>
>> >>>> >><br>
>> >>>> >> /home/kfischer/julia-san/deps/llvm-svn/projects/compiler-rt/lib/msan/msan_interceptors.cc:854<br>
>> >>>> >> #1 0x7f417cc7d7f1 in alloc_big<br>
>> >>>> >> /home/kfischer/julia-san/src/gc.c:355<br>
>> >>>> >> [snip]<br>
>> >>>> >><br>
>> >>>> >> Now, by going through it in the debugger, I see<br>
>> >>>> >><br>
>> >>>> >> (gdb) f 3<br>
>> >>>> >> #3 0x00007f417cea318a in bitvector_any1 (b=0x60c000607240,<br>
>> >>>> >> b@entry=<optimized out>, offs=0, offs@entry=<optimized out>,<br>
>> >>>> >> nbits=256,<br>
>> >>>> >> nbits@entry=<optimized out>)<br>
>> >>>> >> at bitvector.c:177<br>
>> >>>> >> 177 if ((b[0] & mask) != 0) return 1;<br>
>> >>>> >> (gdb) p __msan_print_shadow(&b,8)<br>
>> >>>> >> ff ff ff ff ff ff ff ff<br>
>> >>>> >> o: 3f0010a6 o: 80007666<br>
>> >>>> >><br>
>> >>>> >> which seems to indicate that the local variable b has<br>
>> >>>> >> uninitialized<br>
>> >>>> >> data.<br>
>> >>>> >> I'm having a hard time believing that though, since if I look at<br>
>> >>>> >> the<br>
>> >>>> >> functions before it, the place where it's coming from is<br>
>> >>>> >> initialized:<br>
>> >>>> >><br>
>> >>>> >> #4 0x00007f41755208a8 in julia_isempty248 ()<br>
>> >>>> >> #5 0x00007f417c163e3d in jl_apply (f=0x606000984d60,<br>
>> >>>> >> f@entry=<optimized<br>
>> >>>> >> out>, args=0x7fff9132da20, args@entry=<optimized out>, nargs=1,<br>
>> >>>> >> nargs@entry=<optimized out>) at ./julia.h:1043<br>
>> >>>> >><br>
>> >>>> >> (here's the code of that julia function for reference)<br>
>> >>>> >><br>
>> >>>> >> isempty(s::IntSet) =<br>
>> >>>> >> !s.fill1s && ccall(:bitvector_any1, Uint32, (Ptr{Uint32},<br>
>> >>>> >> Uint64,<br>
>> >>>> >> Uint64), s.bits, 0, s.limit)==0<br>
>> >>>> >><br>
>> >>>> >> Looking at where that value is coming from:<br>
>> >>>> >><br>
>> >>>> >> (gdb) f 5<br>
>> >>>> >> #5 0x00007f417c163e3d in jl_apply (f=0x606000984d60,<br>
>> >>>> >> f@entry=<optimized<br>
>> >>>> >> out>, args=0x7fff9132da20, args@entry=<optimized out>, nargs=1,<br>
>> >>>> >> nargs@entry=<optimized out>) at ./julia.h:1043<br>
>> >>>> >> 1043 return f->fptr((jl_value_t*)f, args, nargs);<br>
>> >>>> >> (gdb) p ((jl_array_t*)((void**)args[0])[1])->data<br>
>> >>>> >> $43 = (void *) 0x60c000607240<br>
>> >>>> >> (gdb) p<br>
>> >>>> >> __msan_print_shadow(((jl_array_t*)((void**)args[0])[1]),0x30)<br>
>> >>>> >> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00<br>
>> >>>> >> 00<br>
>> >>>> >> 00 00<br>
>> >>>> >> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00<br>
>> >>>> >> 00<br>
>> >>>> >> o: d800496 o: d800496 o: d800496 o: d800496 o: d800496 o:<br>
>> >>>> >> d800496 o:<br>
>> >>>> >> d800496 o: d800496 o: d800496 o: d800496 o: d800496 o:<br>
>> >>>> >> d800496<br>
>> >>>> >><br>
>> >>>> >> There are no uninitialized values to be seen anywhere and the `b`<br>
>> >>>> >> value<br>
>> >>>> >> isn't touched before that line, so I'm a little stumped.<br>
>> >>>> >><br>
>> >>>> >> One note I should make is that I did have to implement TLS support<br>
>> >>>> >> myself in<br>
>> >>>> >> MCJIT for this to work (I'll upstream the patch soon), so I may<br>
>> >>>> >> have<br>
>> >>>> >> made a<br>
>> >>>> >> mistake, but I haven't found anything wrong yet. If nothing looks<br>
>> >>>> >> unusual,<br>
>> >>>> >> I'd also appreciate pointers on what to look for in the TLS<br>
>> >>>> >> variables.<br>
>> >>>> >><br>
>> >>>> >> Thank you for your help,<br>
>> >>>> >> Keno<br>
>> >>>> >><br>
>> >>>> >><br>
>> >>>> >> _______________________________________________<br>
>> >>>> >> LLVM Developers mailing list<br>
>> >>>> >> <a href="mailto:LLVMdev@cs.uiuc.edu" target="_blank">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
>> >>>> >> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
>> >>>> >><br>
>> >>><br>
>> >>><br>
>> >><br>
>> ><br>
><br>
><br>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div>