<div dir="ltr">Ok... <div><br></div><div>The particular instance of the problem can be solved with this patch in my code:</div><div><br></div><div><div>+ IRB.SetInsertPoint(Ins);</div><div> IRB.SetCurrentDebugLocation(EntryLoc);</div><div>- IRB.SetInsertPoint(Ins);</div></div><div><br></div><div>(apparently, SetInsertPoint invalidates the previous call to SetCurrentDebugLocation)</div><div><br></div><div>But then there is another problem....</div><div><br></div><div><div>% cat dummy.c</div><div>void foo() {}</div></div><div><br></div><div>% clang -O -c -gmlt -fsanitize-coverage=func,trace-pc-guard -S dummy.c -o -<br></div><div><div><div>.LBB0_1:</div><div> .loc 1 1 0 # dummy.c:1:0</div><div> pushq %rax</div><div>.Lcfi0:</div><div> .cfi_def_cfa_offset 16</div><div> movl $.L__sancov_gen_, %edi</div><div> callq __sanitizer_cov_trace_pc_guard</div></div></div><div><br></div><div>% clang -O -c -gmlt -fsanitize-coverage=func,trace-pc-guard -S dummy.c -mllvm -use-unknown-locations -o -<br></div><div><br></div><div><div>.LBB0_1:</div><div> .loc 1 1 0 is_stmt 0 # dummy.c:1:0</div><div> pushq %rax</div><div><b> .loc 1 0 0 # :0:0</b></div><div>.Lcfi0:</div><div> .cfi_def_cfa_offset 16</div><div> .loc 1 1 0 is_stmt 1 # dummy.c:1:0</div><div> movl $.L__sancov_gen_, %edi</div><div> callq __sanitizer_cov_trace_pc_guard</div></div><div><br></div><div><br></div><div>Then, when I addr2line the resulting binary some of the instructions get this pesky "<b>.loc 1 0 0</b>" for some reason (did not investigate yet)<br></div><div><br></div><div>I am pretty sure that every particular problem like this can be solved with a simple patch, </div><div>but how do we find those problems before the users get upset enough to file a good bug report? </div><div><br></div><div><br></div><div>--kcc </div><div><br></div><div><br></div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Dec 1, 2016 at 4:16 PM, Robinson, Paul <span dir="ltr"><<a href="mailto:paul.robinson@sony.com" target="_blank">paul.robinson@sony.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div class="m_4766027178238925610WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">There is already –mllvm –use-unknown-locations which ought to trigger this. Don't need my patch.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">--paulr<u></u><u></u></span></p>
<p class="MsoNormal"><a name="m_4766027178238925610__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></a></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Kostya Serebryany [mailto:<a href="mailto:kcc@google.com" target="_blank">kcc@google.com</a>]
<br>
<b>Sent:</b> Thursday, December 01, 2016 4:08 PM</span></p><div><div class="h5"><br>
<b>To:</b> Robinson, Paul<br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] Libfuzzer depending on uninitialized debug info<u></u><u></u></div></div><p></p>
</div>
</div><div><div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Thu, Dec 1, 2016 at 3:37 PM, Robinson, Paul <<a href="mailto:paul.robinson@sony.com" target="_blank">paul.robinson@sony.com</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">It might be a wider problem than libfuzzer. I did want to raise the problem asap and libfuzzer is
something we know has the problem.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">If it came across as "libfuzzer is evil" that was not my intent, sorry!</span><u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal">No, no, I did not mean you implied that :) <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Just wanted to make sure everyone understand that this is not libFuzzer-specific. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Looking at lib/Transforms/<wbr>Instrumentation/<wbr>SanitizerCoverage.cpp:<u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal"> DebugLoc EntryLoc;<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> if (IsEntryBB) {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> if (auto SP = F.getSubprogram())<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> EntryLoc = DebugLoc::get(SP-><wbr>getScopeLine(), 0, SP);<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">...<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> } else {<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> EntryLoc = IP->getDebugLoc();<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> }<u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> IRBuilder<> IRB(&*IP);<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> IRB.SetCurrentDebugLocation(<wbr>EntryLoc);<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">So, using this I assumed that the newly generated instructions have proper debug info, <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">and so far it worked. <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">I wonder if you can re-commit your changes under a flag, off-by default, so that everyone interested can play with it? <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"><a name="m_4766027178238925610_m_-3330181366314494229__MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">--paulr</span></a><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Kostya Serebryany [mailto:<a href="mailto:kcc@google.com" target="_blank">kcc@google.com</a>]
<br>
<b>Sent:</b> Thursday, December 01, 2016 2:53 PM<br>
<b>To:</b> Robinson, Paul<br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] Libfuzzer depending on uninitialized debug info</span><u></u><u></u></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<p class="MsoNormal">On Thu, Dec 1, 2016 at 11:08 AM, Robinson, Paul via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal">TL;DR: LibFuzzer appears to depend on debug-info source locations for<br>
whatever IR instrumentation it uses; however, that instrumentation does<br>
not have proper source locations attached to it, leading to potentially<br>
incorrect reporting. The short-term fix is to make sure the debug info<br>
it needs is actually set up; the long-term fix is not to rely on debug<br>
info, because some optimizations will (correctly) erase it.<u></u><u></u></p>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Why is this libFuzzer-specific? <u></u><u></u></p>
</div>
<div>
<div>
<p class="MsoNormal">We were just [un]lucky to detect the problem early with one of the libFuzzer<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">tests that required debug info. <u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Any tool that needs debug info will suffer from the same problem. No? <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border:none;border-left:solid #cccccc 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p class="MsoNormal"><br>
The long version:<br>
<br>
When Clang generates IR with debug info, one thing it does is attach a<br>
source location to most IR instructions. This source location (at least<br>
in principle) is carried through optimizations, SelectionDAG, MachineIR,<br>
assembler source, and ultimately ends up in the "line table" in the<br>
object file. The line table describes a mapping from the virtual<br>
addresses of instructions to source locations, which is very useful to<br>
debuggers and other tools.<br>
<br>
Not all IR instructions have a source location attached to them. When<br>
that happens, no specific line-table record is emitted for any machine<br>
instruction produced from that IR instruction. In DWARF, that means you<br>
assume the instruction belongs to the same source location as the<br>
instruction that precedes it in memory.<br>
<br>
This is a problem when the first instruction in a machine-basic-block has<br>
no explicit source location, because it implicitly inherits the source<br>
location of the last instruction of the basic block that precedes it in<br>
memory. That means, the source location is entirely at the mercy of<br>
block layout and other optimizations.<br>
<br>
In effect, the source location for that instruction is UNINITIALIZED.<br>
<br>
In r288283, I committed a patch that explicitly initialized the line<br>
number for some instructions to line 0. The DWARF spec says that line 0<br>
means there is no specific source location for the instruction. Debuggers<br>
and other tools generally respond to this looking *forward* in the<br>
instruction stream to find the *next* instruction with an explicit non-0<br>
location, rather than backward to the *previous* instruction with an<br>
explicit location.<br>
<br>
This caused a libFuzzer test to fail, because it depended on seeing a<br>
real source location for something, and got line 0 instead. This tells<br>
me libFuzzer is depending on an uninitialized source location. Kostya<br>
backed out that patch for me, but we really want to have it for improved<br>
debugger single-stepping behavior.<br>
<br>
I am unclear on what instrumentation the fuzzer is using, although the<br>
instructions for building it suggest it's ASAN instrumentation. Whatever<br>
it is, either the instrumentation should use its own source-location<br>
information scheme, or it should initialize the debug info that it is<br>
depending on.<br>
<br>
Note that debug info is not necessarily reliable in the face of<br>
optimization. If two blocks with different source locations get merged,<br>
most likely the source location will be zeroed (and that's not my patch,<br>
that's optimization-specific behavior). Therefore, I would recommend<br>
that fuzzer/asan/whoever stop relying on debug info for source locations,<br>
if we want all that to work on optimized code.<br>
<br>
In the short term it's probably easier to find places where the<br>
instrumentation is missing debug info, and add it. But that's not going<br>
to be reliable for optimized code.<br>
--paulr<br>
<br>
______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div></div></div>
</div>
</div>
</blockquote></div><br></div>