<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I suspect that you misunderstand where ASan instrumentation is added. Unlike UBSan, which is added by Clang during initial IR generation,
ASan instrumentation is added late (at the EP_OptimizerLast extension point).</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><a name="_MailEndCompose"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></a></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">You are correct, I did not know that.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I don't see any better way to get the location information at that point than using the existing debug info.</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Let us distinguish between information carried around in metadata, and information emitted to the object file.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Seems like it would be completely feasible to flag a DebugLoc instance as "do not emit" and yet retain it in the metadata, rather than erasing it (overwriting
it with DebugLoc()). Then the ASan instrumentation can extract the file/line info from it, while we still decline to generate it in the DWARF line table. ASan gets the info it needs, the debugger doesn't get the information that will distress the user.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Another possibility is to flag the DebugLoc for a moved/combined instruction as "not a statement" and cause this flag to suppress the DWARF "is_stmt" flag.
That idea would need a little more baking but feels like it could have potential. Some debuggers might have to learn to pay attention to the flag, but that's the debugger's problem.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">--paulr<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Hal Finkel [mailto:hfinkel@anl.gov]
<br>
<b>Sent:</b> Wednesday, December 07, 2016 10:19 AM<br>
<b>To:</b> Robinson, Paul<br>
<b>Cc:</b> David Blaikie; llvm-dev@lists.llvm.org; Reid Kleckner<br>
<b>Subject:</b> Re: [llvm-dev] Debug Locations for Optimized Code<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><o:p> </o:p></span></p>
<div class="MsoNormal" align="center" style="text-align:center"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">
<hr size="3" width="100%" align="center" id="zwchr">
</span></div>
<blockquote style="border:none;border-left:solid #1010FF 1.5pt;padding:0in 0in 0in 4.0pt;margin-left:3.75pt;margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-family:"Helvetica","sans-serif";color:black">From:
</span></b><span style="font-family:"Helvetica","sans-serif";color:black">"Paul Robinson" <<a href="mailto:paul.robinson@sony.com">paul.robinson@sony.com</a>><br>
<b>To: </b>"Reid Kleckner" <<a href="mailto:rnk@google.com">rnk@google.com</a>><br>
<b>Cc: </b>"Hal Finkel" <<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>>, "David Blaikie" <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>>,
<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a><br>
<b>Sent: </b>Wednesday, December 7, 2016 11:01:57 AM<br>
<b>Subject: </b>RE: [llvm-dev] Debug Locations for Optimized Code<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="color:black">I don't see how ASan and debuggers are different. It feels like both need reasonably accurate source location attribution for any instruction. ASan just happens to care more about loads
and stores than interactive stepping debuggers.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Actually they are pretty different in their requirements.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">ASan cares about *accurate* source location info for *specific* instructions, the ones that do something ASan cares about. The source attributions for any
other instruction is irrelevant to ASan. The source attributions for these instructions *must* survive optimization.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Debuggers care about *useful* source location info for *sets* of instructions, i.e. the instructions related to some particular source statement. If that set
is only 90% complete/accurate, instead of 100%, generally that doesn't adversely affect the user experience. If you step past statement A, and happen to execute one or two instructions from the next statement B before you actually stop, generally that is
not important to the user. Debuggers are able to tolerate a moderate amount of slop in the source attributions, because absolute accuracy is not critical to correct operation of the debugger. This is why optimizations can get away with dropping attributions
that are difficult to represent accurately.</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">ASan should be able to encode source info for just the instructions it cares about, e.g. pass an index or other encoded representation to the RT calls. Being
actual parameters, they will survive any correct optimization, unlike today's situation where multiple calls might be merged by an optimization, damaging the correctness of ASan reports. (We've see this exact thing happen.) ASan does not need a line table
mapping all instructions back to their source; it needs a parameter at each call (more or less). It does need a file table, that's the main bit of redundancy with debug info that I see happening.</span><span style="color:black"><o:p></o:p></span></p>
</blockquote>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">I suspect that you misunderstand where ASan instrumentation is added. Unlike UBSan, which is added by Clang during initial IR generation, ASan instrumentation is
added late (at the EP_OptimizerLast extension point). I don't see any better way to get the location information at that point than using the existing debug info.<br>
<br>
-Hal<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">--paulr</span><span style="color:black"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><span style="color:black"><o:p></o:p></span></p>
<div style="border:none;border-left:solid windowtext 1.5pt;padding:0in 0in 0in 4.0pt;border-color:-moz-use-text-color -moz-use-text-color -moz-use-text-color blue">
<div>
<div style="border:none;border-top:solid windowtext 1.0pt;padding:3.0pt 0in 0in 0in;border-color:-moz-use-text-color -moz-use-text-color">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif";color:black"> Reid Kleckner [<a href="mailto:rnk@google.com">mailto:rnk@google.com</a>]
<br>
<b>Sent:</b> Wednesday, December 07, 2016 8:23 AM<br>
<b>To:</b> Robinson, Paul<br>
<b>Cc:</b> Hal Finkel; David Blaikie; <a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] Debug Locations for Optimized Code</span><span style="color:black"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
<div>
<div>
<div>
<p class="MsoNormal"><span style="color:black">On Wed, Dec 7, 2016 at 7:39 AM, Robinson, Paul via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">When we are looking at a situation where an instruction is merely *moved*<br>
from one place to another, retaining the source location and having a<br>
less naïve statement-marking tactic could help the debugging experience<br>
without perturbing other consumers (although one still wonders whether<br>
profiles will get messed up in cases where e.g. a loop invariant gets<br>
hoisted out of a cold loop into a hot predecessor).<br>
<br>
When we are looking at a situation where two instructions are *merged* or<br>
*combined* into one, and the original two instructions had different<br>
source locations, that's a separate problem. In that case there is no<br>
single correct source location for the new instruction, and typically<br>
erasing the source location will give a better debugging experience (also<br>
a less misleading profile).<br>
<br>
My personal opinion is that having sanitizers *rely* on debug info for<br>
accurate source attribution is just asking for trouble. It happens to<br>
work at –O0 but cannot be considered reliable in the face of optimization.<br>
IMO this is a fundamental design flaw; debug info is best-effort and full<br>
of ambiguities, as shown above. Sanitizers need a more reliable<br>
source-of-truth, i.e. they should encode source info into their own<br>
instrumentation.<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">I don't see how ASan and debuggers are different. It feels like both need reasonably accurate source location attribution for any instruction. ASan just happens to care more about loads and stores than interactive
stepping debuggers.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">It really doesn't make sense for ASan to invent another mechanism to track source location information. Any mechanism we build would be so redundant with debug info that, as an implementation detail, we would find
a way to make them use the same storage when possible. With that in mind, maybe we should really find a way to mark source locations as "hoisted" or "sunk" so that we can suppress them from our line tables or do something else clever.<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black"><br>
<br>
<br>
-- <o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:black">Hal Finkel<br>
Lead, Compiler Technology and Programming Languages<br>
Leadership Computing Facility<br>
Argonne National Laboratory<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>