<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0cm;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.diff-banner-path
{mso-style-name:diff-banner-path;}
span.diff-banner-file
{mso-style-name:diff-banner-file;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Consolas",serif;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-IN" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Thanks Vedant for feedback.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">I am trying to analyse using %m (pls expect delay).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Can %m generate profraw files with basename of DSO and/or binaries (instead of the random basename) ?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Thank you.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> Vedant Kumar <vsk@apple.com>
<br>
<b>Sent:</b> Tuesday, 28 September, 2021 21:45<br>
<b>To:</b> Oza, Hiral <Hiral.Oza@netapp.com><br>
<b>Cc:</b> llvm-dev@lists.llvm.org; cfe-dev@lists.llvm.org<br>
<b>Subject:</b> Re: [cfe-dev] compiler-rt RFE to support creating profraw for each Shared Library<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" align="left" width="200" style="width:150.0pt;background:#0067C5">
<tbody>
<tr>
<td nowrap="" style="padding:6.0pt 6.0pt 6.0pt 6.0pt">
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:yellow">NetApp Security WARNING</span></b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:yellow">: This is an external email. Do not click links or open attachments unless
you recognize the sender and know the content is safe. </span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="color:black"><br>
<br>
</span><o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Sep 28, 2021, at 12:08 AM, Oza, Hiral <<a href="mailto:Hiral.Oza@netapp.com">Hiral.Oza@netapp.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Hello Vedant,</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">> What's not clear to me is why dso-specific .profraw files are helpful for code coverage, since merged .profraw's should work just as well.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">> Have you encountered issues using merged .profraws? Could you clarify what's meant by backtracking and instrumenting [the] final binary?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-US">There were problems when decoding merged profraw files. Couldn’t recall exactly error messages. But symbols in the profraws could not at all be matched to their definitions in the .so files.</span><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I think it'd be instructive to dig into these problems a bit more. How were .profraw contents matched to symbols in a .so? Can you share a minimal test case with (say) two .so's that illustrates the issue with merged .profraws?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">To add some context for the line of inquiry: llvm's infrastructure has supported collecting profile data from processes with multiple instrumented DSOs for quite a while. If the existing flow to emit & use merged .profraws for coverage
reporting has stopped working, or is buggy, that would indicate a serious regression which we should fix.<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Also tried %m which worked, except that we had no way to know which .so file corresponded to which .profraw file when decoding them.</span><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Why is it necessary to know the precise mapping of .profraw files to DSOs? Typically, .profraw files are merged together into an indexed .profdata (via `llvm-profdata merge ...`): that in turn supports coverage analysis for all of the DSOs
that contributed profile data.<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">Hence we added %n.</span><o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'd be hesitant towards adding this to the profile runtime as a workaround for a deeper issue.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">thanks,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">vedant<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Thank you.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">-Hiral<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">Vedant Kumar <<a href="mailto:vsk@apple.com">vsk@apple.com</a>><span class="apple-converted-space"> </span><br>
<b>Sent:</b><span class="apple-converted-space"> </span>Friday, 24 September, 2021 00:01<br>
<b>To:</b><span class="apple-converted-space"> </span>Oza, Hiral <<a href="mailto:Hiral.Oza@netapp.com">Hiral.Oza@netapp.com</a>><br>
<b>Cc:</b><span class="apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>;<span class="apple-converted-space"> </span><a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a><br>
<b>Subject:</b><span class="apple-converted-space"> </span>Re: [cfe-dev] compiler-rt RFE to support creating profraw for each Shared Library</span><o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" align="left" width="200" style="width:150.0pt;background:#0067C5">
<tbody>
<tr>
<td nowrap="" style="padding:6.0pt 6.0pt 6.0pt 6.0pt">
<div>
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:yellow">NetApp Security WARNING</span></b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:yellow">: This is an external email. Do not click links or open attachments unless
you recognize the sender and know the content is safe.<span class="apple-converted-space"> </span></span><o:p></o:p></p>
</div>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="color:black"><br>
</span><br>
<br>
<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal">Hi Oza,<span class="apple-converted-space"> </span><o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">The mechanics of the patch are clear. What's not clear to me is why dso-specific .profraw files are helpful for code coverage, since merged .profraw's should work just as well.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Have you encountered issues using merged .profraws? Could you clarify what's meant by backtracking and instrumenting [the] final binary?<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">thanks,<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">vedant<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal">On Sep 22, 2021, at 9:36 PM, Oza, Hiral <<a href="mailto:Hiral.Oza@netapp.com">Hiral.Oza@netapp.com</a>> wrote:<o:p></o:p></p>
</div>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Hello Vedant,<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">> could you share some of the use cases for this feature? > What were the pros/cons of any alternatives you considered (e.g. the %m/%c modes)?<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">We are taking coverage of shared libraries (i.e. Linux .so) and dumping coverage into files corresponding to SO’s filename.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">With this proposed patch, will get coverage as below for sample code (pls refer<span class="apple-converted-space"> </span><span class="diff-banner-path">compiler-rt/test/profile/Linux/</span><span class="diff-banner-file">instrprof-shared-nProfraws.test)</span>:<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="text-indent:36.0pt">libhi.so -> will dump into libhi.so.profraw<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="text-indent:36.0pt">libhello.so -> will dump into libhellp.so.profraw<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="text-indent:36.0pt">main.out -> will dump into main.out.profraw<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Regarding ‘%m or %c: (from<span class="apple-converted-space"> </span><a href="https://clang.llvm.org/docs/SourceBasedCodeCoverage.html"><span style="color:#0563C1">https://clang.llvm.org/docs/SourceBasedCodeCoverage.html</span></a>): %m
is expands instrumented binary’s signature (and with “%Nm” dumps into poll of N profraws) and %c seems very different from proposed “%n”.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Basically with proposed patch (i.e. using “%n”) one can get one-to-one mapping of Shared Library and generated .profraw.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">This makes backtracking and instrumenting final-binary based on feedback easy.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Please let me know in case of further queries or any changes in proposed patch.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Thank you.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">-Hiral<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<div>
<div>
<p class="MsoNormal"><b><span lang="EN-US">From:</span></b><span class="apple-converted-space"><span lang="EN-US"> </span></span><span lang="EN-US">cfe-dev <<a href="mailto:cfe-dev-bounces@lists.llvm.org"><span style="color:#0563C1">cfe-dev-bounces@lists.llvm.org</span></a>><span class="apple-converted-space"> </span><b>On
Behalf Of<span class="apple-converted-space"> </span></b>Oza, Hiral via cfe-dev<br>
<b>Sent:</b><span class="apple-converted-space"> </span>Wednesday, 22 September, 2021 18:56<br>
<b>To:</b><span class="apple-converted-space"> </span><a href="mailto:cfe-dev@lists.llvm.org"><span style="color:#0563C1">cfe-dev@lists.llvm.org</span></a>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org"><span style="color:#0563C1">llvm-dev@lists.llvm.org</span></a>><br>
<b>Subject:</b><span class="apple-converted-space"> </span>[cfe-dev] compiler-rt RFE to support creating profraw for each Shared Library</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" align="left" width="200" style="width:150.0pt;background:#0067C5">
<tbody>
<tr>
<td nowrap="" style="padding:6.0pt 6.0pt 6.0pt 6.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.25pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:yellow">NetApp Security WARNING</span></b><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:yellow">: This is an external email. Do not click links or open attachments unless
you recognize the sender and know the content is safe.</span><o:p></o:p></p>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="color:black"><br>
</span><br>
<br>
<br>
<o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">Greetings!<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Please review our patch compiler-rt RFE patch here --<span class="apple-converted-space"> </span><a href="https://reviews.llvm.org/D110232"><span style="color:#0563C1">https://reviews.llvm.org/D110232</span></a><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">This patch is about creating profraw for each Shared Library.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><patch_info><o:p></o:p></p>
</div>
</div>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">SBCC RFE: Support SBCC for Shared Library</span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">SBCC RFE: Support SBCC for Shared Library</span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Using '%n' format parameter for profile name templates, to insert the full</span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">path of the current shared object into profile file names.</span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> </span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Internally using a linked list to gather all profile dumping functions</span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">(there is one defined in each shared object) so they can be invoked with a</span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">single function call. Ensure the resulting directory paths are created</span><o:p></o:p></pre>
<pre><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">prior to dumping.</span><o:p></o:p></pre>
<div>
<div>
<p class="MsoNormal"></patch_info><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">Thank you in advance for your kind review.<o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal">-Hiral<o:p></o:p></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>