<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Sep 28, 2021, at 12:08 AM, Oza, Hiral <<a href="mailto:Hiral.Oza@netapp.com" class="">Hiral.Oza@netapp.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><meta charset="UTF-8" class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Hello Vedant,<o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">> What's not clear to me is why dso-specific .profraw files are helpful for code coverage, since merged .profraw's should work just as well.<o:p class=""></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="" class="">> Have you encountered issues using merged .profraws? </span>Could you clarify what's meant by backtracking and instrumenting [the] final binary?<o:p class=""></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">There were problems when decoding merged profraw files. Couldn’t recall exactly error messages. But symbols in the profraws could not at all be matched to their definitions in the .so files.</span></div></div></div></blockquote><div><br class=""></div><div>I think it'd be instructive to dig into these problems a bit more. How were .profraw contents matched to symbols in a .so? Can you share a minimal test case with (say) two .so's that illustrates the issue with merged .profraws?</div><div><br class=""></div><div>To add some context for the line of inquiry: llvm's infrastructure has supported collecting profile data from processes with multiple instrumented DSOs for quite a while. If the existing flow to emit & use merged .profraws for coverage reporting has stopped working, or is buggy, that would indicate a serious regression which we should fix.</div><br class=""><blockquote type="cite" class=""><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Also tried %m which worked, except that we had no way to know which .so file corresponded to which .profraw file when decoding them.</span></div></div></div></blockquote><div><br class=""></div><div>Why is it necessary to know the precise mapping of .profraw files to DSOs? Typically, .profraw files are merged together into an indexed .profdata (via `llvm-profdata merge ...`): that in turn supports coverage analysis for all of the DSOs that contributed profile data.</div><br class=""><blockquote type="cite" class=""><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class="">Hence we added %n.</span></div></div></div></blockquote><div><br class=""></div><div>I'd be hesitant towards adding this to the profile runtime as a workaround for a deeper issue.</div><div><br class=""></div><div>thanks,</div><div>vedant</div><br class=""><blockquote type="cite" class=""><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span lang="EN-US" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Thank you.<o:p class=""></o:p></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">-Hiral<span lang="EN-US" class=""><o:p class=""></o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span class=""><o:p class=""> </o:p></span></div><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class=""><span lang="EN-US" class="">From:</span></b><span lang="EN-US" class=""><span class="Apple-converted-space"> </span>Vedant Kumar <<a href="mailto:vsk@apple.com" style="color: blue; text-decoration: underline;" class="">vsk@apple.com</a>><span class="Apple-converted-space"> </span><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>Friday, 24 September, 2021 00:01<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span>Oza, Hiral <<a href="mailto:Hiral.Oza@netapp.com" style="color: blue; text-decoration: underline;" class="">Hiral.Oza@netapp.com</a>><br class=""><b class="">Cc:</b><span class="Apple-converted-space"> </span><a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a>;<span class="Apple-converted-space"> </span><a href="mailto:cfe-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">cfe-dev@lists.llvm.org</a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>Re: [cfe-dev] compiler-rt RFE to support creating profraw for each Shared Library<o:p class=""></o:p></span></div></div></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" align="left" width="200" style="width: 150pt; background-color: rgb(0, 103, 197);"><tbody class=""><tr class=""><td nowrap="" style="padding: 6pt;" class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 10pt; font-family: Arial, sans-serif; color: yellow;" class="">NetApp Security WARNING</span></b><span style="font-size: 10pt; font-family: Arial, sans-serif; color: yellow;" class="">: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.<span class="Apple-converted-space"> </span></span><o:p class=""></o:p></div></td></tr></tbody></table><p class="MsoNormal" style="margin: 0cm 0cm 12pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="" class=""><br class=""><br class=""></span><o:p class=""></o:p></p><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hi Oza,<span class="Apple-converted-space"> </span><o:p class=""></o:p></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">The mechanics of the patch are clear. What's not clear to me is why dso-specific .profraw files are helpful for code coverage, since merged .profraw's should work just as well.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><span style="" class="">Have you encountered issues using merged .profraws? </span>Could you clarify what's meant by backtracking and instrumenting [the] final binary?<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">thanks,<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">vedant<o:p class=""></o:p></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><br class=""><br class=""><o:p class=""></o:p></div><blockquote style="margin-top: 5pt; margin-bottom: 5pt;" class=""><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">On Sep 22, 2021, at 9:36 PM, Oza, Hiral <<a href="mailto:Hiral.Oza@netapp.com" style="color: blue; text-decoration: underline;" class="">Hiral.Oza@netapp.com</a>> wrote:<o:p class=""></o:p></div></div><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hello Vedant,<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">> could you share some of the use cases for this feature? > What were the pros/cons of any alternatives you considered (e.g. the %m/%c modes)?<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">We are taking coverage of shared libraries (i.e. Linux .so) and dumping coverage into files corresponding to SO’s filename.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">With this proposed patch, will get coverage as below for sample code (pls refer<span class="apple-converted-space"> </span><span class="diff-banner-path">compiler-rt/test/profile/Linux/</span><span class="diff-banner-file">instrprof-shared-nProfraws.test)</span>:<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: 36pt;" class="">libhi.so -> will dump into libhi.so.profraw<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: 36pt;" class="">libhello.so -> will dump into libhellp.so.profraw<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif; text-indent: 36pt;" class="">main.out -> will dump into main.out.profraw<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Regarding ‘%m or %c: (from<span class="apple-converted-space"> </span><a href="https://clang.llvm.org/docs/SourceBasedCodeCoverage.html" style="color: blue; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">https://clang.llvm.org/docs/SourceBasedCodeCoverage.html</span></a>): %m is expands instrumented binary’s signature (and with “%Nm” dumps into poll of N profraws) and %c seems very different from proposed “%n”.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Basically with proposed patch (i.e. using “%n”) one can get one-to-one mapping of Shared Library and generated .profraw.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">This makes backtracking and instrumenting final-binary based on feedback easy.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Please let me know in case of further queries or any changes in proposed patch.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Thank you.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">-Hiral<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(225, 225, 225); padding: 3pt 0cm 0cm;" class=""><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class=""><span lang="EN-US" class="">From:</span></b><span class="apple-converted-space"><span lang="EN-US" class=""> </span></span><span lang="EN-US" class="">cfe-dev <<a href="mailto:cfe-dev-bounces@lists.llvm.org" style="color: blue; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">cfe-dev-bounces@lists.llvm.org</span></a>><span class="apple-converted-space"> </span><b class="">On Behalf Of<span class="apple-converted-space"> </span></b>Oza, Hiral via cfe-dev<br class=""><b class="">Sent:</b><span class="apple-converted-space"> </span>Wednesday, 22 September, 2021 18:56<br class=""><b class="">To:</b><span class="apple-converted-space"> </span><a href="mailto:cfe-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">cfe-dev@lists.llvm.org</span></a>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">llvm-dev@lists.llvm.org</span></a>><br class=""><b class="">Subject:</b><span class="apple-converted-space"> </span>[cfe-dev] compiler-rt RFE to support creating profraw for each Shared Library</span><o:p class=""></o:p></div></div></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><table class="MsoNormalTable" border="1" cellspacing="0" cellpadding="0" align="left" width="200" style="width: 150pt; background-color: rgb(0, 103, 197);"><tbody class=""><tr class=""><td nowrap="" style="padding: 6pt;" class=""><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 10pt; font-family: Arial, sans-serif; color: yellow;" class="">NetApp Security WARNING</span></b><span style="font-size: 10pt; font-family: Arial, sans-serif; color: yellow;" class="">: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe.</span><o:p class=""></o:p></div></div></td></tr></tbody></table><p class="MsoNormal" style="margin: 0cm 0cm 12pt; font-size: 11pt; font-family: Calibri, sans-serif;"><span style="" class=""><br class=""></span><br class=""><br class=""><o:p class=""></o:p></p><div class=""><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Greetings!<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Please review our patch compiler-rt RFE patch here --<span class="apple-converted-space"> </span><a href="https://reviews.llvm.org/D110232" style="color: blue; text-decoration: underline;" class=""><span style="color: rgb(5, 99, 193);" class="">https://reviews.llvm.org/D110232</span></a><o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">This patch is about creating profraw for each Shared Library.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><patch_info><o:p class=""></o:p></div></div><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">SBCC RFE: Support SBCC for Shared Library</span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">SBCC RFE: Support SBCC for Shared Library</span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Using '%n' format parameter for profile name templates, to insert the full</span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">path of the current shared object into profile file names.</span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class=""> </span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">Internally using a linked list to gather all profile dumping functions</span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">(there is one defined in each shared object) so they can be invoked with a</span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">single function call. Ensure the resulting directory paths are created</span><o:p class=""></o:p></pre><pre style="margin: 0cm 0cm 0.0001pt; font-size: 10pt; font-family: "Courier New";" class=""><span style="font-size: 11pt; font-family: Calibri, sans-serif;" class="">prior to dumping.</span><o:p class=""></o:p></pre><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""></patch_info><o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class=""> <o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Thank you in advance for your kind review.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0cm; font-size: 11pt; font-family: Calibri, sans-serif;" class="">-Hiral</div></div></div></div></blockquote></div></div></div></div></div></blockquote></div><br class=""></body></html>