<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Jun 16, 2017 at 6:08 PM, Friedman, Eli via llvm-dev <span dir="ltr"><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I've started looking at the state of code coverage recently; we figured LLVM itself would be a good test to figure out how mature it is, so I gave it a shot. My experience:<br>
<br>
1. You have to specify -DLLVM_USE_LINKER=gold (or maybe lld works; I didn't try). If you link with binutils ld, the program will generate broken profile information. Apparently, the linked binary is missing the __llvm_prf_names section. This took me half a day to figure out. This issue isn't documented anywhere, and the only error message I got was "Assertion `!Key.empty()' failed." from llvm-cov.<br>
<br></blockquote><div><br></div><div><br></div><div>I believe the gnu-ld bug is <a href="https://sourceware.org/bugzilla/show_bug.cgi?id=19161">https://sourceware.org/bugzilla/show_bug.cgi?id=19161</a> which is fixed in version 2.26.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
2. The generated binaries are big and slow. Comparing to a build without coverage, llc becomes 8x larger overall (text section becomes roughly 2x larger). And check-llvm-codegen-arm goes from 3 seconds to 250 seconds.<br></blockquote><div><br></div><div>Over last couple of years, the instrumentation and coverage data overhead has reduced greatly. FE based instrumentation in general has larger overhead than IR based instrumentation, but the coverage testing currently only works with FE instrumentation. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
3. The generated profile information takes up a lot of space: llc generates a 90MB profraw file.<br></blockquote><div><br></div><div>This looks like in the normal range of raw profile size.</div><div><br></div><div><br></div><div>David</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
4. When prepare-code-coverage-artifact<wbr>.py invokes llvm-profdata for the profiles generated by "make check", it takes 50GB of memory to process about 1.5GB of profiles. Is it supposed to use that much?<br>
<br>
5. Using prepare-code-coverage-artifact<wbr>.py generates "warning: 229 functions have mismatched data". I'm not sure what's causing this... I guess it has something to do with merging the profile data for multiple binaries? The error message is not very helpful.<br>
<br>
5. The HTML output highlights the semicolon after a break or return statement in some switch statements in red. (For example, LowerADDC_ADDE_SUBC_SUBE in ARMISelLowering.cpp.) Not really important, but annoying.<br>
<br>
6. On the bright side, when it works, the generated coverage information is precise and easy to read.<span class="gmail-HOEnZb"><font color="#888888"><br>
<br>
-Eli<br>
<br>
-- <br>
Employee of Qualcomm Innovation Center, Inc.<br>
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project<br>
<br>
______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
</font></span></blockquote></div><br></div></div>