<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 10, 2019, at 12:45 PM, Max Moroz <<a href="mailto:mmoroz@chromium.org" class="">mmoroz@chromium.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Hmm, I think we've fixed that behavior in <a href="https://reviews.llvm.org/D46478" class="">https://reviews.llvm.org/D46478</a>. For instance, we run 400+ binaries in Chrome, each having its own LLVMFuzzerTestOneInput function, and we're able to combine all .profdata together and generate an aggregate report.</div></div></blockquote><div><br class=""></div>Oh, thanks for pointing that out. This should cover the common case, excepting symbols like "main" that really do create ambiguity.</div><div><br class=""></div><div><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="">However, it feels like there is some other condition which may still lead to a collision. We've hit it in OSS-Fuzz recently (<a href="https://github.com/google/oss-fuzz/issues/2330" class="">https://github.com/google/oss-fuzz/issues/2330</a>), but the developer ended up changing the sources of the binaries and the issue went away. Where are the logs from the buildbot? Are there any warnings reported like the following one:</div></div></div></blockquote><blockquote type="cite" class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class=""><pre style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:11.9px;margin-bottom:16px;margin-top:0px;background-color:rgb(246,248,250);border-radius:3px;line-height:1.45;overflow:auto;padding:16px;color:rgb(36,41,46)" class=""><code style="box-sizing:border-box;font-family:SFMono-Regular,Consolas,"Liberation Mono",Menlo,Courier,monospace;font-size:11.9px;background:transparent;border-radius:3px;margin:0px;padding:0px;border:0px;word-break:normal;display:inline;line-height:inherit;overflow:visible" class="">warning: /out/dumps/simple_round_trip.profdata: Function basic block count change detected (counter mismatch)</code></pre></div></div></blockquote><div>The logs should be visible via the Jenkins job. Here's one: <a href="http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/4092/consoleText" class="">http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/4092/consoleText</a> (I do see a 'counter mismatch' warning here).</div><div><br class=""></div><div>Coverage for a symbol is dropped if there's a mismatch between the hash recorded in the profile and the hash recorded in the code coverage blob. Typically this occurs when the profile is out of sync with the build products, but that's not the case on the bot. It's also possible for there to be conflicting definitions of a function present in different binaries. These definitions can have different numbers of counters or AST hashes due to source-level differences. For this case, I think it's reasonable to not display coverage.</div><div><br class=""></div><div>vedant</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jun 10, 2019 at 11:44 AM Nico Weber <<a href="mailto:thakis@chromium.org" class="">thakis@chromium.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class=""><div dir="ltr" class="">On Mon, Jun 10, 2019 at 2:11 PM <<a href="mailto:vsk@apple.com" target="_blank" class="">vsk@apple.com</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Jun 6, 2019, at 9:56 AM, Nico Weber <<a href="mailto:thakis@chromium.org" target="_blank" class="">thakis@chromium.org</a>> wrote:</div><br class="gmail-m_-6929773844212415040gmail-m_7999966255999581194Apple-interchange-newline"><div class=""><div dir="ltr" style="font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><div dir="ltr" class="">On Wed, Jun 5, 2019 at 1:33 PM <<a href="mailto:vsk@apple.com" target="_blank" class="">vsk@apple.com</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=""><br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Jun 4, 2019, at 4:41 PM, Nico Weber <<a href="mailto:thakis@chromium.org" target="_blank" class="">thakis@chromium.org</a>> wrote:</div><br class="gmail-m_-6929773844212415040gmail-m_7999966255999581194gmail-m_6528325519568439786Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div dir="ltr" class="">On Mon, Jun 3, 2019 at 2:06 PM <<a href="mailto:vsk@apple.com" target="_blank" class="">vsk@apple.com</a>> wrote:<br class=""></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="">Hi Nico,<div class=""><br class=""></div><div class="">Sorry for the delay, I've been OOO. The llvm-cov bot should produce reports for llvm-undname starting today.</div></div></blockquote><div class=""><br class=""></div><div class="">Thanks! It looks like <a href="http://lab.llvm.org:8080/coverage/coverage-reports/index.html" target="_blank" class="">http://lab.llvm.org:8080/coverage/coverage-reports/index.html</a><span class="gmail-m_-6929773844212415040gmail-m_7999966255999581194Apple-converted-space"> </span>now has an "llvm-undname" entry, but <a href="http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html" target="_blank" class="">http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html</a><span class="gmail-m_-6929773844212415040gmail-m_7999966255999581194Apple-converted-space"> </span>still doesn't have an entry for llvm/lib/MicrosoftDemangle.cpp (and neither does <a href="http://lab.llvm.org:8080/coverage/coverage-reports/llvm-undname/index.html" target="_blank" class="">http://lab.llvm.org:8080/coverage/coverage-reports/llvm-undname/index.html</a>).</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">For now, coverage for MicrosoftDemangle.cpp should show up under the <a href="http://lab.llvm.org:8080/coverage/coverage-reports/all/coverage/Users/buildslave/jenkins/workspace/clang-stage2-coverage-R/llvm/lib/Demangle/MicrosoftDemangle.cpp.html" target="_blank" class="">"all" entry</a>.</div><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class="">What I'd ideally want is that the llvm report (<a href="http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html" target="_blank" class="">http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html</a>) just shows coverage data for llvm/lib/MicrosoftDemangle.cpp llvm/lib/MicrosoftDemangleNodes.cpp llvm/include/llvm/Demangle/MicrosoftDemangle.h llvm/include/llvm/Demangle/MicrosoftDemangleNodes.h in addition to the other files that are there (and there's no separate report for llvm-undname).</div></div></div></div></blockquote><div class=""><br class=""></div>This should be fixed on the next successful run.</div></div></blockquote><div class=""><br class=""></div><div class="">Thanks much! I know see the file on <a href="http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html" target="_blank" class="">http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html</a><span class="gmail-m_-6929773844212415040gmail-m_7999966255999581194Apple-converted-space"> </span>, that's great!</div><div class=""><br class=""></div><div class="">Is there a way to see which revision the report was built at?</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">For now, it's possible to look up this up via the <a href="http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/" target="_blank" class="">Jenkins job status page</a> and clicking on the description of the latest successful job (e.g yesterday's <a href="http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/4089/" target="_blank" class="">#4089</a>). Ideally the svn revision would be included in the report, but I haven't worked out the right steps to take in Jenkins to make that happen. This is something I'd like to tackle during the monorepo transition.</div><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><div class="gmail_quote"><div class=""> That page shows 75.13% (1607/2139) line coverage for MicrosoftDemangle.cpp. I added lots of converage ~2 days ago (<a href="https://github.com/llvm/llvm-project/commits/master/llvm/test/Demangle" target="_blank" class="">https://github.com/llvm/llvm-project/commits/master/llvm/test/Demangle</a>) and locally coverage for that file after running just `out/gn/bin/llvm-lit llvm/test/Demangle/` shows line coverage of 83.50% (1801/2157). Is that just due to the coverage report lagging trunk by a few days, or is something else up?</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">The discrepancy may be due to an llvm-cov limitation when processing multiple binaries: it ignores coverage records from a binary if it has a name that's already been seen.</div></div></div></blockquote><div class=""><br class=""></div><div class="">Thanks, that sounds like a good explanation. Is there a PR for this problem?</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=""><div class=""><div class=""><br class=""></div><div class="">vedant</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="font-family:Helvetica;font-size:14px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=""><div class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class="">I figured what the bot does is run `check-llvm` and then pass all binaries that run as part of `llvm-check` to the report generation script, and I had assumed llvm-undname was just missing on that list of all binaries. But maybe that's not how that coverage list is computed?</div></div></div></div></blockquote><div class=""><br class=""></div><div class="">The bot runs check-llvm, but passes a predefined list of binaries to llvm-cov to save time. llvm-undname was missing in the list of binaries to group under the 'all' and 'llvm' entries.</div><div class=""><br class=""></div><br class=""><blockquote type="cite" class=""><div dir="ltr" class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=""><div class="">As for +x permissions on llvm/utils/prepare-code-coverage-artifact.py, it's an oversight that they are missing. I wasn't able to land a permissions change via the new monorepo (I get: "Committed c3b9398d101 to svn", but the commit does not appear). Perhaps you'll have better luck?<br class=""></div></div></blockquote><div class=""><br class=""></div><div class="">I couldn't figure out how to do it via git-svn / `git-llvm push` either, but I added the +x bit in r362561 using an old svn checkout I had lying around.</div></div></div></blockquote><div class=""><br class=""></div>Thanks!</div><div class=""><br class=""></div><div class="">vedant</div><div class=""><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class=""><div class=""></div><div class=""><br class=""></div><div class="">best,</div><div class="">vedant<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On May 31, 2019, at 8:01 PM, Chris Matthews <<a href="mailto:chris.matthews@apple.com" target="_blank" class="">chris.matthews@apple.com</a>> wrote:</div><br class="gmail-m_-6929773844212415040gmail-m_7999966255999581194gmail-m_6528325519568439786gmail-m_-6203657135107623817Apple-interchange-newline"><div class=""><div dir="auto" class="">Probably this job:<div class=""><br class=""></div><div class=""><a href="http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/" target="_blank" class="">lab.llvm.org:8080/green/job/clang-stage2-coverage-R/</a><br class=""><br class=""><div dir="ltr" class="">💬 from 📱</div><div dir="ltr" class=""><br class="">On May 31, 2019, at 3:35 PM, Duncan Exon Smith <<a href="mailto:dexonsmith@apple.com" target="_blank" class="">dexonsmith@apple.com</a>> wrote:<br class=""><br class=""></div><blockquote type="cite" class=""><div dir="ltr" class="">+Chris Matthews, do you know where the configs are stored for this?<div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On 2019 May 31, at 12:39, Chris Bieneman <<a href="mailto:beanz@apple.com" target="_blank" class="">beanz@apple.com</a>> wrote:</div><br class="gmail-m_-6929773844212415040gmail-m_7999966255999581194gmail-m_6528325519568439786gmail-m_-6203657135107623817Apple-interchange-newline"><div class=""><div class="">Hey Nico,<div class=""><br class=""></div><div class="">I'm actually not sure where the configurations for that bot are stored. I suspect Duncan may have a better idea.</div><div class=""><br class=""></div><div class="">I'm reasonably certain that the missing +x is just an oversight.</div><div class=""><br class=""></div><div class="">-Chris<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On May 30, 2019, at 6:24 PM, Nico Weber via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="gmail-m_-6929773844212415040gmail-m_7999966255999581194gmail-m_6528325519568439786gmail-m_-6203657135107623817Apple-interchange-newline"><div class=""><div dir="ltr" class="">Vedant or Chris: Ping :)</div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, May 29, 2019 at 7:56 AM Nico Weber <<a href="mailto:thakis@chromium.org" target="_blank" class="">thakis@chromium.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="">Hi Vedant and Chris,<br class=""><br class="">is the config for<span class="gmail-m_-6929773844212415040gmail-m_7999966255999581194Apple-converted-space"> </span><a href="http://lab.llvm.org:8080/coverage/coverage-reports/index.html" target="_blank" class="">http://lab.llvm.org:8080/coverage/coverage-reports/index.html</a><span class="gmail-m_-6929773844212415040gmail-m_7999966255999581194Apple-converted-space"> </span>public somewhere? If so, where? (I looked in zorg but didn't find it.)<br class=""><br class="">If not, could you add "llvm-undname" to the list of binaries passed to llvm/utils/prepare-code-coverage-artifact.py so that llvm/lib/Demangle/MicrosoftDemangle.cpp (and friends) show up? (If the config is public, I can send you a patch.)<br class=""><br class="">Also, is there a reason llvm/utils/prepare-code-coverage-artifact.py doesn't have +x set, or is that just an oversight?<br class=""><br class="">Thanks,<br class="">Nico<br class=""></div></blockquote></div>_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div></div></div></div></blockquote></div></div></div></blockquote></div></div></div></blockquote></div></div></div></blockquote></div></div></div></blockquote></div></div></blockquote></div></div></div></blockquote></div><br class=""></div></blockquote></div></div>
</blockquote></div>
</div></blockquote></div><br class=""></body></html>