[llvm-dev] Adding llvm-undname to the llvm-cov bot

[via llvm-dev]

> On Jun 6, 2019, at 9:56 AM, Nico Weber <thakis at chromium.org> wrote:
> 
> On Wed, Jun 5, 2019 at 1:33 PM <vsk at apple.com <mailto:vsk at apple.com>> wrote:
> 
> 
>> On Jun 4, 2019, at 4:41 PM, Nico Weber <thakis at chromium.org <mailto:thakis at chromium.org>> wrote:
>> 
>> On Mon, Jun 3, 2019 at 2:06 PM <vsk at apple.com <mailto:vsk at apple.com>> wrote:
>> Hi Nico,
>> 
>> Sorry for the delay, I've been OOO. The llvm-cov bot should produce reports for llvm-undname starting today.
>> 
>> Thanks! It looks like http://lab.llvm.org:8080/coverage/coverage-reports/index.html <http://lab.llvm.org:8080/coverage/coverage-reports/index.html> now has an "llvm-undname" entry, but http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html <http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html> still doesn't have an entry for llvm/lib/MicrosoftDemangle.cpp (and neither does http://lab.llvm.org:8080/coverage/coverage-reports/llvm-undname/index.html <http://lab.llvm.org:8080/coverage/coverage-reports/llvm-undname/index.html>).
> 
> For now, coverage for MicrosoftDemangle.cpp should show up under the "all" entry <http://lab.llvm.org:8080/coverage/coverage-reports/all/coverage/Users/buildslave/jenkins/workspace/clang-stage2-coverage-R/llvm/lib/Demangle/MicrosoftDemangle.cpp.html>.
> 
> 
>> What I'd ideally want is that the llvm report (http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html <http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html>) just shows coverage data for llvm/lib/MicrosoftDemangle.cpp llvm/lib/MicrosoftDemangleNodes.cpp llvm/include/llvm/Demangle/MicrosoftDemangle.h llvm/include/llvm/Demangle/MicrosoftDemangleNodes.h in addition to the other files that are there (and there's no separate report for llvm-undname).
> 
> This should be fixed on the next successful run.
> 
> Thanks much! I know see the file on http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html <http://lab.llvm.org:8080/coverage/coverage-reports/llvm/index.html> , that's great!
> 
> Is there a way to see which revision the report was built at?

For now, it's possible to look up this up via the Jenkins job status page <http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/> and clicking on the description of the latest successful job (e.g yesterday's #4089 <http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/4089/>). Ideally the svn revision would be included in the report, but I haven't worked out the right steps to take in Jenkins to make that happen. This is something I'd like to tackle during the monorepo transition.


>   That page shows 75.13% (1607/2139) line coverage for MicrosoftDemangle.cpp. I added lots of converage ~2 days ago (https://github.com/llvm/llvm-project/commits/master/llvm/test/Demangle <https://github.com/llvm/llvm-project/commits/master/llvm/test/Demangle>) and locally coverage for that file after running just `out/gn/bin/llvm-lit llvm/test/Demangle/` shows line coverage of 83.50% (1801/2157). Is that just due to the coverage report lagging trunk by a few days, or is something else up?

The discrepancy may be due to an llvm-cov limitation when processing multiple binaries: it ignores coverage records from a binary if it has a name that's already been seen.

vedant

>  
>> I figured what the bot does is run `check-llvm` and then pass all binaries that run as part of `llvm-check` to the report generation script, and I had assumed llvm-undname was just missing on that list of all binaries. But maybe that's not how that coverage list is computed?
> 
> The bot runs check-llvm, but passes a predefined list of binaries to llvm-cov to save time. llvm-undname was missing in the list of binaries to group under the 'all' and 'llvm' entries.
> 
> 
>> As for +x permissions on llvm/utils/prepare-code-coverage-artifact.py, it's an oversight that they are missing. I wasn't able to land a permissions change via the new monorepo (I get: "Committed c3b9398d101 to svn", but the commit does not appear). Perhaps you'll have better luck?
>> 
>> I couldn't figure out how to do it via git-svn / `git-llvm push` either, but I added the +x bit in r362561 using an old svn checkout I had lying around.
> 
> Thanks!
> 
> vedant
> 
>>  
>> 
>> best,
>> vedant
>> 
>>> On May 31, 2019, at 8:01 PM, Chris Matthews <chris.matthews at apple.com <mailto:chris.matthews at apple.com>> wrote:
>>> 
>>> Probably this job:
>>> 
>>> lab.llvm.org:8080/green/job/clang-stage2-coverage-R/ <http://lab.llvm.org:8080/green/job/clang-stage2-coverage-R/>
>>> 
>>> 💬 from 📱
>>> 
>>> On May 31, 2019, at 3:35 PM, Duncan Exon Smith <dexonsmith at apple.com <mailto:dexonsmith at apple.com>> wrote:
>>> 
>>>> +Chris Matthews, do you know where the configs are stored for this?
>>>> 
>>>>> On 2019 May  31, at 12:39, Chris Bieneman <beanz at apple.com <mailto:beanz at apple.com>> wrote:
>>>>> 
>>>>> Hey Nico,
>>>>> 
>>>>> I'm actually not sure where the configurations for that bot are stored. I suspect Duncan may have a better idea.
>>>>> 
>>>>> I'm reasonably certain that the missing +x is just an oversight.
>>>>> 
>>>>> -Chris
>>>>> 
>>>>>> On May 30, 2019, at 6:24 PM, Nico Weber via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>>>> 
>>>>>> Vedant or Chris: Ping :)
>>>>>> 
>>>>>> On Wed, May 29, 2019 at 7:56 AM Nico Weber <thakis at chromium.org <mailto:thakis at chromium.org>> wrote:
>>>>>> Hi Vedant and Chris,
>>>>>> 
>>>>>> is the config for http://lab.llvm.org:8080/coverage/coverage-reports/index.html <http://lab.llvm.org:8080/coverage/coverage-reports/index.html> public somewhere? If so, where? (I looked in zorg but didn't find it.)
>>>>>> 
>>>>>> If not, could you add "llvm-undname" to the list of binaries passed to llvm/utils/prepare-code-coverage-artifact.py so that llvm/lib/Demangle/MicrosoftDemangle.cpp (and friends) show up? (If the config is public, I can send you a patch.)
>>>>>> 
>>>>>> Also, is there a reason llvm/utils/prepare-code-coverage-artifact.py doesn't have +x set, or is that just an oversight?
>>>>>> 
>>>>>> Thanks,
>>>>>> Nico
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190610/98db9111/attachment.html>