[llvm-dev] Getting a combined llvm-cov report for a set of tests

[Vedant Kumar via llvm-dev]

Hi,

There isn't yet a way to generate a single coverage report for multiple
binaries. We're planning on adding this functionality to llvm-cov soon. The
patch is up for review here: https://reviews.llvm.org/D25086.

> Maybe it's possible to merge multiple of these covmapping files and generate a report from that data (plus the merged profraw files)?

It also isn't yet possible to merge multiple 'covmapping' files. Do you have
other use-cases for this functionality (apart from using it to generate a
single report given multiple binaries)?

> I found some discussion on this thread, but I am not sure if this is asking the same thing:
> https://groups.google.com/forum/#!topic/llvm-dev/r4qOdN-y9hc

Yes, that thread is about the same topic. The person who started it is the
author of the patch I linked to :).

best,
vedant


> On Oct 14, 2016, at 9:05 AM, Chris Brown via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> I am working on a large software project with many thousands of tests. A common workflow using gcov as a coverage tool is:
> 
> 1) Build code with coverage instrumentation
> 2) Run *all* the tests. Gcov aggregates the line coverage counters together into its .gcda files
> 3) Generate a report for some set of files using those combined gcda files.
> 4) Analyze the report to find files/sections of code that are not covered by any test.
>  
> Using llvm's source-based code coverage, I don't believe I can accomplish steps 3-4. Each binary produces a .profraw file which contains profile counts, but the mapping of those profile counts to files is contained inside the binary.  I can use llvm-profdata merge to merge multiple .profraw files together, but the report can still only be generated for a single binary, and so only the files contained in that binary's __llvm_covmap section will be produced as part of the report.
> 
> Am I correct in this thinking? Is there a way to get a single coverage report for a set of multiple binaries? I discovered the undocumented 'llvm-cov convert-for-testing' command which generates 'covmapping files', which appear to be the contents of __llvm_covmap with an additional header. Maybe it's possible to merge multiple of these covmapping files and generate a report from that data (plus the merged profraw files)? Or perhaps I could use 'llvm-cov export' on each binary individually and manually merge the results somehow (although in that case, I would not be able to 'import' the merge to create the usual report) ? Are there other suggestions, or is this a planned feature in 4.0? I found some discussion on this thread, but I am not sure if this is asking the same thing:
> https://groups.google.com/forum/#!topic/llvm-dev/r4qOdN-y9hc
> 
> Thanks!
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev