[PATCH] Initial code coverage mapping data structures, and reader and writers + C interface for ProfileData library

[Eric Christopher]

On Thu, Jun 26, 2014 at 11:23 AM, Hal Finkel <hfinkel at anl.gov> wrote:
> ----- Original Message -----
>> From: "Philip Reames" <listmail at philipreames.com>
>> To: llvm-commits at cs.uiuc.edu
>> Sent: Thursday, June 26, 2014 12:51:43 PM
>> Subject: Re: [PATCH] Initial code coverage mapping data structures, and reader        and writers + C interface for
>> ProfileData library
>>
>>
>>
>>
>> On 06/26/2014 09:43 AM, Alex L wrote:
>>
>>
>>
>>
>>
>>
>>
>>
>> 2014-06-25 17:57 GMT-07:00 Eric Christopher < echristo at gmail.com > :
>>
>>
>> So, any reason why we want the new coverage system? You didn't seem
>> to
>> highlight what you saw wrong with gcov or the way it works.
>>
>> I saw your original mail and it didn't have a lot of motivation here.
>>
>> Thanks.
>>
>> -eric
>>
>>
>>
>> I am not very familiar with GCOV, but the primary motivation for the
>> new coverage system is to provide very accurate execution counts for
>> a program. This would enable us to provide modern, high quality
>> tools for code coverage analysis.
>>
>> By very accurate I mean that instead of reasoning about code coverage
>> for basic blocks, branches and even lines, we would be able to
>> reason about code coverage for the regions of source code that
>> resemble the corresponding AST. This would enable the coverage tool
>> to locate and mark the exact regions of code that weren't executed,
>> even if the IR for those regions was transformed by the optimizer. I
>> think that GCOV fails to provide coverage for certain constructs
>> when optimizations are enabled. Also, I think that GCOV doesn't
>> really show good coverage for the lines that have multiple regions
>> with different execution counts, and the new system will enable us
>> to create a tool which will have a better way to deal with this
>> particular situation.
>>
>>
>> Also, the GCOV way produces separate mapping files and counter files
>> for each source file/object file, which can be somewhat
>> inconvenient. In the new system we pack the mapping data into the
>> generated IR and allow it to be merged by the linker, and as a
>> result of that all our mapping information is embedded inside the
>> program's executable.
>>
>>
>>
>> The new coverage tool will be able to provide a more interactive
>> experience as well, by showing reports or code coverage only for
>> selected items like certain functions, classes, etc.
>>
>>
>>
>> Also, this new coverage system will provide a library which various
>> code coverage tools can use to make coverage reports without the
>> need to parse the output of GCOV.
>> I find nothing in the above explanation to justify including this
>> code *in LLVM*. Such a tool may be useful, but this really sounds
>> like an independent project (i.e. "replace gcov").
>>
>> Glancing through the code, I also see no real interaction with LLVM.
>> This really seems like an independent profiling library which could
>> be used to provide profiling data to LLVM, but is otherwise
>> unrelated. Correct me if I'm wrong here - I did a *very* quick scan.
>>
>> Your post about this topic on llvm-dev has not generated any
>> consensus. If anything, there seems to be an active disinterest in
>> your proposal.
>
> I'm not sure what you mean by "has not generated any consensus", I saw no discussion at all. That having been said, I know that a number of community members care a lot about coverage quantification (including me), and the gcov zillions-of-files approach clearly does not scale. Really, getting feedback on long RFCs is not easy, and I'd draw no inference from the lack of response to date.
>
> That having been said, I'm not sure what to think about this patch. I think it will be easier to review once we see the code that uses it.
>

My biggest complaint so far is that I don't see anything particularly
compelling or any analysis as to what's wrong or what this gets us.
Until then it's just not interesting.

-eric


>  -Hal
>
>>
>> Given the above, I would oppose the inclusion of this change set.
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> On Wed, Jun 25, 2014 at 4:32 PM, Alex L < arphaman at gmail.com > wrote:
>> > Hi everyone,
>> > This is a first patch that implements the data structures, readers
>> > and
>> > writers used by the new code coverage system. I added the new code
>> > to the
>> > ProfileData library. I also added a very minimal C api for the
>> > ProfileData
>> > library.
>> >
>> > http://reviews.llvm.org/D4301
>> >
>> >
>> >
>> > _______________________________________________
>> > llvm-commits mailing list
>> > llvm-commits at cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>> >
>>
>>
>>
>> _______________________________________________
>> llvm-commits mailing list llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits