[lldb-dev] Code coverage of llvm, clang, lldb & polly

[Konstantin Tokarev]

02.06.2013, 01:36, "Sylvestre Ledru" <sylvestre at debian.org>:
> On 01/06/2013 17:55, Konstantin Tokarev wrote:
>
>>  01.06.2013, 19:39, "Sylvestre Ledru" <sylvestre at debian.org>:
>>>  http://buildd-clang.debian.net/coverage/
>>>  Your patch fixed the issue!
>>>  Thanks to it, the LLVM toolchain is close to the 80 % lines test
>>>  coverage (which is excellent).
>>  Note that line-based coverage is far from being precise.
>
> Could you describe your criticism a bit more ?

You can read through review of code coverage metrics in [1].

In short, if you have a code line

if ((a<b) && (c<d)) doIt();

you will get 100% line coverage for it if condition is ever evaluated,
but you'll get 100% condition/decision coverage only if tests cover 4
different situations:

1) a<b is true, c<d is true
2) a<b is false, c<d is true
3) a<b is true, c<d is false
4) a<b is false, c<d is false

>
>>  To get more realistic estimation of coverage, you need to use coverage tool that
>>  provides decision and condition coverage, e.g. TestCocoon.
>
> Are you aware of any free software providing this feature ? I would be
> happy to try.

I'm using TestCocoon [2]. It is open source tool under GPLv3 license. Unfortunately,
it's effectively dead right now, because it was bought by Froglogic and is developed
as proprietary tool Squish Coco [3] now. On the positive side, Squish Coco is available
for free (as beer) for non-commercial usage [4].

BTW, it would be a great project to port TestCocoon's instrumentation to clang
infrastructure.


[1] http://www.bullseye.com/coverage.html
[2] git://gitorious.org/testcocoon/testcocoon.git
[3] http://www.froglogic.com/squish/coco/
[4] http://www.froglogic.com/squish/coco/non-commercial.php

-- 
Regards,
Konstantin