[cfe-dev] [analyzer] Clang Static Analyzer and Clang Tidy regression tests

[Artem Dergachev via cfe-dev]

Yeah, we should totally document our buildbot utils somewhere.

I believe the main reason why there aren't any public buildbots is that 
most parties target their own code, i.e. they're more interested in 
fixing false positives that hurt them specifically, and these overlap 
very poorly, given how dramatically different any different codebases 
usually are.

There's a historical attempt at 
http://green.lab.llvm.org/green/view/Experimental/job/StaticAnalyzerBenchmarks/ 
- technically it's there but in practice we never got to growing it into 
anything more than an experimental prototype. Also, i heard that Green 
Dragon's CPU power is relatively expensive, so we should go for a 
separate machine if we are to set up a public Static Analyzer reference 
results bot.

On 4/26/19 2:06 AM, Alexander Zaitsev via cfe-dev wrote:
>
> Hi!
>
> For Clang Static Analyzer (CSA) and for Clang Tidy there are a lot of 
> unit-tests. That's great, but from my point of view running CSA and 
> clang tidy *regularly *on some set of real life projects and comparing 
> the results between runs is also important to check any regressions.
>
> Now I know that for CSA there are set of useful scripts in 
> 'clang/utils/analyzer' but didn't find anything similar for clang-tidy.
>
> My questions are:
>
>  1. Are there any buildbots which test CSA and/or Clang Tidy on some
>     set of projects regularly? If there are not any buildbots - is it
>     possible to setup it?
>  2. I didn't find anything about regression testing in official
>     documentation. I think a note about regression testing has to be
>     added to the documentation: how it works now, how to it for your
>     code, etc. Now we have just small note about running unit-tests -
>     https://clang-analyzer.llvm.org/checker_dev_manual.html#testing
>
> Thank you.
>
> -- 
> Best regards,
> Alexander Zaitsev
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190429/0fcf21fd/attachment.html>