[cfe-dev] Proposal: Integrate static analysis test suites

[Anna Zaks via cfe-dev]

> On Jan 29, 2016, at 4:33 PM, Philip Reames <listmail at philipreames.com <mailto:listmail at philipreames.com>> wrote:
> 
> 
> 
> On 01/28/2016 05:53 AM, Aaron Ballman via cfe-dev wrote:
>> On Thu, Jan 28, 2016 at 2:31 AM, Anna Zaks <ganna at apple.com <mailto:ganna at apple.com>> wrote:
>> <snip..>
>>> This is by design. Many more people have compiler as part of their daily
>>> flow so it’s best to have such errors being reported by the compiler.
>>> Having the analyzer produce all of the compiler warnings is likely to be too
>>> nosy for the users.
>> Personally, I find that design to lead to a confusing user experience.
>> When I run the analyzer, my mental model is that I am running the
>> compiler plus some additional analyses. When I don't get compiler
>> warnings that I would otherwise get, it feels like I (as the user)
>> have configured things improperly and done something wrong. Put
>> another way: the point to running a static analyzer is to find out
>> what's wrong with some code, so it's surprising that we would disable
>> some of those notices of what's wrong that would otherwise be enabled
>> by default.
>> 
>> Perhaps my mental model is in the minority, but it's another anecdote
>> to remember if this design is ever reconsidered again.
> I'd also find the current design slightly confusing.  I generally don't expect to see *fewer* warnings when I tell the compiler to work harder unless the original warning really was a false positive.

By calling "$clang —analyze” you are not calling the compiler and asking it to work harder. You are calling another tool that is not going to compile for you but rather provide deep static code analysis. Calling "clang —analyze" could call the compiler behind the scenes and report the compiler warnings in addition to the static analyzer issues. However, when warnings from both tools are merged in a straightforward way on command line, the user experience could be confusing. For example, both tools report some issues such as warning on code like this:
  int j = 5/0; // warning: Division by zero
                   // warning: division by zero is undefined [-Wdivision-by-zero]

Most importantly, end users should never invoke the analyzer by calling “clang —analyze” since “clang —analyze” is an implementation detail of the static analyzer. The only documented user facing clang static analysis tool is scan-build (see http://clang-analyzer.llvm.org <http://clang-analyzer.llvm.org/>). Here are some reasons for that. For one, it is almost impossible to understand why the static analyzer warns without examining the error paths. Second, the analyzer could be extended to perform whole project analysis in the future and "clang —analyze" works with a single TU at a time.

I agree that the best user experience is to report all warnings in one place, while still differentiating which warning was reported by which tool. It would be awesome if the results from all bug finding tools such as the clang static analyzer, the compiler, and clang-tidy would be reported through the same interface.

The CodeChecker team is working on a solution for that and I hope we can incorporate their technology in LLVM/clang.

> 
> Philip

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160129/99da5b0f/attachment.html>