[cfe-dev] Proposal: Integrate static analysis test suites

[Sean Silva via cfe-dev]

On Fri, Jan 29, 2016 at 8:32 PM, Anna Zaks via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

>
> On Jan 29, 2016, at 4:33 PM, Philip Reames <listmail at philipreames.com>
> wrote:
>
>
>
> On 01/28/2016 05:53 AM, Aaron Ballman via cfe-dev wrote:
>
> On Thu, Jan 28, 2016 at 2:31 AM, Anna Zaks <ganna at apple.com> wrote:
> <snip..>
>
> This is by design. Many more people have compiler as part of their daily
> flow so it’s best to have such errors being reported by the compiler.
> Having the analyzer produce all of the compiler warnings is likely to be
> too
> nosy for the users.
>
> Personally, I find that design to lead to a confusing user experience.
> When I run the analyzer, my mental model is that I am running the
> compiler plus some additional analyses. When I don't get compiler
> warnings that I would otherwise get, it feels like I (as the user)
> have configured things improperly and done something wrong. Put
> another way: the point to running a static analyzer is to find out
> what's wrong with some code, so it's surprising that we would disable
> some of those notices of what's wrong that would otherwise be enabled
> by default.
>
> Perhaps my mental model is in the minority, but it's another anecdote
> to remember if this design is ever reconsidered again.
>
> I'd also find the current design slightly confusing.  I generally don't
> expect to see *fewer* warnings when I tell the compiler to work harder
> unless the original warning really was a false positive.
>
>
> By calling "$clang —analyze” you are not calling the compiler and asking
> it to work harder. You are calling another tool that is not going to
> compile for you but rather provide deep static code analysis. Calling
> "clang —analyze" could call the compiler behind the scenes and report the
> compiler warnings in addition to the static analyzer issues. However, when
> warnings from both tools are merged in a straightforward way on command
> line, the user experience could be confusing. For example, both tools
> report some issues such as warning on code like this:
>   int j = 5/0; // warning: Division by zero
>                    // warning: division by zero is undefined
> [-Wdivision-by-zero]
>
> Most importantly, end users should never invoke the analyzer by calling
> “clang —analyze” since “clang —analyze” is an implementation detail of the
> static analyzer. The only documented *user facing* clang static analysis
> tool is scan-build (see http://clang-analyzer.llvm.org).
>

--analyze is in `clang -help`. Also, clang-check advertises a `-analyze`
option which was clearly intentionally added. So it seems spurious to say
that the only user-facing way to invoke the analyzer is scan-build.

In fact, anecdotally I seem to remember --analyze as being considered a
user-facing option. I'm pretty sure that if I go digging back through old
devmtg slides or whatnot I'll find a presentation recommending its use.

-- Sean Silva


> Here are some reasons for that. For one, it is almost impossible to
> understand why the static analyzer warns without examining the error paths.
> Second, the analyzer could be extended to perform whole project analysis in
> the future and "clang —analyze" works with a single TU at a time.
>
> I agree that the best user experience is to report all warnings in one
> place, while still differentiating which warning was reported by which
> tool. It would be awesome if the results from all bug finding tools such as
> the clang static analyzer, the compiler, and clang-tidy would be reported
> through the same interface.
>
> The CodeChecker team is working on a solution for that and I hope we can
> incorporate their technology in LLVM/clang.
>
>
> Philip
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160129/624dfd36/attachment.html>