[cfe-dev] Proposal: Integrate static analysis test suites

Sun Jan 31 16:51:54 PST 2016

>
> I guess, I should have said “advertised and recommended”, not
> “documented”. A lot of low-level options are documented in 'clang -help'.
>

Ahh, this is starting to make proper sense. Perhaps it'd be a good idea to
note (in help) that certain options aren't intended to be used *directly*?

Our team has always been careful about not recommending end users to run
> 'clang —analyze' directly, mainly, because their workflow would stop
> working if the analyzer ever adds whole program analysis. (Of cause LLVM
> community knows about the clang option but they are not average end users.)
>

Despite the hard work, I think it's still non-obvious to the average user
that they need to use scan-build :(

Sincerely,
Alexander Riccio
--
"Change the world or go home."
about.me/ariccio

<http://about.me/ariccio>
If left to my own devices, I will build more.
⁂

On Sat, Jan 30, 2016 at 3:13 AM, Anna Zaks <ganna at apple.com> wrote:

>
> On Jan 29, 2016, at 9:12 PM, Sean Silva <chisophugis at gmail.com> wrote:
>
>
>
> On Fri, Jan 29, 2016 at 8:32 PM, Anna Zaks via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>>
>> On Jan 29, 2016, at 4:33 PM, Philip Reames <listmail at philipreames.com>
>> wrote:
>>
>>
>>
>> On 01/28/2016 05:53 AM, Aaron Ballman via cfe-dev wrote:
>>
>> On Thu, Jan 28, 2016 at 2:31 AM, Anna Zaks <ganna at apple.com> wrote:
>> <snip..>
>>
>> This is by design. Many more people have compiler as part of their daily
>> flow so it’s best to have such errors being reported by the compiler.
>> Having the analyzer produce all of the compiler warnings is likely to be
>> too
>> nosy for the users.
>>
>> Personally, I find that design to lead to a confusing user experience.
>> When I run the analyzer, my mental model is that I am running the
>> compiler plus some additional analyses. When I don't get compiler
>> warnings that I would otherwise get, it feels like I (as the user)
>> have configured things improperly and done something wrong. Put
>> another way: the point to running a static analyzer is to find out
>> what's wrong with some code, so it's surprising that we would disable
>> some of those notices of what's wrong that would otherwise be enabled
>> by default.
>>
>> Perhaps my mental model is in the minority, but it's another anecdote
>> to remember if this design is ever reconsidered again.
>>
>> I'd also find the current design slightly confusing.  I generally don't
>> expect to see *fewer* warnings when I tell the compiler to work harder
>> unless the original warning really was a false positive.
>>
>>
>> By calling "$clang —analyze” you are not calling the compiler and asking
>> it to work harder. You are calling another tool that is not going to
>> compile for you but rather provide deep static code analysis. Calling
>> "clang —analyze" could call the compiler behind the scenes and report the
>> compiler warnings in addition to the static analyzer issues. However, when
>> warnings from both tools are merged in a straightforward way on command
>> line, the user experience could be confusing. For example, both tools
>> report some issues such as warning on code like this:
>>   int j = 5/0; // warning: Division by zero
>>                    // warning: division by zero is undefined
>> [-Wdivision-by-zero]
>>
>> Most importantly, end users should never invoke the analyzer by calling
>> “clang —analyze” since “clang —analyze” is an implementation detail of the
>> static analyzer. The only documented *user facing* clang static analysis
>> tool is scan-build (see http://clang-analyzer.llvm.org).
>>
>
> --analyze is in `clang -help`.
>
>
> I guess, I should have said “advertised and recommended”, not
> “documented”. A lot of low-level options are documented in 'clang -help'.
>
> Also, clang-check advertises a `-analyze` option which was clearly
> intentionally added.
>
>
> Is it advertised in places other than help? Again, I do not think we can
> provide a good user experience for consuming the analyzer results without
> showing the paths, which 'clang-check' does not do.
>
> So it seems spurious to say that the only user-facing way to invoke the
> analyzer is scan-build.
>
> In fact, anecdotally I seem to remember --analyze as being considered a
> user-facing option. I'm pretty sure that if I go digging back through old
> devmtg slides or whatnot I'll find a presentation recommending its use.
>
>
> Our team has always been careful about not recommending end users to run
> 'clang —analyze' directly, mainly, because their workflow would stop
> working if the analyzer ever adds whole program analysis. (Of cause LLVM
> community knows about the clang option but they are not average end users.)
>
>
> -- Sean Silva
>
>
>> Here are some reasons for that. For one, it is almost impossible to
>> understand why the static analyzer warns without examining the error paths.
>> Second, the analyzer could be extended to perform whole project analysis in
>> the future and "clang —analyze" works with a single TU at a time.
>>
>> I agree that the best user experience is to report all warnings in one
>> place, while still differentiating which warning was reported by which
>> tool. It would be awesome if the results from all bug finding tools such as
>> the clang static analyzer, the compiler, and clang-tidy would be reported
>> through the same interface.
>>
>> The CodeChecker team is working on a solution for that and I hope we can
>> incorporate their technology in LLVM/clang.
>>
>>
>> Philip
>>
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160131/a4562c20/attachment.html>