[cfe-dev] Idea for better invoking static analysis via command line

Anna Zaks via cfe-dev cfe-dev at lists.llvm.org
Fri Feb 5 15:34:01 PST 2016 

> On Feb 4, 2016, at 3:52 PM, <Alexander G. Riccio> <test35965 at gmail.com> <Alexander G. Riccio> wrote:
> 
> I'm still not quite used to the archaic mailing list format, but I thought the proper thing to do when replying to a message on a mailing list is to do a reply all? I totally missed these three messages, because they were part of the silly "batched" messages. 
> 
> Consider making "no detailed analysis" an option for -enable-analyze-pass to help with these use cases
> 
> Eh? Do you mean less detailed output or less detailed analysis done by clang?
> 
> Another possible (not mutually exclusive) extension point to add static
> analysis into would be through the Tooling interface and compilation
> databases - this would allow just the analysis to be run, without making it
> part of a build (no one would make static analysis part of an interactive
> build, it's too slow, right? the only reason it was integrated into the
> build with scan-build was because it was the best way to discover the build
> commands - but the Tooling/Compilation Database system allows us to
> separate build discovery from tool execution)
> 
> I'm not (yet) terribly familiar with the tooling interface. What exactly do you mean? Also: what do you mean by "interactive build"?
>  
> Adding a flag to a build is also a much lower barrier to entry to get
> started.  
> 
> Example #1: I don't have Perl. Not many machines have Perl. That makes scan-build problematic.
> 
> "Not many" is relative to Python <http://www.fastcompany.com/3026446/the-fall-of-perl-the-webs-most-promising-language>. Python ate the world <http://www.talyarkoni.org/blog/2013/11/18/the-homogenization-of-scientific-computing-or-why-python-is-steadily-eating-other-languages-lunch/>.
>  
> If the work is done right, then the combined compile+analyze execution
> could be faster than a compile action followed by an analyze action.  If
> you're willing to give up "#ifndef __clang_analyzer__", then the AST
> from the compile can be reused by the analyzer.

We cannot drop support for “#ifndef __clang_analyzer__”, which many users rely on for false positive suppression.

> 
> Not just faster, but also better. Currently the analyzer makes some inlining decisions on its own; currently the analyzer can't make use of any kind of analysis or folding done by the optimization passes. 

Since the analyzer works on the AST level, it will not be able to utilize the llvm analysis.

> 
> I'll CC some other static analysis (more knowledgeable & important that me) people.
> 
> Message: 2
> Date: Tue, 2 Feb 2016 10:49:55 -0600
> From: "Craig, Ben via cfe-dev" <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
> To: David Blaikie <dblaikie at gmail.com <mailto:dblaikie at gmail.com>>
> Cc: Clang Dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
> Subject: Re: [cfe-dev] Idea for better invoking static analysis via
>         command line
> Message-ID: <56B0DE33.9020107 at codeaurora.org <mailto:56B0DE33.9020107 at codeaurora.org>>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> The judgment of "how much slower is too much slower" is pretty
> subjective, and depends on the project.  Increasing a project's build
> from 30 seconds to 5 minutes is probably too much for that project.
> Increasing a project's build from 45 minutes to 55 minutes is probably
> fine though.
> 
> Adding a flag to a build is also a much lower barrier to entry to get
> started.  There are a lot of build systems out there, and scan-build and
> compilation databases only work easily with a few of those systems.
> Adding a build flag is pretty easy on every build system that I've ever
> worked with.
> 
> If the work is done right, then the combined compile+analyze execution
> could be faster than a compile action followed by an analyze action.  If
> you're willing to give up "#ifndef __clang_analyzer__", then the AST
> from the compile can be reused by the analyzer.  Even if you aren't
> willing to give up that feature, doing a compile of a file immediately
> followed by an analysis of the same file is probably going to be faster
> than doing all the compiles then all the analyzes due to disk caching
> effects.
> 
> On 2/2/2016 10:23 AM, David Blaikie wrote:
> > Another possible (not mutually exclusive) extension point to add
> > static analysis into would be through the Tooling interface and
> > compilation databases - this would allow just the analysis to be run,
> > without making it part of a build (no one would make static analysis
> > part of an interactive build, it's too slow, right? the only reason it
> > was integrated into the build with scan-build was because it was the
> > best way to discover the build commands - but the Tooling/Compilation
> > Database system allows us to separate build discovery from tool execution)
> >
> > On Tue, Feb 2, 2016 at 6:19 AM, Craig, Ben via cfe-dev
> > <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org> <mailto:cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>> wrote:
> >
> >     I'm all for this idea.  There is precedent for this in other tools
> >     (Visual Studio's /analyze).  I think it also greatly reduces the
> >     need for build interposition via scan-build.
> >
> >     I would ask that you think carefully about the output format of
> >     the detailed analysis for -enable-analyze-pass. If people are
> >     using -enable-analyze-pass on most of their builds, then plist and
> >     html reports are likely to go unread for the most part.  Consider
> >     making "no detailed analysis" an option for -enable-analyze-pass
> >     to help with these use cases.
> >
> >
> >     On 1/29/2016 9:04 PM, <Alexander G. Riccio> via cfe-dev wrote:
> >>     As mentioned by myself, Aaron Ballman, and Philip Reames, in a
> >>     reply to "Proposal: Integrate static analysis test suites", the
> >>     fact that static analysis generates a totally different set of
> >>     warnings than compilation (not a superset), is surprising to some.
> >>
> >>     One possibility, in order to preserve the current behavior for
> >>     any tools that rely on this, is to add an option to clang,
> >>     something like "-enable-analyze-pass" that the user can specify
> >>     to run analysis AND compilation.
> >>
> >>     Thoughts?
> >>
> >>     Sincerely,
> >>     Alexander Riccio
> >>     --
> >>     "Change the world or go home."
> >>     about.me/ariccio <http://about.me/ariccio> <http://about.me/ariccio <http://about.me/ariccio>>
> >>
> >>     <http://about.me/ariccio <http://about.me/ariccio>>
> >>     If left to my own devices, I will build more.
> >>     ⁂
> >>
> >>
> >>     _______________________________________________
> >>     cfe-dev mailing list
> >>     cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org> <mailto:cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
> >>     http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
> >
> >     --
> >     Employee of Qualcomm Innovation Center, Inc.
> >     Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
> >
> >
> >     _______________________________________________
> >     cfe-dev mailing list
> >     cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org> <mailto:cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
> >     http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
> >
> >
> 
> --
> Employee of Qualcomm Innovation Center, Inc.
> Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
> 
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160202/e3ce5c8a/attachment-0001.html <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160202/e3ce5c8a/attachment-0001.html>>
>  
> Message: 1
> Date: Tue, 2 Feb 2016 08:23:51 -0800
> From: David Blaikie via cfe-dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
> To: "Craig, Ben" <ben.craig at codeaurora.org <mailto:ben.craig at codeaurora.org>>
> Cc: Clang Dev <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
> Subject: Re: [cfe-dev] Idea for better invoking static analysis via
>         command line
> Message-ID:
>         <CAENS6Eu-2WDA=f6qre94bQKv3paQsfMd6Krs6OF+z0OGXPwPag at mail.gmail.com <mailto:f6qre94bQKv3paQsfMd6Krs6OF%2Bz0OGXPwPag at mail.gmail.com>>
> Content-Type: text/plain; charset="utf-8"
> 
> Another possible (not mutually exclusive) extension point to add static
> analysis into would be through the Tooling interface and compilation
> databases - this would allow just the analysis to be run, without making it
> part of a build (no one would make static analysis part of an interactive
> build, it's too slow, right? the only reason it was integrated into the
> build with scan-build was because it was the best way to discover the build
> commands - but the Tooling/Compilation Database system allows us to
> separate build discovery from tool execution)
> 
> On Tue, Feb 2, 2016 at 6:19 AM, Craig, Ben via cfe-dev <
> cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>> wrote:
> 
> > I'm all for this idea.  There is precedent for this in other tools (Visual
> > Studio's /analyze).  I think it also greatly reduces the need for build
> > interposition via scan-build.
> >
> > I would ask that you think carefully about the output format of the
> > detailed analysis for -enable-analyze-pass.  If people are using
> > -enable-analyze-pass on most of their builds, then plist and html reports
> > are likely to go unread for the most part.  Consider making "no detailed
> > analysis" an option for -enable-analyze-pass to help with these use cases.
> >
> >
> > On 1/29/2016 9:04 PM, <Alexander G. Riccio> via cfe-dev wrote:
> >
> > As mentioned by myself, Aaron Ballman, and Philip Reames, in a reply to
> > "Proposal: Integrate static analysis test suites", the fact that static
> > analysis generates a totally different set of warnings than compilation
> > (not a superset), is surprising to some.
> >
> > One possibility, in order to preserve the current behavior for any tools
> > that rely on this, is to add an option to clang, something like
> > "-enable-analyze-pass" that the user can specify to run analysis AND
> > compilation.
> >
> > Thoughts?
> >
> > Sincerely,
> > Alexander Riccio
> > --
> > "Change the world or go home."
> > about.me/ariccio <http://about.me/ariccio>
> >
> > <http://about.me/ariccio <http://about.me/ariccio>>
> > If left to my own devices, I will build more.
> > ⁂
> >
> >
> > _______________________________________________
> > cfe-dev mailing listcfe-dev at lists.llvm. <mailto:listcfe-dev at lists.llvm.orghttp>orghttp <mailto:listcfe-dev at lists.llvm.orghttp>://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
> >
> >
> > --
> > Employee of Qualcomm Innovation Center, Inc.
> > Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
> >
> >
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160202/c94f40c1/attachment-0001.html <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160202/c94f40c1/attachment-0001.html>>
> 
> ------------------------------
> 
>  
> Message: 3
> Date: Tue, 2 Feb 2016 08:19:03 -0600
> From: "Craig, Ben via cfe-dev" <cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>>
> To: cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
> Subject: Re: [cfe-dev] Idea for better invoking static analysis via
>         command line
> Message-ID: <56B0BAD7.5030309 at codeaurora.org <mailto:56B0BAD7.5030309 at codeaurora.org>>
> Content-Type: text/plain; charset="utf-8"; Format="flowed"
> 
> I'm all for this idea.  There is precedent for this in other tools
> (Visual Studio's /analyze).  I think it also greatly reduces the need
> for build interposition via scan-build.
> 
> I would ask that you think carefully about the output format of the
> detailed analysis for -enable-analyze-pass.  If people are using
> -enable-analyze-pass on most of their builds, then plist and html
> reports are likely to go unread for the most part.  Consider making "no
> detailed analysis" an option for -enable-analyze-pass to help with these
> use cases.
> 
> On 1/29/2016 9:04 PM, <Alexander G. Riccio> via cfe-dev wrote:
> > As mentioned by myself, Aaron Ballman, and Philip Reames, in a reply
> > to "Proposal: Integrate static analysis test suites", the fact that
> > static analysis generates a totally different set of warnings than
> > compilation (not a superset), is surprising to some.
> >
> > One possibility, in order to preserve the current behavior for any
> > tools that rely on this, is to add an option to clang, something like
> > "-enable-analyze-pass" that the user can specify to run analysis AND
> > compilation.
> >
> > Thoughts?
> >
> > Sincerely,
> > Alexander Riccio
> > --
> > "Change the world or go home."
> > about.me/ariccio <http://about.me/ariccio> <http://about.me/ariccio <http://about.me/ariccio>>
> >
> > <http://about.me/ariccio <http://about.me/ariccio>>
> > If left to my own devices, I will build more.
> > ⁂
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at lists.llvm.org <mailto:cfe-dev at lists.llvm.org>
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev>
> 
> Sincerely,
> Alexander Riccio
> --
> "Change the world or go home."
> about.me/ariccio <http://about.me/ariccio>
> 
>  <http://about.me/ariccio>
> If left to my own devices, I will build more.
> ⁂
> 
> On Fri, Jan 29, 2016 at 10:04 PM, <Alexander G. Riccio> <test35965 at gmail.com <mailto:test35965 at gmail.com>> wrote:
> As mentioned by myself, Aaron Ballman, and Philip Reames, in a reply to "Proposal: Integrate static analysis test suites", the fact that static analysis generates a totally different set of warnings than compilation (not a superset), is surprising to some.
> 
> One possibility, in order to preserve the current behavior for any tools that rely on this, is to add an option to clang, something like "-enable-analyze-pass" that the user can specify to run analysis AND compilation.
> 
> Thoughts?
> 
> Sincerely,
> Alexander Riccio
> --
> "Change the world or go home."
> about.me/ariccio <http://about.me/ariccio>
> 
>  <http://about.me/ariccio>
> If left to my own devices, I will build more.
> ⁂

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20160205/571fc79e/attachment.html>

More information about the cfe-dev mailing list