[PATCH] D57858: [analyzer] Add a new frontend flag to display all checker options

Daniel Krupp via Phabricator via cfe-commits cfe-commits at lists.llvm.org
Mon May 13 08:37:09 PDT 2019


dkrupp added a comment.

In D57858#1499414 <https://reviews.llvm.org/D57858#1499414>, @Szelethus wrote:

> In D57858#1498640 <https://reviews.llvm.org/D57858#1498640>, @NoQ wrote:
>
> > So, like, the global picture is as follows. In our case the Driver (i.e., --analyze) is not much more user facing than frontend flags. It's still fairly unusable directly, as our Static Analyzer is generally not a command-line tool. The real user-facing stuff is the GUIs such as scan-build or CodeChecker. These GUIs decide themselves on what options they want to expose. For instance, you have a right to decide that CodeChecker shouldn't support the aggressive mode of the move-checker and don't expose it as an option in your GUI. In this sense it's not really useful to provide a centralized `-help` of all user-facing options.
> >
> > But it sounds as if you want to change this situation and provide a single source of truth on what are the user-facing options. Particular GUIs can still ignore them, but you don't want to hardcode flags in CodeChecker, but instead you want to rely on clang to provide the list of supported options for you and, as a side effect, for scan-build users (if you also add a scan-build help flag). I'm totally in favor of crystallizing such list of user-facing flags, and this patch sounds like a good step in that direction, assuming that non-user-facing options are hidden.
>
>
> That describes my intention quite well :)
>
> > I'm still in favor of hiding alpha checkers (as they are for development only, much like debug flags; i'd recommend hiding them in the CodeChecker UI as well)
> > 
> > Now, why do we care about frontend/driver flags when they're unusable by definition? That's because we have a mental trauma after seeing a few powerusers actively explore those flags, observe that they don't work, and then tell everybody that the Analyzer is broken. So there's a threshold, based on a tiny but painful bit of practical experience, that says that documentation of developer-only features on llvm.org or in code comments is fine, but documentation printed by the released binary itself is not fine.
>
> What you say sounds very reasonable. Still, I am kind of hesitant about hiding all alpha checkers: I initially intended to hide only are developer-only checkers (modeling, debug). I guess if we define alpha checkers (as you stated numerous times) as incomplete, under development, are missing half their limbs and crash if you look at them the wrong way, sure, they belong in the developer-only category. But checkers such as mine (UninitializedObjectChecker), for the longest time were very stable, have been enabled by default for our internal projects, despite only recently moving out of alpha.
>
> Then agaaain, if we're that stubborn about alpha checkers, we could might as well dig them out of `-analyzer-checker-help-hidden`, and leave the rest there. Untangling what alpha checkers depend on one another could be solved by making yet another frontend flag that would display checker dependencies, which would be super easy since D54438 <https://reviews.llvm.org/D54438>, or create an `-analyzer-checker-help-alpha` flag that would display alpha, but not developer-only checkers. @dkrupp @o.gyorgy Do you have any feelings on this?
>
> > and we should probably automatically hide options of checker that are hidden.
>
> Checker options are a different kind of animal entirely. I think we should definitely let the user fine-tune some modeling checkers, for instance, `unix.DynamicMemoryModeling:Optimistic`, despite us not really wanting to let  anyone (even developers really) mess around whether `unix.DynamicMemoryModeling` should be enabled. While that specific option is, to put it nicely, a little esoteric, making some decisions the analyzer makes less conservative, or limiting state splits to help performance may be desirable in the future.
>
> Let's move the rest of the discussion directly related to hiding checker options to D61839 <https://reviews.llvm.org/D61839>!


Yes, it would be great if the clang static analyzer would be the ultimate source of information with respect to the checkers and checker options. Then we would not need to split this info between the front-end (scanbuild, codechecker) and the analyzer.

Some alpha checkers are considerably more mature than others and are quite usable. In our experience, there are some users who are keen to run these checkers on their code and report back any false positives to us. So in this sense these are not "developer only" checkers. So I think we should let the users list them, read their descriptions and try them out. Some of them will come back with useful feedback as to how to improve them further. Some users would not care if the checker gives some more false positives than the "mature" checkers if they can catch some true positives with them.

@Szelethus  has a good point in supporting three checker and option categories
-stable: checkers that are stable and which report "small number of false positives". (-analyzer-checker-help)
-experimental: checkers in alpha stage, with more false positives than usual, but which still should not crash. (-analyzer-checker-help-experimental or -analyzer-checker-help-alpha or -analyzer-checker-help-hidden ). We can add a disclaimer for this option.
-developer: modeling checkers, debug checkers that is used by checker/analyzer developer (-analyzer-checker-help-devel)

we could create the same help categories for checker options and non-checker configuration.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D57858/new/

https://reviews.llvm.org/D57858





More information about the cfe-commits mailing list