[PATCH] D71963: clang-tidy doc: Add the severities description

Sun Dec 29 12:05:02 PST 2019

aaron.ballman added a comment.

In D71963#1798212 <https://reviews.llvm.org/D71963#1798212>, @sylvestre.ledru wrote:

> I do agree that they are subjective and not perfect.
>
> However, I found the classification extremely useful when you look at the results on big projects.
>  I have been using codechecker (where the severities are coming from) for Firefox and its has been extremely useful to evaluate the importance of the checkers.

IMO, that usefulness comes from consistency when picking a severity. I share the concern that these are pretty subjective descriptions currently. For instance, the guidance you give in this patch is somewhat different than the guidance picked by CERT (https://wiki.sei.cmu.edu/confluence/display/c/How+this+Coding+Standard+is+Organized#HowthisCodingStandardisOrganized-RiskAssessment) and this will lead to discrepancies if it hasn't already.

>> For instance, the CERT rules all come with a severity specified by the rule itself
> 
> Did you see some difference?

I've not looked for them specifically yet (tbh, this severity thing caught me off guard, I didn't see the reviews for adding it), but my concern comes from the fact that the process of picking severity already differs between what's written and one of the coding standards we have checks for.

>> it if each coding standard has drastically different ideas about severity
> 
> Do you have some examples of this occurrence?

Not off the top of my head. I think it would be useful for someone to look at the coding standards we currently have clang-tidy checks for to see if those standards specify a severity for their rules. From there, we can see what commonalities there are between the coding standards and see if we can come up with a heuristic for picking a severity that roughly matches. Or maybe we should only specify a severity when one is picked by a coding standard and not attempt to determine our own.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D71963/new/

https://reviews.llvm.org/D71963