[PATCH] D70439: [Analyzer][Docs][NFC] Add CodeChecker to the command line tools

Tue Dec 3 19:13:55 PST 2019

NoQ added a comment.

Hey, thanks for waiting on me! I'm slow these days, just 50 more mails to go >.<

This review strikes me as a wiki material. I wonder what's the community stance on making wikis. #showerthoughs

================
Comment at: clang/www/analyzer/codechecker.html:13
+<div id="page">
+<!--#include virtual="menu.html.incl"-->
+<div id="content">
----------------
Note related to your patch, but SSI seem to be completely broken these days; previously the dropdown menus header kept working on the front page but now even that's missing. Patches are very welcome >.<

================
Comment at: clang/www/analyzer/codechecker.html:23
+<pre class="code_example">
+CodeChecker check -b "cd ~/your-project && make clean && make" -o ~/results
+</pre>
----------------
I think you should start with installing CodeChecker. I.e.,
1. Clone it from github: `git clone ...`.
2. ???

================
Comment at: clang/www/analyzer/codechecker.html:28
+<p>
+The <tt>check</tt> command will print an overview of the issues found in your project by the analyzers.
+</p>
----------------
I'm confused. I obtained an overview, but what are the steps that i need to do if i want to actually view the issues? I have to start the web server anyway, right? Then what's the point of of invoking check separately? Can you invoke check from the web server directly?

I wish this document looked more like a step-by-step guide on how to obtain the results (or how to set up a collaborative server). Right now it's a collection of seemingly unrelated solutions for sub-problems that I don't immediately understand how to combine in order to obtain the desired result. 

================
Comment at: clang/www/analyzer/command-line.html:18
+
+<p>The following tools are used commonly to run the analyzer from the command line.
+Both tools are wrapper scripts to drive the analysis and the underlying invocations of the Clang compiler:
----------------
The first thing that we want to underline here is that the user should ABSOLUTELY NOT try to read warnings from the command line. I still see a lot of users who try to read scan-build's standard output and understand warnings from there without seeing any path notes.

I suggest the following intro:

"Static Analyzer is by design a GUI tool. Its purpose is to find buggy execution paths in the program, and such paths are very hard to comprehend by looking at a non-interactive standard output. It is possible, however, to invoke the Static Analyzer from the command line in order to obtain analysis results, and then later view them interactively in a graphical interface."

================
Comment at: clang/www/analyzer/command-line.html:19
+<p>The following tools are used commonly to run the analyzer from the command line.
+Both tools are wrapper scripts to drive the analysis and the underlying invocations of the Clang compiler:
+<ol>
----------------
Do you plan to eventually mention clang-tidy as well?

================
Comment at: clang/www/analyzer/command-line.html:22-27
+  <ul>
+    <li>Preferred on macOS.</li>
+    <li>In tree, part of the LLVM project.</li>
+    <li>Provides limited and unsupported <a href="https://clang.llvm.org/docs/analyzer/user-docs/CrossTranslationUnit.html">Cross Translation Unit (CTU) analysis</a> capabilities.
+    </li>
+  </ul>
----------------
WDYT of the following:

Scan-Build is an old and simple command-line tool that emits static analyzer warnings as HTML files while compiling your project. You can view analysis results in your web browser.
- Useful for individual developers who simply want to view static analysis results at their desk, or in very simple collaborative environment.
- Works on all major platforms (Windows, Linux, macOS) and is available as a package in many linux distributions.
- Does not include support for cross-translation-unit analysis.

================
Comment at: clang/www/analyzer/command-line.html:28
+  </ul>
+  <li><a href="codechecker.html">CodeChecker</a></li>
+  <ul>
----------------
Let's describe the tool a bit as well. Maybe something like this:

CodeChecker is a web server that runs the Static Analyzer on your projects on demand and maintains a database of issues.

- Perfect for managing large amounts of Static Analyzer warnings in a collaborative environment.
- Generally much more feature-rich than scan-build.
- (more stuff here)

================
Comment at: clang/www/analyzer/command-line.html:31
+    <li>Preferred on Linux.</li>
+    <li>Out-of-tree, not part of the LLVM project, hosted on github.</li>
+    <li>On Linux there are many projects for embedded devices which are built only with GCC.
----------------
LLVM is now also hosted on github, so not sure what the message is. We should probably either provide a link or just say that it's out-of-tree.

================
Comment at: clang/www/analyzer/command-line.html:32-33
+    <li>Out-of-tree, not part of the LLVM project, hosted on github.</li>
+    <li>On Linux there are many projects for embedded devices which are built only with GCC.
+    CodeChecker converts these GCC invocations to the appropriate Clang invocations, this way the CSA can work on these projects seamlessly.
+    </li>
----------------
[[ https://github.com/llvm/llvm-project/blob/llvmorg-9.0.0/clang/tools/scan-build/libexec/ccc-analyzer#L350 | The same is true for scan-build ]].

Also, while it is possible that one of the tools does a strictly better job than the other, but the problem is, i think, generally unsolvable, because there's no perfect correspondence between GCC flags and Clang flags. So i think we should avoid strong statements here ("work seamlessly") - even intercepting the build system (let alone setting correct compiler flags) is an unsolvable problem in general.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D70439/new/

https://reviews.llvm.org/D70439