[cfe-dev] [CrossTU] [CTU] Status of Cross Translation Unit Static Analysis, and a new build server

[Gábor Márton via cfe-dev]

Hi,

First, we try to find `clang-extdef-mapping` next to the used `clang`
binary. Then we search in the PATH for `clang-extdef-mapping`.
We suspect that when you do not specify clang-10 then
`clang-extdef-mapping` is not found in the directory where your clang
selected by the PATH resides (provided by the Debian package?).
Anyway, thanks for the issue report. We are considering to give better
guidelines when the `clang-extdef-mapping` is not found. Currently we just
bail out with a simple "unrecognized arguments" error in this case, which
is indeed not that user friendly.

Thanks,
Gabor

On Mon, Nov 18, 2019 at 6:33 PM Roman Lebedev <lebedev.ri at gmail.com> wrote:

> On Mon, Nov 18, 2019 at 8:18 PM Gábor Márton via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
> >
> > Dear Clang Community,
> >
> >
> > TLDR; I'd like to encourage you to experiment with CTU static analysis
> because it has evolved a lot this year! Usage documentation is available
> online.
> >
> >
> >
> > This year, we have landed several ASTImporter patches that made it
> possible to analyze even C++11/14 projects with reasonable stability. I can
> confidently assure you that the upstream master of llvm/llvm-project is as
> stable as our downstream fork. We've made enormous efforts to properly
> implement the error handling of ASTImporter and this way we could
> dramatically improve the stability of CTU. We do have a few more patches
> that we still want to land in the future, but they are not error handling
> related. From Clang version 10, we no longer plan to maintain our CTU
> downstream fork.
> >
> >
> > We have set up a publicly available Jenkins build server that
> continuously analyzes the following C and C++ projects:
> >
> > - Tmux (C)
> >
> > - Curl (C)
> >
> > - Redis (C)
> >
> > - Xerces (C++14)
> >
> > - Bitcoin (C++11)
> >
> > - Protobuf (C++11/C++14)
> >
> > CTU analysis results are compared to non-CTU results, both for C
> projects and C++ projects. CTU always results in more findings, and the
> false-positive/true-positive ratio remains roughly the same [1]. We are
> monitoring the analysis job, and if an assertion/crash happens then we are
> going to get in touch with the author of the commit that plausibly caused
> the error. We also have a buildbot patch in Phabricator which is ought to
> analyse only one simple C project (Tmux). We decided to install  Jenkins
> because of ownership and flexibility. The buildbot code is pretty
> convoluted and review and communication are very slow. The buildbot also
> uses CodeChecker as a dependency, changes to its workflow requires changes
> in the buildbot configuration as well.
> >
> >
> > Notes:
> >
> > Please note that our primary target for CTU is Linux, and we encourage
> everybody to use CodeChecker for CTU (scan-build is no longer supported by
> CTU developers). Projects that use templates heavily may expect an increase
> in analysis time. Also note that CTU can be very memory consuming, e.g. in
> case of the analysis run on LLVM code we've seen 10GB of resident memory
> usage for one process. Thus, it may be useful to set a maximum limit for
> the loaded ASTUnits (e.g. -analyzer-config ctu-import-threshold=8, with
> CodeChecker you need to edit an saargs file).
>
> I was initially going to write a mail that CodeChecker trunk + clang
> trunk isn't sufficient, since it complained for me:
> $ clang --version
> clang version 10.0.0-+20191116100608+584704c725a-1~exp1~20191116211218.2875
> Target: x86_64-pc-linux-gnu
> Thread model: posix
> InstalledDir: /usr/local/bin
> $ CodeChecker analyze --ctu build-Clang-SANITIZE/compile_commands.json
> -o clang-sa-reports
> usage: CodeChecker [-h] {analyze} ...
> CodeChecker: error: unrecognized arguments: --ctu
>
> Nut apparently the detection of clang to use is simply broken.
> Because if i follow
> https://github.com/Ericsson/codechecker#configuring-clang-version
> and specify clang-10 it appears to start working. I guess i should file a
> bug.
> Also, https://github.com/Ericsson/codechecker/issues/1841 is *really*
> inconvenient :/
>
> Roman.
>
> > [1] 2017 EuroLLVM Developers’ Meeting: G. Horvath “Cross Translational
> Unit Analysis in Clang ...”
> >
> >
> > Cheers,
> >
> > Gabor Marton
> >
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20191118/cc39d2be/attachment.html>