[cfe-dev] How do I use Clang Static Analyzer custom checkers in standing alone tool, or invoke the Clang Static Analyzer from such a tool?

Sun Jul 21 04:27:32 PDT 2019

Hi!

Good luck on your assignment, sounds fun!

On Sun, 21 Jul 2019 at 00:49, Konstantin Rebrov via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> I found the file CoreEngine.h which seems to contain the definitions of
> the data structures used by the Static Analyzer's Engine. Would it be a
> good idea to use it for my project? How can I run the Static Analyzer's
> Engine directly from my application? Is there an example of how to use
> CoreEngine or something like that?
>

I think writing a standalone checker would be a better approach. If you
really need to touch the analyzer's core, I think patching the analyzer
directly would be better approach (unless you are constrained to using a
specific version of Clang)

> On Sat, Jul 20, 2019 at 1:19 PM Konstantin Rebrov <
> krebrov at mail.csuchico.edu> wrote:
>
>> Hello fellow software developers.
>>
>> I have a question. I am working on a software development project. In
>> this project I am creating a standing alone refactoring tool which uses the
>> LLVM/Clang libraries. This software development project entails source to
>> source rewriting of C files, performing refactorings on them. These
>> refactorings include: removing unsupported functions, rewriting problematic
>> constructs, removal of global variables, turning dynamic memory allocations
>> of arrays into variable length stack arrays. I am using AST matchers for
>> this task. My code includes the following libraries:
>> #include "clang/ASTMatchers/ASTMatchFinder.h"
>> #include "clang/Tooling/Core/Replacement.h"
>> #include "clang/Tooling/Refactoring.h"
>> #include "llvm/Support/CommandLine.h"
>>
>> Basically my refactoring tool is a C++ command line application. It takes
>> a list of *.c source files, uses the AST matchers and call back classes to
>> perform the refactorings on these files, and then save the modifications to
>> those same files. In order to make the structure of my project more clear,
>> I am uploading the main C++ source file. This file includes all the header
>> files for the call back classes which implement the refactorings.
>>
>> refactoring_tool.cpp
>> RemovePointerMatchCallback.h
>>
>> This refactoring tool runs on a large legacy code base of C files. For
>> performing the obvious refactorings I have been using the AST matchers. Now
>> I've exhausted their capabilities, meaning that I have completed
>> implementing the obvious refactorings using AST matchers. Now for
>> implementing the more advanced refactorings I need to use the Clang Static
>> Analyzer libraries. I have been studying how to create my own checkers for
>> the Clang Static Analyzer using this lecture:
>> 2012 LLVM Developers’ Meeting: A. Zaks & J. Rose “Building a Checker in
>> 24 hours”
>> <nabble_embed>https://www.youtube.com/watch?v=kdxlsP5QVPw</nabble_embed>
>>
>> I also bought the book "Getting Started with LLVM Core Libraries" by
>> Bruno Cardoso Lopes and Rafael Auler. Chapter 9 of that book teaches you
>> how to write your own checker for the Clang Static Analyzer.
>>
>> If I understood correctly, the usual way to use your own custom checkers
>> would be to install them into the Clang Static Analyzer. Then you run the
>> Clang Static Analyzer as you would usually do it, using an IDE like XCode
>> or via the command line itself: clang --analyze -Xanalyzer
>> -analyzer-checker=core source.c
>>
>> I have some kind of an issue here. In order to make you understand, I
>> should explain you my assignment. I need to detect redundant computations
>> and function calls returning some result. They are assigned to some
>> variable, and then that variable is never referenced:
>> variable = sqrt(pow(variable, 3) + pow(variable, 3) * 8) / 2;
>>
>> Don't ask me why that code is like that, I didn't write it. There are
>> multiple examples such as this. The code base that I'm refactoring is also
>> supposed to be optimized for super high performance, so we can't have
>> redundant calculations like this, when the result is never even used.
>> Creating a refactoring tool to search and replace these bad constructs is
>> my job.
>>
>> When I run code such as the above through the Clang Static Analyzer, it
>> shows me this message:
>> warning: Value stored to 'variable' is never read
>>
>> I need to tap into the power of the Static Analyzer in detecting
>> problematic code constructs such as these. I'm trying to make my own
>> checker that would not only just detect such occurences and emit a warning,
>> but also to create replacements of these occurences with nothing. I want my
>> checker to just edit the source file that it's processing and perform the
>> replacements just like the AST matcher call backs do.
>>
>> My direct supervisor said that the whole C++ refactoring tool that I'm
>> writing has to be a single executable standing alone, and that it needs to
>> process the source code files and perform the refactorings in place. Now in
>> the main function of this refactoring tool I'm invoking the Clang Compiler
>> and providing AST matchers to it:
>> // Run the tool
>>         auto result =
>> tool.runAndSave(newFrontendActionFactory(&mf).get());
>>         if (result != 0) {
>>                 errs() << "Error in the Refactoring Tool: " << result <<
>> "\n";
>>                 return result;
>>         }
>>
>> As I will be also writing a custom static analysis checker, I also need
>> to add this to the main function, and I need to have some way to invoke the
>> Clang Static Analyzer from the main function of the executable. Is there a
>> function similar to clang::tooling::RefactoringTool::runAndSave(), but for
>> the Clang Static Analyzer, that will install my custom static analysis
>> checkers and run the Clang Static Analyzer over the legacy source code
>> files provided in the command line of my executable tool? This is what I
>> need.
>>
>
I think some of our unit tests might be what you're looking for:
https://github.com/llvm/llvm-project/blob/master/clang/unittests/StaticAnalyzer/RegisterCustomCheckersTest.cpp

> If there is no such "hot and ready to eat" function, how does the Clang
>> Static Analyzer work, what kind of code in the LLVM libraries starts up the
>> Clang Static Analyzer and runs it over the input files? If it's necessary,
>> I will create such a function to invoke the Clang Static Analyzer from my
>> code, myself and use it like that. I am not afraid to dig into the LLVM
>> source code to see how can I duplicate the startup and file processing
>> behaviors of the Clang Static Analyzer in my own code. But I think that
>> would be the hard way of doing things. If there is an easier way to invoke
>> the Clang Static Analyzer with my custom checkers programatically, please
>> let me know.
>>
>> And I want my static analysis checkers to actually perform the
>> replacements, to remove the problematic code constructs are they are found.
>> To do this I think that I would need to know the SourceLocation of these
>> detected code constructs. It is not enough for me to simply print a warning
>> to the screen. The C code base is HUGE and it would take an inordinate
>> amount of time to perform the replacements manually.
>> The lecture from the 2012 LLVM Developers’ Meeting that I watched, Anna
>> Zaks and Jordan Rose didn't really explain how to perform the replacements
>> inside the static analysis checkers. They just had their custom checker
>> print a warning to the screen. However, as I explained, my requirements are
>> different.
>>
>
Uhh, I can't say anything smart on that, but I vaguely remember folks
talking about similar desires in the mailing list before. You mentioned you
made great progress on an AST-only based solution, did you also manage to
make refactorings with it?

> Also, one of the requirements is that my application needs to perform the
>> replacements in place. The company needs a standing alone internal tool.
>> The whole process needs to be seamless. My application needs to be easily
>> used by the other engineers. They are electrical engineers and wouldn't
>> know any better, so that's why I'm automating all this stuff for them.
>>
>> What changes should I add to my Makefile to be able to link to the Clang
>> Static Analyzer's API? I want to use it in my project.
>>
>
It seems like to me that you'd prefer to statically link your checker
against clang, and not load it as a checker plugin (which we don't even
support that much), but the CMakefile here might be useful:
https://github.com/Szelethus/minimal_csa_plugin

> So far I've achieved some large amount of progress using the AST matchers
>> and now I need to extend my application with the static analysis checkers
>> as for my direct supervisor's assignment. Please don't scold me too much if
>> I am wrong somewhere. This is my first real development job, and first time
>> using the Clang/LLVM libraries.
>>
>
Please follow up if you have any questions! :)

Cheers,
Kristóf

> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20190721/e5bf8919/attachment.html>