[PATCH] D75665: [analyzer] On-demand parsing capability for CTU

[Endre Fülöp via Phabricator via cfe-commits]

gamesh411 added a comment.

This patch adds an alternative way of loading ASTs to provide the CTU definitions needed during analysis.
The additional approach is to use Tooling with a user-provided compile_commands.json,
and look up the needed file from the output of the ext-def-mapping tool (which is now used for both approaches).

As the CTUDir string prefix is only needed in case of the old approach, I have refactored the external definition mapping storage to NOT include that.

Experience with diagnostics generated by the on-demand approach shows that location information inside plists contain file paths as they were in the compile_commands.json.
I would say, a uniform handling of paths is desirable, so an ArgAdjuster is added to the Clang Tool which implements the path normalization step.

The following test still fails. As far as I can tell, this is due to the C language being laxer. The concrete failure:

  ******************** TEST 'Clang :: Analysis/ctu-on-demand-parsing.c' FAILED ********************
  Script:
  --
  : 'RUN: at line 1';   rm -rf /home/gamesh411/projects/clang-rwd/tools/clang/test/Analysis/Output/ctu-on-demand-parsing.c.tmp && mkdir /home/gamesh411/projects/clang-rwd/tools/clang/test/Analysis/Output/ctu-on-demand-parsing.c.tmp
  : 'RUN: at line 2';   mkdir -p /home/gamesh411/projects/clang-rwd/tools/clang/test/Analysis/Output/ctu-on-demand-parsing.c.tmp/ctudir2
  : 'RUN: at line 3';   echo '[{"directory":"/home/gamesh411/projects/llvm-project/clang/test/Analysis/Inputs","command":"clang -x c -std=c89 -c ctu-other.c","file":"ctu-other.c"}]' | sed -e 's/\\/\\\\/g' > /home/gamesh411/projects/clang-rwd/tools/clang/test/Analysis/Output/ctu-on-demand-parsing.c.tmp/ctudir2/compile_commands.json
  : 'RUN: at line 4';   /home/gamesh411/projects/clang-rwd/bin/clang-extdef-mapping /home/gamesh411/projects/llvm-project/clang/test/Analysis/Inputs/ctu-other.c > /home/gamesh411/projects/clang-rwd/tools/clang/test/Analysis/Output/ctu-on-demand-parsing.c.tmp/ctudir2/externalDefMap.txt
  : 'RUN: at line 5';   /home/gamesh411/projects/clang-rwd/bin/clang -cc1 -internal-isystem /home/gamesh411/projects/clang-rwd/lib/clang/11.0.0/include -nostdsysteminc -triple x86_64-pc-linux-gnu -fsyntax-only -x c -std=c89 -analyze    -analyzer-checker=core,debug.ExprInspection    -analyzer-config experimental-enable-naive-ctu-analysis=true    -analyzer-config ctu-dir=/home/gamesh411/projects/clang-rwd/tools/clang/test/Analysis/Output/ctu-on-demand-parsing.c.tmp/ctudir2    -analyzer-config ctu-on-demand-parsing=true    -analyzer-config ctu-on-demand-parsing-database="/home/gamesh411/projects/clang-rwd/tools/clang/test/Analysis/Output/ctu-on-demand-parsing.c.tmp/ctudir2/compile_commands.json"    -verify /home/gamesh411/projects/llvm-project/clang/test/Analysis/ctu-on-demand-parsing.c
  --
  Exit Code: 1
  
  Command Output (stderr):
  --
  warning: treating 'c' input as 'c++' when in C++ mode, this behavior is deprecated [-Wdeprecated]
  warning: unknown warning option '-Wno-maybe-uninitialized'; did you mean '-Wno-uninitialized'? [-Wunknown-warning-option]
  /home/gamesh411/projects/llvm-project/clang/test/Analysis/Inputs/ctu-other.c:47:26: error: 'DataType' cannot be defined in a parameter type
  int structInProto(struct DataType {int a;int b; } * d) {
                           ^
  1 warning and 1 error generated.
  Error while processing /home/gamesh411/projects/llvm-project/clang/test/Analysis/Inputs/ctu-other.c.

Would it be ok to remove those tests? Currently, I see no solution for solving this test case, any insight is welcome.



================
Comment at: clang/unittests/CrossTU/CrossTranslationUnitTest.cpp:176
 
-TEST(CrossTranslationUnit, CTUDirIsHandledCorrectly) {
-  llvm::StringMap<std::string> Index;
----------------
CTUDir is no longer prepended to the mapping by default, as on-demand parsing does not use the CTUDir prefix.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75665/new/

https://reviews.llvm.org/D75665