[cfe-dev] How to use libtooling to parse multiple files at once? and succesfully find stddef.h?

rayjcwu rayjcwu at gmail.com
Mon Nov 4 23:45:02 PST 2013 

Poking here and there but I still can't figure out why. 
I take away the environment variables C_INCLUDE_PATH. Take tempname.c for
example, if I use verbose mode in make file, I have 

=====
depbase=`echo tempname.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\ 
        /home/jcwu/repos/llvm-release/Release/bin/clang -DHAVE_CONFIG_H -I.
-I../src     -O2 -Wall -MT tempname.o -MD -MP -MF $depbase.Tpo -c -o
tempname.o tempname.c &&\ 
        mv -f $depbase.Tpo $depbase.Po 
=====

Which is almost the same with the command in compile_commands.json

====
/home/jcwu/repos/llvm-release/Release/bin/clang -DHAVE_CONFIG_H -I. -I../src
-O2 -Wall -MT tempname.o -MD -MP -MF .deps/tempname.Tpo -c -o tempname.o
tempname.c
====

Original compile_commands.json has a lot of stddef.h not found error. I add
paths generated by -v option, so for this output

====
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 .
 ../src
* /usr/local/include             
* /home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include
* /usr/include/x86_64-linux-gnu
* /usr/include
End of search list.
====

I put those paths marked with * with -I option in compile_commands.json.
There are still ot of errors, take this error message for example

===
Processing: /home/jcwu/repos/wget-1.14/lib/tempname.c.
tempname.c:21:11: fatal error: cannot open file './config.h': No such file
or directory
# include <config.h>
          ^
1 error generated.
Error while processing /home/jcwu/repos/wget-1.14/lib/tempname.c.
===

The modified section for this file in compile_commands.json is

===
  {
    "directory": "/home/jcwu/repos/wget-1.14/lib",
    "command": "/home/jcwu/repos/llvm-release/Release/bin/clang
-DHAVE_CONFIG_H -I. -I../src -I/usr/local/include
-I/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include
-I/usr/include/x86_64-linux-gnu -I/usr/include  -O2 -Wall -MT tempname.o -MD
-MP -MF .deps/tempname.Tpo -c -o tempname.o tempname.c",
    "file": "/home/jcwu/repos/wget-1.14/lib/tempname.c"
  },

===

I could run that command without any trouble. adding -v option, I have

===
clang version 3.4 (trunk 192426)
Target: x86_64-unknown-linux-gnu
Thread model: posix
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.6
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.6.3
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.6
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.6.3
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.6
 "/home/jcwu/repos/llvm-release/Release/bin/clang" -cc1 -triple
x86_64-unknown-linux-gnu -emit-obj -disable-free -disable-llvm-verifier
-main-file-name tempname.c -mrelocation-model static -fmath-errno
-masm-verbose -mconstructor-aliases -munwind-tables -target-cpu x86-64
-target-linker-version 2.22 -momit-leaf-frame-pointer -v -coverage-file
/home/jcwu/repos/wget-1.14/lib/tempname.o -resource-dir
/home/jcwu/repos/llvm-release/Release/bin/../lib/clang/3.4 -dependency-file
.deps/tempname.Tpo -sys-header-deps -MP -MT tempname.o -D HAVE_CONFIG_H -I .
-I ../src -I /usr/local/include -I
/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include -I
/usr/include/x86_64-linux-gnu -I /usr/include -cxx-isystem
/usr/local/include -cxx-isystem
/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include -cxx-isystem
/usr/include/x86_64-linux-gnu -cxx-isystem /usr/include -cxx-isystem
/usr/src/linux-headers-3.5.0-42-generic/include/config/i2o/
-internal-isystem /usr/local/include -internal-isystem
/home/jcwu/repos/llvm-release/Release/bin/../lib/clang/3.4/include
-internal-externc-isystem /usr/include/x86_64-linux-gnu
-internal-externc-isystem /include -internal-externc-isystem /usr/include
-O2 -Wall -fdebug-compilation-dir /home/jcwu/repos/wget-1.14/lib
-ferror-limit 19 -fmessage-length 205 -mstackrealign -fobjc-runtime=gcc
-fdiagnostics-show-option -fcolor-diagnostics -vectorize-loops
-vectorize-slp -o tempname.o -x c tempname.c
clang -cc1 version 3.4 based upon LLVM 3.4svn default target
x86_64-unknown-linux-gnu
ignoring nonexistent directory "/include"
ignoring duplicate directory "/usr/local/include"
  as it is a non-system directory that duplicates a system directory
ignoring duplicate directory
"/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include"
  as it is a non-system directory that duplicates a system directory
ignoring duplicate directory "/usr/include/x86_64-linux-gnu"
  as it is a non-system directory that duplicates a system directory
ignoring duplicate directory "/usr/include"
  as it is a non-system directory that duplicates a system directory
#include "..." search starts here:
#include <...> search starts here:
 .
 ../src
 /usr/local/include
 /home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
===

searching for a while and I could find out the <config.h> is actually
../src/config.h.
However, it is already specified in -I../src, I don't know why clang
libtooling cannot find it.
Since it's complaing "cannot open file './config.h'", I also tried to remove
-I., but obviously it's not the case.
If I change the command to the one generated by -v option, the error
messages become

===
error: unknown argument: '-cc1'
error: unknown argument: '-triple'
error: unknown argument: '-main-file-name'
error: unknown argument: '-mrelocation-model'
error: unknown argument: '-masm-verbose'
error: unknown argument: '-mconstructor-aliases'
error: unknown argument: '-munwind-tables'
error: unknown argument: '-target-cpu'
error: unknown argument: '-target-linker-version'
error: unknown argument: '-coverage-file'
error: unknown argument: '-sys-header-deps'
error: unknown argument: '-fdebug-compilation-dir'
error: unknown argument: '-ferror-limit'
error: unknown argument: '-vectorize-slp'
clang version 3.4 (trunk 192426)
Target: x86_64-unknown-linux-gnu
Thread model: posix
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.6
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.6.3
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.6
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.6.3
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/4.6
warning: x86_64-unknown-linux-gnu: 'linker' input unused
warning: static: 'linker' input unused
warning: x86-64: 'linker' input unused
warning: 2.22: 'linker' input unused
warning: /home/jcwu/repos/wget-1.14/lib/tempname.o: 'linker' input unused
warning: /usr/local/include: 'linker' input unused
warning: /home/jcwu/repos/llvm-release/Release/bin/../lib/clang/3.4/include:
'linker' input unused
warning: /usr/include/x86_64-linux-gnu: 'linker' input unused
warning: /include: 'linker' input unused
warning: /usr/include: 'linker' input unused
warning: /home/jcwu/repos/wget-1.14/lib: 'linker' input unused
warning: 19: 'linker' input unused
warning: 205: 'linker' input unused
error: unable to handle compilation, expected exactly one compiler job in '
"/home/jcwu/repos/clang.libtooling.annotator/main" "-cc1" "-triple"
"x86_64-unknown-linux-gnu" "-fsyntax-only" "-disable-free"
"-disable-llvm-verifier" "-main-file-name" "tempname.c" "-mrelocation-model"
"static" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases"
"-munwind-tables" "-target-cpu" "x86-64" "-target-linker-version" "2.22"
"-momit-leaf-frame-pointer" "-v" "-resource-dir"
"/home/jcwu/repos/llvm-release/Release/bin/../lib/clang/3.4" "-MP" "-MT"
"tempname.o" "-D" "HAVE_CONFIG_H" "-I" "." "-I" "../src" "-I"
"/usr/local/include" "-I"
"/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include" "-I"
"/usr/include/x86_64-linux-gnu" "-I" "/usr/include" "-internal-isystem"
"/usr/local/include" "-internal-isystem"
"/home/jcwu/repos/llvm-release/Release/bin/../lib/clang/3.4/include"
"-internal-externc-isystem" "/usr/include/x86_64-linux-gnu"
"-internal-externc-isystem" "/include" "-internal-externc-isystem"
"/usr/include" "-O2" "-Wall" "-fdebug-compilation-dir"
"/home/jcwu/repos/wget-1.14/lib" "-ferror-limit" "19" "-fmessage-length"
"205" "-backend-option" "-force-align-stack" "-mstackrealign"
"-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics"
"-vectorize-loops" "-vectorize-slp" "-x" "c" "tempname.c"; 
"/home/jcwu/repos/clang.libtooling.annotator/main" "-cc1" "-triple"
"x86_64-unknown-linux-gnu" "-fsyntax-only" "-disable-free"
"-disable-llvm-verifier" "-main-file-name" "tempname.c" "-mrelocation-model"
"static" "-fmath-errno" "-masm-verbose" "-mconstructor-aliases"
"-munwind-tables" "-target-cpu" "x86-64" "-target-linker-version" "2.22"
"-momit-leaf-frame-pointer" "-v" "-resource-dir"
"/home/jcwu/repos/llvm-release/Release/bin/../lib/clang/3.4" "-MP" "-MT"
"tempname.o" "-D" "HAVE_CONFIG_H" "-I" "." "-I" "../src" "-I"
"/usr/local/include" "-I"
"/home/jcwu/repos/llvm-release/Release/lib/clang/3.4/include" "-I"
"/usr/include/x86_64-linux-gnu" "-I" "/usr/include" "-internal-isystem"
"/usr/local/include" "-internal-isystem"
"/home/jcwu/repos/llvm-release/Release/bin/../lib/clang/3.4/include"
"-internal-externc-isystem" "/usr/include/x86_64-linux-gnu"
"-internal-externc-isystem" "/include" "-internal-externc-isystem"
"/usr/include" "-O2" "-Wall" "-fdebug-compilation-dir"
"/home/jcwu/repos/wget-1.14/lib" "-ferror-limit" "19" "-fmessage-length"
"205" "-backend-option" "-force-align-stack" "-mstackrealign"
"-fobjc-runtime=gcc" "-fdiagnostics-show-option" "-fcolor-diagnostics"
"-vectorize-loops" "-vectorize-slp" "-x" "c" "tempname.c"; '
Error while processing /home/jcwu/repos/wget-1.14/lib/tempname.c.
===

Maybe it's not the correct way to use it in compile_commands.json



--
View this message in context: http://clang-developers.42468.n3.nabble.com/How-to-use-libtooling-to-parse-multiple-files-at-once-and-succesfully-find-stddef-h-tp4035389p4035618.html
Sent from the Clang Developers mailing list archive at Nabble.com.

More information about the cfe-dev mailing list