[cfe-dev] python libclang

[Gregory Szorc]

While multiprocessing.Pool offers an alluring API and promises of faster
performance, it has many gotchas and can blow up in many ways. I suggest
getting your tool to work right without multiprocessing.Pool. Only then
should you attempt to use multiprocessing.Pool.

On 2/11/2013 7:53 AM, Laszlo Nagy wrote:
> Hi,
>
> I'm trying to use python binding to libclang to generate include
> graphs. I took the example program
> (bindings/python/examples/cindex/cindex-includes.py) and add support
> to read a JSON compilation database, therefore I can have include
> graph for a whole project.
>
> Now, I'm running against the Clang project. And it fails... I compiled
> Clang with itself. Without any problem... My tool read the
> `compile_commands.json` generated by cmake and pass all arguments
> (except the compiler and the module). So it supposes to run the same
> as it was by clang... And this is not happen with any modules, it
> compiles all LLVM code and the majority of Clang.
>
> Any hints how can I fix it? Or make it reproducible for a bug report?
>
> Regards,
> Laszlo
>
>
> The related JSON file entry looks like this:
>
> {
>   "directory":
> "/home/laszlona/Programming/llvm-svn.src/build/tools/clang/lib/Serialization",
>   "command":
> "/home/laszlona/Programming/llvm-svn/usr/local/bin/clang++  
> -D_GNU_SOURCE -D_DEBUG -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS
> -D__STDC_LIMIT_MACROS -D_GNU_SOURCE -DCLANG_ENABLE_ARCMT
> -DCLANG_ENABLE_REWRITER -DCLANG_ENABLE_STATIC_ANALYZER  -fPIC
> -fvisibility-inlines-hidden -Wno-nested-anon-types
> -Wcovered-switch-default -fno-common -Woverloaded-virtual -Wcast-qual
> -fno-strict-aliasing -pedantic -Wno-long-long -Wall -W
> -Wno-unused-parameter -Wwrite-strings -fno-rtti -O2 -g
> -I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/lib/Serialization
> -I/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization -I/home/laszlona/Programming/llvm-svn.src/tools/clang/include
> -I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/include
> -I/home/laszlona/Programming/llvm-svn.src/build/include
> -I/home/laszlona/Programming/llvm-svn.src/include    -Wall -W
> -Wno-unused-parameter -Wwrite-strings -Wmissing-field-initializers
> -pedantic -Wno-long-long -fno-exceptions -o
> CMakeFiles/clangSerialization.dir/ASTWriterStmt.cpp.o -c
> /home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization/ASTWriterStmt.cpp",
>   "file":
> "/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization/ASTWriterStmt.cpp"
> },
>
>
> The error report looks like this:
>
> python:
> /home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Sema/SemaInit.cpp:3153:
> clang::OverloadingResult
> TryRefInitWithConversionFunction(clang::Sema&, const
> clang::InitializedEntity&, const clang::InitializationKind&,
> clang::Expr*, bool, clang::InitializationSequence&): Assertion
> `!S.CompareReferenceRelationship(Initializer->getLocStart(), T1, T2,
> DerivedToBase, ObjCConversion, ObjCLifetimeConversion) && "Must have
> incompatible references when binding via conversion"' failed.
> libclang: crash detected during parsing: {
>   'source_filename' :
> '/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization/ASTWriterStmt.cpp'
>   'command_line_args' : ['-D_GNU_SOURCE', '-D_DEBUG',
> '-D__STDC_CONSTANT_MACROS', '-D__STDC_FORMAT_MACROS',
> '-D__STDC_LIMIT_MACROS', '-D_GNU_SOURCE', '-DCLANG_ENABLE_ARCMT',
> '-DCLANG_ENABLE_REWRITER', '-DCLANG_ENABLE_STATIC_ANALYZER', '-fPIC',
> '-fvisibility-inlines-hidden', '-Wno-nested-anon-types',
> '-Wcovered-switch-default', '-fno-common', '-Woverloaded-virtual',
> '-Wcast-qual', '-fno-strict-aliasing', '-pedantic', '-Wno-long-long',
> '-Wall', '-W', '-Wno-unused-parameter', '-Wwrite-strings',
> '-fno-rtti', '-O2', '-g',
> '-I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/lib/Serialization',
> '-I/home/laszlona/Programming/llvm-svn.src/tools/clang/lib/Serialization',
> '-I/home/laszlona/Programming/llvm-svn.src/tools/clang/include',
> '-I/home/laszlona/Programming/llvm-svn.src/build/tools/clang/include',
> '-I/home/laszlona/Programming/llvm-svn.src/build/include',
> '-I/home/laszlona/Programming/llvm-svn.src/include', '-Wall', '-W',
> '-Wno-unused-parameter', '-Wwrite-strings',
> '-Wmissing-field-initializers', '-pedantic', '-Wno-long-long',
> '-fno-exceptions', '-o',
> 'CMakeFiles/clangSerialization.dir/ASTWriterStmt.cpp.o', '-c'],
>   'unsaved_files' : [],
>   'options' : 0,
> }
> Traceback (most recent call last):
>   File "./bin/include_graph_to_gexf.py", line 107, in <module>
>     pool.apply(compile_module_to_queue, args=(q, m))
>   File "/usr/lib64/python2.7/multiprocessing/pool.py", line 220, in apply
>     return self.apply_async(func, args, kwds).get()
>   File "/usr/lib64/python2.7/multiprocessing/pool.py", line 528, in get
>     raise self._value
> AssertionError
> Exception ctypes.ArgumentError: "argument 1: <type
> 'exceptions.AttributeError'>: 'TranslationUnit' object has noProcess
> Process-2:
> Traceback (most recent call last):
>   File "/usr/lib64/python2.7/multiprocessing/process.py", line 258, in
> _bootstrap
>     self.run()
>   File "/usr/lib64/python2.7/multiprocessing/process.py", line 114, in run
>     self._target(*self._args, **self._kwargs)
>   File "./bin/include_graph_to_gexf.py", line 76, in
> build_graph_from_queue
>     for e in iter(queue.get, 'STOP'):
>   File "<string>", line 2, in get
>   File "/usr/lib64/python2.7/multiprocessing/managers.py", line 759,
> in _callmethod
>     kind, result = conn.recv()
> EOFError
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20130213/46f83eef/attachment.html>