[cfe-dev] issue with libclang and Python bindings

[Brian Cain via cfe-dev]

On Sun, Nov 5, 2017 at 2:39 PM, Trevor Laughlin via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> Hello,
>
>
>
> I’m using libclang and its Python bindings to build a binding generator to
> a large C++ library using pybind11. All is going well but I am experiencing
> a strange issue and I’m hoping someone on this list has an idea.
> Unfortunately, I’m not able to reproduce the issue on a small scale so I’ll
> just describe it as best I can and if anyone has any thoughts please share.
>
>
>
> I have a single include file (“all_includes.h”) that includes all the
> headers of the C++ library. This file is parsed by libclang and I use the
> results to write pybind11 code via the libclang Python bindings. I don’t
> get any translation errors or warnings. In some cases, certain types
> (usually parameter declarations) get resolved to “int” instead of the
> actual type like “std::list<TypeA>”.
>
>
>

What kind of behavior do you get when you use a command-line clang or other
C++ compiler with a source file that includes "all_includes.h"?

I don't know much about libclang's operation but here's a wild guess: are
you getting implicit int parameters for undeclared functions?  If so it
would seem that libclang might think this is C source and not C++.  Going
out on a big limb: maybe this is also why you have trouble with types using
the scope resolution operator?  Are you giving a header file to libclang as
if it were a translation unit for compilation?  If that's the case then
does the behavior change if you call it "all_includes.hpp" or
"all_includes.H"?  Have you tried explicitly indicating the language w/" -x
"?   See
https://clang.llvm.org/docs/ClangCommandLineReference.html#introduction
towards the end.


> If my “all_inlcudes.h” has only the header file where the issue occurs,
> the types are resolved correctly. It’s only when I include all my headers
> (or a large number of them) does this strange behavior start to appear. I
> can’t figure out a root reason or pattern for it happening, the only thing
> I’ve noticed is it seems to happen with things in a namespace liked
> “std::list<>” or “TypeA::value”.
>
>
>

Is it possible that these are from preprocessor macros and they're getting
omitted when some critical definitions are absent/present?



> One other thing, the C++ library makes heavy use of class templates. In
> libclang, these class declarations have no children (i.e., constructors,
> methods, etc.). In order to generate the binding code I have to retrieve
> the cursor to the class template, write its code then replace the template
> parameters with a string replace method. Is there a reason these class
> declarations from class templates don’t resolve the children of the class
> template with the template parameters already in place? It would make the
> binding generation process much easier and robust.
>
>
>
> For either of these issues, my next step is to rewrite my binding
> generation tool in C++ and use the clang libraries like LibTooling. They
> seem to provide more functionality anyway, but the Python version is so
> close I’d thought it’d be worth asking before I go down that path.
>
>
>
>
>
I would try to bisect the differences between your libclang usage and some
known-good compiler configuration.  Are the definitions the same?  "clang++
-dM -E -x c++ /dev/null > cmdline_defs"  -- Is the preprocessed output the
same?  "clang++ -E all_includes.h > all_includes.ii"
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20171105/b147bdfb/attachment.html>