[cfe-dev] [cfe-commits] [PATCH 2/2] [clang.py] Implement Token API

Gregory Szorc gregory.szorc at gmail.com
Thu Jul 12 18:41:07 PDT 2012 

 From the command line, set LD_LIBRARY_PATH to where your compiled 
LLVM/Clang libraries (libclang.so) are. Then, try to run the unit tests:

$ LD_LIBRARY_PATH=/usr/local/lib nosetests -v 
/home/gps/llvm/tools/clang/bindings/python

If that works, then the libclang shared library being loaded isn't the 
right one. You should seek out and destroy references to the old 
version. If that doesn't work, you need to recompile a fresh source pull 
of LLVM/Clang and load that libclang.

Gregory

On 7/12/12 5:52 PM, Carlos Andrade wrote:
> Sorry I guess I lost you on this one. Where can I change this variable?
>
> I am currently setting this on my ~/.profile
>
> PYTHONPATH="/Users/carlosandrade/Desktop/llvm/tools/clang/bindings/python:$PYTHONPATH"
> export PYTHONPATH
>
> I only found on read me file after running grep this:
>
> $ env PYTHONPATH=$(echo ~/llvm/tools/clang/bindings/python/) \
>       LD_LIBRARY_PATH=$(llvm-config --libdir) \
>   nosetests -v
>
>
> But I am a little clueless on how to go about this!
>
> Thanks,
>
>
> Carlos Andrade
> http://carlosandrade.co
>
>
>
> 2012/7/12 Gregory Szorc <gregory.szorc at gmail.com 
> <mailto:gregory.szorc at gmail.com>>
>
>     I just ran the most recent Python bindings on my OS X build and it
>     works fine.
>
>     I suspect an older libclang is being loaded by the Python
>     bindings. Try:
>
>     LD_LIBRARY_PATH=/path/to/llvm/lib python
>
>     You may also want to debug the library loading code and ensure the
>     proper path is being picked up. You can always through a print
>     statement in there or something.
>
>     Gregory
>
>
>     On 7/12/12 5:25 PM, Carlos Andrade wrote:
>>     Hi,
>>
>>     I was using the old version of lib clang python interface and
>>     have been following up on this. I checked llvm, clang and the
>>     compiler as recommended on the get started of the clang webpage
>>     (http://clang.llvm.org/get_started.html). After changing the
>>     python path to the new build I am now getting an error while
>>     trying to run the code.
>>
>>     Traceback (most recent call last):
>>       File "aura.py", line 7, in <module>
>>         import clang.cindex
>>       File
>>     "/Users/carlosandrade/Desktop/llvm/tools/clang/bindings/python/clang/cindex.py",
>>     line 2816, in <module>
>>         register_functions(lib)
>>       File
>>     "/Users/carlosandrade/Desktop/llvm/tools/clang/bindings/python/clang/cindex.py",
>>     line 2407, in register_functions
>>     lib.clang_CompilationDatabase_dispose.argtypes = [c_object_p]
>>       File
>>     "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py",
>>     line 378, in __getattr__
>>         func = self.__getitem__(name)
>>       File
>>     "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py",
>>     line 383, in __getitem__
>>         func = self._FuncPtr((name_or_ordinal, self))
>>     AttributeError: dlsym(0x1003b21b0,
>>     clang_CompilationDatabase_dispose): symbol not found
>>
>>
>>     Does anyone know what is that?
>>
>>     Thanks,
>>
>>
>>     Carlos Andrade
>>     http://carlosandrade.co
>>
>>
>>
>>     2012/7/11 Manuel Klimek <klimek at google.com
>>     <mailto:klimek at google.com>>
>>
>>         On Tue, Jul 10, 2012 at 3:27 PM, Manuel Klimek
>>         <klimek at google.com <mailto:klimek at google.com>> wrote:
>>         > On Mon, Jul 9, 2012 at 8:13 AM, Gregory Szorc
>>         <gregory.szorc at gmail.com <mailto:gregory.szorc at gmail.com>> wrote:
>>         >> On 7/1/12 11:48 PM, Manuel Klimek wrote:
>>         >>
>>         >> +        lib.clang_tokenize(tu, extent, byref(tokens_memory),
>>         >> +  byref(tokens_count))
>>         >>
>>         >> Strange indent. Hm, after reading further I see that that
>>         kind of indent is
>>         >> quite common. Is there a guideline for how to indent this?
>>         >>
>>         >> It appears my editor is indenting 8 spaces when wrapping
>>         lines. I'm pretty
>>         >> sure that's provided by a popular "macro" settings file to
>>         handle Python.
>>         >> I'll change it to 4 spaces, as that seems more conventional.
>>         >>
>>         >>
>>         >>
>>         >> --- /dev/null
>>         >> +++ b/bindings/python/clang/enumerations.py
>>         >>
>>         >> +TokenKinds = [
>>         >> +    ('PUNCTUATION', 0),
>>         >> +    ('KEYWORD', 1),
>>         >> +    ('IDENTIFIER', 2),
>>         >> +    ('LITERAL', 3),
>>         >> +    ('COMMENT', 4),
>>         >> +]
>>         >>
>>         >> This doesn't seem to match the comment. Those "TokenKinds"
>>         are different
>>         >> from what I see in clang/Basic/TokenKinds.def. Am I
>>         missing something?
>>         >>
>>         >> What doesn't match the comment?
>>         >>
>>         >> Also, these come from
>>         >>
>>         http://clang.llvm.org/doxygen/group__CINDEX__LEX.html#gaf63e37eee4280e2c039829af24bbc201,
>>         >> not the low-level tokens from TokenKinds.def.
>>         >
>>         > Oh, wow. Can you add a comment on where these TokenKinds
>>         come from.
>>         > With the overloaded meaning it's really confusing.
>>
>>         And apart from that lgtm.
>>
>>         >
>>         > Thanks!
>>         > /Manuel
>>         >
>>         >>
>>         >>
>>         >> On Sat, Jun 30, 2012 at 4:16 AM, Gregory Szorc
>>         <gregory.szorc at gmail.com <mailto:gregory.szorc at gmail.com>>
>>         >> wrote:
>>         >>>
>>         >>> Updated patch attached.
>>         >>>
>>         >>> * Added Cursor.get_tokens()
>>         >>> * Refactored TokenGroup out of TranslationUnit to support
>>         above
>>         >>> * Refactored with TranslationUnit.get_* API changes from
>>         last patch.
>>         >>>
>>         >>> On Fri, Jun 29, 2012 at 12:18 AM, Gregory Szorc
>>         <gregory.szorc at gmail.com <mailto:gregory.szorc at gmail.com>>
>>         >>> wrote:
>>         >>> > This exposes all of libclang's token functions in the
>>         Python bindings.
>>         >>> >
>>         >>> > I'm putting the TokenKind enumerations in a new module,
>>         >>> > clang.enumerations. I plan to eventually move all existing
>>         >>> > enumerations there so they are all consolidated. And,
>>         maybe one day,
>>         >>> > we can even generate that file automatically by parsing
>>         the libclang
>>         >>> > header files. I know Anders has code around somewhere
>>         that does
>>         >>> > this...
>>         >>> >
>>         >>> > ---
>>         >>> >  bindings/python/clang/cindex.py          | 177
>>         >>> > +++++++++++++++++++--
>>         >>> >  bindings/python/clang/enumerations.py          |  32 ++++
>>         >>> >  bindings/python/tests/cindex/test_token_kind.py    |
>>          43 +++++
>>         >>> >  bindings/python/tests/cindex/test_tokens.py        |
>>          52 ++++++
>>         >>> >  .../python/tests/cindex/test_translation_unit.py   |
>>          24 ++-
>>         >>> >  5 files changed, 312 insertions(+), 16 deletions(-)
>>         >>> >  create mode 100644 bindings/python/clang/enumerations.py
>>         >>> >  create mode 100644
>>         bindings/python/tests/cindex/test_token_kind.py
>>         >>> >  create mode 100644
>>         bindings/python/tests/cindex/test_tokens.py
>>         >>>
>>         >>> _______________________________________________
>>         >>> cfe-commits mailing list
>>         >>> cfe-commits at cs.uiuc.edu <mailto:cfe-commits at cs.uiuc.edu>
>>         >>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>>         >>>
>>         >>
>>         >>
>>
>>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20120712/1b6860e5/attachment.html>

More information about the cfe-dev mailing list