[cfe-commits] [PATCH 2/2] [clang.py] Implement Token API

[Gregory Szorc]

On 7/1/12 11:48 PM, Manuel Klimek wrote:
> +        lib.clang_tokenize(tu, extent, byref(tokens_memory),
> +                byref(tokens_count))
>
> Strange indent. Hm, after reading further I see that that kind of 
> indent is quite common. Is there a guideline for how to indent this?
It appears my editor is indenting 8 spaces when wrapping lines. I'm 
pretty sure that's provided by a popular "macro" settings file to handle 
Python. I'll change it to 4 spaces, as that seems more conventional.

>
> --- /dev/null
> +++ b/bindings/python/clang/enumerations.py
>
> +TokenKinds = [
> +    ('PUNCTUATION', 0),
> +    ('KEYWORD', 1),
> +    ('IDENTIFIER', 2),
> +    ('LITERAL', 3),
> +    ('COMMENT', 4),
> +]
>
> This doesn't seem to match the comment. Those "TokenKinds" are 
> different from what I see in clang/Basic/TokenKinds.def. Am I missing 
> something?
>
What doesn't match the comment?

Also, these come from 
http://clang.llvm.org/doxygen/group__CINDEX__LEX.html#gaf63e37eee4280e2c039829af24bbc201, 
not the low-level tokens from TokenKinds.def.
>
> On Sat, Jun 30, 2012 at 4:16 AM, Gregory Szorc 
> <gregory.szorc at gmail.com <mailto:gregory.szorc at gmail.com>> wrote:
>
>     Updated patch attached.
>
>     * Added Cursor.get_tokens()
>     * Refactored TokenGroup out of TranslationUnit to support above
>     * Refactored with TranslationUnit.get_* API changes from last patch.
>
>     On Fri, Jun 29, 2012 at 12:18 AM, Gregory Szorc
>     <gregory.szorc at gmail.com <mailto:gregory.szorc at gmail.com>> wrote:
>     > This exposes all of libclang's token functions in the Python
>     bindings.
>     >
>     > I'm putting the TokenKind enumerations in a new module,
>     > clang.enumerations. I plan to eventually move all existing
>     > enumerations there so they are all consolidated. And, maybe one day,
>     > we can even generate that file automatically by parsing the libclang
>     > header files. I know Anders has code around somewhere that does
>     > this...
>     >
>     > ---
>     >  bindings/python/clang/cindex.py                    | 177
>     +++++++++++++++++++--
>     >  bindings/python/clang/enumerations.py              |  32 ++++
>     >  bindings/python/tests/cindex/test_token_kind.py    |  43 +++++
>     >  bindings/python/tests/cindex/test_tokens.py        |  52 ++++++
>     >  .../python/tests/cindex/test_translation_unit.py   |  24 ++-
>     >  5 files changed, 312 insertions(+), 16 deletions(-)
>     >  create mode 100644 bindings/python/clang/enumerations.py
>     >  create mode 100644 bindings/python/tests/cindex/test_token_kind.py
>     >  create mode 100644 bindings/python/tests/cindex/test_tokens.py
>
>     _______________________________________________
>     cfe-commits mailing list
>     cfe-commits at cs.uiuc.edu <mailto:cfe-commits at cs.uiuc.edu>
>     http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20120708/4c7bac3d/attachment.html>