[cfe-commits] [PATCH 2/2] [clang.py] Implement Token API

[Manuel Klimek]

+        lib.clang_tokenize(tu, extent, byref(tokens_memory),
+                byref(tokens_count))

Strange indent. Hm, after reading further I see that that kind of indent is
quite common. Is there a guideline for how to indent this?

+    @staticmethod
+    def register(value, name):
+        """Register a new TokenKind enumeration.
+
+This should only be called at module load time by code within this
+package.
+"""

Indent.

--- /dev/null
+++ b/bindings/python/clang/enumerations.py

+TokenKinds = [
+    ('PUNCTUATION', 0),
+    ('KEYWORD', 1),
+    ('IDENTIFIER', 2),
+    ('LITERAL', 3),
+    ('COMMENT', 4),
+]

This doesn't seem to match the comment. Those "TokenKinds" are different
from what I see in clang/Basic/TokenKinds.def. Am I missing something?

Cheers,
/Manuel

On Sat, Jun 30, 2012 at 4:16 AM, Gregory Szorc <gregory.szorc at gmail.com>wrote:

> Updated patch attached.
>
> * Added Cursor.get_tokens()
> * Refactored TokenGroup out of TranslationUnit to support above
> * Refactored with TranslationUnit.get_* API changes from last patch.
>
> On Fri, Jun 29, 2012 at 12:18 AM, Gregory Szorc <gregory.szorc at gmail.com>
> wrote:
> > This exposes all of libclang's token functions in the Python bindings.
> >
> > I'm putting the TokenKind enumerations in a new module,
> > clang.enumerations. I plan to eventually move all existing
> > enumerations there so they are all consolidated. And, maybe one day,
> > we can even generate that file automatically by parsing the libclang
> > header files. I know Anders has code around somewhere that does
> > this...
> >
> > ---
> >  bindings/python/clang/cindex.py                    | 177
> +++++++++++++++++++--
> >  bindings/python/clang/enumerations.py              |  32 ++++
> >  bindings/python/tests/cindex/test_token_kind.py    |  43 +++++
> >  bindings/python/tests/cindex/test_tokens.py        |  52 ++++++
> >  .../python/tests/cindex/test_translation_unit.py   |  24 ++-
> >  5 files changed, 312 insertions(+), 16 deletions(-)
> >  create mode 100644 bindings/python/clang/enumerations.py
> >  create mode 100644 bindings/python/tests/cindex/test_token_kind.py
> >  create mode 100644 bindings/python/tests/cindex/test_tokens.py
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20120702/7088586a/attachment.html>