<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">I already have this Python code written
in a personal branch and just need to integrate it with the
official bindings. This involves a little refactoring to make the
API nicer to consume, etc.<br>
<br>
I'll try to submit a patch for review by the end of next week.<br>
<br>
While I'm at it, I may also add support for the comment APIs
recently added to libclang since they are somewhat related.<br>
<br>
Gregory<br>
<br>
On 6/28/12 11:20 AM, Carlos Andrade wrote:<br>
</div>
<blockquote
cite="mid:CA+KzQ4HijfEWhTSdgPDWocDx+CJ-W3sfVjsbV_3u-PKHuEYjEw@mail.gmail.com"
type="cite">Dears,<br>
<br>
I recently was able to make my -- very messed up - c version to
output the values of the cursors I wanted on libclang and very
much appreciate to know if there is any chance that the methods I
listed on the title of this e-mail <b>clang_tokenize</b> and <b>clang_getTokenSpelling
</b>would be available anytime soon. <br>
<br>
More specifically what I am looking for is this. If I recurse the
tree using python interface I get at the very most to the best of
my knowledge:<br>
<br>
<b>piece of source sample: </b> int loren = 2 <b>+</b> 2;<br>
<br>
<b>Cursor representing the binary operator (using <u>cursor.kind</u>,
<u>cursor.spelling</u>, <u>cursor.displayname</u>, <u>cursor.location</u>,
<u>cursor.hash</u>):</b><br>
<br>
CursorKind.BINARY_OPERATOR <b>None</b> <SourceLocation file
'simple.c', line 11, column 14> 3289119033<br>
type: TypeKind.INT <br>
canonical type: TypeKind.INT <br>
<br>
<br>
As you can see, I <u>cant</u> distinguish what is the binary
operator (I get None). And this apply for other cursors as well. <br>
<br>
However running the C code snipet from libclang which I found
somewhere on stackoverflow with my code I was able to extract the
token and obtain such information with some effort (I get the null
instead of None but since I have access to the tokens associated
with the Cursor I can reach it). <br>
<br>
The output is: BinaryOperator (null)||| Start: Line: 11 Column:
14 Offset: 103 |||token = 2 token = + token = 2 token = ;<br>
<br>
<u>See code highlighted here:</u> <a moz-do-not-send="true"
href="http://paste2.org/p/2062173">http://paste2.org/p/2062173</a><br>
---------------------------------------------------------------------------------------------------<br>
if (kindType.kind != CXType_Invalid)<br>
{<br>
CXSourceRange range = clang_getCursorExtent(cursor);<br>
CXToken *tokens = 0;<br>
unsigned int nTokens = 0;<br>
<b>clang_tokenize</b>(TU, range, &tokens,
&nTokens);<br>
for (unsigned int i = 0; i < nTokens; i++)<br>
{<br>
CXString spelling = <b>clang_getTokenSpelling</b>(TU,
tokens[i]);<br>
printf("token = %s\n", clang_getCString(spelling));<br>
clang_disposeString(spelling);<br>
}<br>
clang_disposeTokens(TU, tokens, nTokens);<br>
}<br>
---------------------------------------------------------------------------------------------------<br>
<br>
<br>
I search on cindex and on the website branch (<a
moz-do-not-send="true"
href="http://llvm.org/svn/llvm-project/cfe/branches/tooling/bindings/python/clang/cindex.py">http://llvm.org/svn/llvm-project/cfe/branches/tooling/bindings/python/clang/cindex.py</a>)
but i didnt seen anything related to tokens so far. <br>
<br>
My motivation for asking this is that getting to see the tokens is
just a small fraction of my interest on it and I would like to
stick with python rather than trying it directly on C. Also I am
mostly done on my python code version where this is the only part
holding me off on moving forward while on C there are many things
I don't see yet a way to do it like I do on Python.<br>
<br>
Thank you very much for your attention.<br>
<br>
Best Regards,<br>
<br clear="all">
Carlos Andrade<br>
<div><a moz-do-not-send="true" href="http://carlosandrade.co"
target="_blank">http://carlosandrade.co</a></div>
<br>
<br>
<br>
<div class="gmail_quote">2012/6/18 Carlos Andrade <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:carlosviansi@gmail.com"
target="_blank">carlosviansi@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
Thank you Gregory! I am very happy to know that! The available
methods on 3.1 work just as fine on the code so I am fine for
now! I might be using other functionalities in a near feature
from lib clang which might not be on the current python
interface so I will for sure check on this :)
<div>
<br>
</div>
<div>I very much appreciate your offer on fast-tracking any
specific features and will keep that in mind! </div>
<div><br>
</div>
<div>Best Regards,</div>
<div><br clear="all">
Carlos Andrade<br>
<div><a moz-do-not-send="true"
href="http://carlosandrade.co" target="_blank">http://carlosandrade.co</a></div>
<div>
<div class="h5">
<br>
<br>
<br>
<div class="gmail_quote">2012/6/18 Gregory Szorc <span
dir="ltr"><<a moz-do-not-send="true"
href="mailto:gregory.szorc@gmail.com"
target="_blank">gregory.szorc@gmail.com</a>></span><br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On 6/18/12 10:13 AM, David Röthlisberger wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<blockquote class="gmail_quote" style="margin:0
0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
The reasons I stayed on C was because I saw
few comments that python was still catching up
on making available the methods provided by
libclang,<br>
</blockquote>
That's true in the sense that any new feature
must be added to the C<br>
interface before it can be added to the python
bindings; that is just<br>
the nature of foreign bindings. But I don't know
how far behind the<br>
python bindings are currently lagging (if at
all). Gregory Szorc (CCd)<br>
is one of the maintainers of the python bindings
-- perhaps he can<br>
comment.<br>
<br>
</blockquote>
</div>
I have a branch of the Clang bindings that are
nearly feature complete at <a
moz-do-not-send="true"
href="https://github.com/indygreg/clang/tree/python_features"
target="_blank">https://github.com/indygreg/clang/tree/python_features</a>.
However, I /think/ that tree may be busted right
now, so use at your own risk.<br>
<br>
For the past ~6 months I've been slowly moving
patches from my repository into the mainline. It has
been a long process. Now that I have Manuel as a
reliable reviewer, things could start moving faster.
I've just been busy with other projects.<br>
<br>
If there is a particular feature missing from the
in-tree Python bindings, I probably have code for it
somewhere. If anyone asks kindly, I can probably
fast-track specific features to the main tree.<span><font
color="#888888"><br>
<br>
Gregory<br>
</font></span></blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</blockquote>
<br>
</body>
</html>