[cfe-dev] AST and Tokens?

[John Bailey]

Hi All,

I'm hoping that someone can help point me in the right direction - I 
seem to be going round in circles at the moment.

I'm using ClangTool coupled with a RecursiveASTVisitor to examine some 
C-code [1].  In addition to the AST I'd like to be able to examine a 
token stream of the source.  I've found a couple of examples ([2], [3]) 
of using the pre-processor to lex the file, however I was hoping that as 
the file is already being lexed in order to generate the AST I'd just be 
able to hook in to that process somehow rather than (what seems to be) 
significantly duplicating that effort.

Any suggestions gratefully accepted.

Many Thanks,

John

[1] https://github.com/bright-tools/ccsm/blob/master/src/ccsm.cpp
[2] http://amnoid.de/tmp/clangtut/tut.html
[3] https://github.com/loarabia/Clang-tutorial/wiki/TutorialOrig