[cfe-dev] libclang: Get tokens after preprocessing ("as the compiler sees it")

[Alexei Svitkine]

You can get raw tokens from clang via:

clang::Preprocessor* pp = /* ... */;
clang::Token token;
pp->Lex(token);
while (token.isNot(clang::tok::eof)) {
  /* do something with token */
  pp->Lex(token);
}

I forget if that gives you tokens after or before preprocessing. If
it's before, you can always preprocess the input first with
"clang++/g++ -E" - though there's probably a better way through API.

-Alexei