[cfe-dev] Unable to get tokens, whenever macros are involved

[Suryavanshi Abhinav Singh]

Hello Everyone,

I am witting tools for source code analysis using clang AST. Currently
I am using python bindings as an interface to clang.

There is this one test case that's breaking my analysis,
the problem occurs whenever I am listing tokens.

here are the sample codes:

    ast_exp.h
    ----------
    #define ERROR_CODE 2
    #define STATIC static
    typedef int int32_t;
    
    ast_exp.c
    ----------
    #include "ast_exp.h"
    STATIC int32_t x = ERROR_CODE;

    ast_exp_2.c
    ----------
    #define ERROR_CODE 2
    #define STATIC static
    typedef int int32_t;
    STATIC int32_t x = ERROR_CODE;

now whenever I try to list tokens for the variable declaration line,
following happens:
 * ast_exp.c # 2
	* No tokens are found
 * ast_exp_2.c # 4
	* All tokens are listed correctly.
Am I doing something wrong, or is this a limitation of clang or its
python binding?

I have expressed this problem on stackoverflow too:
http://stackoverflow.com/questions/26029160/clang-macro-expansion-missing-tokens

Hope its not a bug.
--
Best,
Abhi