[cfe-dev] Source information for types

[Argyrios Kyrtzidis]

On Aug 9, 2009, at 7:21 AM, Enea Zaffanella wrote:

> Argyrios Kyrtzidis wrote:
>> On Aug 8, 2009, at 2:38 AM, Enea Zaffanella wrote:
>>>>> Another issue that could be solved by following this proposal is  
>>>>> the ability to distinguish different syntactic representations  
>>>>> of the same type; namely, distinguishing "unsigned" from  
>>>>> "unsigned int" from "int unsigned" and the like.
>>>> What are the use cases for this, do you also need it to check  
>>>> adherence to coding standards ?
>>>
>>> Yes. (In this particular case, I suspect that coding standard  
>>> checkers are the only application.)
>> Is re-lexing the type specifier a reasonable solution for this ?
>> My concern is the impact on memory consumption for type source  
>> info. Ideally memory usage for type source info should be  
>> reasonable enough to consider having them on all the time.
>> -Argiris
>
> Let us see: according to the C standard, you can freely reorder type  
> specifiers and mix them with type qualifiers (which can occur  
> several times) and storage class specifiers (which can occur only  
> once).
> So, the worst cases should look like the following:
>
> void foo() {
>  /* ... */
>  const long const int extern const unsigned volatile long *p1;
>
>  const long const int const unsigned typedef volatile long * PType;
>  const PType p2 = p1;
>
>  p1 = (int long const unsigned volatile long * restrict) p2;
> }
>
> Well, things would be easy, provided we can see the sequence of  
> preprocessed tokens. Because the tokens above may be the result of  
> (several levels of) macro expansions ...

For the "adherence to coding standards" case, isn't it more  
appropriate to see the tokens without macro expansion ? e.g

int unsigned x;
COOL_MACRO y;

"COOL_MACRO" may expand to "int unsigned" but whether the compiler  
sees  "int" or "int unsigned" doesn't matter, we are really interested  
in what the user wrote, right ?

> here is where I am afraid I am missing knowledge about the  
> underlying clang infrastructure, so I need some help.
>
> Is it possible to start the re-lexing from the source location of a  
> particular token (e.g., the TypeSpecStartLoc of the declaration of  
> p1 above) so as to obtain the exact sequence of the preprocessed  
> tokens?
> Can you point me to some existing code snippet where such a re- 
> lexing is done?

Not sure if it's feasible to get preprocessed tokens from a specific  
source location, though, as I mentioned, getting the lexed tokens may  
be enough for the majority of cases.
We will probably provide helper methods to re-lex type specs and get  
additional info, once the type source info infrastructure is in place.

-Argiris