[cfe-dev] #define behavior

['Neil Booth']

Hartmut Kaiser wrote:-

> > Standard C works on tokens, not text, so the code is invalid as
> > * and = are two separate tokens.
> 
> The preprocessor works on preprocessor tokens (translation phase 4), which
> are different from the tokens the compiler is working on. The preprocessor
> tokens are converted into tokens at translation phase 7 only. Any
> preprocessor token not matching the syntax of a token is an error at this
> point.

Well, is undefined behaviour.  The standard has no concept of error per
se, only "must be diagnosed" and "failure to successfully translate",
the latter applying to #error exclusively.

> Further, IIUC concatenating two tokens (using ##) forming another (invalid)
> pp token results in undefined behaviour in C90/C++ but is allowed in
> C99/C++09. Gcc issues a warning here (I'm not sure about clang) for C90/C++.

There is no difference between C90 and C99 here.  I'm the person that
made it a hard error in GCC; it is not a warning.  That is and was my
preference.  :)
 
> Juxtapositioning two preprocessor tokens generally doesn't create a new
> preprocessor token. So the two preprocessor tokens '*' and '=' are converted
> to tokens separately (in translation step 7), resulting in the syntax error
> above.
> 
> When the textual representation of the preprocessed input is to be generated
> (i.e. gcc -E) all good preprocessors _insert_ additional whitespace between
> two preprocessor tokens, which otherwise would form another token based on
> juxtaposition, just to avoid creating the syntax of a different token from
> juxtapositioning two preprocessor tokens. 
> 
> IMHO, gcc and clang (and FWIW, wave) behave correctly.

Absolutely.  I was just expressing it in plain English.

Neil.