[cfe-dev] Clang macro expansion differs from GCC

[David Peixotto]

Hi Chris,

I took a look at modifying the preprocessor output and it was pretty
straightforward. I have attached a patch that makes the preprocessed output
less conservative in adding spaces between periods. It will now add a space to
avoid turning ".. ." into "...", but does not always add a space between two
consecutive dots.

The change was done by keeping track of the last two tokens that were output
and testing against those two tokens to avoid creating "...". Previously only
the last output token was tracked and tested against, which caused any two
consecutive . tokens to have a space between them.

The patch passes all the tests in the tests subdirectory. I modified one test
to reflect the output changing from ". . ." to ".. .". This output happens to
match the expected output in the test comment. I also added a test for the
original case that was causing me problems.

Please let me know if there are more tests I should run or anything else that
needs to happen for the patch to be accepted. Thanks!

-David


On Apr 8, 2010, at 11:48 AM, Chris Lattner wrote:

> 
> On Apr 8, 2010, at 8:34 AM, David Peixotto wrote:
> 
>> I have found a discrepancy in behavior between the clang preprocessor and gcc. 
>> 
>> $ cat t.c 
>> #define X 0 .. 1
>> 
>> 0 .. 1 // Ok
>> X      // Bad: space between ..
>> 
>> $ gcc -E t.c 
>> <snip>
>> 
>> 0 .. 1
>> 0 .. 1
>> 
>> $ clang -E t.c 
>> <snip>
>> 
>> 0 .. 1
>> 0 . . 1
>> 
>> The problem is that when ".." is in the macro and the macro gets expanded, it adds a space between the dots. As you can see, the space is not added when the ".." is entered directly in the text. GCC does not insert a space in either case. 
>> 
>> I'm not sure this qualifies as a bug, but the reason I need this to work is that I am trying to compile GHC (a Haskell compiler) with clang. It uses GCC to preprocess c-- files (http://www.cminusminus.org) before parsing them and ".." is a valid operator in c--. Adding the extra space messes up the subsequent parsing phase.
> 
> Hi David,
> 
> I'm not particularly interested in fixing this: people abusing the C preprocessor to preprocess things that aren't C don't get much sympathy from me.  However, if you are able to produce a patch that works, doesn't break anything else, and doesn't affect performance, I'd be happy to apply it.
> 
>> 
>> I'd be happy to take a look at fixing this myself if someone could point me in the right direction.
> 
> 
> lib/Frontend/PrintPreprocessedOutput.cpp is the code that outputs a .i file, the "AvoidConcat" stuff is what you want.  It is what decides to add spaces to prevent three . tokens from looking like a ... for example.
> 
> -Chris
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20100411/0e7ad646/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PPdots.patch
Type: application/octet-stream
Size: 4869 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20100411/0e7ad646/attachment.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20100411/0e7ad646/attachment-0001.html>