[cfe-dev] Specific clang-cl preprocessor emulation problem

[Edward Diener via cfe-dev]

On 4/15/2016 6:15 PM, James Dennett via cfe-dev wrote:
> On Fri, Apr 15, 2016 at 3:04 PM, Edward Diener via cfe-dev
> <cfe-dev at lists.llvm.org
> <mailto:cfe-dev at lists.llvm.org>> wrote:
>
>     I have reported bug 27380 located at
>     https://llvm.org/bugs/show_bug.cgi?id=27380 for a clang-cl
>     preprocessor problem as it emulates the VC++ preprocessor. The
>     problem can be illustrated wit the code:
>
>     #define TEST_MACRO(x,y) TEST_EXPAND(2, 0, TEST_SOMETHING(x,y,1))
>     #define TEST_SOMETHING(x,y,z) 2
>
>     #define TEST_EXPAND(...) TEST_EXPAND_I(TEST_OVR(TEST_NAME,
>     __VA_ARGS__), (__VA_ARGS__))
>     #define TEST_OVR(name,...) TEST_SOMENAME
>     #define TEST_EXPAND_I(m, args) TEST_EXPAND_II(m, args)
>     #define TEST_EXPAND_II(m, args) TEST_CAT(m ## args,)
>     #define TEST_SOMENAME(x,y,z) ;
>
>     #define TEST_CAT(a, b) TEST_CAT_I(a, b)
>     #define TEST_CAT_I(a, b) TEST_CAT_II(~, a ## b)
>     #define TEST_CAT_II(p, res) res
>
>     int main()
>              {
>              TEST_MACRO(1,2)
>          }
>
>     which compiles correctly with VC++14 but with the latest clang-cl
>     built from the trunk gives:
>
>     test_clang.cpp(17,2):  error: pasting formed 'TEST_SOMENAME(', an
>     invalid preprocessing token [-Winvalid-token-paste]
>              TEST_MACRO(1,2)
>              ^
>     test_clang.cpp(2,25):  note: expanded from macro 'TEST_MACRO'
>     #define TEST_MACRO(x,y) TEST_EXPAND(2, 0, TEST_SOMETHING(x,y,1))
>                              ^
>     test_clang.cpp(5,26):  note: expanded from macro 'TEST_EXPAND'
>     #define TEST_EXPAND(...) TEST_EXPAND_I(TEST_OVR(TEST_NAME,
>     __VA_ARGS__), (__VA_ARGS__))
>                               ^
>     test_clang.cpp(7,32):  note: expanded from macro 'TEST_EXPAND_I'
>     #define TEST_EXPAND_I(m, args) TEST_EXPAND_II(m, args)
>                                     ^
>     test_clang.cpp(8,44):  note: expanded from macro 'TEST_EXPAND_II'
>     #define TEST_EXPAND_II(m, args) TEST_CAT(m ## args,)
>
>     The bug is instrumental in keeping Boost PP from working correctly
>     with clang-cl, and therefore keeping many other Boost libraries,
>     which use Boost PP, from working with clang-cl. There appear to be
>     other problems but until this gets fixed I won't know for sure,
>     since fixing this problem may make any others which have shown up go
>     away.
>
>     Thanks for listening !
>
>
> Did you try disabling -Winvalid-token-paste (-Wno-invalid-token-paste)?
> Possibly it should be disabled by default in MSVC emulation mode, but
> macro_paste_identifier_error.c explicitly turns it off as well as
> specifying -fms-extensions.

Thanks ! Your suggestion of using -Wno-invalid-token-paste worked.

However:

1) If clang-cl is trying to emulate the non-standard VC++ preprocessor, 
and clearly the VC++ preprocessor does not think that the token pasting 
is invalid doesn't it behoove clang-cl to also follow this behavior ? If 
it is not following the behavior of the VC++ preprocessor, whether that 
preprocessor is correct or not according to the C++ standard, then it is 
simply failing its mission.

2) I have never believed it correct for clang-cl to emulate the 
non-standard VC++ preprocessor, except in cases in which it must do so 
in order to process Windows or VC++ header files, but since it has 
decided to do so in all situations there should be no need for the 
end-user to pass a compiler option to have clang-cl get this right.

3) I do not know what your reference to macro_paste_identifier_error.c 
means. I assume that is a source file used to build clang-cl.