[cfe-dev] Clang-cl.exe and the VC++ preprocessor

[Edward Diener]

On 7/11/2014 1:12 PM, Richard Smith wrote:
> On Fri, Jul 11, 2014 at 8:32 AM, Jonathan 'Rynn' Sauer
> <jonathan.sauer at gmx.de
> <mailto:jonathan.sauer at gmx.de>> wrote:
>
>     Hello,
>
>      > Also it's worth pointing out that the idea is not to emulate VC++
>     in every respect.  Only in every respect where not doing so will
>     break vast amounts of code.  There are plenty of places where not
>     emulating VC++ will *fix* vasts amounts of code.  I imagine the goal
>     is to depart with VC++ in those respects.
>      >
>      > There are examples where not emulating VC++'s bugs means **we
>     can't even include standard windows headers**.  Surely we agree that
>     a Windows compiler that cannot include <windows.h> would not be
>     particularly useful.
>
>     Could it be an option to limit the emulation of VC++'s preprocessor
>     to system headers and macros defined
>     there, similar to how clang suppresses warnings in these cases?
>
>
> I don't think so, no. Clang defines _MSC_VER, so code that has separate
> preprocessor magic for cl.exe versus everything else should use the
> cl.exe path. As a result, we'd break such code if we didn't act like cl.exe.
>
>
> In boost's case, the problem is that
>
> http://www.boost.org/doc/libs/1_55_0/boost/preprocessor/seq/detail/binary_transform.hpp
>
> does not take the MSVC-preprocessor-compatible code path, because:
>
> http://www.boost.org/doc/libs/1_55_0/boost/preprocessor/config/config.hpp
>
> ... explicitly shoots itself in the foot:
>
> #    if defined _MSC_VER && _MSC_VER >= 1400 && !(defined __EDG__ ||
> defined __GCCXML__ || defined __CUDACC__ || defined __PATHSCALE__ ||
> defined __clang__ || defined __DMC__ || defined __CODEGEARC__ || defined
> __BORLANDC__ || defined __MWERKS__ || defined __SUNPRO_CC || defined
> __HP_aCC || defined __MRC__ || defined __SC__ || defined __IBMCPP__ ||
> defined __PGI)
> #        define BOOST_PP_VARIADICS_MSVC 1
> #    endif
>
> I'm not sure what that list of compilers is for (if _MSC_VER is defined,
> why don't you assume the compiler acts like MSC?), but removing "||
> defined __clang__" from the list should fix the issue.

Before I answer you about Boost PP I first want to apologize to 
everybody about getting so heated about clang-cl emulating a VC++ 
preprocessor bug. I have my reasons, as I think everyone could see, for 
not wanting clang to emulate the VC++ preprocessor in any way but I 
admit from the clang-cl bug report I looked at ( 
http://llvm.org/bugs/show_bug.cgi?id=14981 ) that trying to process the 
VC11 header file xstddef with a conforming C++ standard preprocessor 
will not work. So I do understand why it was necessary to emulate a VC++ 
preprocessor bug in order to process the file.

The code you have mentioned above only deals with variadic macro support 
in Boost PP. If you look more closely at it variadic macro support is 
always turned on for clang.

As for C++ preprocessor support in Boost PP clang gets the 
BOOST_PP_CONFIG_STRICT() flag, denoting a C++ standard supporting 
compiler and not the BOOST_PP_CONFIG_MSVC() when using clang-cl. The 
first and simplest reason for this was the very reasonable assumption 
that clang is producing a C++ standard conformant preprocessor. But  the 
secondary reason for this, even assuming that the clang-cl preprocessor 
must work to some extent as if it were VC++, is that Boost PP support 
for VC++ has a huge number of workarounds for various VC++ preprocessor 
"bugs" which cause VC++ not to be able to expand macros in the correct 
way for a C++ standard conforming preprocessor. The greater number of 
those workarounds were crafted by Paul Mensonides when he designed and 
wrote the Boost PP library. I crafted a few when working with Paul to 
add variadic macro support to Boost PP, and even more recently on the 
'develop' branch to solve some more VC++ related preprocessor issues 
with variadic macros.

We could obviously change the code very easily to have clang-cl act like 
the VC++ preprocessor in every respect by simply setting the 
BOOST_PP_CONFIG_MSVC() flag for clang-cl. But it will then be involved 
in all the workarounds for VC++ and I have no idea "a priori" if these 
VC++ workarounds in Boost PP for clang-cl will make things any better. 
My guess is that it will make things much worse. Please try to realize 
that the one workaround I have seen in clang-cl to solve the 14981 bug 
above is just a drop in the bucket of all the C++ preprocessor 
non-conformant behavior of VC++. And I highly doubt whether clang-cl 
emulates all the "bugs" in the VC++ preprocessor. Nor is it even 
possible in most cases to see what the VC++ preprocessor is doing when 
it does not expand macros correctly in complicated situations of 
preprocessor metaprogramming, so it is nevertheless endlessly 
frustrating trying to get it to work correctly which is often a matter 
of trying one trick or another.

This is just a way of saying that the shotgun approach in Boost PP of 
having clang-cl act like VC++ is almost certainly wrong. And needless to 
say neither I, and almost certainly neither Paul Mensonides, wants to 
spend any time working with another C++ preprocessor "like", or 
"similar", or "emulating" in certain respects but not all the VC++ 
preprocessor as far as Boost PP code is involved. But if some clang 
developer and preprocessing guru would like to join the Boost PP team 
and hack away at the code to get clang-cl to work in Boost PP in some 
sort of VC++ emulation mode, please feel free to do so. But having 
worked with the VC++ preprocessor in both Boost PP and my own VMD 
library, I wouldn't wish such work on my worst enemy.

My suggestion to clang-cl developers working with the preprocessor 
follows the thinking of Matt Calabrese in his post. If it is possible it 
would be optimal if a #pragma could control the expansion of macros 
between whatever is the VC++ emulation which clang-cl currentlty employs 
and the conforming C++ preprocessor which clang is in its other builds. 
I do not know how feasible this is but I hope it is so. In this way we 
could surround the code in Boost PP when clang-cl is being used in such 
a way that C++ standard preprocessing can be done while the VC++ 
emulation mode is used to compile the incorrect C++ preprocessor code in 
VC11's xstddef and wherever such erroneous stuff occurs in other 
Microsoft header files.