<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">+libcxx-dev …<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">“Why do you not use `lit` to run the test suite?”<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I don’t work at Microsoft, but I have tried to smash MSVC and libc++ tests together repeatedly. Lit doesn’t make it easy. I’ve never gotten something nice enough to upstream. The libc++ lit infrastructure is heavily tuned towards gcc
and clang, and doesn’t do a good job at isolating compiler differences. Instead, the logic is spread all over the place.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Here is one such quick-and-dirty attempt from a year and a half ago…
<a href="https://github.com/ben-craig/libcxx/commit/fea52ceee736c97e74194b3f38660a7b0f73db83">
https://github.com/ben-craig/libcxx/commit/fea52ceee736c97e74194b3f38660a7b0f73db83</a> , note that this is after shuffling around a whole bunch of code to attempt to put compiler configuration behind an “abstract interface”.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I didn’t feel like subjecting myself to that again, so when I made my freestanding branch tests, I just rolled my own ninja generator written in Ruby. Now I only rerun tests that weren’t successful, or have had a change.
<a href="https://github.com/ben-craig/kernel_test_harness/tree/msvc_stl">https://github.com/ben-craig/kernel_test_harness/tree/msvc_stl</a> .<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> libcxx-dev <libcxx-dev-bounces@lists.llvm.org> <b>
On Behalf Of </b>Louis Dionne via libcxx-dev<br>
<b>Sent:</b> Tuesday, December 11, 2018 9:57 AM<br>
<b>To:</b> Stephan T. Lavavej <stl@exchange.microsoft.com><br>
<b>Cc:</b> Billy O'Neal (VC LIBS) <bion@microsoft.com>; Jonathan Wakely <cxx@kayari.org>; Casey Carter <cacarter@microsoft.com>; libcxx-dev@lists.llvm.org<br>
<b>Subject:</b> Re: [libcxx-dev] How to check for a feature-test macro<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Dec 10, 2018, at 20:14, Casey Carter <<a href="mailto:cacarter@microsoft.com">cacarter@microsoft.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">I don't have context for the original issue, but our test selector does inspect some of the LIT control comments. Specifically, it skips tests with any of the comments:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<blockquote style="margin-left:30.0pt;margin-right:0in;caret-color: rgb(0, 0, 0);font-variant-caps: normal;text-align:start;-webkit-text-stroke-width: 0px;word-spacing:0px">
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">// REQUIRES: c++11<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">// REQUIRES: c++11 || c++14<o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">// REQUIRES: c++98 || c++03<o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">// REQUIRES: c++98 || c++03 || c++11 || c++14<o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">// UNSUPPORTED: c++14, c++17, c++2a<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">So if were talking about something that doesn't touch those, it won't affect our typical usage.<o:p></o:p></span></p>
</div>
<div id="signature">
<div id="divtagdefaultwrapper">
<div>
<p class="MsoNormal" style="background:white"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="1440" style="width:15.0in" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b>From:</b><span class="apple-converted-space"> </span>Stephan T. Lavavej<br>
<b>Sent:</b><span class="apple-converted-space"> </span>Monday, December 10, 2018 15:43<br>
<b>To:</b><span class="apple-converted-space"> </span>Jonathan Wakely; <a href="mailto:ldionne@apple.com">
ldionne@apple.com</a><br>
<b>Cc:</b><span class="apple-converted-space"> </span>Richard Smith; Marshall Clow;
<a href="mailto:libcxx-dev@lists.llvm.org">libcxx-dev@lists.llvm.org</a>; Billy O'Neal (VC LIBS); Casey Carter<br>
<b>Subject:</b><span class="apple-converted-space"> </span>RE: [libcxx-dev] How to check for a feature-test macro<span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"> <o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica",sans-serif">(+Casey who also works on this)<br>
<br>
When running libcxx's tests against MSVC's STL, we have an MSVC-internal skipped_tests.txt (full content available if requested) which says:<br>
<br>
# *** MISSING STL FEATURES ***<br>
# C++20 P0122R7 "<span>"<br>
upstream\test\std\containers\views\types.pass.cpp<br>
[...]<br>
<br>
# C++20 P0355R7 "<chrono> Calendars And Time Zones"<br>
upstream\test\std\utilities\time\days.pass.cpp<br>
[...]<br>
<br>
Updating this requires minimal effort. It's more problematic when existing tests are modified to test new features, since we have to skip the entire test even if only a small part is affected.<br>
<br>
Guarding tests with feature-test macros in their source code (as opposed to "the LIT level", which I believe is machinery that we don't use) would be ideal, I think.<o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Why do you not use `lit` to run the test suite?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">As I understand it, MSVC would prefer disabling tests that are not supported using `#if !defined(<feature>)`. This can be made to work, but what I don't like is that it removes visibility into what tests are passing and which ones are not
supported. Right now, we can easily tell how many tests are unsupported by an implementation because those are marked by LIT as `UNSUPPORTED`. If we start using feature test macros to disable tests, we lose that.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Louis<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Helvetica",sans-serif"><br>
The annoying thing is when features are added without corresponding feature-test macros. There's a bunch of those right now. We'll probably need the skip-tests approach for those.<br>
<br>
STL<br>
<br>
-----Original Message-----<br>
From: Jonathan Wakely <<a href="mailto:cxx@kayari.org">cxx@kayari.org</a>><span class="apple-converted-space"> </span><br>
Sent: Monday, December 10, 2018 12:10 PM<br>
To: <a href="mailto:ldionne@apple.com">ldionne@apple.com</a><br>
Cc: Richard Smith <<a href="mailto:richard@metafoo.co.uk">richard@metafoo.co.uk</a>>; Marshall Clow <<a href="mailto:mclow.lists@gmail.com">mclow.lists@gmail.com</a>>;
<a href="mailto:libcxx-dev@lists.llvm.org">libcxx-dev@lists.llvm.org</a>; Stephan T. Lavavej <<a href="mailto:stl@exchange.microsoft.com">stl@exchange.microsoft.com</a>>; Billy O'Neal (VC LIBS) <<a href="mailto:bion@microsoft.com">bion@microsoft.com</a>><br>
Subject: Re: [libcxx-dev] How to check for a feature-test macro<br>
<br>
On Mon, 10 Dec 2018 at 19:47, Louis Dionne wrote:<br>
> We should also see how other implementations that use our test suite feel about this (CCing them).<br>
<br>
I only use your tests a few times a year, run by hand. If I get a load of FAILs because I've not implemented something yet and you have, that's not a problem. I don't expect to get 100% green when I run your tests.<o:p></o:p></span></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>