<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">The “gigantic patch” will probably be a few weeks out, but yes, I will absolutely try and get a small, representative patch first.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Louis Dionne <ldionne@apple.com> <br>
<b>Sent:</b> Wednesday, November 18, 2020 3:27 PM<br>
<b>To:</b> Ben Craig <ben.craig@ni.com><br>
<b>Cc:</b> libcxx-dev@lists.llvm.org; Stephan T. Lavavej <stl@exchange.microsoft.com><br>
<b>Subject:</b> [EXTERNAL] Re: RFC: freestanding libc++ lit tests<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal">On Nov 17, 2020, at 20:21, Ben Craig <<a href="mailto:ben.craig@ni.com">ben.craig@ni.com</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">I think I like it. I'll restate it to check for understanding.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">As a first pass, I'll apply an "UNSUPPORTED: freestanding" to all the facilities that the standard lists as hosted.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">As we identify areas of vendor extension, we will label that area in the negative. For example, assume some crazy vendor had a freestanding platform that supported iostreams. At that point, we would add
a libcpp-has-no-iostreams feature. We would then remove "UNSUPPORTED:freestanding" from the associated tests and add "UNSUPPORTED:libcpp-has-no-iostreams" in its place.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">This would have the effect of temporarily breaking the tests of all the vendors that don't support iostreams on freestanding. But that's fine, they can fix things by adding libcpp-has-no-iostreams to their
feature list. They could also be pleasantly surprised to discover that their platform happens to support iostreams.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">I'm not thrilled with the double negative in "UNSUPPORTED:libcpp-has-no-iostreams", but the workflow gets the job done.<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Yes, you understand correctly. However, we should actually aim to define `freestanding` as a conjunction of finer grained features. For example, we could add a new Lit parameter `--param freestanding=[True|False]`, and that would be equivalent
to implicitly setting other parameters like `--param has-localization=False --param has-random_device=False`, etc. Those finer-grained parameters don't exist in Lit today, they are currently tied to libc++ and set automatically based on what we find in libc++'s
__config_site, but it would be easy to separate them from libc++ and add official support for them in the test suite.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I imagine there's going to be pretty gigantic patches coming up. Can you please open up a small one so we can discuss the specifics before you modify the whole test suite? It'll save time for everyone.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Cheers and thanks for your interest in improving the test suite,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Louis<o:p></o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="1440" style="width:15.0in" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b>From:</b><span class="apple-converted-space"> </span>Louis Dionne <<a href="mailto:ldionne@apple.com">ldionne@apple.com</a>><br>
<b>Sent:</b><span class="apple-converted-space"> </span>Tuesday, November 17, 2020 3:55 PM<br>
<b>To:</b><span class="apple-converted-space"> </span>Ben Craig <<a href="mailto:ben.craig@ni.com">ben.craig@ni.com</a>><br>
<b>Cc:</b><span class="apple-converted-space"> </span><a href="mailto:libcxx-dev@lists.llvm.org">libcxx-dev@lists.llvm.org</a> <<a href="mailto:libcxx-dev@lists.llvm.org">libcxx-dev@lists.llvm.org</a>>; Stephan T. Lavavej <<a href="mailto:stl@exchange.microsoft.com">stl@exchange.microsoft.com</a>><br>
<b>Subject:</b><span class="apple-converted-space"> </span>[EXTERNAL] Re: RFC: freestanding libc++ lit tests<span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"> <o:p></o:p></span></p>
</div>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">On Nov 15, 2020, at 12:19, Ben Craig <<a href="mailto:ben.craig@ni.com">ben.craig@ni.com</a>> wrote:<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">I'd like to start annotating the libc++ lit tests to indicate which can run in freestanding vs. hosted, while also allowing for vendor extensions to freestanding. I'm going to need to add a new feature to
lit though.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style="font-size:12.0pt">Recommendation:</span></b><span style="font-size:12.0pt"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">Mark all the hosted tests as "REQUIRES: hosted". Most platforms would have the "hosted" feature set automatically. Freestanding tests would not have any new REQUIRES attribute added.<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">I would like an approach where we add the minimum number of REQUIRES. Instead, I think using UNSUPPORTED is better. The reason is that by default, when you're porting the
test suite to a new platform, you just don't define any lit features, and ideally that enables the full test suite. You can then see what fails, and see what features you actually need to define.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">If you're using `REQUIRES:`, then the majority of tests will be disabled by default, if you don't define any Lit features. Of course, both are logically equivalent, but I
just find that using `UNSUPPORTED: freestanding` provides more ergonomics than `REQUIRES: hosted`.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><br>
<br>
<o:p></o:p></span></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:12.0pt">To handle vendor extensions, things get a little more involved. I would like to add a "REQUIRES_ONE" tag, so that we can put expressions like "REQUIRES_ONE: hosted, darwinKernel" and "REQUIRES_ONE: hosted,
winKernel". The expression would be satisfied so long as at least one of the listed features is present.<o:p></o:p></span></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">This can be handled with `// REQUIRES: hosted || darwinKernel`. Boolean expressions in REQUIRES work.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">All that being said, my actual preference to handle freestanding is to add finer-grained features to describe features that are not available on a specific platform. These
features are generic and not target specific. For example, I added a mode of libc++ for platforms that don't support localization, and added the corresponding libcpp-has-no-localization Lit feature. The nice part about this feature is that some platform that
supports everything but localization can simply use that, however if another platform is more constrained, they can intersect such features by defining multiple ones: libcpp-has-no-localization, libcpp-has-no-threads, libcpp-has-no-random_device, etc.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">I think that's the most extensible way of supporting this. And then, freestanding can simply be an alias of some specific set of Lit features, like no localization, no random
device, no this, and no that.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">What do you think?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Helvetica",sans-serif">Louis<o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>