<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal">> There are often ways to safely use STL containers w/o exceptions (Especially when custom allocators are provided).</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Yes, but those ways generally involve going to terminate() on OOM. That's not OK in a driver.</p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="mso-element:para-border-div;border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="border:none;padding:0in"><b>From: </b><a href="mailto:eric@efcs.ca">Eric Fiselier</a><br>
<b>Sent: </b>Monday, March 27, 2017 4:37 PM<br>
<b>To: </b><a href="mailto:ben.craig@ni.com">Ben Craig</a><br>
<b>Cc: </b><a href="mailto:cfe-dev@lists.llvm.org">via cfe-dev</a>; <a href="mailto:mclow.lists@gmail.com">
mclow.lists@gmail.com</a>; <a href="mailto:bion@microsoft.com">Billy O'Neal (VC LIBS)</a>;
<a href="mailto:stl@exchange.microsoft.com">Stephan T. Lavavej</a>; <a href="mailto:Casey@carter.net">
Casey@Carter.net</a><br>
<b>Subject: </b>Re: [libc++] RFC: Add Windows kernel support and partial MSVC 2015 support</p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Sat, Mar 25, 2017 at 9:18 AM, Ben Craig <<a href="mailto:ben.craig@ni.com" target="_blank">ben.craig@ni.com</a>> wrote:<o:p></o:p></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">Abstract:<br>
I would like to add Windows x86 and x64 kernel support to libc++. My initial<br>
compiler would be MSVC 2015 Update 3, though MSVC 2017 shouldn't be difficult to<br>
add after the fact.<br>
<br>
I would like to know if this is a port that the libc++ maintainers are willing<br>
to accept.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Before responding in depth I need to ask this question: Why not use Clang/C2 or simply Clang+LLVM?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I'm assuming they don't support targeting the kernel?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">There seem to be three separate changes going on here, and each of the changes<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">will benefits and challenges. For that reason I think it's best to consider them<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">separately. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">(1) Porting to MSVC<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(2) Supporting the Windows Kernel C library.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(3) Implementation a Freestanding configuration of libc++<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">(4) Implementing the Kernel-space compatible configuration described in the original email.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">In general I'm happy to accept specific or niche libc++ changes as long as they<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">aren't detrimental to the overall libc++ quality and implementation complexity.<o:p></o:p></p>
</div>
<div>
<div>
<p class="MsoNormal">Changes to better support (1) or (3) would be beneficial to the larger libc++ audience and I would b<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">happy to upstream them. However I have large concerns regarding the changes required for (2) and (4)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">as I suspect they won't satisfy the above requirement.<o:p></o:p></p>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">For example What if the Windows Kernel C library is incomplete, or significantly different from<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">existing implementations, and supporting it requires re-implementing the missing parts within libc++?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">These portions would be virtually untestable outside of the Windows Kernel environment and would<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">quickly become unmaintained. Having such code in Libc++ could quickly become a detriment.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">My main concern with (4) is the limited feature set that has been proposed. Specifically<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">how it limits the libc++ internals to the same feature set and the changes that would be<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">needed to support and maintain it.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">First Libc++ cannot reasonably limit itself to the proposed language and library feature set since<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">it must be free to use "restricted" features within the implementation of non-restricted ones whenever<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">it is beneficial to the implementation. The burden of maintaining a working restricted feature set could<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">not fall on upstream maintainers.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Second, the changes required to either guard restricted features using #ifdef or remove restricted features<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">by re-structuring the headers would be vast and would require constant maintenance. Frankly I don't<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">see how libc++ or its existing users could benefit from upstreaming these changes (For example adding<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">#ifdef guards for every usage of `operator new`).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I think it would be much better to get as much of libc++ compiling as possible, even if it depends on restricted<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">features, and then finding another mechanism to restrict the features end-users are allowed to use (Such as clang-tidy).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">This would eliminate the need to restructure headers or spam out internal #ifdef guards.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">This means that string, vector, and the non-array containers won't be ported.<br>
Any class or function that requires throwing exceptions to meet their standards<br>
required behavior will be omitted. That rules out a lot of classes that<br>
allocate memory.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">There are often ways to safely use STL containers w/o exceptions (Especially when<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">custom allocators are provided).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">Avoiding allocations allows us to sidestep one other large issue. In the<br>
kernel, not all memory is equal. There are several memory pools to choose from,<br>
but the two most common memory pools are the pageable pool and the non-pageable<br>
pool. There is no clear correct answer for which pool a global operator new<br>
should use, so we simply won't require an allocating new to be present for our<br>
implementation. Placement new shall remain though.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Containers don't use new/delete directly but instead go through the specified allocator,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">allowing containers to change the allocation behavior on a per-object basis. Not <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">supporting containers because of global operator new's behavior seems misguided.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">My employer has significant experience using C++ in the kernel. We have been<br>
using a modified version of STLPort for quite some time and learned a lot about<br>
C++ library usage in the kernel, often the hard way. The big, obvious lesson<br>
is that a lot of the STL is either difficult, or impossible to work with when<br>
exceptions are off and std::terminate is undesired. There's nothing wrong with<br>
sorting in the kernel though. <o:p></o:p></p>
</blockquote>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><br>
Challenges:<br>
* Header partitioning.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Libc++ prefers larger monolithic headers over many well-partitioned headers. The idea is that hitting the filesystem<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">multiple times is slower than processing the single include file. Any proposed changes should keep this in mind.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"> * I don't have an exact list of what functions and classes I will be<br>
keeping. I do know that some of the headers I want to bring along have<br>
parts that I won't be keeping. For instance, many of the algorithms<br>
allocate a temporary buffer in order to meet performance requirements.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I don't think partitioning <algorithm> into smaller headers would be beneficial<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">to existing libc++ users (If that's what you're suggesting). <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"> * I'll also need to figure out how to not drag along unwanted header<br>
dependencies.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I don't see how libc++ could upstream changes since it requires every header<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">it currently includes (modulo bugs).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">* Testing.<br>
* Installing code so that it can be run in the kernel takes several seconds<br>
for each binary. <o:p></o:p></p>
</blockquote>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"> * There is no facility in the Windows kernel for running a program starting<br>
at "main" in the context of the kernel.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I suspect this will be the largest hurdle to get the test-suite running. My only<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">idea for handling this would be to write a script to rename main() to some<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">unique function name and creating a separate test-driver that calls the re-named main.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">This could also allow us to combine multiple tests into a single executable, avoiding<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">the cost of installing every test manually.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"> * The 32-bit Windows kernel requires a default calling convention of<br>
stdcall, but warns if "main" is not cdecl.<br>
* A common failure mode for libc++ tests is to crash or assert. That<br>
will "blue-screen" a machine.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I could envision a fix which replaces `<assert.h>` when compiling the tests, but that<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">would be a hack. Maybe the Windows Kernel C library provides a mechanism for replacing<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">the assert handler?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> .<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">* MSVC compiler feature set<br>
* No #include_next. I'll need to use computed include paths, like an<br>
STL using caveman.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Using hard-coded kernel paths is not a change I see being upstream-able. However<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">we might be able to convince MSVC to implement #include_next if we can provide<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">strong enough rational for why we need it.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"> * Limited expression SFINAE. Some areas in the code are straightforward<br>
to fix, and others are not.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm violently against working around MSVC template bugs. Every time I've seen<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">it done it makes the implementation worse.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Another point is that libc++ doesn't have an <atomic> implementation for MSVC<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">so adding MSVC support would required adding one.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">* C-runtime<br>
* The Windows kernel has its own implementation of the C-runtime. I don't<br>
know all the details on it. I suspect (but don't know) that it is<br>
derived from Dinkumware, but I know that it is not the same C-runtime<br>
as used in user mode.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I'm very curious to know the changes required to support the kernel C runtime.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Hopefully it's similar enough to other supported C libraries that very few changes<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">are needed.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I hope my response has been helpful, and it hasn't scared you away from using libc++.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">These are my initial reactions and are in no way absolute. Please let me know if I can<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">clarify anything or if I got anything wrong.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">/Eric<o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>