<div dir="ltr"><div><br></div><div><br></div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">On Sat, Mar 25, 2017 at 9:18 AM, Ben Craig <span dir="ltr"><<a href="mailto:ben.craig@ni.com" target="_blank">ben.craig@ni.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Abstract:<br>
I would like to add Windows x86 and x64 kernel support to libc++. My initial<br>
compiler would be MSVC 2015 Update 3, though MSVC 2017 shouldn't be difficult to<br>
add after the fact.<br>
<br>
I would like to know if this is a port that the libc++ maintainers are willing<br>
to accept.<br></blockquote><div><br></div><div>Before responding in depth I need to ask this question: Why not use Clang/C2 or simply Clang+LLVM?</div><div>I'm assuming they don't support targeting the kernel?</div><div><br></div><div><div>There seem to be three separate changes going on here, and each of the changes</div><div>will benefits and challenges. For that reason I think it's best to consider them</div><div>separately. </div><div><br></div><div>(1) Porting to MSVC</div><div>(2) Supporting the Windows Kernel C library.</div><div>(3) Implementation a Freestanding configuration of libc++</div><div>(4) Implementing the Kernel-space compatible configuration described in the original email.</div><div><br></div><div>In general I'm happy to accept specific or niche libc++ changes as long as they</div><div>aren't detrimental to the overall libc++ quality and implementation complexity.</div><div><div>Changes to better support (1) or (3) would be beneficial to the larger libc++ audience and I would b</div><div>happy to upstream them. However I have large concerns regarding the changes required for (2) and (4)</div><div>as I suspect they won't satisfy the above requirement.</div></div></div><div><br></div><div>For example What if the Windows Kernel C library is incomplete, or significantly different from</div><div>existing implementations, and supporting it requires re-implementing the missing parts within libc++?</div><div>These portions would be virtually untestable outside of the Windows Kernel environment and would</div><div>quickly become unmaintained. Having such code in Libc++ could quickly become a detriment.</div><div><br></div><div>My main concern with (4) is the limited feature set that has been proposed. Specifically</div><div>how it limits the libc++ internals to the same feature set and the changes that would be</div><div>needed to support and maintain it.</div><div><br></div><div>First Libc++ cannot reasonably limit itself to the proposed language and library feature set since</div><div>it must be free to use "restricted" features within the implementation of non-restricted ones whenever</div><div>it is beneficial to the implementation. The burden of maintaining a working restricted feature set could</div><div>not fall on upstream maintainers.</div><div><br></div><div>Second, the changes required to either guard restricted features using #ifdef or remove restricted features</div><div>by re-structuring the headers would be vast and would require constant maintenance. Frankly I don't</div><div>see how libc++ or its existing users could benefit from upstreaming these changes (For example adding</div><div>#ifdef guards for every usage of `operator new`).</div><div><br></div><div>I think it would be much better to get as much of libc++ compiling as possible, even if it depends on restricted</div><div>features, and then finding another mechanism to restrict the features end-users are allowed to use (Such as clang-tidy).</div><div>This would eliminate the need to restructure headers or spam out internal #ifdef guards.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
This means that string, vector, and the non-array containers won't be ported.<br>
Any class or function that requires throwing exceptions to meet their standards<br>
required behavior will be omitted. That rules out a lot of classes that<br>
allocate memory.</blockquote><div><br></div><div>There are often ways to safely use STL containers w/o exceptions (Especially when</div><div>custom allocators are provided).</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
Avoiding allocations allows us to sidestep one other large issue. In the<br>
kernel, not all memory is equal. There are several memory pools to choose from,<br>
but the two most common memory pools are the pageable pool and the non-pageable<br>
pool. There is no clear correct answer for which pool a global operator new<br>
should use, so we simply won't require an allocating new to be present for our<br>
implementation. Placement new shall remain though.</blockquote><div><br></div><div>Containers don't use new/delete directly but instead go through the specified allocator,</div><div>allowing containers to change the allocation behavior on a per-object basis. Not </div><div>supporting containers because of global operator new's behavior seems misguided.</div><div> </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
My employer has significant experience using C++ in the kernel. We have been<br>
using a modified version of STLPort for quite some time and learned a lot about<br>
C++ library usage in the kernel, often the hard way. The big, obvious lesson<br>
is that a lot of the STL is either difficult, or impossible to work with when<br>
exceptions are off and std::terminate is undesired. There's nothing wrong with<br>
sorting in the kernel though. </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<br>
Challenges:<br>
* Header partitioning.<br></blockquote><div><br></div><div>Libc++ prefers larger monolithic headers over many well-partitioned headers. The idea is that hitting the filesystem</div><div>multiple times is slower than processing the single include file. Any proposed changes should keep this in mind.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* I don't have an exact list of what functions and classes I will be<br>
keeping. I do know that some of the headers I want to bring along have<br>
parts that I won't be keeping. For instance, many of the algorithms<br>
allocate a temporary buffer in order to meet performance requirements.<br></blockquote><div><br></div><div>I don't think partitioning <algorithm> into smaller headers would be beneficial</div><div>to existing libc++ users (If that's what you're suggesting). </div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* I'll also need to figure out how to not drag along unwanted header<br>
dependencies.<br></blockquote><div><br></div><div>I don't see how libc++ could upstream changes since it requires every header</div><div>it currently includes (modulo bugs).</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* Testing.<br>
* Installing code so that it can be run in the kernel takes several seconds<br>
for each binary. </blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* There is no facility in the Windows kernel for running a program starting<br>
at "main" in the context of the kernel.<br></blockquote><div><br></div><div>I suspect this will be the largest hurdle to get the test-suite running. My only</div><div>idea for handling this would be to write a script to rename main() to some</div><div>unique function name and creating a separate test-driver that calls the re-named main.</div><div><br></div><div>This could also allow us to combine multiple tests into a single executable, avoiding</div><div>the cost of installing every test manually.</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* The 32-bit Windows kernel requires a default calling convention of<br>
stdcall, but warns if "main" is not cdecl.<br>
* A common failure mode for libc++ tests is to crash or assert. That<br>
will "blue-screen" a machine.<br></blockquote><div><br></div><div>I could envision a fix which replaces `<assert.h>` when compiling the tests, but that</div><div>would be a hack. Maybe the Windows Kernel C library provides a mechanism for replacing</div><div>the assert handler?</div><div><br></div><div> .</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* MSVC compiler feature set<br>
* No #include_next. I'll need to use computed include paths, like an<br>
STL using caveman.<br></blockquote><div><br></div><div>Using hard-coded kernel paths is not a change I see being upstream-able. However</div><div>we might be able to convince MSVC to implement #include_next if we can provide</div><div>strong enough rational for why we need it.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* Limited expression SFINAE. Some areas in the code are straightforward<br>
to fix, and others are not.<br></blockquote><div><br></div><div>I'm violently against working around MSVC template bugs. Every time I've seen</div><div>it done it makes the implementation worse.</div><div><br></div><div>Another point is that libc++ doesn't have an <atomic> implementation for MSVC</div><div>so adding MSVC support would required adding one.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
* C-runtime<br>
* The Windows kernel has its own implementation of the C-runtime. I don't<br>
know all the details on it. I suspect (but don't know) that it is<br>
derived from Dinkumware, but I know that it is not the same C-runtime<br>
as used in user mode.<br></blockquote><div><br></div><div>I'm very curious to know the changes required to support the kernel C runtime.</div><div>Hopefully it's similar enough to other supported C libraries that very few changes</div><div>are needed.</div><div><br></div><div>I hope my response has been helpful, and it hasn't scared you away from using libc++.</div><div>These are my initial reactions and are in no way absolute. Please let me know if I can</div><div>clarify anything or if I got anything wrong.</div><div><br></div><div>/Eric</div></div><br></div></div>