[cfe-dev] [libc++] RFC: Add Windows kernel support and partial MSVC 2015 support

[Ben Craig via cfe-dev]

Abstract:
I would like to add Windows x86 and x64 kernel support to libc++.  My initial
compiler would be MSVC 2015 Update 3, though MSVC 2017 shouldn't be difficult to
add after the fact.

I would like to know if this is a port that the libc++ maintainers are willing
to accept.

Motivation:
C++ is intended to be useful for systems programming, and the kernel is as
"systems" as it gets.  There are a fair number of language features that lean
heavily on the presence of the library.  Initializer lists require library
support.  R-value references require std::move, std::forward, and others, or
require users to duplicate their functionality.  Variadic templates are
extremely cumbersome to work with if std::tuple is not available.  std::atomic
provides a much better interface than compiler intrinsics for working with the
C++ memory model.

A subset of the C++ language can be used (with caution) in the Windows kernel,
but the library remains out of reach.  The C++ library that ships with MSVC
won't compile at all in the kernel, even for headers that ostensibly make sense,
like <atomic>.

Since the vendor's C++ library doesn't work in the kernel, might as well try
and make libc++ work there instead.

There's no reason that the Windows ARM kernel couldn't be addressed as well.
However, my employer has no significant interest in that environment, and I have
no access to devices to test with for that environment.  I would be glad to help
review code to support the Windows ARM kernel though.

Strategy:
I will not be porting the entirety of libc++.  I will have exceptions and RTTI
turned off.  I plan on porting the freestanding headers [17.5.1.3,2 compliance],
minus <exception>, and <typeinfo>.  I also plan on porting the portions of
<algorithm>, <array>, <tuple>, and <utility> that can naturally be expressed in
an exception-free and RTTI-free environment.  I can be convinced that other
headers should also be included.

This means that string, vector, and the non-array containers won't be ported.
Any class or function that requires throwing exceptions to meet their standards
required behavior will be omitted.  That rules out a lot of classes that
allocate memory.

Avoiding allocations allows us to sidestep one other large issue.  In the
kernel, not all memory is equal.  There are several memory pools to choose from,
but the two most common memory pools are the pageable pool and the non-pageable
pool.  There is no clear correct answer for which pool a global operator new
should use, so we simply won't require an allocating new to be present for our
implementation.  Placement new shall remain though.

I don't believe I will need to port libc++abi, libunwind, and compiler-rt.

In some ways, this is a good first step for getting MSVC support.  The
functionality that is most coupled to POSIX-like operating systems is all in
the "hosted" portion of the library.

Prior Work:
Lawrence Crowl and Alberto Ganesh Barbati proposed this paper back in the C++11
time frame with mixed results:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3256.html

My employer has significant experience using C++ in the kernel.  We have been
using a modified version of STLPort for quite some time and learned a lot about
C++ library usage in the kernel, often the hard way.  The big, obvious lesson
is that a lot of the STL is either difficult, or impossible to work with when
exceptions are off and std::terminate is undesired.  There's nothing wrong with
sorting in the kernel though.

Challenges:
* Header partitioning.
    * I don't have an exact list of what functions and classes I will be 
      keeping.  I do know that some of the headers I want to bring along have
      parts that I won't be keeping.  For instance, many of the algorithms
      allocate a temporary buffer in order to meet performance requirements.
    * I'll also need to figure out how to not drag along unwanted header
      dependencies.
* I don't know if anyone has ever attempted to get cmake to target a Windows
  kernel environment.  I don't think I will be trying to do this either.  I
  think I will be using a script similar in spirit to buildit.sh.
* Testing.
    * New 'requires' code will be needed to only attempt to run the tests that
      I care about.  I plan on doing this by white listing headers, and having
      the test runner examine the test to see if any unsupported headers are in
      the test.
    * Installing code so that it can be run in the kernel takes several seconds
      for each binary.
    * There is no facility in the Windows kernel for running a program starting
      at "main" in the context of the kernel.
    * The 32-bit Windows kernel requires a default calling convention of
      stdcall, but warns if "main" is not cdecl.
    * A common failure mode for libc++ tests is to crash or assert.  That
      will "blue-screen" a machine.
    * The existing lit test infrastructure assumes flags similar to gcc and
      llvm.  These assumptions are scattered throughout the code, and not in a
      centralized location.
* MSVC compiler feature set
    * No #include_next.  I'll need to use computed include paths, like an
      STL using caveman.
    * Limited expression SFINAE.  Some areas in the code are straightforward
      to fix, and others are not.
* C-runtime
    * The Windows kernel has its own implementation of the C-runtime.  I don't
      know all the details on it.  I suspect (but don't know) that it is
      derived from Dinkumware, but I know that it is not the same C-runtime
      as used in user mode.
      
I have a very early prototype that is poorly tested, and won't work at all
for existing supporting platforms.  Basically, I took the quick and dirty
approach of commenting out / deleting code that got in the way of the Windows
kernel.  It can still be somewhat instructive in the kinds of things I expect
to need to do (eventually) to the libc++ code base.  The prototype can
be found here: https://github.com/ben-craig/libcxx/tree/ntddk .


Thanks,
Ben Craig