[cfe-dev] [libc++] RFC: Add Windows kernel support and partial MSVC 2015 support

[Nico Weber via cfe-dev]

On Tue, Mar 28, 2017 at 11:28 AM, Ben Craig via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> >> There are often ways to safely use STL containers w/o exceptions
> (Especially when custom allocators are provided).
>
> > Yes, but those ways generally involve going to terminate() on OOM.
> That's not OK in a driver.
>
> There is another way, but it isn't conforming, it's easy to get wrong in
> the library, and it's easy to get wrong for users.  It's basically the
> alternative that my company has used with STL Port.
>
> You take an allocator (maybe even the default allocator), and you return
> nullptr on failure.  You also add a bool member (possibly static) that
> indicates whether the allocator has failed.  Then you check that bool
> anytime you use an STL container in a way that could fail.
>
>
>
> This approach sort-of works with std::vector, especially if you develop
> reserve() paranoia.  It sort-of works with std::string if you ignore all
> the temporaries that std::string operations tend to generate.  It doesn't
> work very well at all for the node based containers, because they tend to
> dereference the node immediately after allocating it.
>
>
>
> It's not really an approach I want to endorse.  My company has some
> home-rolled, STL inspired containers that use error codes, and sometimes
> even require "T" to have specific constructors that take those error codes
> by reference.  That approach is still error-prone, but less error-prone
> than using an STL container in a way that it wasn't intended.
>
>
>
> > Before responding in depth I need to ask this question: Why not use
> Clang/C2 or simply Clang+LLVM?
>
> > I'm assuming they don't support targeting the kernel?
>
> I haven't even attempted to use Clang targeting the Windows kernel.  I did
> attempt to point Clang's static analyzer at a kernel component once, and I
> got some decent results... plus a lot of compiler errors that I hadn't
> worked through.  I've done nothing with Clang/C2 so far.
>
> There are some strange MSVC flags required to build for the kernel.  I
> didn't see any web documentation for making stdcall the default calling
> convention,
>

clang-cl implements /Gz as far as I can tell.


> or the dreaded /Zc:wchar_t-, which turns wchar_t into an unsigned short.
>

This isn't implemented, but probably isn't too hard to add, if you want to
give it a shot.


> Even if I did use Clang though, that would only address one of the four
> sub-features you identified, the port to MSVC.
>
>
>
> > However I have large concerns regarding the changes required for [Kernel
> C library] and [Kernel subset] as I suspect they won't satisfy the above
> requirement.
>
> Understood.  This was feedback I was expecting.  Subsets and niche
> platforms are a maintenance burden.
>
>
>
> > For example What if the Windows Kernel C library is incomplete, or
> significantly different from existing implementations [...]
>
> Depends on what it is.  If it is something niche, I will likely want to
> just not support that particular use.  For example, floating point use
> "works" in x86 kernel mode, with a pile of caveats and extra code.  If
> supporting that becomes annoying from my side, I'll likely just drop
> float.h and cfloat support.  Basically, if the kernel C library doesn't
> have support for it, I probably shouldn't be doing it...
>
>
>
> ... unless it's using the STL in the kernel.  Then, I'm totally in the
> right :)
>
>
>
> > My main concern with (4) is the limited feature set that has been
> proposed. Specifically
>
> > how it limits the libc++ internals to the same feature set and the
> changes that would be
>
> > needed to support and maintain it.
>
> >
>
> > First Libc++ cannot reasonably limit itself to the proposed language and
> library feature set since
>
> > it must be free to use "restricted" features within the implementation
> of non-restricted ones whenever
>
> > it is beneficial to the implementation. The burden of maintaining a
> working restricted feature set could
>
> > not fall on upstream maintainers.
>
> I'm not going to make any promises... but would a public build bot address
> your concerns?  What if the restricted feature set got standardized as the
> C++2x definition of a freestanding implementation?
>
>
>
> I can definitely see maintaining a restricted include set to be
> burdensome  I will have to think on how to address that in a reasonable
> way.  Your clang-tidy suggestion may be useful here.
>
>
>
> I don't think maintaining a restricted use of features will be
> particularly hard though.  For <array>, <tuple>, <utility>, and
> <algorithm>, do you foresee changes where an existing function doesn't
> throw, but starts to throw?  Or doesn't currently allocate, but starts to
> allocate?  If we were talking about the containers, or iostreams, or regex,
> I would definitely agree that the implementation flexibility is needed.
> I'm not sure I agree when it's a smaller list of low level headers though.
>
>
>
> >
>
> > Second, the changes required to either guard restricted features using
> #ifdef or remove restricted features
>
> > by re-structuring the headers would be vast and would require constant
> maintenance. Frankly I don't
>
> > see how libc++ or its existing users could benefit from upstreaming
> these changes (For example adding
>
> > #ifdef guards for every usage of `operator new`).
>
>
>
> I hope to use this project as a way to provide implementation experience
> to revise the definition of a freestanding implementation.  I think that
> C++11 got it wrong, and that the current definition isn't useful.  I also
> believe that other kernel, embedded, and bare-metal users would want the
> same feature set that I'm looking for.  This feature set would have been
> useful to me on other projects that weren't targeting the Windows Kernel.
>
>
>
> So for the cost, I agree that it will require a substantial amount of
> work.  Let's say that it's somewhere between _LIBCPP_HAS_NO_THREADS and
> _LIBCPP_NO_EXCEPTIONS amount of work.  There is still benefit for other
> users though.
>
>
>
> > I think it would be much better to get as much of libc++ compiling as
> possible, even if it depends on restricted
>
> > features, and then finding another mechanism to restrict the features
> end-users are allowed to use (Such as clang-tidy).
>
> > This would eliminate the need to restructure headers or spam out
> internal #ifdef guards.
>
> One thing I want to avoid is turning straightforward mistakes into errors
> that are even more cryptic than what C++ users have grown accustomed to.  I
> don't really want the failure condition of inplace_merge to be a linker
> error complaining about a missing operator new.  clang-tidy might help with
> that.  #ifdefs are a straightforward way to address that concern, though it
> requires a substantial amount of effort.  Would you be opposed to
> widespread static_asserts?
>
>
>
> > I don't think partitioning <algorithm> into smaller headers would be
> beneficial
>
> > to existing libc++ users (If that's what you're suggesting).
>
> The three obvious options are 1) do nothing, 2) #ifdefs around the
> existing declarations and definitions, and 3) #ifdefs around #includes that
> pull in the supported feature set.  I could also put static_asserts in
> select areas that I know will cause trouble.
>
>
>
> >> * I'll also need to figure out how to not drag along unwanted header
> dependencies.
>
>
>
> > I don't see how libc++ could upstream changes since it requires every
> header
>
> > it currently includes (modulo bugs).
>
> The straightforward (and labor intensive) way is with #ifdef
> _LIBCPP_HAS_NO_ALLOC
>
>
>
>
>
> > My only idea for handling this would be to write a script to rename
> main() to some
>
> > unique function name and creating a separate test-driver that calls the
> re-named main.
>
> I had similar thoughts.  Alternatively, I could just run execution tests
> overnight.  I can call main just fine, I just can't call 5000 different
> functions each called main.
>
>
>
> > I could envision a fix which replaces `<assert.h>` when compiling the
> tests, but that
>
> > would be a hack. Maybe the Windows Kernel C library provides a mechanism
> for replacing
>
> > the assert handler?
>
> I think the best solution, long term, is to use an "expect" macro instead
> of an assert macro, similar to google test.  I'm not sure that I want to
> take that on though.  "assert" is used in more than 4000 tests.  I'll have
> to look at this more later.
>
>
>
> > Using hard-coded kernel paths is not a change I see being upstream-able.
> However
>
> > we might be able to convince MSVC to implement #include_next if we can
> provide
>
> > strong enough rational for why we need it.
>
> These wouldn't be hard coded paths.  The code would look something like...
>
> #if __LIBCPP_HAS_INCLUDE_NEXT
>
> #include_next <limits.h>
>
> #else
>
> #include __LIBCPP_HACK_INCLUDE_NEXT(limits.h)
>
> #endif
>
> __config would have the definition of __LIBCPP_HACK_INCLUDE_NEXT, and
>
> config_site.in would have the relative path needed to glue everything
> together ("../../km/crt" in my case).
>
>
>
> > I'm violently against working around MSVC template bugs. Every time I've
> seen
>
> > it done it makes the implementation worse.
>
> That's fair.  I will say that not _all_ of the workarounds make worse code,
>
> but quite a few do.  This may end up being what forces me to MSVC 2017.
>
>
>
> One example of a workaround that I don't think made things worse...
>
> struct _LIBCPP_TYPE_VIS __check_tuple_constructor_fail {
>
>      template <class ...>
>
> -    static constexpr bool __enable_default() { return false; }
>
> +    static constexpr bool __enable_default = false;
>
> > Another point is that libc++ doesn't have an <atomic> implementation for
> MSVC
>
> > so adding MSVC support would required adding one.
>
> Yep, I've got one implemented (and untested).  This is the biggest reason
> why
>
> I excluded ARM support explicitly.
>
>
>
> > I hope my response has been helpful, and it hasn't scared you away from
> using libc++.
>
> > These are my initial reactions and are in no way absolute. Please let me
> know if I can
>
> > clarify anything or if I got anything wrong.
>
> The response has been helpful and useful.  No complaints here on the tone
> or style of your reply.
>
>
>
> Thanks,
>
> Ben Craig
>
>
>
> *From:* Billy O'Neal (VC LIBS) [mailto:bion at microsoft.com]
> *Sent:* Monday, March 27, 2017 6:42 PM
> *To:* Eric Fiselier <eric at efcs.ca>; Ben Craig <ben.craig at ni.com>
> *Cc:* via cfe-dev <cfe-dev at lists.llvm.org>; mclow.lists at gmail.com;
> Stephan T. Lavavej <stl at exchange.microsoft.com>; Casey at Carter.net
> *Subject:* RE: [libc++] RFC: Add Windows kernel support and partial MSVC
> 2015 support
>
>
>
> > There are often ways to safely use STL containers w/o exceptions
> (Especially when custom allocators are provided).
>
>
>
> Yes, but those ways generally involve going to terminate() on OOM. That's
> not OK in a driver.
>
>
>
> *From: *Eric Fiselier <eric at efcs.ca>
> *Sent: *Monday, March 27, 2017 4:37 PM
> *To: *Ben Craig <ben.craig at ni.com>
> *Cc: *via cfe-dev <cfe-dev at lists.llvm.org>; mclow.lists at gmail.com; Billy
> O'Neal (VC LIBS) <bion at microsoft.com>; Stephan T. Lavavej
> <stl at exchange.microsoft.com>; Casey at Carter.net <Casey at carter.net>
> *Subject: *Re: [libc++] RFC: Add Windows kernel support and partial MSVC
> 2015 support
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Mar 25, 2017 at 9:18 AM, Ben Craig <ben.craig at ni.com> wrote:
>
> Abstract:
> I would like to add Windows x86 and x64 kernel support to libc++.  My
> initial
> compiler would be MSVC 2015 Update 3, though MSVC 2017 shouldn't be
> difficult to
> add after the fact.
>
> I would like to know if this is a port that the libc++ maintainers are
> willing
> to accept.
>
>
>
> Before responding in depth I need to ask this question: Why not use
> Clang/C2 or simply Clang+LLVM?
>
> I'm assuming they don't support targeting the kernel?
>
>
>
> There seem to be three separate changes going on here, and each of the
> changes
>
> will benefits and challenges. For that reason I think it's best to
> consider them
>
> separately.
>
>
>
> (1) Porting to MSVC
>
> (2) Supporting the Windows Kernel C library.
>
> (3) Implementation a Freestanding configuration of libc++
>
> (4) Implementing the Kernel-space compatible configuration described in
> the original email.
>
>
>
> In general I'm happy to accept specific or niche libc++ changes as long as
> they
>
> aren't detrimental to the overall libc++ quality and implementation
> complexity.
>
> Changes to better support (1) or (3) would be beneficial to the larger
> libc++ audience and I would b
>
> happy to upstream them. However I have large concerns regarding the
> changes required for (2) and (4)
>
> as I suspect they won't satisfy the above requirement.
>
>
>
> For example What if the Windows Kernel C library is incomplete, or
> significantly different from
>
> existing implementations, and supporting it requires re-implementing the
> missing parts within libc++?
>
> These portions would be virtually untestable outside of the Windows Kernel
> environment and would
>
> quickly become unmaintained. Having such code in Libc++ could quickly
> become a detriment.
>
>
>
> My main concern with (4) is the limited feature set that has been
> proposed. Specifically
>
> how it limits the libc++ internals to the same feature set and the changes
> that would be
>
> needed to support and maintain it.
>
>
>
> First Libc++ cannot reasonably limit itself to the proposed language and
> library feature set since
>
> it must be free to use "restricted" features within the implementation of
> non-restricted ones whenever
>
> it is beneficial to the implementation. The burden of maintaining a
> working restricted feature set could
>
> not fall on upstream maintainers.
>
>
>
> Second, the changes required to either guard restricted features using
> #ifdef or remove restricted features
>
> by re-structuring the headers would be vast and would require constant
> maintenance. Frankly I don't
>
> see how libc++ or its existing users could benefit from upstreaming these
> changes (For example adding
>
> #ifdef guards for every usage of `operator new`).
>
>
>
> I think it would be much better to get as much of libc++ compiling as
> possible, even if it depends on restricted
>
> features, and then finding another mechanism to restrict the features
> end-users are allowed to use (Such as clang-tidy).
>
> This would eliminate the need to restructure headers or spam out internal
> #ifdef guards.
>
>
>
>
>
> This means that string, vector, and the non-array containers won't be
> ported.
> Any class or function that requires throwing exceptions to meet their
> standards
> required behavior will be omitted.  That rules out a lot of classes that
> allocate memory.
>
>
>
> There are often ways to safely use STL containers w/o exceptions
> (Especially when
>
> custom allocators are provided).
>
>
>
>
>
> Avoiding allocations allows us to sidestep one other large issue.  In the
> kernel, not all memory is equal.  There are several memory pools to choose
> from,
> but the two most common memory pools are the pageable pool and the
> non-pageable
> pool.  There is no clear correct answer for which pool a global operator
> new
> should use, so we simply won't require an allocating new to be present for
> our
> implementation.  Placement new shall remain though.
>
>
>
> Containers don't use new/delete directly but instead go through the
> specified allocator,
>
> allowing containers to change the allocation behavior on a per-object
> basis. Not
>
> supporting containers because of global operator new's behavior seems
> misguided.
>
>
>
>
>
> My employer has significant experience using C++ in the kernel.  We have
> been
> using a modified version of STLPort for quite some time and learned a lot
> about
> C++ library usage in the kernel, often the hard way.  The big, obvious
> lesson
> is that a lot of the STL is either difficult, or impossible to work with
> when
> exceptions are off and std::terminate is undesired.  There's nothing wrong
> with
> sorting in the kernel though.
>
>
> Challenges:
> * Header partitioning.
>
>
>
> Libc++ prefers larger monolithic headers over many well-partitioned
> headers. The idea is that hitting the filesystem
>
> multiple times is slower than processing the single include file. Any
> proposed changes should keep this in mind.
>
>
>
>     * I don't have an exact list of what functions and classes I will be
>       keeping.  I do know that some of the headers I want to bring along
> have
>       parts that I won't be keeping.  For instance, many of the algorithms
>       allocate a temporary buffer in order to meet performance
> requirements.
>
>
>
> I don't think partitioning <algorithm> into smaller headers would be
> beneficial
>
> to existing libc++ users (If that's what you're suggesting).
>
>
>
>
>
>     * I'll also need to figure out how to not drag along unwanted header
>       dependencies.
>
>
>
> I don't see how libc++ could upstream changes since it requires every
> header
>
> it currently includes (modulo bugs).
>
>
>
> * Testing.
>     * Installing code so that it can be run in the kernel takes several
> seconds
>       for each binary.
>
>     * There is no facility in the Windows kernel for running a program
> starting
>       at "main" in the context of the kernel.
>
>
>
> I suspect this will be the largest hurdle to get the test-suite running.
> My only
>
> idea for handling this would be to write a script to rename main() to some
>
> unique function name and creating a separate test-driver that calls the
> re-named main.
>
>
>
> This could also allow us to combine multiple tests into a single
> executable, avoiding
>
> the cost of installing every test manually.
>
>
>
>
>
>
>
>     * The 32-bit Windows kernel requires a default calling convention of
>       stdcall, but warns if "main" is not cdecl.
>     * A common failure mode for libc++ tests is to crash or assert.  That
>       will "blue-screen" a machine.
>
>
>
> I could envision a fix which replaces `<assert.h>` when compiling the
> tests, but that
>
> would be a hack. Maybe the Windows Kernel C library provides a mechanism
> for replacing
>
> the assert handler?
>
>
>
>  .
>
> * MSVC compiler feature set
>     * No #include_next.  I'll need to use computed include paths, like an
>       STL using caveman.
>
>
>
> Using hard-coded kernel paths is not a change I see being upstream-able.
> However
>
> we might be able to convince MSVC to implement #include_next if we can
> provide
>
> strong enough rational for why we need it.
>
>
>
>
>
>     * Limited expression SFINAE.  Some areas in the code are
> straightforward
>       to fix, and others are not.
>
>
>
> I'm violently against working around MSVC template bugs. Every time I've
> seen
>
> it done it makes the implementation worse.
>
>
>
> Another point is that libc++ doesn't have an <atomic> implementation for
> MSVC
>
> so adding MSVC support would required adding one.
>
>
>
>
>
> * C-runtime
>     * The Windows kernel has its own implementation of the C-runtime.  I
> don't
>       know all the details on it.  I suspect (but don't know) that it is
>       derived from Dinkumware, but I know that it is not the same C-runtime
>       as used in user mode.
>
>
>
> I'm very curious to know the changes required to support the kernel C
> runtime.
>
> Hopefully it's similar enough to other supported C libraries that very few
> changes
>
> are needed.
>
>
>
> I hope my response has been helpful, and it hasn't scared you away from
> using libc++.
>
> These are my initial reactions and are in no way absolute. Please let me
> know if I can
>
> clarify anything or if I got anything wrong.
>
>
>
> /Eric
>
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170328/8b005e6e/attachment.html>