[libcxx-dev] Why does libcxx include its own files this way?

Sun Jan 20 11:06:35 PST 2019

From: Eric Fiselier <eric at efcs.ca>

On Sat, Jan 19, 2019 at 11:16 PM Olivier Giroux <OGiroux at nvidia.com<mailto:OGiroux at nvidia.com>> wrote:
>> Are you trying to make libc++ work when another STL is first on the include path?
Yes, this is a big part of the issue. I can’t practically replace all the host STLs, nor make all the host STLs grow accelerator support overnight. So I introduce my STL in namespace::std:: and under a <subdir/name> include path, if you recall, but then the libcxx self-includes go to the wrong place.

A couple of questions:

  1.  why `simt::std` and not `std::simt`? ie. leading with the fact it's  a standard library.

I don’t have strong feelings about this. Do you think the alternative is simpler?

2) How important is it that you expose `<simt/foo>` to the user? Could `<foo>` act like a header that did `#include_next <foo> #include <simt/foo>`?

That could also work. Noting that Windows requires the kludge #include <../include/foo> instead of #include_next <foo>. (Potentially that could become just #include <foo> if the libcxx headers can be isolated in a subdirectory and use #include "name" when they mean to find each other, and not the host.)

>> Google wraps the STL headers with their own intermediate headers that add or adjust declarations as needed
In a nutshell, that’s what we do too, but it’s convoluted and brittle in some cases.
Can you provide an example cases where it's problematically brittle?

In hindsight, I was just repeating the previous point that it’s convoluted. It’s hard to keep it all straight in all the compiler scenarios we need to deal with, so there’s a lot of whack-a-mole play. Obviously testing is the only real answer to that, but one shouldn’t discount the discount one gets from simplicity.
Given your use-case long predates ours… it sounds like either a change like this one would have to live in our fork, or we choose to keep and maintain the redirection wrappers.

Olivier

From: Eric Fiselier <eric at efcs.ca<mailto:eric at efcs.ca>>
Date: Saturday, January 19, 2019 at 7:26 PM
To: Olivier Giroux <OGiroux at nvidia.com<mailto:OGiroux at nvidia.com>>
Cc: Duncan Exon Smith <dexonsmith at apple.com<mailto:dexonsmith at apple.com>>, Libc++ Dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>>
Subject: Re: [libcxx-dev] Why does libcxx include its own files this way?

> It looks to me like the use of #include <name> is an attempt to enforce a reservation on this name in the filesystem of the build machine.

It's not enforcing a reservation on the header name. But it's critically important that libc++'s includes are found in the <angled> include path, before almost all headers.
Otherwise our "#include_next" directives don't work.

Additionally, Google wraps the STL headers with their own intermediate headers that add or adjust declarations as needed. This depends on our internal includes
being picked up first for `#include <foo>`. Your change would break that.

I'm not sure exactly what you're trying to do here. Are you trying to make libc++ work when another STL is first on the include path?

On Sat, Jan 19, 2019 at 11:53 AM Olivier Giroux via libcxx-dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>> wrote:
I’ve tested replacing #include <name> with #include "name" for all names in the table https://en.cppreference.com/w/cpp/freestanding and all names in the transitive dependency closure in libcxx (e.g. __config), and except for a small gotcha in the /experimental subdirectory (where some need to become "../name") this passes all tests on Mac. Obviously this is a 145-file patch, a couple lines per.

Are you interested in this patch?

Olivier

From: libcxx-dev <libcxx-dev-bounces at lists.llvm.org<mailto:libcxx-dev-bounces at lists.llvm.org>> on behalf of Olivier Giroux via libcxx-dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>>
Reply-To: Olivier Giroux <OGiroux at nvidia.com<mailto:OGiroux at nvidia.com>>
Date: Friday, January 18, 2019 at 1:28 PM
To: Duncan Exon Smith <dexonsmith at apple.com<mailto:dexonsmith at apple.com>>
Cc: Bruno Cardoso Lopes <bruno.cardoso at gmail.com<mailto:bruno.cardoso at gmail.com>>, Libc++ Dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>>
Subject: Re: [libcxx-dev] Why does libcxx include its own files this way?

Ok. If nobody stops me then I’ll make a patch. I’ve looked at it before and it’s a big patch. Would it be acceptable if I started with the Freestanding headers as a trial balloon?

Olivier

From: <dexonsmith at apple.com<mailto:dexonsmith at apple.com>> on behalf of Duncan Exon Smith <dexonsmith at apple.com<mailto:dexonsmith at apple.com>>
Date: Friday, January 18, 2019 at 10:10 AM
To: Olivier Giroux <OGiroux at nvidia.com<mailto:OGiroux at nvidia.com>>
Cc: Libc++ Dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>>, Bruno Cardoso Lopes <bruno.cardoso at gmail.com<mailto:bruno.cardoso at gmail.com>>
Subject: Re: [libcxx-dev] Why does libcxx include its own files this way?

+libcxx-dev, which got dropped
+Bruno

On 2019 Jan  18, at 07:21, Olivier Giroux <OGiroux at nvidia.com<mailto:OGiroux at nvidia.com>> wrote:

But you agree that it does, though, right?

Hmm.  To be concrete, this is what I was worried about:

#!/bin/bash

run() { echo "$*" >&2; "$@" || exit 1; }
run mkdir -p quote
run cat </dev/null >quote/type_traits
run /usr/bin/printf "%s\n" \
    "#include <vector>" \
    "std::vector<int> g;" \
    > t.cpp
run clang -iquote quote -fsyntax-only t.cpp

which should give this output:

$ bash t.sh
mkdir -p quote
cat
/usr/bin/printf %s\n #include <vector> std::vector<int> g;
clang -iquote quote -fsyntax-only t.cpp

(i.e., no errors)

I thought if we switched to quoted "type_traits" everywhere I'd see a spew of errors.  However, I'd forgotten that quoted includes check relative to the includer *first*, before hitting the search paths.  So I was wrong here and this testcase doesn't expose a problem with your suggestion.

One other concern I have is that the funny #include_next dance between libc++ headers, clang headers, and libc (especially on Darwin-derived SDKs) continues work for modules.  I kind of doubt your idea would cause a problem there since libc++ should be the outer most layer, but it's pretty subtle and I'd want to be sure.  Bruno might be able to check that out for you if you make a patch.

Bruno, are there other concerns for modules as well?

That’s actually a problem that I spend a bunch of time working around in my CUDA standard library port. I don’t want to interfere with <atomic> so I make <cuda/atomic> but then that header wants to include <type_traits> instead of the <cuda/type_traits> that I need it to find. If it included "type_traits" instead, it would all just work.

So I have work-arounds for that, and obviously I can also flip <> to "" in my fork, but do you want to see ""?

Makes sense?

Olivier

From: <dexonsmith at apple.com<mailto:dexonsmith at apple.com>> on behalf of Duncan Exon Smith <dexonsmith at apple.com<mailto:dexonsmith at apple.com>>
Date: Friday, January 18, 2019 at 5:46 AM
To: Olivier Giroux <OGiroux at nvidia.com<mailto:OGiroux at nvidia.com>>
Cc: Libc++ Dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>>
Subject: Re: [libcxx-dev] Why does libcxx include its own files this way?

I don’t think we want Libc++ to pick up files in -iquote search paths.

On Jan 17, 2019, at 22:20, Olivier Giroux via libcxx-dev <libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>> wrote:
Inside of libcxx, you can see #include <name> being used when the library is _clearly_ intending for it to find the file that #include "name" would find. I say it’s clear because, if any other file in the world were chosen, then the library would likely fail to compile.

It looks to me like the use of #include <name> is an attempt to enforce a reservation on this name in the filesystem of the build machine. If true, then it’s kind of an overreach. I’d personally much rather if these sideways includes used #include "name" instead.

Is there a better explanation for why this is done this way?

Olivier

________________________________
This email message is for the sole use of the intended recipient(s) and may contain confidential information.  Any unauthorized review, use, disclosure or distribution is prohibited.  If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message.
________________________________
_______________________________________________
libcxx-dev mailing list
libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/libcxx-dev

_______________________________________________
libcxx-dev mailing list
libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/libcxx-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/libcxx-dev/attachments/20190120/5c1815dd/attachment-0001.html>