[llvm-dev] Compiler support in libc++

Wed Mar 3 21:09:13 PST 2021

On Wed, Mar 3, 2021 at 5:52 PM Louis Dionne <ldionne at apple.com> wrote:

>
>
> On Mar 3, 2021, at 14:17, Mehdi AMINI <joker.eph at gmail.com> wrote:
>
>
>
> On Wed, Mar 3, 2021 at 10:32 AM David Blaikie <dblaikie at gmail.com> wrote:
>
>> On Wed, Mar 3, 2021 at 9:31 AM Mehdi AMINI via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Hi,
>>>
>>> It seems to me that this would require one extra stage of bootstrap in
>>> CI for many buildbots.
>>> For example, today I have a Linux bot with a clang-8 host compiler and
>>> libstdc++. The goal is to ensure that MLIR (but it is applicable to any
>>> project) builds with clang and libc++ at the top of the main branch.
>>> So the setup is:
>>> - stage1: build clang/libc++ with host clang-8/libstdc++
>>> - stage2: build test "anything" using stage1 (`ninja check-all` in the
>>> monorepo for example, but applicable to any other external project)
>>>
>>
>
>> With this proposal, the setup would be:
>>>
>>> - stage1: build just clang with host clang-8/libstdc++
>>> - stage2: build clang/libc++ with stage1 clang and host libstdc++
>>> - stage3: build test "anything" using stage2 (`ninja check-all` in the
>>> monorepo for example, but applicable to any other external project)
>>>
>>
>> Would it be possible to change the build system so that libc++ can be
>> built like compiler-rt, using the just-built clang? That would then avoid
>> the need for the extra stage? (though it would bottleneck the usual build a
>> bit - not being able to start the libc++ build until after clang build)
>>
>
> That's a good point:
>  - stage1: build just clang with host clang-8/libstdc++
> - stage1.5: build libc++ with stage1 clang
> - stage 2: assemble toolchain with clang from stage1 and libc++ from stage2
> - stage3: build test "anything" using stage2 (`ninja check-all` in the
> monorepo for example, but applicable to any other external project)
>
> Since this "stage 2" is the new "stage1", I believe that this should be
> made completely straightforward to achieve. Ideally it should boil down to
> a single standard CMake invocation to produce this configuration.
>
>
> I think the Runtimes build is exactly what you’re looking for. With the
> runtimes build, you say:
>
>     $ cmake -S "${MONOREPO_ROOT}/llvm" -B "${BUILD_DIR}” \
>         -DLLVM_ENABLE_PROJECTS="clang” \
>         -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi” \
>         -DLLVM_RUNTIME_TARGETS="x86_64-unknown-linux-gnu”
>
> And then you can just do:
>
>     $ make -C $BUILD_DIR cxx
>
> That will bootstrap Clang and then build libc++ with the just-built Clang.
> I don’t know whether you consider that to be one or two stages, but it
> happens automatically in that single CMake invocation. And since building
> libc++ is basically trivial, this takes approximately the same time as
> building Clang only.
>

Yes, thanks, this config is just perfectly fitting here!

>
>
>
>>
>> & again, this isn't so much a proposal of change, but one of documenting
>> the current state of things - which reveals the current situations are sort
>> of unsupported? (though it also goes against the claim that they're
>> untested) - so I'll be curious to hear from the libc++ folks about this for
>> sure.
>>
>
> Right: I'm absolutely not convinced by the "we're documenting the current
> state of things" actually.
> In particular my take in general on what we call "supported" is a policy
> that "we revert if we break a supported configuration" and "we accept
> patches to fix a supported configuration". So the change here is that
> libc++ would not accept to revert when they break an older toolchain, and
> we wouldn't accept patches to libc++ to fix it.
> We don't necessarily have buildbots for every configuration that we claim
> LLVM is supporting, yet this is the policy, and I'm quite wary of defining
> the "current state of things" based exclusively on the current public
> buildbots setup.
>
>
> To be clear, what we do today to “fix” older compilers is usually to mark
> failing tests in the test suite with XFAIL or UNSUPPORTED annotations. We
> don’t actually provide a good level of support for those compilers. There’s
> also other things that we simply can’t fix, like the fact that a libc++
> built with a compiler that doesn’t know about char8_t (for example) won’t
> produce the RTTI for char8_t in the dylib, and hence will produce a dylib
> where some random uses of char8_t will break down. This is just an example,
> but my point is that it’s far better to clarify the support policy to
> something that *we know* will work, and that we can commit to supporting.
> There's a small upfront cost for people running build bots right now, but
> once things are setup it’ll just be better for everyone.
>
>
>
>>
>>
>>>
>>> The only way to avoid adding a stage in the bootstrap is to keep
>>> updating the bots with a very recent host clang (I'm not convinced that
>>> increasing the cost of maintenance for CI / infra is good in general).
>>>
>>> We should aim for a better balance: it is possible that clang-5 is too
>>> old (I don't know?), but there are people (like me, and possibly others)
>>> who are testing HEAD with older compiler (clang-8 here) and it does not
>>> seem broken at the moment (or the recent years), I feel there should be a
>>> strong motivation to break it.
>>>
>>
> Libc++ on Clang 8 doesn’t look broken because it builds. And it builds
> because you’ve been pinging us on Phabricator when we break you with a
> change, and we add a “workaround” that makes it build. But there’s no
> guarantee about the “quality" of the libc++ that you get in that case
> though. That’s exactly what we want to avoid - you get something that
> “kinda works”, yet we still have to insert random workarounds in the code.
> It’s a lose/lose situation.
>

To be fair there has been exactly *one* breakage caused by libc++ over the
last 2 years: while there may be issue in corner cases like you mentions,
it seems to work fine for many projects (including clang/llvm/mlir
bootstrap since this is what I've been testing).

>
> Could we find something more intermediate here? Like a time-based support
>>> (2 years?) or something based on the latest Ubuntu release or something
>>> like that. That would at least keep the cost of upgrading bots a bit more
>>> controlled (and avoid a costly extra stage of bootstrap).
>>>
>>
> As I said above, I don’t think there’s any extra stage of bootstrap. The
> only difference is that you build your libc++ using the Clang you just
> built, instead of against the system compiler. In both cases you need to
> build both Clang and libc++ anyway.
>
> Furthermore, we specifically support the last released Clang. If you were
> in a situation where you didn’t want to build Clang but wanted to build
> libc++, you’d just have to download a sufficiently recent Clang release and
> use that.
>
> Louis
>
>
>
>>> On Tue, Mar 2, 2021 at 7:10 AM Louis Dionne via llvm-dev <
>>> llvm-dev at lists.llvm.org> wrote:
>>>
>>>>
>>>>
>>>> > On Mar 1, 2021, at 15:41, Joerg Sonnenberger via llvm-dev <
>>>> llvm-dev at lists.llvm.org> wrote:
>>>> >
>>>> > On Mon, Mar 01, 2021 at 12:40:36PM -0500, Louis Dionne via llvm-dev
>>>> wrote:
>>>> >> However, for a library like libc++, things are a bit different.
>>>> >
>>>> > So how does this prevent the libstdc++ mess that you need to lock step
>>>> > the RTL with the compiler and more importantly, get constantly screwed
>>>> > over when you need to upgrade or downgrade the compiler in a complex
>>>> > environment like an actual Operating System?
>>>>
>>>> Could you please elaborate on what issue you’re thinking about here? As
>>>> someone who ships libc++ as part of an operating system and SDK (which
>>>> isn’t necessarily in perfect lockstep with the compiler), I don’t see any
>>>> issues. The guarantee that you can still use a ~6 months old Clang is
>>>> specifically intended to allow for that use case, i.e. shipping libc++ as
>>>> part of an OS instead of a toolchain.
>>>>
>>>>
>>>> > I consider this proposal a major step backwards...
>>>>
>>>> To be clear, we only want to make official the level of support that we
>>>> already provide in reality. As I explained in my original email, if you’ve
>>>> been relying on libc++ working on much older compilers, I would suggest
>>>> that you stop doing so because nobody is testing that and we don’t really
>>>> support it, despite what the documentation says. So IMO this can’t be a
>>>> step backwards, since we already don’t support these compilers, we just
>>>> pretend that we do.
>>>>
>>>> Louis
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210303/29a34303/attachment.html>