[libcxx-dev] How to check for a feature-test macro

Tue Dec 11 09:11:12 PST 2018

In addition to that we didn't use it because we wanted to plug the tests into our existing distributed test harness system (first Gauntlet, and now Contest) that we use to test all the other compiler and libraries bits.

Billy3
________________________________
From: Ben Craig <ben.craig at ni.com>
Sent: Tuesday, December 11, 2018 08:19 AM
To: Louis Dionne; Stephan T. Lavavej; libcxx-dev at lists.llvm.org
Cc: Billy O'Neal (VC LIBS); Jonathan Wakely; Casey Carter
Subject: RE: [libcxx-dev] How to check for a feature-test macro

+libcxx-dev …

“Why do you not use `lit` to run the test suite?”

I don’t work at Microsoft, but I have tried to smash MSVC and libc++ tests together repeatedly.  Lit doesn’t make it easy.  I’ve never gotten something nice enough to upstream.  The libc++ lit infrastructure is heavily tuned towards gcc and clang, and doesn’t do a good job at isolating compiler differences.  Instead, the logic is spread all over the place.

Here is one such quick-and-dirty attempt from a year and a half ago… https://github.com/ben-craig/libcxx/commit/fea52ceee736c97e74194b3f38660a7b0f73db83<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fben-craig%2Flibcxx%2Fcommit%2Ffea52ceee736c97e74194b3f38660a7b0f73db83&data=02%7C01%7Cbion%40microsoft.com%7Ce737584415fe4dcaf2e008d65f847c12%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636801419961225892&sdata=9rCiOe1KO5O02tI2AAImbFQGSkOfsCxzf9hEpccxtes%3D&reserved=0> , note that this is after shuffling around a whole bunch of code to attempt to put compiler configuration behind an “abstract interface”.

I didn’t feel like subjecting myself to that again, so when I made my freestanding branch tests, I just rolled my own ninja generator written in Ruby.  Now I only rerun tests that weren’t successful, or have had a change.  https://github.com/ben-craig/kernel_test_harness/tree/msvc_stl<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fben-craig%2Fkernel_test_harness%2Ftree%2Fmsvc_stl&data=02%7C01%7Cbion%40microsoft.com%7Ce737584415fe4dcaf2e008d65f847c12%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636801419961235896&sdata=CdUielGGNrFD0KZCiCUrpmbjSImEdO1kijhDXpP1roA%3D&reserved=0> .

From: libcxx-dev <libcxx-dev-bounces at lists.llvm.org> On Behalf Of Louis Dionne via libcxx-dev
Sent: Tuesday, December 11, 2018 9:57 AM
To: Stephan T. Lavavej <stl at exchange.microsoft.com>
Cc: Billy O'Neal (VC LIBS) <bion at microsoft.com>; Jonathan Wakely <cxx at kayari.org>; Casey Carter <cacarter at microsoft.com>; libcxx-dev at lists.llvm.org
Subject: Re: [libcxx-dev] How to check for a feature-test macro

On Dec 10, 2018, at 20:14, Casey Carter <cacarter at microsoft.com<mailto:cacarter at microsoft.com>> wrote:

I don't have context for the original issue, but our test selector does inspect some of the LIT control comments. Specifically, it skips tests with any of the comments:

// REQUIRES: c++11

// REQUIRES: c++11 || c++14

// REQUIRES: c++98 || c++03

// REQUIRES: c++98 || c++03 || c++11 || c++14

// UNSUPPORTED: c++14, c++17, c++2a

So if were talking about something that doesn't touch those, it won't affect our typical usage.

________________________________

From: Stephan T. Lavavej
Sent: Monday, December 10, 2018 15:43
To: Jonathan Wakely; ldionne at apple.com<mailto:ldionne at apple.com>
Cc: Richard Smith; Marshall Clow; libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>; Billy O'Neal (VC LIBS); Casey Carter
Subject: RE: [libcxx-dev] How to check for a feature-test macro

(+Casey who also works on this)

When running libcxx's tests against MSVC's STL, we have an MSVC-internal skipped_tests.txt (full content available if requested) which says:

# *** MISSING STL FEATURES ***
# C++20 P0122R7 "<span>"
upstream\test\std\containers\views\types.pass.cpp
[...]

# C++20 P0355R7 "<chrono> Calendars And Time Zones"
upstream\test\std\utilities\time\days.pass.cpp
[...]

Updating this requires minimal effort. It's more problematic when existing tests are modified to test new features, since we have to skip the entire test even if only a small part is affected.

Guarding tests with feature-test macros in their source code (as opposed to "the LIT level", which I believe is machinery that we don't use) would be ideal, I think.

Why do you not use `lit` to run the test suite?

As I understand it, MSVC would prefer disabling tests that are not supported using `#if !defined(<feature>)`. This can be made to work, but what I don't like is that it removes visibility into what tests are passing and which ones are not supported. Right now, we can easily tell how many tests are unsupported by an implementation because those are marked by LIT as `UNSUPPORTED`. If we start using feature test macros to disable tests, we lose that.

Louis

The annoying thing is when features are added without corresponding feature-test macros. There's a bunch of those right now. We'll probably need the skip-tests approach for those.

STL

-----Original Message-----
From: Jonathan Wakely <cxx at kayari.org<mailto:cxx at kayari.org>>
Sent: Monday, December 10, 2018 12:10 PM
To: ldionne at apple.com<mailto:ldionne at apple.com>
Cc: Richard Smith <richard at metafoo.co.uk<mailto:richard at metafoo.co.uk>>; Marshall Clow <mclow.lists at gmail.com<mailto:mclow.lists at gmail.com>>; libcxx-dev at lists.llvm.org<mailto:libcxx-dev at lists.llvm.org>; Stephan T. Lavavej <stl at exchange.microsoft.com<mailto:stl at exchange.microsoft.com>>; Billy O'Neal (VC LIBS) <bion at microsoft.com<mailto:bion at microsoft.com>>
Subject: Re: [libcxx-dev] How to check for a feature-test macro

On Mon, 10 Dec 2018 at 19:47, Louis Dionne wrote:
> We should also see how other implementations that use our test suite feel about this (CCing them).

I only use your tests a few times a year, run by hand. If I get a load of FAILs because I've not implemented something yet and you have, that's not a problem. I don't expect to get 100% green when I run your tests.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/libcxx-dev/attachments/20181211/4c7c3165/attachment-0001.html>