[libcxx-commits] [PATCH] D93971: Add freestanding parameter to libcxx lit

[Ben Craig via Phabricator via libcxx-commits]

bcraig added a comment.

Messed with this some over the weekend, and realized there's ambiguity in the request.  I also found that there are some challenges with each of the reasonable things that I think you could be requesting.  Note that all 3 approaches result in failing tests, and fixing those tests will take time.

Potential path 1:
Test an (as much as possible) conforming freestanding libcxx configuration with the new -freestanding parameter.  This seems mostly useful, but it doesn't do much to test the above code, as there is already code that will take libc++ configuration macros and turn them into features.  This approach also has a ton of failures, from things like printf usage in helper functions.  I got these results...

  Unsupported        : 1664
  Passed             : 4726
  Expectedly Failed  :   46
  Failed             :   71
  Unexpectedly Passed:    3

Potential path 2:
Test a "useful" conforming freestanding libcxx configuration (i.e. turn off exceptions and RTTI).  This has most of the same problems as above, but changes which tests need to be adjusted.

Potential path 3:
Narrowly test the specific changes in this request.  The easiest way to do that is to build a hosted libc++, and run the freestanding tests against it.  This results in a few test failures.  Some of them are of dubious value.  For example, there's a test that makes sure the lit feature and the `_LIBCPP_HAS_NO_THREADS` preprocessor value match.  The advantage of this approach is that it tests the above changes, as old code paths would take configuration values and automatically add the associated lit features.  There were something like 4 FAILS and 4 XPASSes with this approach.

I think both paths 1 and 2 make sense long term.  They start "actually" testing the new freestanding parameter as soon as we start sprinkling `UNSUPPORTED:libcpp-any-freestanding` all over the place, but it will likely be months before we get a passing CI run.

Any thoughts on what you would like to see?  Is it ok to have a broken CI bot for a while?  Or do I need to fix the tests as part of this change as well?


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D93971/new/

https://reviews.llvm.org/D93971