<html><head><meta http-equiv="Content-Type" content="text/html; charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi,<div class=""><br class=""></div><div class="">That's only true with the old format -- the new format handles .fail.cpp tests properly (i.e. clang-verify when supported, otherwise not). Support for the old format should be removed and folks should move to the new format, which handles custom executors much better actually. For example, the new format will honour custom executors when performing feature tests to see whether e.g. locales or non-lockfree atomics are supported on the target.</div><div class=""><br class=""></div><div class="">The issue up to now was how to specify a custom executor from the CMake build, but that should be solved since this commit:</div><div class=""><br class=""></div><div class=""><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">commit 96e6cbbf941d0f937b7e823433d4c222967a1817</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">Author: Louis Dionne <<a href="mailto:ldionne@apple.com" class="">ldionne@apple.com</a>></span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">Date: Thu Jun 11 15:41:38 2020 -0400</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""><br class=""></span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> [libc++] Allow specifying arbitrary custom executors with the new format</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""><br class=""></span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> The integration between CMake and executor selection in the new format</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> wasn't very flexible -- only the default executor and SSH executors were</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> supported.</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""><br class=""></span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> This patch makes it possible to specify arbitrary executors with the new</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> format. With the new testing format, a custom executor is just a script</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> that gets called with a command-line to execute, and some arguments like</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> --env, --codesign_identity and --execdir. As such, the default executor</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> is just run.py.</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""><br class=""></span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> Remote execution with the SSH executor can be achived by specifying</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> LIBCXX_EXECUTOR="<path-to-ssh.py> --host <host>". Similarly, arbitrary</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> scripts can be provided.</span></font></div></div><div class=""><br class=""></div><div class="">It should now be more flexible and easier to specify arbitrary executors. You will need to change a few things since executors are now arbitrary scripts instead of subclasses of the Python executor, but that should be fairly straightforward. Can we work together to make sure you're able to migrate to the new format?</div><div class=""><br class=""></div><div class="">Cheers,</div><div class="">Louis<br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jun 11, 2020, at 08:06, Mikhail Maltsev <<a href="mailto:Mikhail.Maltsev@arm.com" class="">Mikhail.Maltsev@arm.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi all,<br class=""><br class="">After the following commit<br class=""><br class="">"commit 2fd7d364cd999c743b5bdefa7ee9e5630f3564ad<br class="">Author: Louis Dionne <<a href="mailto:ldionne@apple.com" class="">ldionne@apple.com</a>><br class="">Date: Mon Apr 27 12:49:01 2020 -0400<br class=""><br class="">[libc++] Make the verify-support feature implicit<br class=""><br class="">Tests that require support for Clang-verify are already marked as such<br class="">explicitly by their extension, which is .verify.cpp. Requiring the use<br class="">of an explicit Lit feature is, after thought, not really helpful.<br class=""><br class="">This is a change in design: we have been bitten in the past by tests not<br class="">being enabled when we thought they were. However, the issue was mostly<br class="">with file extensions being ignored. The fix for that is not to blindly<br class="">require explicit features all the time, but instead to report all files<br class="">that are in the suite but that don't match any known test format. This<br class="">can be implemented in a follow-up patch."<br class=""><br class="">All '.fail.cpp' started being run without the -verify command line <br class="">option. This reduced test coverage: the test suite is now only checking <br class="">that compilation of the .fail.cpp tests fails, but does not check the <br class="">locations of the actual errors and warnings.<br class=""><br class="">It this intentional? Just in case: we are using the 'use_old_format' lit <br class="">feature because the new format does not support custom executors (e.g. <br class="">simulators) yet.<br class=""><br class="">-- <br class="">Regards,<br class=""> Mikhail Maltsev<br class=""></div></div></blockquote></div><br class=""></div></body></html>