[libcxx] [llvm] [libc++] Optionally support filecheck-based tests (PR #165769)

[Louis Dionne via llvm-commits]

ldionne wrote:

> > If a user wants to set up FileCheck locally to work on libc++, they shouldn't have to configure LLVM and build it. It's a fragile and complicated process.
> 
> Can you elaborate on why this process is fragile? I don't think configuring LLVM/building FileCheck should be more fragile than building libc++.

Fragile in the sense that configuring and building LLVM (or parts of it) is not a simple process. It may not be super difficult to do from the command-line (just use the right command-line invocation), however what's involved under the hood is not simple. For that reason, I envision people potentially running into issues trying to do that and the ergonomics wouldn't be great.


> > If we productized FileCheck into something standalone that can be installed with e.g. Homebrew, I'd much prefer use the LLVM version. But since I don't see that happening, I think it makes more sense to use an easily installable equivalent of it.
> 
> I don't think this would be too difficult to do. We could even make it installable via pip.

I looked into it in the past and couldn't figure out how to do it because `FileCheck` was pulling in stuff from LLVM support libraries. If we're able to extract it into its own utility that can be installed separately, that would be amazing. However, I would like not to block this improvement to our test suite on such a refactoring unless someone signs up to do it in the short term. The longer we wait until we can use FileCheck in libc++, the more test coverage we're just not going to have because it would require FileCheck.

https://github.com/llvm/llvm-project/pull/165769