[libcxx-commits] [PATCH] D154987: [lit] Implement PYTHON directive and config.prologue

[Aaron Ballman via Phabricator via libcxx-commits]

aaron.ballman added a comment.

I definitely understand the desire to keep lit as simple and focused as we can, and from that perspective, adding support for a full scripting language seems like a bad idea. However, lit also has to meet our testing needs and in its current form, we're missing a direct way to support our uses for defect report testing in C and C++. Because there are thousands of defect reports in C++ alone, we group the tests together in blocks of roughly 100 tests within a single file. However, we need to run those tests semi-hermetically because we do not want an earlier test failure to impact the diagnostics for later tests (this causes us to miss diagnostics that would otherwise be issued and can hide compiler or testing bugs). Additionally, each of these tests needs to be run in all of the various language modes we support and we'd like it to be easy to add new language standard modes to the set without having to update each test file directly. The proposed support in this patch will help solve those problems in a way that I think is pretty straightforward yet powerful enough to be used to solve testing problems for other projects as well (such as the issue flang is having that @MaskRay pointed out).


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D154987/new/

https://reviews.llvm.org/D154987