[PATCH] D150856: [lit] Add %{for-each-file} substitution

[Vlad Serebrennikov via Phabricator via llvm-commits]

Endill added a comment.

In D150856#4423840 <https://reviews.llvm.org/D150856#4423840>, @jhenderson wrote:

> Your point about the potential for cartesian products is fair, and mentioning things like DEFINE and REDEFINE made me wonder whether we could simply have a special lit suffix (or prefix), a bit like FileCheck suffixes, that simply tells lit to expand foreach directives. In that way, a user can see that the line will be potentially invoked multiple times without having to spot the substitution in the line Something like `RUN-FOREACH` (which could later be extrapolated as `DEFINE-FOREACH` etc).
>
> FWIW, I can see a lot of value for "for each" style syntax in various tests I've written, although it's not always about the input file, and may even be about a sequence of invocations. However, if we start following that chain of thought, we end up making lit directives a Turing-complete language (after all, if you want a generic "FOR", you'll probably quickly want "IF" etc to handle special cases), and I'm not convinced we want to fall down that hole.

We can do it the following way:

  // RUN: rm -rf %t && mkdir %t
  // RUN: split-file --allow-comments --add-file-extension=cpp %s %t
  // DEFINE-FOR-EACH-FILE: %{file} = %t
  // RUN: %clang_cc1 -std=c++11 %{file} -verify -fexceptions -fcxx-exceptions -pedantic-errors
  // RUN: %clang_cc1 -std=c++11 %{file} -verify -fexceptions -fcxx-exceptions -pedantic-errors
  // RUN: %clang_cc1 -std=c++14 %{file} -verify -fexceptions -fcxx-exceptions -pedantic-errors
  // RUN: %clang_cc1 -std=c++17 %{file} -verify -fexceptions -fcxx-exceptions -pedantic-errors
  // RUN: %clang_cc1 -std=c++20 %{file} -verify -fexceptions -fcxx-exceptions -pedantic-errors
  // RUN: %clang_cc1 -std=c++23 %{file} -verify -fexceptions -fcxx-exceptions -pedantic-errors

>From there, we can disallow RUN lines with FOR-EACH defines on syntax level, requiring something along the lines of RUN-FOR-EACH instead. Though I don't have a strong opinion whether we should.
It also opens a way to `DEFINE-FOR-EACH` with list of arbitrary strings, which Aaron mentioned. Seems useful, but I'm not ready yet to advocate that we need this for DR testing, even for language modes.

I understand your concern about moving towards Turing-completeness, but we're not changing anything for majority of tests that are fine with handful of RUN directives. Those Turing-complete features are going to replace dozens of RUN lines (I can image us running codegen DR tests for many triples in each language mode), and I don't see simplicity of lit serving us well in those important use cases.



================
Comment at: llvm/docs/CommandGuide/lit.rst:559
                          Otherwise, %T but with a single leading ``/`` removed.
+ %{for-each-file <path>} Invokes RUN command for each file in ``<path>``. Accepts wildcards.
  ======================= ==============
----------------
jhenderson wrote:
> It's completely unclear what "Accepts wildcards" means in this context. I wonder if an example might be appropriate.
Would it be more clear if I explicitly mention asterisk and double asterisk as allowed in path?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D150856/new/

https://reviews.llvm.org/D150856