[PATCH] D37421: [XRay] [test-suite] Add LNT support to retref-bench benchmarks.

[Matthias Braun via Phabricator via llvm-commits]

MatzeB added a comment.

In https://reviews.llvm.org/D37421#876105, @hfinkel wrote:

> In https://reviews.llvm.org/D37421#875900, @MatzeB wrote:
>
> > I found two problems with this at the moment:
> >
> > - The list of litsupport plugins is currently global. Thus enabling this microbenchmarking plugin will make the non-microbenchmarking tests fail. It should be easy enough to change this to a local setting so that `lit.local.cfg` files can override the plugin list (just have to convince myself that this is a good feature to have :).
>
>
> What's the design space here? Would it be better to be able to name plugins in the .test file?




- The majority of the plugins depend on the users needs and should be chosen by cmake/cmake flags. This is what happens today.
- We could modify the plugin list further depending on the test. `lit.local.cfg` would apply new settings for a directory and all subdirectories. This would be in line with what we already do to influence behavior between SingleSource/MultiSource/External (the compiletime files end up with different names and the reference files must/mustn't contain the `exit XX` line at the end). Enhancing this with a way to request additional plugins for a subdirectory would feel most natural to me.
- It wouldn't be harder to do it per .test file but I'm not convinced we need it at that granularity and it seems less discoverable to me than a setting in `lit.local.cfg`.


https://reviews.llvm.org/D37421