[llvm-dev] RFC: FileCheck Enhancements

[Vedant Kumar via llvm-dev]

> On Sep 13, 2016, at 4:50 PM, Chris Lattner <clattner at apple.com> wrote:
> 
> On Sep 13, 2016, at 11:16 AM, Vedant Kumar <vsk at apple.com> wrote:
>>> Instead of doing this, has anyone considered baking “modes” into Filecheck to support the important clients (e.g. LLVM IR, MC, clang, etc)?  This would mean that a test could just add a “--mode=llvmir” flag to filecheck and get a bunch of baked in patterns, potentially with magic syntax.  Something like this would avoid having to redundantly enter "%[0-9]+” a kajillion times all over the place.
>> 
>> I hadn't considered adding modes to FileCheck. That seems like it could work
>> pretty well.
>> 
>> Even if that's the route we're going to take, I'd prefer that the FileCheck
>> modes simply expose pre-defined patterns instead of introducing magic syntax.
>> It would be a shame to have to hack FileCheck to use check patterns for clients
>> that aren't LLVM IR, clang, etc.
> 
> Why?
> 
> This is a tradeoff: the more extensible you make filecheck (by introducing ever more complicated DSLs into it) the more esoteric it gets, and the less likely that someone will be able to actually UNDERSTAND a random filecheck invocation that they come across.
> 
> The advantage of adding a few hard coded modes to filecheck is that we can have specific standardized solutions to important use cases that the LLVM project has.  FileCheck lives in service of LLVM and its downstream projects, it shouldn’t aspire to be some general tool that needs a DSL.  If a downstream project outside of the LLVM project (e.g. the Rust compiler) wanted its own mode, I think it would be fine to add it, and having a gaggle of modes seems preferable to me than building in a complex DSL for describing named patterns.
> 
> -Chris

Let me start by saying that adding a few hard-coded patterns to FileCheck could
work well for many of llvm's tests. I'd be perfectly fine with that approach.

My thinking was that if it isn't easy to teach FileCheck about new patterns,
it's less likely that people will do it. I'm not interested in making FileCheck
a general-purpose testing tool, but I would like to use custom patterns to
simplify in-tree tests that aren't in *.ll or *.s files.

I don't think that all extensions to the FileCheck syntax would cause serious
maintainability issues. Being able to define patterns in some tests could
actually make them more readable. However, if there isn't a way to extend
FileCheck without creating serious readability issues, I agree that we
shouldn't do it.

vedant