[llvm-dev] [RFC] Compiled regression tests.

[David Greene via llvm-dev]

Michael Kruse via llvm-dev <llvm-dev at lists.llvm.org> writes:

> Unfortunately, most tests we have do not even use placeholders for
> metadata nodes, including those generated by update_test_checks.py. I
> looked into improving the script in this regard, but its
> implementation is function-centric, making it difficult to add
> module-level placeholders.

I actually have an implementation that does this for the update scripts.
It's on my queue to upstream.  In addition I have a number of
enhancements that allow the update scripts to generate more focused,
robust tests.  The first of these is D68230 which is currently stuck and
I'd appreciate some more eyes on it.

> As a result, I estimate that I had to invest about twice the time to
> update/fix tests than writing the code changes themselves. Due to my
> experience, I find updating FileCheck tests very frustrating.
> I'd prefer not to test whether specific metadata nodes are present,
> but whether the LLVM API to query them (such as `isAnnotatedParallel`)
> returns the expected result.

I think both could be useful.  With my update script enhancements I was
able to successfully match specific metadata nodes and check them
against metadata definitions at the Module level, though I'll admit my
use-case is probably simpler than yours.

                -David