[llvm-dev] [RFC] Compiled regression tests.

[Michael Kruse via llvm-dev]

Am Do., 2. Juli 2020 um 12:41 Uhr schrieb Hal Finkel <hfinkel at anl.gov>:
> I think that, to a large extent, we're on the same page on this aspect.
> It's a question of reuse and stability. If there's a principled way to
> design an output that will be reused across many tests and can
> reasonably be believed will remain relatively stable, then we should do
> that. If not, then unit tests are better. The question is: do we have so
> many such unit tests that we want a special way to construct them from
> IR files (instead of, I suppose, just having the IR in a string in the
> code)? I don't know.

Implementing another print mechanism and justifying its presence
significantly adds to the burden of writing robust and understandable
tests that  often could be simple call of an already existing API
function in a unittest.
Inserting a layer of text representations and matching it using
FileCheck feels like "unittesting with extra steps", adding the
previously mentioned shortcoming of FileCheck-based testing to the
system.

Another example from Polly:

    Arrays {
        i32 MemRef0[*]; // Element size 4
    }
    Arrays (Bounds as pw_affs) {
        i32 MemRef0[*]; // Element size 4
    }

This emits the same information twice. The latter is the more
high-level representation that should be more useful to users, but the
former is still printed as well because there are existing regression
tests that expect the former format. For humans trying to understand
this it makes the output more verbose.

Michael