[llvm-dev] [RFC] Compiled regression tests.

[Michael Kruse via llvm-dev]

Am Mi., 24. Juni 2020 um 11:19 Uhr schrieb Mehdi AMINI <joker.eph at gmail.com>:
>
> Hi,
>
> On Tue, Jun 23, 2020 at 6:34 PM Michael Kruse via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>
>> Hello LLVM community,
>>
>> For testing IR passes, LLVM currently has two kinds of tests:
>>  1. regression tests (in llvm/test); .ll files invoking opt, and
>> matching its text output using FileCheck.
>>  2. unittests (in llvm/unittests); Google tests containing the IR as a
>> string, constructing a pass pipeline, and inspecting the output using
>> code.
>>
>> I propose to add an additional kind of test, which I call "compiled
>> regression test", combining the advantages of the two.
>
>
> You expand below on the mechanism you'd like to implement, but I am a bit puzzled about the motivation right now?

See https://reviews.llvm.org/D82426 and
http://lists.llvm.org/pipermail/llvm-dev/2020-June/142706.html for
more motivation.


> I'm failing to see what kind of IR-level test (unittests are relevant for data-structures and non-IR internals IMO) we would implement this way that we can't just implement with lit/FileCheck?

My argument is not that those tests cannot be written using FileCheck,
but tend (not all of them) to check more than relevant, may be truisms
and are difficult to reverse-engineer and to update when something
changes.

Michael