[llvm-dev] [RFC] Compiled regression tests.

[Hal Finkel via llvm-dev]

On 7/1/20 1:43 PM, David Greene wrote:
> Hal Finkel via llvm-dev <llvm-dev at lists.llvm.org> writes:
>
>> I definitely agree that we should not be trying to do this kind of
>> checking using textual metadata-node matching in FileCheck. The
>> alternative already available is to add an analysis pass with some kind
>> of verifier output. This output, not the raw metadata itself, can be
>> checked by FileCheck. We also need to check the verification code, but
>> at least that's something we can keep just in one place. For parallel
>> annotations, we already have such a thing (we can run opt -loops
>> -analyze; e.g., in
>> test/Analysis/LoopInfo/annotated-parallel-complex.ll). We also do this
>> kind of thing for the cost model (by running with -cost-model -analyze).
>> To what extent would making more-extensive use of this technique address
>> the use cases you're trying to address?
> Analysis tests are definitely useful but I don't think they're
> sufficient.  By their nature, analysis tests run in a "sanitized"
> environment.  The test specifies exactly what is analyzed and no other
> passes are run.


Yes, but we can also combine them with a transformation pipeline.


>
>  From my quick glance at Michael's example test, it's checking that the
> inlining pass does something specific with metadata:
>
> ; Check that the access groups (llvm.access.group) are correctly merged.
>
> I suppose one could create an "analysis" that just dumps out metadata
> information.  Is that what you have in mind?  That seems like a lot of
> overhead just to write a test to check metadata.


Yes. From my perspective, it depends on how much resuse you get out of 
that work. Can the "dump out metadata information" part be reused for 
many tests? And is there a reason not to use a traditional unit test?

  -Hal


>
>                        -David

-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory