[LLVMdev] LLVM unit and regression tests, enabled targets and conditions

[NAKAMURA Takumi]

Good afternoon, Galina.
This is cool suggestion, I feel. :)

2010/11/24 Galina Kistanova <gkistanova at gmail.com>:

> We have 3 types of unit and regression tests:
> 1.) Target-independent tests
> 2.) Target-dependent tests
> 3.) Conditional tests

Shall we think of potentially "host-dependent" tests?

With one of my tasks, we should know llc -march=x86-64 behaves host-dependent.
I took the way to specify explicit -mtriple=x86_64-{linux|win32} then.
(are they needed; darwin, cygwin, or mingw32? :p)
You know, Win32 x64(aka Win64) ABI is different from AMD64 ABI.

> To follow this pattern, I think, we need to extend the declarative
> part by adding an explicit dependency statement. Technically, we need
> to declare a dependency (or a run condition), and for what targets the
> test should be run.
>
> I propose
>
> A.) A new condition statement, something like
>
> ; DEPENDS: <condition>
> or
> ; REQUIRE: <condition>

Lit has "REQUIRES:" for features.ToT llvm/test does not use "features".
(clang/test uses it, and one of my patches is under review)

I am so poor for English, though, I would like to suggest "ASSUMES:".

> For example, the test/DebugInfo/2010-08-04-StackVariable.ll test could
> have the declarative part as
>
> ; REQUIRE: llc -mtriple=arm-apple-darwin    < /dev/null
> ; REQUIRE: llc -mtriple=x86_64-apple-darwin < /dev/null
> ; RUN: llc -O0 -mtriple=arm-apple-darwin    < %s | grep DW_OP_fbreg
> ; RUN: llc -O0 -mtriple=x86_64-apple-darwin < %s | grep DW_OP_fbreg
> ; Use DW_OP_fbreg in variable's location expression if the variable is
> in a stack slot.

I guess something label would be needed to extend to selective test. eg.

; ASSUMES-ARM: llc -mtriple=arm-apple-darwin    < /dev/null
; ASSUMES-X64: llc -mtriple=x86_64-apple-darwin < /dev/null
; RUN-ARM: llc -O0 -mtriple=arm-apple-darwin    < %s | grep DW_OP_fbreg
; RUN-X64: llc -O0 -mtriple=x86_64-apple-darwin < %s | grep DW_OP_fbreg

Anyway, I think, lit can know host and targets_to_build (and
subtargets) as literals.
With the case above, running llc might be overkill.

> B.) $each_target placeholder, which could be used in the test run
> command to indicate that this is the type 2 test. This placeholder
> will be replaced with the appropriate value at the test run time.
>
> For example, if the test/CodeGen/Generic/2003-07-07-BadLongConst.ll
> will be defined as
>
> ; RUN: llc -march=$each_target < %s

It's cool. Also I suggest matrix stuff, eg.

; $arch = i686,x86_64
; $os = darwin,linux,mingw32,win32
; RUN: llc -mtriple=$arch-$os < %s

To expect running 8 tests.

I wonder it would be too heavyweight. To attempt matrices, we might
need matrix-test mode for llc, or building dedicated-codegen-tester.


Thank you, ...Takumi