[llvm-dev] Testing "normal" cross-compilers versus GPU backends

[Mehdi Amini via llvm-dev]

> On Sep 8, 2015, at 4:56 PM, Robinson, Paul <Paul_Robinson at playstation.sony.com> wrote:
> 
>>> Mehdi - Could you list a couple of the features that show up in generic
>> tests which your target doesn't support by design?  After reading through
>> the discussion to this point, I don't really have a feel for that.
>> Additionally, do you have any sense for how common that set is across GPU
>> targets?
>> 
>> This is what I was referring to in my last email:
>> "To be “correct”, tests should “require” (in the lit sense) features like
>> “function call”, etc. However this is a significant amount of work with
>> not that much added value.
>> This is what Takumi did to disable one test that check for “debug_frame”
>> which are not emitted on Windows.”
>> 
>> I could try to survey all the failing tests and abstract the “features”
>> they are using, but this does not seem like a small enough task that I can
>> commit to an ETA now or have an estimate about when it’ll come on top of
>> my stack.
>> 
>> This is why I tried to offer an alternative with
>> http://reviews.llvm.org/D12660 ; to identify the tests that rely on the
>> default triple using a generic default_triple feature.
>> 
>> 
>>> Paul - If we introduce the type of flag your talking about, we should
>> probably have a discussion about which classes of backends LLVM supports
>> and try to create a *documented* list of feature expectations somewhere.
>>> 
>>> p.s. Maybe I missed something, but why did simply using "XFAIL: AMDGPU"
>> get rejected?  That seems like the easiest fix.
>> 
>> You haven't missed anything IIUC.
>> If we were to document the list of feature expectations from the backends,
>> the “default_triple” would be applied to the tests that makes use of these
>> features.
>> 
>> —
>> Mehdi
> 
> Hmmm "default_triple" is certainly not an expectation or feature of any
> target; rather, it is an expectation of certain tools when used in certain
> ways (mainly: llc or llvm-mc without an explicit triple) and therefore it
> is an expectation of how certain tests are _written_.  This has little to
> do with the _features_ verified by those tests.
> 
> In particular, configuring a GPU-only LLVM with a default triple for that 
> GPU will still fail all the same tests that Mehdi tripped over.  Inventing
> "default_triple" lets us trick the test suite into skipping the tests that
> the GPU target would fail anyway.

I agree with your description as of now.

“default_triple” does not make sense as a feature now, but the way I see it is that it goes along with the definition of what is the minimum set of IR-level “features" a backend must support.

Tests that are relying on the default_triple (and marked with the appropriate require) can use any “feature” in this set and expects to be portable across backends.
If a class of backends is more restrictive, then on a case by case it could be up to this particular backend to express more fine-grain feature restriction, the same way as the debug_frame feature was added.

For instance if “target-independent” tests that need a backend are allowed to perform arbitrary function calls, then it might be a possibility to add a (enabled by default) “function_calls” feature and it would be up to the maintainers of the backends that makes use of this feature to add the “REQUIRE” line to the relevant tests and disable it in lit.local.cfg for their arch.

Hope it makes sense.

— 
Mehdi


> 
> This is not to say "default_triple" is a bad thing; it lets us support a
> kind of configuration that Mehdi has persuaded me is valid, and I'm happy
> to have it.  But it is not a target-feature-based solution.
> 
> It is not impossible that if we defined a "GPU" feature set, we might find
> that it would exclude many of the same tests that "default_triple" excludes.
> But that would be essentially by chance, not because "default_triple" has
> any inherent meaning for targets.
> --paulr