[llvm-dev] Testing "normal" cross-compilers versus GPU backends

[Robinson, Paul via llvm-dev]

> > Mehdi - Could you list a couple of the features that show up in generic
> tests which your target doesn't support by design?  After reading through
> the discussion to this point, I don't really have a feel for that.
> Additionally, do you have any sense for how common that set is across GPU
> targets?
> 
> This is what I was referring to in my last email:
> "To be “correct”, tests should “require” (in the lit sense) features like
> “function call”, etc. However this is a significant amount of work with
> not that much added value.
> This is what Takumi did to disable one test that check for “debug_frame”
> which are not emitted on Windows.”
> 
> I could try to survey all the failing tests and abstract the “features”
> they are using, but this does not seem like a small enough task that I can
> commit to an ETA now or have an estimate about when it’ll come on top of
> my stack.
> 
> This is why I tried to offer an alternative with
> http://reviews.llvm.org/D12660 ; to identify the tests that rely on the
> default triple using a generic default_triple feature.
> 
> 
> > Paul - If we introduce the type of flag your talking about, we should
> probably have a discussion about which classes of backends LLVM supports
> and try to create a *documented* list of feature expectations somewhere.
> >
> > p.s. Maybe I missed something, but why did simply using "XFAIL: AMDGPU"
> get rejected?  That seems like the easiest fix.
> 
> You haven't missed anything IIUC.
> If we were to document the list of feature expectations from the backends,
> the “default_triple” would be applied to the tests that makes use of these
> features.
> 
> —
> Mehdi

Hmmm "default_triple" is certainly not an expectation or feature of any
target; rather, it is an expectation of certain tools when used in certain
ways (mainly: llc or llvm-mc without an explicit triple) and therefore it
is an expectation of how certain tests are _written_.  This has little to
do with the _features_ verified by those tests.

In particular, configuring a GPU-only LLVM with a default triple for that 
GPU will still fail all the same tests that Mehdi tripped over.  Inventing
"default_triple" lets us trick the test suite into skipping the tests that
the GPU target would fail anyway.

This is not to say "default_triple" is a bad thing; it lets us support a
kind of configuration that Mehdi has persuaded me is valid, and I'm happy
to have it.  But it is not a target-feature-based solution.

It is not impossible that if we defined a "GPU" feature set, we might find
that it would exclude many of the same tests that "default_triple" excludes.
But that would be essentially by chance, not because "default_triple" has
any inherent meaning for targets.
--paulr