[llvm-dev] Testing "normal" cross-compilers versus GPU backends

[Robinson, Paul via llvm-dev]

> > On Sep 8, 2015, at 4:56 PM, Robinson, Paul
> <Paul_Robinson at playstation.sony.com> wrote:
> >
> >>> Mehdi - Could you list a couple of the features that show up in
> generic
> >> tests which your target doesn't support by design?  After reading
> through
> >> the discussion to this point, I don't really have a feel for that.
> >> Additionally, do you have any sense for how common that set is across
> GPU
> >> targets?
> >>
> >> This is what I was referring to in my last email:
> >> "To be “correct”, tests should “require” (in the lit sense) features
> like
> >> “function call”, etc. However this is a significant amount of work with
> >> not that much added value.
> >> This is what Takumi did to disable one test that check for
> “debug_frame”
> >> which are not emitted on Windows.”
> >>
> >> I could try to survey all the failing tests and abstract the “features”
> >> they are using, but this does not seem like a small enough task that I
> can
> >> commit to an ETA now or have an estimate about when it’ll come on top
> of
> >> my stack.
> >>
> >> This is why I tried to offer an alternative with
> >> http://reviews.llvm.org/D12660 ; to identify the tests that rely on the
> >> default triple using a generic default_triple feature.
> >>
> >>
> >>> Paul - If we introduce the type of flag your talking about, we should
> >> probably have a discussion about which classes of backends LLVM
> supports
> >> and try to create a *documented* list of feature expectations
> somewhere.
> >>>
> >>> p.s. Maybe I missed something, but why did simply using "XFAIL:
> AMDGPU"
> >> get rejected?  That seems like the easiest fix.
> >>
> >> You haven't missed anything IIUC.
> >> If we were to document the list of feature expectations from the
> backends,
> >> the “default_triple” would be applied to the tests that makes use of
> these
> >> features.
> >>
> >> —
> >> Mehdi
> >
> > Hmmm "default_triple" is certainly not an expectation or feature of any
> > target; rather, it is an expectation of certain tools when used in
> certain
> > ways (mainly: llc or llvm-mc without an explicit triple) and therefore
> it
> > is an expectation of how certain tests are _written_.  This has little
> to
> > do with the _features_ verified by those tests.
> >
> > In particular, configuring a GPU-only LLVM with a default triple for
> that
> > GPU will still fail all the same tests that Mehdi tripped over.
> Inventing
> > "default_triple" lets us trick the test suite into skipping the tests
> that
> > the GPU target would fail anyway.
> 
> I agree with your description as of now.
> 
> “default_triple” does not make sense as a feature now, but the way I see
> it is that it goes along with the definition of what is the minimum set of
> IR-level “features" a backend must support.

We differ there.  The REQUIRES feature-string set is not about what the 
target supports; it's about what the specific test needs, which might 
include some target feature(s) and might include some tool feature(s) and 
might include some other thing entirely.

Consider the "native" feature-string. It says nothing about the target,
or any tool; it merely tells the test about the relationship between the 
default target and the host.  There is no piece of code in the target that 
defines this relationship; it's entirely about the environment.  The 
"default_triple" feature-string is similar: it tells the test that there 
is a default triple, or not.  Nothing in your target itself can influence 
whether there is a default triple, and I see no code in any target that
even looks for a default triple.

Something like "object-emission" or "debug_frame" or "function_calls" on
the other hand, would very much be a target-related feature. You could
readily identify target code that does or does not support each of these.

It can be confusing because we mash all these different kinds of feature
strings into one big set and use the same REQUIRES syntax to let each test
specify what it needs; but we will be much better off remembering that we
have multiple kinds of feature-strings, and not think that "default_triple"
has anything to do with what the target actually supports.
--paulr

> 
> Tests that are relying on the default_triple (and marked with the
> appropriate require) can use any “feature” in this set and expects to be
> portable across backends.
> If a class of backends is more restrictive, then on a case by case it
> could be up to this particular backend to express more fine-grain feature
> restriction, the same way as the debug_frame feature was added.
> 
> For instance if “target-independent” tests that need a backend are allowed
> to perform arbitrary function calls, then it might be a possibility to add
> a (enabled by default) “function_calls” feature and it would be up to the
> maintainers of the backends that makes use of this feature to add the
> “REQUIRE” line to the relevant tests and disable it in lit.local.cfg for
> their arch.
> 
> Hope it makes sense.
> 
> —
> Mehdi
> 
> 
> >
> > This is not to say "default_triple" is a bad thing; it lets us support a
> > kind of configuration that Mehdi has persuaded me is valid, and I'm
> happy
> > to have it.  But it is not a target-feature-based solution.
> >
> > It is not impossible that if we defined a "GPU" feature set, we might
> find
> > that it would exclude many of the same tests that "default_triple"
> excludes.
> > But that would be essentially by chance, not because "default_triple"
> has
> > any inherent meaning for targets.
> > --paulr