[llvm-dev] Testing "normal" cross-compilers versus GPU backends

[Tom Stellard via llvm-dev]

On Thu, Sep 03, 2015 at 02:07:54AM -0700, Mehdi Amini wrote:
> 
> > On Sep 3, 2015, at 12:18 AM, Robinson, Paul <Paul_Robinson at playstation.sony.com> wrote:
> > 
> > 
> > 
> >> -----Original Message-----
> >> From: Mehdi Amini [mailto:mehdi.amini at apple.com]
> >> Sent: Wednesday, September 02, 2015 7:10 PM
> >> To: Robinson, Paul
> >> Cc: llvm-dev at lists.llvm.org; tom at stellard.net; NAKAMURA Takumi
> >> Subject: Re: Testing "normal" cross-compilers versus GPU backends
> >> 
> >> Hi Paul,
> >> 
> >> Thanks for the summary!
> >> 
> >>> On Sep 2, 2015, at 5:44 PM, Robinson, Paul
> >> <Paul_Robinson at playstation.sony.com> wrote:
> >>> 
> >>> This note arose from https://urldefense.proofpoint.com/v2/url?u=http-
> >> 3A__reviews.llvm.org_D12506&d=BQIFAg&c=eEvniauFctOgLOKGJOplqw&r=v-
> >> ruWq0KCv2O3thJZiK6naxuXK8mQHZUmGq5FBtAmZ4&m=Wr0uOhkAp_10X4edWwxZQ9V8L97j8e
> >> o6cR_1Ia-gMOw&s=OOTP9DnL-TWV1zvy9EcU0Z6yfTq5lBjhE-LvYlWMJ3Y&e=  but the
> >> reviewers
> >>> felt that we needed a broader audience, because the proposed patch
> >>> really didn't solve the entire problem and we had no better ideas.
> >>> 
> >>> Mehdi Amini needs to build LLVM with just a GPU backend, and still have
> >>> "ninja check" Just Work.  Commits r243958-243960 tried to accomplish
> >>> that; however they are too big a hammer, and cause much simpler cross
> >>> environments (like mine) to exclude big chunks of very useful tests
> >>> (including my favorite, DebugInfo).
> >>> 
> >>> FYI, my main cross environment is building on X86 Windows but using
> >>> a default target triple for PS4 (which is also X86).
> >>> 
> >>> I experimented with building LLVM with just the ARM backend (running on
> >>> an X86 workstation) and setting the default triple to some ARM value.
> >>> "ninja check" worked fine (without Mehdi's series of commits), so the
> >>> normal kind of cross-compiler environment seems to be happy with how
> >>> things were set up originally.
> >>> 
> >>> Mehdi reports building LLVM with the X86 and AMDGPU backends, setting
> >>> the default triple to "amdgcn--amdhsa", and getting 200-some failures.
> >>> 
> >>> (This does make me wonder about AMDGPU testing in general; how does that
> >>> work?  The only places I see lit checks for AMDGPU are in the usual
> >>> target-dependent places.)
> >> 
> >> I don’t understand this interrogation about how do you do testing in
> >> general. The same way you don’t process tests/CodeGen/X86/* with the ARM
> >> backend, you can’t process any random IR through these backends.
> > 
> > You said you had 200+ failures with AMDGPU.  Are the AMD folks simply
> > tolerating the 200 failures, and you don't want to?  I should hope there
> > is more to it than that.
> 
> Well, I don’t know, they might just run `ninja check` with the default triple set to X86?
> (which I would consider being working around a buggy test suite)
> 

I always enable AMDGPU and X86 when I build, so I've never run into
this problem.

-Tom


> > 
> >> 
> >> IMO, the problem is in general about tests that are written without
> >> specifying a triple, that will be executed with the default triple.
> >> 
> >> Most of these tests were written with X86 (or ARM) in mind, and there is
> >> no guarantee that they will behave as intended with every possible triple.
> >> The DataLayout for instance has to be the one from the target, and is not
> >> portable.
> >> I think a "portable backend test” is pretty rare in general.
> > 
> > It depends on what the test is trying to do.  I'm sure it is quite common
> > for IR tests to behave essentially the same way regardless of target.
> 
> IR tests != backend test (I may miss your point here, it’s late…).
> 
> 
> > We have lots of tests (the ones you chose to mark "native") that had been
> > working fine with ARM, X86, PPC, SPARC, and whatever.
> > 
> > The fact that they don't work with your backend is different from saying
> > those tests can't possibly work for any cross-compiler.
> 
> I believe this is far from what I said, or at least from what I had in mind.
> 
> >  But the latter is what your patch implemented, and it is preventing useful testing.
> 
> I don’t disagree with that statement, and I agree that it should be fixed.
> It doesn’t mean that I think the previous situation was better though. 
> The current over-conservative state seems at least more correct than having only a subset of the targets that can pass `ninja check`.
> 
> Are you attending the LLVM bay area social tonight by any chance?
> 
> Best,
> 
> — 
> Mehdi
> 
> 
> 
> > --paulr
> > 
> >> 
> >> You can run all the tests by setting the default triple to X86 and
> >> compiling in the X86 backend, but that’s just a trick to make the tests
> >> happy.
> >> Alternatively, and this is what I tried to do, blacklisting these tests
> >> that “pretends” to be “portable” but are not.
> >> 
> >> 
> >>> Mehdi's solution was:
> >>> - In lit.cfg, change the existing "native" feature definition from
> >>> "host-triple == target-triple" to also check "and the corresponding
> >>> backend is included."(**)
> >> 
> >> I agree that we could remove the condition on the host backend included
> >> and deemed it a unsupported, but the build system needs to reject this
> >> configuration.
> >> 
> >> 
> >>> - Make piles of tests that seemed inapplicable to GPUs depend on the
> >>> "native" feature (through REQUIRES: or in the lit.local.cfg).
> >> 
> >> Nitpick: the GPU is just an example, any other backends can be affected.
> >> It seems that these test are “lying” about the target they will be able to
> >> run on (like if they would run on “anything”).
> >> 
> >> 
> >>> - Build LLVM with just the GPU backend, and not set a target triple
> >>> (i.e., it's set to the host triple, typically an X86-something).(*)
> >>> Thus lit.cfg sees matching host and target triples, but the X86
> >>> backend is missing, and so the "native" feature is not set.
> >>> 
> >>> [The "native" feature was invented to identify environments where
> >>> JIT would work properly. The "host-triple == target-triple" condition
> >>> isn't exactly right, but it works well enough.]
> >>> 
> >>> The major problem is that these new "native" dependencies are incorrect.
> >>> For example the DebugInfo tests don't really require it; they work fine
> >>> as long as the default triple has the corresponding backend included,
> >>> as my ARM-on-X86 experiment demonstrated.
> >> 
> >> Are they are guarantee’d to work with a default triple set to any of the
> >> possible (in-tree) backend?
> >> (I don’t know enough about these tests, which is definitively why I
> >> included them in the “big hammer” solution)
> >> 
> >> Thanks,
> >> 
> >> —
> >> Mehdi
>