[llvm-dev] Testing "normal" cross-compilers versus GPU backends

Thu Sep 3 02:07:54 PDT 2015

> On Sep 3, 2015, at 12:18 AM, Robinson, Paul <Paul_Robinson at playstation.sony.com> wrote:
> 
> 
> 
>> -----Original Message-----
>> From: Mehdi Amini [mailto:mehdi.amini at apple.com]
>> Sent: Wednesday, September 02, 2015 7:10 PM
>> To: Robinson, Paul
>> Cc: llvm-dev at lists.llvm.org; tom at stellard.net; NAKAMURA Takumi
>> Subject: Re: Testing "normal" cross-compilers versus GPU backends
>> 
>> Hi Paul,
>> 
>> Thanks for the summary!
>> 
>>> On Sep 2, 2015, at 5:44 PM, Robinson, Paul
>> <Paul_Robinson at playstation.sony.com> wrote:
>>> 
>>> This note arose from https://urldefense.proofpoint.com/v2/url?u=http-
>> 3A__reviews.llvm.org_D12506&d=BQIFAg&c=eEvniauFctOgLOKGJOplqw&r=v-
>> ruWq0KCv2O3thJZiK6naxuXK8mQHZUmGq5FBtAmZ4&m=Wr0uOhkAp_10X4edWwxZQ9V8L97j8e
>> o6cR_1Ia-gMOw&s=OOTP9DnL-TWV1zvy9EcU0Z6yfTq5lBjhE-LvYlWMJ3Y&e=  but the
>> reviewers
>>> felt that we needed a broader audience, because the proposed patch
>>> really didn't solve the entire problem and we had no better ideas.
>>> 
>>> Mehdi Amini needs to build LLVM with just a GPU backend, and still have
>>> "ninja check" Just Work.  Commits r243958-243960 tried to accomplish
>>> that; however they are too big a hammer, and cause much simpler cross
>>> environments (like mine) to exclude big chunks of very useful tests
>>> (including my favorite, DebugInfo).
>>> 
>>> FYI, my main cross environment is building on X86 Windows but using
>>> a default target triple for PS4 (which is also X86).
>>> 
>>> I experimented with building LLVM with just the ARM backend (running on
>>> an X86 workstation) and setting the default triple to some ARM value.
>>> "ninja check" worked fine (without Mehdi's series of commits), so the
>>> normal kind of cross-compiler environment seems to be happy with how
>>> things were set up originally.
>>> 
>>> Mehdi reports building LLVM with the X86 and AMDGPU backends, setting
>>> the default triple to "amdgcn--amdhsa", and getting 200-some failures.
>>> 
>>> (This does make me wonder about AMDGPU testing in general; how does that
>>> work?  The only places I see lit checks for AMDGPU are in the usual
>>> target-dependent places.)
>> 
>> I don’t understand this interrogation about how do you do testing in
>> general. The same way you don’t process tests/CodeGen/X86/* with the ARM
>> backend, you can’t process any random IR through these backends.
> 
> You said you had 200+ failures with AMDGPU.  Are the AMD folks simply
> tolerating the 200 failures, and you don't want to?  I should hope there
> is more to it than that.

Well, I don’t know, they might just run `ninja check` with the default triple set to X86?
(which I would consider being working around a buggy test suite)

> 
>> 
>> IMO, the problem is in general about tests that are written without
>> specifying a triple, that will be executed with the default triple.
>> 
>> Most of these tests were written with X86 (or ARM) in mind, and there is
>> no guarantee that they will behave as intended with every possible triple.
>> The DataLayout for instance has to be the one from the target, and is not
>> portable.
>> I think a "portable backend test” is pretty rare in general.
> 
> It depends on what the test is trying to do.  I'm sure it is quite common
> for IR tests to behave essentially the same way regardless of target.

IR tests != backend test (I may miss your point here, it’s late…).

> We have lots of tests (the ones you chose to mark "native") that had been
> working fine with ARM, X86, PPC, SPARC, and whatever.
> 
> The fact that they don't work with your backend is different from saying
> those tests can't possibly work for any cross-compiler.

I believe this is far from what I said, or at least from what I had in mind.

>  But the latter is what your patch implemented, and it is preventing useful testing.

I don’t disagree with that statement, and I agree that it should be fixed.
It doesn’t mean that I think the previous situation was better though. 
The current over-conservative state seems at least more correct than having only a subset of the targets that can pass `ninja check`.

Are you attending the LLVM bay area social tonight by any chance?

Best,

— 
Mehdi

> --paulr
> 
>> 
>> You can run all the tests by setting the default triple to X86 and
>> compiling in the X86 backend, but that’s just a trick to make the tests
>> happy.
>> Alternatively, and this is what I tried to do, blacklisting these tests
>> that “pretends” to be “portable” but are not.
>> 
>> 
>>> Mehdi's solution was:
>>> - In lit.cfg, change the existing "native" feature definition from
>>> "host-triple == target-triple" to also check "and the corresponding
>>> backend is included."(**)
>> 
>> I agree that we could remove the condition on the host backend included
>> and deemed it a unsupported, but the build system needs to reject this
>> configuration.
>> 
>> 
>>> - Make piles of tests that seemed inapplicable to GPUs depend on the
>>> "native" feature (through REQUIRES: or in the lit.local.cfg).
>> 
>> Nitpick: the GPU is just an example, any other backends can be affected.
>> It seems that these test are “lying” about the target they will be able to
>> run on (like if they would run on “anything”).
>> 
>> 
>>> - Build LLVM with just the GPU backend, and not set a target triple
>>> (i.e., it's set to the host triple, typically an X86-something).(*)
>>> Thus lit.cfg sees matching host and target triples, but the X86
>>> backend is missing, and so the "native" feature is not set.
>>> 
>>> [The "native" feature was invented to identify environments where
>>> JIT would work properly. The "host-triple == target-triple" condition
>>> isn't exactly right, but it works well enough.]
>>> 
>>> The major problem is that these new "native" dependencies are incorrect.
>>> For example the DebugInfo tests don't really require it; they work fine
>>> as long as the default triple has the corresponding backend included,
>>> as my ARM-on-X86 experiment demonstrated.
>> 
>> Are they are guarantee’d to work with a default triple set to any of the
>> possible (in-tree) backend?
>> (I don’t know enough about these tests, which is definitively why I
>> included them in the “big hammer” solution)
>> 
>> Thanks,
>> 
>> —
>> Mehdi