r254251 - ARM v8.1a adds Advanced SIMD instructions for Rounding Double Multiply

[James Molloy via cfe-commits]

Hi,

FWIW, I'm happy with moving these tests into the test-suite. They do make
sense to be expressed in a compiler-neutral manner, even though some may
test clang-specific behaviour.

There is a UnitTests directory already, and adding the ability to run LIT
tests should be simple within the new CMake framework.

James
On Tue, 1 Dec 2015 at 19:09, David Blaikie <dblaikie at gmail.com> wrote:

> On Tue, Dec 1, 2015 at 10:43 AM, Renato Golin <renato.golin at linaro.org>
> wrote:
>
>> On 1 December 2015 at 18:29, David Blaikie <dblaikie at gmail.com> wrote:
>> > Are they things the test-suite couldn't (either technically or
>> > philosophically) cover, or only that it doesn't cover it at the moment,
>> but
>> > could do so?
>>
>> IMO, it's a philosophical issue. The test-suite is a whole-program
>> execution, and all that matters is the end result, the output of the
>> program, not the program itself.
>>
>> Those tests are mostly large (> 3 lines of code) so, if we were trying
>> to objdump and match for instruction patterns, any optimisation
>> variance could destroy them, as well as you may find the pattern
>> elsewhere in the same object, not belonging to the function you want
>> (inlining, etc).
>
>
> Not sure I follow - I'm not suggesting adding objdump/instruction checking
> to existing large programs in the test-suite, but adding other tests to the
> test-suite that do this and have appropriate input for it to be a
> reliable/useful form of testing.
>
>
>> All in all, all the problems with reducing C code
>> would be pertinent here, so a big hammer to kill small issues. On the
>> other hand, the Clang based asm/obj examples are all small, and
>> hand-crafted to suit the problem being tested.
>>
>> We could, however, add a new directory in the test-suite for that kind
>> of tests, like we already have for regressions in C/C++.
>
>
> If a separate directory makes it more clear what the purpose is, sure,
> seems fine to me, but I don't have strong or informed opinions on exactly
> how the test-suite is laid out.
>
>
>> That would
>> dissociate the front-end from the back-end, and essentially any
>> compiler used would have to abide by the rules of the test (which may
>> not be true for some compilers).
>>
>
> This is perhaps an interesting part/point: In theory I would think that
> these tests should be able to be phrased in a compiler-agnostic way, since
> they're testing the compiler as a whole and testing behavior that the
> source code demands, no?
>
>
>> I don't mind where they end up, really, but we need that kind of test
>> somewhere. Clang is an easy place because we know we have it built
>> when the Clang repo is checked out.
>>
>>
>> > That would sort of defeat the point of having the testing and projects
>> > separated though - it would tie the tests together and produce most of
>> the
>> > undesirable outcomes of having single end-to-end tests.
>>
>> Right, I agree, but since the relationship between Clang and LLVM is
>> non-trivial, and there are lots of changes that need to be done on
>> both sides, I can't see why we shouldn't have tests that span across
>> projects.
>>
>> Clearly, the sanitizer tests need Clang and Compiler-RT. Some Libc++
>> tests need Compiler-RT, others need libunwind, on ARM I need Clang to
>> build it. There are a lot of dependencies that are not there by
>> accident, but by design.
>>
>> The more you move tests away from the big targets (LLVM, Clang), the
>> less they're ran by people committing patches, and the harder will be
>> to pick up the failures. Having said that, a decent buildbot coverage
>> would account for most of those problems, but it would also put a
>> stress on slow/expensive/experimental hardware support. It's a
>> balance, I think.
>>
>
> Why would it require slow/expensive/experimental hardware? (& if it does
> require that, how do you expect average developers to run them on a regular
> basis?)
>
>
>>
>>
>> > (it would mean that at least the pure (or at least non-Clang) LLVM
>> developer
>> > would have a test to run where they would not if the test remained in
>> Clang
>> > only)
>>
>> That is a good point. Right now, RT/libc++ tests in that category are
>> controlled by enabling/disabling support via CMake flags. We may be
>> able to do the same with Clang (if LLVM is not built with it, disable
>> asm/obj tests). Moving them all to a regression package in the
>> test-suite would be another option, but one that would take a lot
>> longer...
>>
>> cheers,
>> --renato
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20151201/3fca826a/attachment.html>