[LLVMdev] [cfe-dev] RFC: Change tests to run with fixed (not-host dependent) triple

[David Blaikie]

On Mon, Dec 3, 2012 at 1:37 AM, Renato Golin <rengolin at systemcall.org> wrote:
> Apart from what has already being said, which I agree mostly, I have
> some specific comments:
>
>
>> a. We change the test suite to run in such a way that all tools default to
>> an "unknown" host triple.
>
> The assumptions will be different and I believe the refactoring of the
> tests to make them pass on all currently passing architectures will
> not be trivial. Good luck! ;)

I'm not sure what you mean by "refactoring of the tests to make them
pass on all currently passing architectures" - if we force tests
without specified triples to use an "unknown" triple, the tests would
need to be refactored, but why would they need to be refactored to
account for all currently passing architectures? They should behave
exactly the same (since they won't inherit the host triple that they
do now) on all test machines/architectures, right?

>> b. If someone feels there is missing coverage in some area, we add increased
>> tests for that area (which get run on all platforms).
>
> If you mean adding "unknown" triple tests as much as possible, I agree.

I believe this statement was about the reverse: if certain
target-specific coverage is required it would have to be done by
writing a test that specifies /that/ target triple, not by (as is
sometimes the case currently) writing a test that picks up the host
triple & then running it on a machine with the desired triple to get
coverage for.

>> c. If there is some reason that running with an "unknown" host triple is
>> undesirable, I propose that we set the default test triple to be
>> "x86_64-pc-linux-gnu", and require deviations to be specified.
>
> Here, I don't agree. I don't see why one platform should be the
> default over another.

Because we would need/want a default of some kind. The argument here
is: "If we can't choose some agnostic default for all tests, we should
choose a non-agnostic default" - the only alternative position is that
we don't choose a default but instead force every test to specify an
arbitrary triple. I don't think this is substantially more valuable,
though it is the current state of affairs among the tests that do have
triples specified (that they are "random" in the sense that they're
usually whatever architecture the developer is working on at the time
- so we have lots of linux ones, lots of darwin ones, and a smattering
of ARM)

> A similar concept, but less prejudicial is: If the tests require
> platform-specific features, platform-specific tests should be added.
> *Only* those added will be assumed tested (for the sakes of
> validation). Other architectures can add similar tests on their own
> triples, if necessary / desired.

Yes, that's the intent. This clause (c) was only applicable to the
approach overall (ie: if there are no (or no substantial) tests that
can be expressed with an unknown triple then we can't really use that
as a default - we'd have to pick some other default, or have no
default at all & force every test to specify a triple). For specific
tests that need to test platform-specific things, that is what Daniel
was saying in (b) - add coverage for that area by writing a test with
an explicit triple.

> In a nutshell, "unknown" defaults to nothing at all, whether running
> on Intel, ARM, MIPS or whatever. If you default "unknown" to the host
> architecture, or to a specific architecture, the benefits of having an
> "unknown" target is void, IMO.
>
>
> --
> cheers,
> --renato
>
> http://systemcall.org/
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev