[LLVMdev] [cfe-dev] RFC: Change tests to run with fixed (not-host dependent) triple

[Daniel Dunbar]

On Sat, Dec 1, 2012 at 1:06 AM, Chandler Carruth <chandlerc at google.com>wrote:

> On Fri, Nov 30, 2012 at 8:04 PM, Chris Lattner <clattner at apple.com> wrote:
>
>> I'm ok with this in principle, but how about with the nuance that some
>> tests (eg test/codegen) explicitly opt into march=native?
>>
>
> I'd really like the default behavior to be something that forces the test
> to either be independent of the targeted triple, or explicitly set a
> target. I like the default being unknown.
>
> I wonder, would the ability to run the entire test suite with all of the
> 'default' triples (that lit sets to unknown in normal runs) instead set to
> the host, or to a specific triple maybe be a useful extra form of checking?
> This would let both humans and build bots find bugs and discrepancies
> specific to a particular target.
>

I'd prefer not to do this universally (i.e., run the whole test suite that
way), but what I do think would be useful is to add enough test suite
support to be able to easily run the same test on multiple triples (or even
all configured ones).

My primary goal is to have the tests that individual developers be
equivalent (independent of the target they are running on).

 - Daniel

We could even have a common test target that build bots use which runs all
> the tests both in the default, and in the host-triple mode so that we force
> people to converge on target independent tests or explicit triples.
>
>
>>
>> -Chris
>>
>> On Nov 30, 2012, at 12:16 PM, Daniel Dunbar <daniel at zuster.org> wrote:
>>
>> > Hi all,
>> >
>> > We consistently see test failures arising because by default many of
>> our tests run in a mode where the tool (clang or llc) pick host-dependent
>> behavior. This makes it easy for developers to write tests that pass on
>> their system, but fail for other developers.
>> >
>> > There is some utility in this behavior, as it gives us (unintended)
>> testing coverage of some things, but overall I think it is a net loss for
>> productivity.
>> >
>> > I propose:
>> >
>> > a. We change the test suite to run in such a way that all tools default
>> to an "unknown" host triple.
>> >
>> > b. If someone feels there is missing coverage in some area, we add
>> increased tests for that area (which get run on all platforms).
>> >
>> > c. If there is some reason that running with an "unknown" host triple
>> is undesirable, I propose that we set the default test triple to be
>> "x86_64-pc-linux-gnu", and require deviations to be specified.
>> >
>> > Comments?
>> >
>> >  - Daniel
>> >
>> > _______________________________________________
>> > LLVM Developers mailing list
>> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20121203/886f8041/attachment.html>