[LLVMdev] teaching FileCheck to handle variations in order

[Krzysztof Parzyszek]

On 9/7/2012 4:08 PM, Chandler Carruth wrote:
>
> Fundamentally, I agree with you that these ordering dependencies are
> unfortunate in tests. However, I would make a few observations:

I disagree.  A testcase should test what it was meant to test, and 
nothing else.  Instruction ordering, for example, is not something that 
we can expect to remain the same.  Various optimizations can affect it 
(changes in the scheduler being the most obvious case), causing spurious 
failures that would need to be investigated and addressed.

As a matter of fact, we should make an effort to make our tests 
independent of any unrelated (but valid) optimizations that may be added 
at some point in the future.  I realize that this is not always possible 
to achieve completely, but the more precise we get with the tests, the 
better off we will be in the long run.

I don't think that FileCheck is modified as frequently as the compiler 
itself, so the added complexity (which apparently is not large in this 
case) is not likely to add a significant recurring cost.

-K

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, 
hosted by The Linux Foundation