[llvm-dev] Uncovering non-determinism in LLVM - The Next Steps

[Robinson, Paul via llvm-dev]

Side-tracking on one thing:

Also for LLVM testing purposes, I believe any case where the output text/bits are different are usually fixed so the tests are reliable.

FWIW: In the cases i cited the tests can already be made reliable in the face of these differences :)

With LLVM's infrastructure mostly around FileCheck I've found making tests order-independent to be somewhat difficult. Do you have particular techniques in mind? (eg: I care that one BB has feature X and one BB has feature Y - I'm not sure how to do that with LLVM's test infrastructure)

Assuming this requires multiple CHECK statements per feature of interest (minimally, one to identify the BB and one to check for the presence of the feature) what you need to do is capture the output and run it through FileCheck twice, with a different prefix for each set of checks.
That's the only way I'm aware of to make *sequences* of checks order-independent.
You can often make individual CHECKs order-independent with -DAG (2nd prize winner of most-misleading-suffix-name) but not always.

Back to your regularly scheduled debate now.
--paulr
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170707/c13c95ed/attachment.html>