[PATCH] D23156: [ADT] Stop trying to test every combination of values in a triple in every permutation.

[Chandler Carruth via llvm-commits]

chandlerc added a comment.

In https://reviews.llvm.org/D23156#506253, @dblaikie wrote:

> Do you need to use permutations?
>
> So the original version tried every value in every slot against every other value in every slot.


More than that, see below:

> Given, say, 4 slots each with M, N, O, and P possible values, that gives us M * N * O * P test cases.


So, the original version, ignoring the 3-slot special case and just focusing on the 4 slot case did:

(M * N * O * P) * 4!

> - but we can reasonably identify that the parsing of the value in a given slot is independent of the values in the other slots, so this now tests every value in every slot regardless of the values in other slots?

>   - that gets us down to (M + N + O + P) * 4 tests (assuming 4 slots for simplicity)


Correct in general. IT gets us down to: (M + N + O + P) * 4! tests.

> - would it be reasonable to assume that the parsing of a value is independent of which slot it is in? So we shouldn't then test that every value can appear in every slot/ordering?


It might, but that just saves 4! from this and wouldn't catch stateful bugs in the parsing. So I'm pretty happy to try and parse each entry in each position.

Mostly this is because the runtime is now so fast that it seems to not matter.

Does that make sense?


https://reviews.llvm.org/D23156