[Lldb-commits] [lldb] [lldb] change how unsupported tests are aggregated (PR #185675)

[Jonas Devlieghere via lldb-commits]

JDevlieghere wrote:

> The change in the diff is explained by the fact that many of the tests on Windows are considered UNSUPPORTED whereas they are really XFAIL.

Gotcha, so what we really want here is to say that XFAILs should be considered more important than UNSUPPORTED, which I think is reasonable. 

> I don't think returning multiple results is supported yet, the output of the `execute` method is expected to be a tuple with the first element being the return code and the second the output: https://github.com/charles-zablit/llvm-project/blob/4727ed3c09c0765373c426d0059097bc92023c65/llvm/utils/lit/lit/worker.py#L94-L99

You're right, looks like we still do the reduction below, which explains the issue you're describing, if there are no passing tests to take priority.

```
        # Return the lit result code with the maximum occurrence. Only look at
        # the first element and rely on the original order to break ties.
        return max(lit_results, key=operator.itemgetter(0))[1], output
```

So maybe the correct solution here is to gather all the test results in `lit_results` like we do today, and have a reduction function that prioritizes certain results over others. Top off my head, I would expect something like:

```
UNRESOLVED > FAIL > XPASS > PASS > XFAIL > UNSUPPORTED
```

https://github.com/llvm/llvm-project/pull/185675