[Lldb-commits] [PATCH] D24629: Allow for tests to be disabled at runtime

[Todd Fiala via lldb-commits]

tfiala accepted this revision.
tfiala added a comment.
This revision is now accepted and ready to land.

I am accepting this with one strong reservation which I will explicitly call out here:

- If somebody checks in changes that are broken, and claims they missed it because they have an xfail exclusion file and didn't catch it, I will rip this out.  If the xfails are hard to setup, it is likely that this is a code smell for needing better decorators to more precisely home in on the cases that are failing.  Often times version checks are helpful.

I do get the utility this would afford for bring-up of different scenarios, though.  Hence I see that being useful enough to have it as an escape hatch.


================
Comment at: packages/Python/lldbsuite/test/configuration.py:107-108
@@ +106,4 @@
+skip_methods = None
+xfail_files = None
+xfail_methods = None
+
----------------
The skip seems okay.  The xfail seems *very* dangerous.  Nobody else is going to get these xfails.  We're setting ourselves up for having people check in tests that are broken.  It allows for a workflow where the user "thinks they're done", when they're not.


https://reviews.llvm.org/D24629