[Lldb-commits] [PATCH] D75864: Add a decorator option to skip tests based on a default setting

Tue Mar 10 09:12:30 PDT 2020

labath accepted this revision.
labath added a comment.
This revision is now accepted and ready to land.

In D75864#1914795 <https://reviews.llvm.org/D75864#1914795>, @aprantl wrote:

> In D75864#1913876 <https://reviews.llvm.org/D75864#1913876>, @labath wrote:
>
> > If a test requires a specific value of a setting, would it make more sense to just (re)set its value at the start of a test?
>
>
> This is not meant for requiring a setting, but to run the testsuite once in normal mode, and then again with --setting "experimental-feature-that-doesn't-work-for-all-tests-yet=true" and have a way to mark tests as not yet supported with that feature.

Right, that makes sense, though it makes me cry a bit every time we add a new option to the test decorators...

================
Comment at: lldb/test/API/sanity/TestSettingSkipping.py:28
+  def testNotExisting(self):
+    self.assertTrue(True, "This test should run!")
+
----------------
JDevlieghere wrote:
> aprantl wrote:
> > JDevlieghere wrote:
> > > This won't trip if the tests doesn't run. If you make the assert trip and XFAIL the test it should catch it. 
> > I'm not sure how that would work. Can you post an example? I want to test the skipIf decorator — how would I XFAIL it?
> Yeah I think it has the same problem, skipping takes priority over XFAIL, so it wouldn't matter. 
At one point, Todd added some tests for the test harness. There are still some remnants of it in `packages/Python/lldbsuite/test/test_runner/` but they're hopelessly out of date by now (and even when they were new they only seemed to work for Todd).

If we wanted to write self-tests something like that would seem appropriate. I'm not sure these tests are very useful as they stand now because they could pass or fail depending on what kind of --setting values I pass to dotest.

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D75864/new/

https://reviews.llvm.org/D75864