[libcxx-commits] [PATCH] D154987: [lit] Implement PYTHON directive and config.prologue

[Joel E. Denny via Phabricator via libcxx-commits]

jdenny added a comment.

@ldionne: It looks like this patch has revealed an existing bug in `libcxx/test/libcxx/selftest/sh.cpp/empty.sh.cpp`, but I'm not sure how best to handle it.  Can you help?

That test intends to check the case when no RUN line is specified, but its comments accidentally have an embedded RUN line:

  // XFAIL: *
  
  // Make sure the test DOES NOT pass if it has no 'RUN:' steps

As a result, the actual lit error is a shell parse error not an error about missing RUN lines.  From the pre-commit CI fails above:

  Script:
  --
  %dbg(RUN: at line 11) ' steps
  --
  Exit Code: 1
  
  Command Output (stdout):
  --
  shell parser error on: ": 'RUN: at line 11';  ' steps"
  
  --

The `XFAIL` directive turns that into an expected test fail.  That's why the test has not caused the test suite to fail before this patch.  After this patch, a shell parse error produces an unresolved test status not a failed test status, so the test suite now fails.  That seems like a good side effect of this patch as it catches this malformed test.

I tried to fix the test so it exercises what it was meant to: no RUN lines.  To do so, I removed the `:` after `RUN`.  Both //before and after// this patch, that produces an unresolved test status, and again that seems like the right lit behavior for a malformed test.  However, again, that means the test suite now fails.  How would you like to handle that?


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D154987/new/

https://reviews.llvm.org/D154987