<div dir="ltr">On Fri, Feb 1, 2013 at 8:13 AM, David Blaikie <span dir="ltr"><<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Thu, Jan 31, 2013 at 5:39 PM, Daniel Dunbar <<a href="mailto:daniel@zuster.org">daniel@zuster.org</a>> wrote:<br>

> Author: ddunbar<br>
> Date: Thu Jan 31 19:39:42 2013<br>
> New Revision: 174135<br>
><br>
> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=174135&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=174135&view=rev</a><br>
> Log:<br>
> LitTestCommand: If the test command fails, always report failure.<br>
<br>
</div>Have you seen this in the wild? Where/why?</blockquote><div><br></div><div style>Yes. This is just a good fallback, otherwise its possible for a failure to go unnoticed.</div><div style> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
 Shouldn't we do something a<br>
bit more severe if this happens - seems to represent a fairly<br>
confusing situation for any user.<br></blockquote><div><br></div><div style>Reporting failure in the buildbot is the appropriate thing to do, in my opinion.</div><div style><br></div><div style>This situation can easily come up if someone put in the wrong lit command, or anything else where lit or the tests would fail to run but not necessarily produce any test failures.</div>
<div style><br></div><div style> - Daniel</div><div style><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="HOEnZb"><div class="h5"><br>
><br>
> Modified:<br>
>     zorg/trunk/zorg/buildbot/commands/LitTestCommand.py<br>
><br>
> Modified: zorg/trunk/zorg/buildbot/commands/LitTestCommand.py<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/commands/LitTestCommand.py?rev=174135&r1=174134&r2=174135&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/commands/LitTestCommand.py?rev=174135&r1=174134&r2=174135&view=diff</a><br>

> ==============================================================================<br>
> --- zorg/trunk/zorg/buildbot/commands/LitTestCommand.py (original)<br>
> +++ zorg/trunk/zorg/buildbot/commands/LitTestCommand.py Thu Jan 31 19:39:42 2013<br>
> @@ -76,8 +76,14 @@ class LitTestCommand(Test):<br>
>      self.addLogObserver('stdio', self.logObserver)<br>
><br>
>    def evaluateCommand(self, cmd):<br>
> +    # Always report failure if the command itself failed.<br>
> +    if cmd.rc != 0:<br>
> +      return FAILURE<br>
> +<br>
> +    # Otherwise, report failure if there were failures in the log.<br>
>      if self.logObserver.failed:<br>
> -        return FAILURE<br>
> +      return FAILURE<br>
> +<br>
>      return SUCCESS<br>
><br>
>    def describe(self, done=False):<br>
> @@ -97,6 +103,10 @@ class StepProxy(object):<br>
>    def addCompleteLog(self, name, text):<br>
>      self.logs.append((name, text))<br>
><br>
> +class RemoteCommandProxy(object):<br>
> +  def __init__(self, rc):<br>
> +    self.rc = rc<br>
> +<br>
>  class TestLogObserver(unittest.TestCase):<br>
>    def parse_log(self, text):<br>
>      observer = LitLogObserver()<br>
> @@ -115,5 +125,18 @@ PASS: test-three (3 of 3)<br>
>      self.assertEqual(obs.resultCounts, { 'FAIL' : 1, 'PASS' : 2 })<br>
>      self.assertEqual(obs.step.logs, [('test-two', 'FAIL: test-two (2 of 3)')])<br>
><br>
> +class TestCommand(unittest.TestCase):<br>
> +  def parse_log(self, text):<br>
> +    cmd = LitTestCommand()<br>
> +    cmd.logObserver.step = StepProxy()<br>
> +    for ln in text.split('\n'):<br>
> +      cmd.logObserver.outLineReceived(ln)<br>
> +    return cmd<br>
> +<br>
> +  def test_command_status(self):<br>
> +    # If the command failed, the status should always be error.<br>
> +    cmd = self.parse_log("")<br>
> +    self.assertEqual(cmd.evaluateCommand(RemoteCommandProxy(1)), FAILURE)<br>
> +<br>
>  if __name__ == '__main__':<br>
>    unittest.main()<br>
><br>
><br>
> _______________________________________________<br>
> llvm-commits mailing list<br>
> <a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br>
</div></div></blockquote></div><br></div></div>