<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jan 9, 2013 at 1:27 PM, David Blaikie <span dir="ltr"><<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im">On Mon, Jan 7, 2013 at 2:18 PM, Daniel Dunbar <<a href="mailto:daniel@zuster.org">daniel@zuster.org</a>> wrote:<br>
> They are similar, but you deleted the wrong one I think. The LitTestCommand<br>
> came later and some stuff was factored out into the StandardizedTest class,<br>
> and the code is somewhat cleaner. I think it would be better to delete<br>
> ClangTestCommand and use LitTestCommand instead.<br>
<br>
</div>Reverted in r172002.<br>
<br>
Actually it was StandardizedTest that I was trying to eliminate. To<br>
improve DejaGNUCommand to report specific result types that match<br>
DejaGNU's behavior (rather than pass/fail/warn, include the full<br>
"expected pass/fail, unexpected pass/fail, unresolved, etc" results)<br>
which will involve pulling down the common functionality from<br>
StandardizedTest into DejaGNUCommand and playing with it a bit since<br>
it won't be so general anymore. I also want to have DejaGNUCommand<br>
report failures in a way more similar to the way Clang failures are<br>
reported - by including a full log of each failed test in a separate<br>
log (rather than a log by result type with only those results & none<br>
of the context) - I'm not sure if the log format is common enough<br>
between DejaGNU and Clang that we can get that level of functionality<br>
shared between the two. I'm open to ideas, though.<br></blockquote><div><br></div><div style>I'm confused, doesn't the DejaGNU parser already separate things into pass, xpass, xfail, unsupported, etc.?</div><div style>
<br></div><div style> - Daniel</div><div style> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
I mentioned some of this on the lab list a few weeks ago before<br>
starting - perhaps I wasn't clear about the direction I was heading.<br>
<span class="HOEnZb"><font color="#888888"><br>
- David<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
><br>
> - Daniel<br>
><br>
><br>
> On Sat, Jan 5, 2013 at 10:18 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>> wrote:<br>
>><br>
>> On Sat, Jan 5, 2013 at 12:15 AM, NAKAMURA Takumi <<a href="mailto:geek4civic@gmail.com">geek4civic@gmail.com</a>><br>
>> wrote:<br>
>> > David, as I addressed in PR11317, I don't agree to delete<br>
>> > LitTestCommand.<br>
>> > <a href="http://llvm.org/bugs/show_bug.cgi?id=11317" target="_blank">http://llvm.org/bugs/show_bug.cgi?id=11317</a><br>
>> > (It could be closed as RESOLVED. I could file new one instead.)<br>
>> ><br>
>> > FYI, my builders are using LitTestCommand.<br>
>><br>
>> Is there any particular reason? Perhaps we could discuss the relevant<br>
>> differences between the two & figure out if/how they can be<br>
>> aggregated. They seemed similar enough to me that it was a fairly<br>
>> arbitrary choice based on which one was used more often. I'm open to<br>
>> other ideas.<br>
>><br>
>> ><br>
>> > ...Takumi<br>
>> ><br>
>> > 2013/1/3 David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>>:<br>
>> >> Author: dblaikie<br>
>> >> Date: Wed Jan 2 15:50:47 2013<br>
>> >> New Revision: 171416<br>
>> >><br>
>> >> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=171416&view=rev" target="_blank">http://llvm.org/viewvc/llvm-project?rev=171416&view=rev</a><br>
>> >> Log:<br>
>> >> Remove redundant LitTestCommand in favor of ClangTestCommand.<br>
>> >><br>
>> >> The latter could be renamed to LitTestCommand (as per the FIXIT) at<br>
>> >> some point.<br>
>> >><br>
>> >> My first intention, though, is to smoosh StandardTest into<br>
>> >> ClangTestCommand and<br>
>> >> then make it a bit more functional, including reporting all kinds of<br>
>> >> lit<br>
>> >> results rather than just warn/pass/fail.<br>
>> >><br>
>> >> Removed:<br>
>> >> zorg/trunk/zorg/buildbot/commands/LitTestCommand.py<br>
>> >> Modified:<br>
>> >> zorg/trunk/zorg/buildbot/builders/ClangBuilder.py<br>
>> >> zorg/trunk/zorg/buildbot/builders/LNTBuilder.py<br>
>> >> zorg/trunk/zorg/buildbot/commands/__init__.py<br>
>> >><br>
>> >> Modified: zorg/trunk/zorg/buildbot/builders/ClangBuilder.py<br>
>> >> URL:<br>
>> >> <a href="http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/builders/ClangBuilder.py?rev=171416&r1=171415&r2=171416&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/builders/ClangBuilder.py?rev=171416&r1=171415&r2=171416&view=diff</a><br>
>> >><br>
>> >> ==============================================================================<br>
>> >> --- zorg/trunk/zorg/buildbot/builders/ClangBuilder.py (original)<br>
>> >> +++ zorg/trunk/zorg/buildbot/builders/ClangBuilder.py Wed Jan 2<br>
>> >> 15:50:47 2013<br>
>> >> @@ -12,7 +12,6 @@<br>
>> >> from zorg.buildbot.commands import DejaGNUCommand<br>
>> >> from zorg.buildbot.commands.BatchFileDownload import BatchFileDownload<br>
>> >> from zorg.buildbot.commands.ClangTestCommand import ClangTestCommand<br>
>> >> -from zorg.buildbot.commands.LitTestCommand import LitTestCommand<br>
>> >> from zorg.buildbot.PhasedBuilderUtils import GetLatestValidated,<br>
>> >> find_cc<br>
>> >><br>
>> >> def getClangBuildFactory(<br>
>> >> @@ -745,12 +744,8 @@<br>
>> >> # Save artifacts of this build for use by other builders.<br>
>> >> f = uploadArtifacts(f)<br>
>> >> # Run the LLVM and Clang regression tests.<br>
>> >> - f.addStep(LitTestCommand(name='run.llvm.tests',<br>
>> >> haltOnFailure=True,<br>
>> >> + f.addStep(ClangTestCommand(name='check-all', haltOnFailure=True,<br>
>> >> command=['make', '-j',<br>
>> >> WithProperties('%(jobs)s'),<br>
>> >> - 'VERBOSE=1'], description=['llvm',<br>
>> >> 'tests'],<br>
>> >> - workdir='%s/test' % clang_build_dir))<br>
>> >> - f.addStep(LitTestCommand(name='run.clang.tests',<br>
>> >> haltOnFailure=True,<br>
>> >> - command=['make', '-j',<br>
>> >> WithProperties('%(jobs)s'),<br>
>> >> - 'VERBOSE=1'], description=['clang',<br>
>> >> 'tests'],<br>
>> >> - workdir='%s/tools/clang/test' %<br>
>> >> clang_build_dir))<br>
>> >> + 'VERBOSE=1'], description=['all',<br>
>> >> 'tests'],<br>
>> >> + workdir=clang_build_dir))<br>
>> >> return f<br>
>> >><br>
>> >> Modified: zorg/trunk/zorg/buildbot/builders/LNTBuilder.py<br>
>> >> URL:<br>
>> >> <a href="http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/builders/LNTBuilder.py?rev=171416&r1=171415&r2=171416&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/builders/LNTBuilder.py?rev=171416&r1=171415&r2=171416&view=diff</a><br>
>> >><br>
>> >> ==============================================================================<br>
>> >> --- zorg/trunk/zorg/buildbot/builders/LNTBuilder.py (original)<br>
>> >> +++ zorg/trunk/zorg/buildbot/builders/LNTBuilder.py Wed Jan 2 15:50:47<br>
>> >> 2013<br>
>> >> @@ -124,7 +124,7 @@<br>
>> >> if parallel:<br>
>> >> args.extend(['-j', WithProperties(jobs)])<br>
>> >> args.extend(nt_flags)<br>
>> >> - f.addStep(zorg.buildbot.commands.LitTestCommand.LitTestCommand(<br>
>> >> +<br>
>> >> f.addStep(zorg.buildbot.commands.ClangTestCommand.ClangTestCommand(<br>
>> >> name='lnt.nightly-test', command=args, haltOnFailure=True,<br>
>> >> description=['nightly test'], workdir='tests',<br>
>> >> logfiles={'configure.log' : 'nt/build/configure.log',<br>
>> >><br>
>> >> Removed: zorg/trunk/zorg/buildbot/commands/LitTestCommand.py<br>
>> >> URL:<br>
>> >> <a href="http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/commands/LitTestCommand.py?rev=171415&view=auto" target="_blank">http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/commands/LitTestCommand.py?rev=171415&view=auto</a><br>
>> >><br>
>> >> ==============================================================================<br>
>> >> --- zorg/trunk/zorg/buildbot/commands/LitTestCommand.py (original)<br>
>> >> +++ zorg/trunk/zorg/buildbot/commands/LitTestCommand.py (removed)<br>
>> >> @@ -1,50 +0,0 @@<br>
>> >> -import re<br>
>> >> -import StandardizedTest<br>
>> >> -<br>
>> >> -class LitTestCommand(StandardizedTest.StandardizedTest):<br>
>> >> - kTestLineRE = re.compile(r'([^ ]*): (.*) \(.*.*\)')<br>
>> >> - kTestFailureLogStartRE = re.compile(r"""\*{4,80} TEST '(.*)' .*""")<br>
>> >> - kTestFailureLogStopRE = re.compile(r"""\*{10,80}""")<br>
>> >> -<br>
>> >> - def parseLog(self, lines):<br>
>> >> - results = []<br>
>> >> - results_by_name = {}<br>
>> >> - failureLogs = []<br>
>> >> - lines = self.getLog('stdio').readlines()<br>
>> >> -<br>
>> >> - it = iter(lines)<br>
>> >> - inFailure = None<br>
>> >> - for ln in it:<br>
>> >> - # See if we are inside a failure log.<br>
>> >> - if inFailure:<br>
>> >> - inFailure[1].append(ln)<br>
>> >> - if self.kTestFailureLogStopRE.match(ln):<br>
>> >> - name,log = inFailure<br>
>> >> - if name not in results_by_name:<br>
>> >> - raise ValueError,'Invalid log result with no status line!'<br>
>> >> - results_by_name[name][2] = ''.join(log) + '\n'<br>
>> >> - inFailure = None<br>
>> >> - continue<br>
>> >> -<br>
>> >> - ln = ln.strip()<br>
>> >> - if not ln:<br>
>> >> - continue<br>
>> >> -<br>
>> >> - # Check for test failure logs.<br>
>> >> - m = self.kTestFailureLogStartRE.match(ln)<br>
>> >> - if m:<br>
>> >> - inFailure = (m.group(1), [ln])<br>
>> >> - continue<br>
>> >> -<br>
>> >> - # Otherwise expect a test status line.<br>
>> >> - m = self.kTestLineRE.match(ln)<br>
>> >> - if m:<br>
>> >> - code, name = m.group(1),m.group(2)<br>
>> >> - results.append( [code, name, None] )<br>
>> >> - results_by_name[name] = results[-1]<br>
>> >> -<br>
>> >> - if inFailure:<br>
>> >> - raise ValueError,("Unexpected clang test running output, "<br>
>> >> - "unterminated failure log!")<br>
>> >> -<br>
>> >> - return results<br>
>> >><br>
>> >> Modified: zorg/trunk/zorg/buildbot/commands/__init__.py<br>
>> >> URL:<br>
>> >> <a href="http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/commands/__init__.py?rev=171416&r1=171415&r2=171416&view=diff" target="_blank">http://llvm.org/viewvc/llvm-project/zorg/trunk/zorg/buildbot/commands/__init__.py?rev=171416&r1=171415&r2=171416&view=diff</a><br>
>> >><br>
>> >> ==============================================================================<br>
>> >> --- zorg/trunk/zorg/buildbot/commands/__init__.py (original)<br>
>> >> +++ zorg/trunk/zorg/buildbot/commands/__init__.py Wed Jan 2 15:50:47<br>
>> >> 2013<br>
>> >> @@ -1,7 +1,6 @@<br>
>> >> import AnalyzerCompareCommand<br>
>> >> import BatchFileDownload<br>
>> >> import ClangTestCommand<br>
>> >> -import LitTestCommand<br>
>> >> import DejaGNUCommand<br>
>> >> import GTestCommand<br>
>> >><br>
>> >><br>
>> >><br>
>> >> _______________________________________________<br>
>> >> llvm-commits mailing list<br>
>> >> <a href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a><br>
>> >> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a><br>
><br>
><br>
</div></div></blockquote></div><br></div></div>