[LLVMbugs] [Bug 969] NEW: llc-beta results count as fail if they didn't run

[bugzilla-daemon at cs.uiuc.edu]

http://llvm.org/bugs/show_bug.cgi?id=969

           Summary: llc-beta results count as fail if they didn't run
           Product: Test Suite
           Version: trunk
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: Nightly Tester
        AssignedTo: unassignedbugs at nondot.org
        ReportedBy: rspencer at x10sys.com


I don't run the llc-beta tests as part of my nightly tester. That is I don't
specify the -enable-llcbeta option to NewNightlyTester.pl. The test results web
site reports every single test as having llc-beta failed. That creates a
can't-see-the-tree-for-the-forest situation when reviewing results. It gets very
easy to miss a failed test because every test has at least one failure: llc-beta. 

I don't think this situation is useful or correct. If --enable-llcbeta was not
specified on the command line, the nightly tester should inform the server of
that situation and:

1. Not report llc-beta tests as failed
2. Not include an llc-beta summary ("Tests Significantly Changed for LLC-BETA")
3. Not include any llc-beta related columns in the detailed reports.



------- You are receiving this mail because: -------
You are on the CC list for the bug, or are watching someone who is.