[llvm-bugs] [Bug 30666] New: Consider changing lit core to support setup vs. execute distinction

[via llvm-bugs]

https://llvm.org/bugs/show_bug.cgi?id=30666

            Bug ID: 30666
           Summary: Consider changing lit core to support setup vs.
                    execute distinction
           Product: Test Suite
           Version: trunk
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: lit
          Assignee: unassignedbugs at nondot.org
          Reporter: modocache at gmail.com
                CC: daniel at zuster.org, llvm-bugs at lists.llvm.org
    Classification: Unclassified

(This task was copied from
https://reviews.llvm.org/diffusion/L/browse/llvm/tags/RELEASE_390/final/utils/lit/TODO.
It was originally written by Daniel Dunbar.)

Many of the existing test formats are cleanly divided into two phases, once
parses the test format and extracts XFAIL and REQUIRES information, etc., and
the other code actually executes the test.

We could make this distinction part of lit's core infrastructure and that would
enable a couple things:

* The REQUIREs handling could be lifted to the core, which is nice.

* This would provide a clear place to insert subtest support, because the setup
phase could be responsible for providing subtests back to the core. That would
provide part of the infrastructure to parallelize them, for example, and would
probably interact well with other possible features like parameterized tests.

* This affords a clean implementation of --no-execute.

* One possible downside could be for test formats that cannot determine their
subtests without having executed the test. Supporting such formats would either
force the test to actually be executed in the setup stage (which might be ok,
as long as the API was explicitly phrased to support that), or would mean we
are forced into supporting subtests as return values from the execute phase.

Any format can just keep all of its code in execute, presumably, so the only
cost of implementing this is its impact on the API and futures changes.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20161012/ee46d365/attachment.html>