<html>
<head>
<base href="https://llvm.org/bugs/" />
</head>
<body><table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Bug ID</th>
<td><a class="bz_bug_link
bz_status_NEW "
title="NEW --- - Consider changing lit to always load suites, then resolve command line arguments"
href="https://llvm.org/bugs/show_bug.cgi?id=30670">30670</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>Consider changing lit to always load suites, then resolve command line arguments
</td>
</tr>
<tr>
<th>Product</th>
<td>Test Suite
</td>
</tr>
<tr>
<th>Version</th>
<td>trunk
</td>
</tr>
<tr>
<th>Hardware</th>
<td>All
</td>
</tr>
<tr>
<th>OS</th>
<td>All
</td>
</tr>
<tr>
<th>Status</th>
<td>NEW
</td>
</tr>
<tr>
<th>Severity</th>
<td>enhancement
</td>
</tr>
<tr>
<th>Priority</th>
<td>P
</td>
</tr>
<tr>
<th>Component</th>
<td>lit
</td>
</tr>
<tr>
<th>Assignee</th>
<td>unassignedbugs@nondot.org
</td>
</tr>
<tr>
<th>Reporter</th>
<td>modocache@gmail.com
</td>
</tr>
<tr>
<th>CC</th>
<td>daniel@zuster.org, llvm-bugs@lists.llvm.org
</td>
</tr>
<tr>
<th>Classification</th>
<td>Unclassified
</td>
</tr></table>
<p>
<div>
<pre>(This task was copied from
<a href="https://reviews.llvm.org/diffusion/L/browse/llvm/tags/RELEASE_390/final/utils/lit/TODO">https://reviews.llvm.org/diffusion/L/browse/llvm/tags/RELEASE_390/final/utils/lit/TODO</a>.
It was originally written by Daniel Dunbar.)
Currently we expect each input argument to be a path on disk; we do a recursive
search to find the test suite for each item, but then we only do a local search
based at the input path to find tests. Additionally, for any path that matches
a file on disk we explicitly construct a test instance (bypassing the formats
on discovery implementation).
This has a couple problems:
* The test format doesn't have control over the test instances that result from
file paths.
* It isn't possible to specify virtual tests as inputs. For example, it is not
possible to specify an individual subtest to run with the googletest format.
* The test format doesn't have full control over the discovery of tests in
subdirectories.
Instead, we should move to a model whereby first all of the input specifiers
are resolved to test suites, and then the resolution of the input specifier is
delegated to each test suite. This could take a couple forms:
* We could resolve to test suites, then fully load each test suite, then have a
fixed process to map input specifiers to tests in the test suite (presumably
based on path-in-suite derivations). This has the benefit of being consistent
across all test formats, but the downside of requiring loading the entire test
suite.
* We could delegate all of the resolution of specifiers to the test suite. This
would allow formats that anticipate large test suites to manage their own
resolution for better performance. We could provide a default resolution
strategy that was similar to what we do now (start at subpaths for directories,
but allow the test format control over what happens for individual tests).</pre>
</div>
</p>
<hr>
<span>You are receiving this mail because:</span>
<ul>
<li>You are on the CC list for the bug.</li>
</ul>
</body>
</html>