[llvm-bugs] [Bug 30668] New: Allow 'lit' driver to cooperate with test formats and suites to add options (or at least sanitize accepted params)

[via llvm-bugs]

https://llvm.org/bugs/show_bug.cgi?id=30668

            Bug ID: 30668
           Summary: Allow 'lit' driver to cooperate with test formats and
                    suites to add options (or at least sanitize accepted
                    params)
           Product: Test Suite
           Version: trunk
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: lit
          Assignee: unassignedbugs at nondot.org
          Reporter: modocache at gmail.com
                CC: daniel at zuster.org, llvm-bugs at lists.llvm.org
    Classification: Unclassified

(This task was copied from
https://reviews.llvm.org/diffusion/L/browse/llvm/tags/RELEASE_390/final/utils/lit/TODO.
It was originally written by Daniel Dunbar.)

We have started to use the --params method more and more extensively, and it is
cumbersome and error prone. Additionally, there are currently various options
``lit`` honors that should more correctly be specified as belonging to the
ShTest test format.

It would be really nice if we could allow test formats and test suites to add
their own options to be parsed. The difficulty here, of course, is that we
don't know what test formats or test suites are in use until we have parsed the
input specifiers. For test formats we could ostensibly require all the possible
formats to be registered in order to have options, but for test suites we would
certainly have to load the suite before we can query it for what options it
understands.

That leaves us with the following options:

* Currently we could almost get away with parsing the input specifiers without
having done option parsing first (the exception is ``--config-prefix``) but
that isn't a very extensible design.

* We could make a distinction in the command line syntax for test format and
test suite options. For example, we could require something like::

    lit -j 1 -sv input-specifier -- --some-format-option

which would be relatively easy to implement with optparser (I think).

* We could allow fully interspersed arguments by first extracting the options
lit knows about and parsing them, then dispatching the remainder to the
formats. This seems the most convenient for users, who are unlikely to care
about (or even be aware of) the distinction between the generic lit
infrastructure and format or suite specific options.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20161012/e86316a4/attachment.html>