[llvm-bugs] [Bug 30667] New: Eliminate duplicate execution models for lit ShTest tests

[via llvm-bugs]

https://llvm.org/bugs/show_bug.cgi?id=30667

            Bug ID: 30667
           Summary: Eliminate duplicate execution models for lit ShTest
                    tests
           Product: Test Suite
           Version: trunk
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: lit
          Assignee: unassignedbugs at nondot.org
          Reporter: modocache at gmail.com
                CC: daniel at zuster.org, llvm-bugs at lists.llvm.org
    Classification: Unclassified

(This task was copied from
https://reviews.llvm.org/diffusion/L/browse/llvm/tags/RELEASE_390/final/utils/lit/TODO.
It was originally written by Daniel Dunbar.)

Currently, the ShTest format uses tests written with shell-script like syntax,
and executes them in one of two ways. The first way is by converting them into
a bash script and literally executing externally them using bash. The second
way is through the use of an internal shell parser and shell execution code
(built on the subprocess module). The external execution mode is used on most
Unix systems that have bash, the internal execution mode is used on Windows.

Having two ways to do the same thing is error prone and leads to unnecessary
complexity in the testing environment. Additionally, because the mode that
converts scripts to bash doesn't try and validate the syntax, it is possible to
write tests that use bash shell features unsupported by the internal shell.
Such tests won't work on Windows but this may not be obvious to the developer
writing the test.

Another limitation is that when executing the scripts externally, the ShTest
format has no idea which commands fail, or what output comes from which
commands, so this limits how convenient the output of ShTest failures can be
and limits other features (for example, knowing what temporary files were
written).

We should eliminate having two ways of executing the same tests to reduce
platform differences and make it easier to develop new features in the ShTest
module. This is currently blocked on:

* The external execution mode is faster in some situations, because it avoids
being bottlenecked on the GIL. This can hopefully be obviated simply by using
--use-processes.

* Some tests in LLVM/Clang are explicitly disabled with the internal shell
(because they use features specific to bash). We would need to rewrite these
tests, or add additional features to the internal shell handling to allow them
to pass.

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20161012/b52c6f5a/attachment.html>