[llvm] r333614 - [lit] Report line number for failed RUN command
Kostya Serebryany via llvm-commits
llvm-commits at lists.llvm.org
Wed May 30 21:00:38 PDT 2018
Yep, thanks!
On Wed, May 30, 2018 at 8:47 PM Joel E. Denny <jdenny.ornl at gmail.com> wrote:
> r333620 fixes it for me. If there are more buildbot troubles, I'll take a
> look in the morning.
>
> Joel
>
> On Wed, May 30, 2018 at 10:39 PM, Joel E. Denny <jdenny.ornl at gmail.com>
> wrote:
>
>> Hi Kostya,
>>
>> Thanks. It seems check-all doesn't run that. I'll work on it.
>>
>> Joel
>>
>> On Wed, May 30, 2018 at 10:20 PM, Kostya Serebryany <kcc at google.com>
>> wrote:
>>
>>> Hi,
>>>
>>> This change broke one of the libFuzzer tests:
>>> ninja check-fuzzer (or just ninja fuzzer && llvm-lit -sv
>>> ~/llvm/projects/compiler-rt/test/fuzzer/sigusr.test)
>>> hangs now.
>>>
>>> Reverting this change fixes the test.
>>>
>>> Please take a look and either fix, revert, or suggest a fix in
>>> projects/compiler-rt/test/fuzzer/sigusr.test
>>>
>>> --kcc
>>>
>>>
>>>
>>>
>>>
>>> On Wed, May 30, 2018 at 5:59 PM Joel E. Denny via llvm-commits <
>>> llvm-commits at lists.llvm.org> wrote:
>>>
>>>> Author: jdenny
>>>> Date: Wed May 30 17:55:32 2018
>>>> New Revision: 333614
>>>>
>>>> URL: http://llvm.org/viewvc/llvm-project?rev=333614&view=rev
>>>> Log:
>>>> [lit] Report line number for failed RUN command
>>>>
>>>> (Relands r333584, reverted in 333592.)
>>>>
>>>> When debugging test failures with -vv (or -v in the case of the
>>>> internal shell), this makes it easier to locate the RUN line that
>>>> failed. For example, clang's test/Driver/linux-ld.c has 892 total RUN
>>>> lines, and clang's test/Driver/arm-cortex-cpus.c has 424 RUN lines
>>>> after concatenation for line continuations.
>>>>
>>>> When reading the generated shell script, this also makes it easier to
>>>> locate the RUN line that produced each command.
>>>>
>>>> To support reporting RUN line numbers in the case of the internal
>>>> shell, this patch extends the internal shell to support the null
>>>> command, ":", except pipelines are not supported.
>>>>
>>>> To support reporting RUN line numbers in the case of windows cmd.exe
>>>> as the external shell, this patch extends -vv to set "echo on" instead
>>>> of "echo off" in bat files. (Support for windows cmd.exe as a lit
>>>> external shell will likely be dropped later, but I found out too
>>>> late.)
>>>>
>>>> Reviewed By: delcypher, asmith, stella.stamenova, jmorse, lebedev.ri,
>>>> rnk
>>>>
>>>> Differential Revision: https://reviews.llvm.org/D44598
>>>>
>>>> Added:
>>>>
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/basic.txt
>>>>
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/line-continuation.txt
>>>>
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/lit.local.cfg
>>>>
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/basic.txt
>>>>
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/line-continuation.txt
>>>>
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/lit.local.cfg
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/lit.cfg
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-shell/colon-error.txt
>>>> llvm/trunk/utils/lit/tests/shtest-run-at-line.py
>>>> Modified:
>>>> llvm/trunk/docs/CommandGuide/lit.rst
>>>> llvm/trunk/utils/lit/lit/TestRunner.py
>>>> llvm/trunk/utils/lit/tests/lit.cfg
>>>> llvm/trunk/utils/lit/tests/max-failures.py
>>>> llvm/trunk/utils/lit/tests/shtest-format.py
>>>> llvm/trunk/utils/lit/tests/shtest-output-printing.py
>>>> llvm/trunk/utils/lit/tests/shtest-shell.py
>>>> llvm/trunk/utils/lit/tests/unit/TestRunner.py
>>>>
>>>> Modified: llvm/trunk/docs/CommandGuide/lit.rst
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/docs/CommandGuide/lit.rst?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/docs/CommandGuide/lit.rst (original)
>>>> +++ llvm/trunk/docs/CommandGuide/lit.rst Wed May 30 17:55:32 2018
>>>> @@ -85,6 +85,10 @@ OUTPUT OPTIONS
>>>> Echo all commands to stdout, as they are being executed.
>>>> This can be valuable for debugging test failures, as the last echoed
>>>> command
>>>> will be the one which has failed.
>>>> + :program:`lit` normally inserts a no-op command (``:`` in the case of
>>>> bash)
>>>> + with argument ``'RUN: at line N'`` before each command pipeline, and
>>>> this
>>>> + option also causes those no-op commands to be echoed to stdout to
>>>> help you
>>>> + locate the source line of the failed command.
>>>> This option implies ``--verbose``.
>>>>
>>>> .. option:: -a, --show-all
>>>>
>>>> Modified: llvm/trunk/utils/lit/lit/TestRunner.py
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/lit/TestRunner.py?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/lit/TestRunner.py (original)
>>>> +++ llvm/trunk/utils/lit/lit/TestRunner.py Wed May 30 17:55:32 2018
>>>> @@ -40,6 +40,17 @@ kUseCloseFDs = not kIsWindows
>>>> kAvoidDevNull = kIsWindows
>>>> kDevNull = "/dev/null"
>>>>
>>>> +# A regex that matches %dbg(ARG), which lit inserts at the beginning
>>>> of each
>>>> +# run command pipeline such that ARG specifies the pipeline's source
>>>> line
>>>> +# number. lit later expands each %dbg(ARG) to a command that behaves
>>>> as a null
>>>> +# command in the target shell so that the line number is seen in lit's
>>>> verbose
>>>> +# mode.
>>>> +#
>>>> +# This regex captures ARG. ARG must not contain a right parenthesis,
>>>> which
>>>> +# terminates %dbg. ARG must not contain quotes, in which ARG might be
>>>> enclosed
>>>> +# during expansion.
>>>> +kPdbgRegex = '%dbg\(([^)\'"]*)\)'
>>>> +
>>>> class ShellEnvironment(object):
>>>>
>>>> """Mutable shell environment containing things like CWD and env
>>>> vars.
>>>> @@ -789,6 +800,13 @@ def _executeShCmd(cmd, shenv, results, t
>>>> results.append(cmdResult)
>>>> return cmdResult.exitCode
>>>>
>>>> + if cmd.commands[0].args[0] == ':':
>>>> + if len(cmd.commands) != 1:
>>>> + raise InternalShellError(cmd.commands[0], "Unsupported:
>>>> ':' "
>>>> + "cannot be part of a pipeline")
>>>> + results.append(ShellCommandResult(cmd.commands[0], '', '', 0,
>>>> False))
>>>> + return 0;
>>>> +
>>>> procs = []
>>>> default_stdin = subprocess.PIPE
>>>> stderrTempFiles = []
>>>> @@ -982,7 +1000,8 @@ def _executeShCmd(cmd, shenv, results, t
>>>>
>>>> def executeScriptInternal(test, litConfig, tmpBase, commands, cwd):
>>>> cmds = []
>>>> - for ln in commands:
>>>> + for i, ln in enumerate(commands):
>>>> + ln = commands[i] = re.sub(kPdbgRegex, ": '\\1'", ln)
>>>> try:
>>>> cmds.append(ShUtil.ShParser(ln, litConfig.isWindows,
>>>> test.config.pipefail).parse())
>>>> @@ -1066,9 +1085,16 @@ def executeScript(test, litConfig, tmpBa
>>>> mode += 'b' # Avoid CRLFs when writing bash scripts.
>>>> f = open(script, mode)
>>>> if isWin32CMDEXE:
>>>> - f.write('@echo off\n')
>>>> - f.write('\nif %ERRORLEVEL% NEQ 0 EXIT\n'.join(commands))
>>>> + for i, ln in enumerate(commands):
>>>> + commands[i] = re.sub(kPdbgRegex, "echo '\\1' > nul", ln)
>>>> + if litConfig.echo_all_commands:
>>>> + f.write('@echo on\n')
>>>> + else:
>>>> + f.write('@echo off\n')
>>>> + f.write('\n at if %ERRORLEVEL% NEQ 0 EXIT\n'.join(commands))
>>>> else:
>>>> + for i, ln in enumerate(commands):
>>>> + commands[i] = re.sub(kPdbgRegex, ": '\\1'", ln)
>>>> if test.config.pipefail:
>>>> f.write('set -o pipefail;')
>>>> if litConfig.echo_all_commands:
>>>> @@ -1301,7 +1327,9 @@ class IntegratedTestKeywordParser(object
>>>> self.parser = parser
>>>>
>>>> if kind == ParserKind.COMMAND:
>>>> - self.parser = self._handleCommand
>>>> + self.parser = lambda line_number, line, output: \
>>>> + self._handleCommand(line_number,
>>>> line, output,
>>>> + self.keyword)
>>>> elif kind == ParserKind.LIST:
>>>> self.parser = self._handleList
>>>> elif kind == ParserKind.BOOLEAN_EXPR:
>>>> @@ -1332,7 +1360,7 @@ class IntegratedTestKeywordParser(object
>>>> return (not line.strip() or output)
>>>>
>>>> @staticmethod
>>>> - def _handleCommand(line_number, line, output):
>>>> + def _handleCommand(line_number, line, output, keyword):
>>>> """A helper for parsing COMMAND type keywords"""
>>>> # Trim trailing whitespace.
>>>> line = line.rstrip()
>>>> @@ -1351,6 +1379,14 @@ class IntegratedTestKeywordParser(object
>>>> else:
>>>> if output is None:
>>>> output = []
>>>> + pdbg = "%dbg({keyword} at line {line_number})".format(
>>>> + keyword=keyword,
>>>> + line_number=line_number)
>>>> + assert re.match(kPdbgRegex + "$", pdbg), \
>>>> + "kPdbgRegex expected to match actual %dbg usage"
>>>> + line = "{pdbg} && {real_command}".format(
>>>> + pdbg=pdbg,
>>>> + real_command=line)
>>>> output.append(line)
>>>> return output
>>>>
>>>>
>>>> Added:
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/basic.txt
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/basic.txt?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> ---
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/basic.txt
>>>> (added)
>>>> +++
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/basic.txt
>>>> Wed May 30 17:55:32 2018
>>>> @@ -0,0 +1,6 @@
>>>> +# These commands must run under both bash and windows cmd.exe (with
>>>> GnuWin32
>>>> +# tools).
>>>> +
>>>> +# RUN: true
>>>> +# RUN: false
>>>> +# RUN: true
>>>>
>>>> Added:
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/line-continuation.txt
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/line-continuation.txt?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> ---
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/line-continuation.txt
>>>> (added)
>>>> +++
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/line-continuation.txt
>>>> Wed May 30 17:55:32 2018
>>>> @@ -0,0 +1,12 @@
>>>> +# These commands must run under both bash and windows cmd.exe (with
>>>> GnuWin32
>>>> +# tools).
>>>> +
>>>> +# RUN: echo 'foo bar' \
>>>> +# RUN: | FileCheck %s
>>>> +# RUN: echo \
>>>> +# RUN: 'foo baz' \
>>>> +# RUN: | FileCheck %s
>>>> +# RUN: echo 'foo bar' \
>>>> +# RUN: | FileCheck %s
>>>> +
>>>> +# CHECK: foo bar
>>>>
>>>> Added:
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/lit.local.cfg
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/lit.local.cfg?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> ---
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/lit.local.cfg
>>>> (added)
>>>> +++
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/external-shell/lit.local.cfg
>>>> Wed May 30 17:55:32 2018
>>>> @@ -0,0 +1,2 @@
>>>> +import lit.formats
>>>> +config.test_format = lit.formats.ShTest(execute_external=True)
>>>>
>>>> Added:
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/basic.txt
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/basic.txt?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> ---
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/basic.txt
>>>> (added)
>>>> +++
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/basic.txt
>>>> Wed May 30 17:55:32 2018
>>>> @@ -0,0 +1,3 @@
>>>> +# RUN: true
>>>> +# RUN: false
>>>> +# RUN: true
>>>>
>>>> Added:
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/line-continuation.txt
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/line-continuation.txt?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> ---
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/line-continuation.txt
>>>> (added)
>>>> +++
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/line-continuation.txt
>>>> Wed May 30 17:55:32 2018
>>>> @@ -0,0 +1,11 @@
>>>> +# RUN: : first line continued \
>>>> +# RUN: to second line
>>>> +# RUN: echo 'foo bar' \
>>>> +# RUN: | FileCheck %s
>>>> +# RUN: echo \
>>>> +# RUN: 'foo baz' \
>>>> +# RUN: | FileCheck %s
>>>> +# RUN: echo 'foo bar' \
>>>> +# RUN: | FileCheck %s
>>>> +
>>>> +# CHECK: foo bar
>>>>
>>>> Added:
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/lit.local.cfg
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/lit.local.cfg?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> ---
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/lit.local.cfg
>>>> (added)
>>>> +++
>>>> llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/internal-shell/lit.local.cfg
>>>> Wed May 30 17:55:32 2018
>>>> @@ -0,0 +1,2 @@
>>>> +import lit.formats
>>>> +config.test_format = lit.formats.ShTest(execute_external=False)
>>>>
>>>> Added: llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/lit.cfg
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/lit.cfg?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/lit.cfg (added)
>>>> +++ llvm/trunk/utils/lit/tests/Inputs/shtest-run-at-line/lit.cfg Wed
>>>> May 30 17:55:32 2018
>>>> @@ -0,0 +1,2 @@
>>>> +config.name = 'shtest-run-at-line'
>>>> +config.suffixes = ['.txt']
>>>>
>>>> Added: llvm/trunk/utils/lit/tests/Inputs/shtest-shell/colon-error.txt
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/Inputs/shtest-shell/colon-error.txt?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/Inputs/shtest-shell/colon-error.txt
>>>> (added)
>>>> +++ llvm/trunk/utils/lit/tests/Inputs/shtest-shell/colon-error.txt Wed
>>>> May 30 17:55:32 2018
>>>> @@ -0,0 +1,3 @@
>>>> +# Check error on an unsupported ":". (cannot be part of a pipeline)
>>>> +#
>>>> +# RUN: : | echo "hello"
>>>>
>>>> Modified: llvm/trunk/utils/lit/tests/lit.cfg
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/lit.cfg?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/lit.cfg (original)
>>>> +++ llvm/trunk/utils/lit/tests/lit.cfg Wed May 30 17:55:32 2018
>>>> @@ -71,3 +71,14 @@ for attribute in ('llvm_tools_dir', 'lit
>>>> if directory:
>>>> path = os.path.pathsep.join((directory, path))
>>>> config.environment['PATH'] = path
>>>> +
>>>> +# These substitutions are needed only in tests where the external
>>>> shell is used
>>>> +# and could be either bash or windows cmd.exe. Substitutions are
>>>> expected to
>>>> +# be expanded in double quotes.
>>>> +isWin32CMDEXE = lit_config.isWindows and not lit_config.getBashPath()
>>>> +if isWin32CMDEXE:
>>>> + config.substitutions.append(('%{pdbg0}', "echo '"))
>>>> + config.substitutions.append(('%{pdbg1}', "' > nul"))
>>>> +else:
>>>> + config.substitutions.append(('%{pdbg0}', ": '"))
>>>> + config.substitutions.append(('%{pdbg1}', "'"))
>>>>
>>>> Modified: llvm/trunk/utils/lit/tests/max-failures.py
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/max-failures.py?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/max-failures.py (original)
>>>> +++ llvm/trunk/utils/lit/tests/max-failures.py Wed May 30 17:55:32 2018
>>>> @@ -8,7 +8,7 @@
>>>> #
>>>> # END.
>>>>
>>>> -# CHECK: Failing Tests (26)
>>>> +# CHECK: Failing Tests (27)
>>>> # CHECK: Failing Tests (1)
>>>> # CHECK: Failing Tests (2)
>>>> # CHECK: error: Setting --max-failures to 0 does not have any effect.
>>>>
>>>> Modified: llvm/trunk/utils/lit/tests/shtest-format.py
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/shtest-format.py?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/shtest-format.py (original)
>>>> +++ llvm/trunk/utils/lit/tests/shtest-format.py Wed May 30 17:55:32 2018
>>>> @@ -39,6 +39,7 @@
>>>> #
>>>> # CHECK: Command Output (stdout):
>>>> # CHECK-NEXT: --
>>>> +# CHECK-NEXT: $ ":" "RUN: at line 1"
>>>> # CHECK-NEXT: $ "printf"
>>>> # CHECK-NEXT: # command output:
>>>> # CHECK-NEXT: line 1: failed test output on stdout
>>>>
>>>> Modified: llvm/trunk/utils/lit/tests/shtest-output-printing.py
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/shtest-output-printing.py?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/shtest-output-printing.py (original)
>>>> +++ llvm/trunk/utils/lit/tests/shtest-output-printing.py Wed May 30
>>>> 17:55:32 2018
>>>> @@ -16,12 +16,15 @@
>>>> #
>>>> # CHECK: Command Output
>>>> # CHECK-NEXT: --
>>>> +# CHECK-NEXT: $ ":" "RUN: at line 1"
>>>> # CHECK-NEXT: $ "true"
>>>> +# CHECK-NEXT: $ ":" "RUN: at line 2"
>>>> # CHECK-NEXT: $ "echo" "hi"
>>>> # CHECK-NEXT: # command output:
>>>> # CHECK-NEXT: hi
>>>> #
>>>> -# CHECK: $ "wc" "missing-file"
>>>> +# CHECK: $ ":" "RUN: at line 3"
>>>> +# CHECK-NEXT: $ "wc" "missing-file"
>>>> # CHECK-NEXT: # redirected output from
>>>> '{{.*(/|\\\\)}}basic.txt.tmp.out':
>>>> # CHECK-NEXT: missing-file{{.*}} No such file or directory
>>>> # CHECK: note: command had no output on stdout or stderr
>>>>
>>>> Added: llvm/trunk/utils/lit/tests/shtest-run-at-line.py
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/shtest-run-at-line.py?rev=333614&view=auto
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/shtest-run-at-line.py (added)
>>>> +++ llvm/trunk/utils/lit/tests/shtest-run-at-line.py Wed May 30
>>>> 17:55:32 2018
>>>> @@ -0,0 +1,70 @@
>>>> +# Check that -vv makes the line number of the failing RUN command
>>>> clear.
>>>> +# (-v is actually sufficient in the case of the internal shell.)
>>>> +#
>>>> +# RUN: not %{lit} -j 1 -vv %{inputs}/shtest-run-at-line > %t.out
>>>> +# RUN: FileCheck --input-file %t.out -Dpdbg0="%{pdbg0}"
>>>> -Dpdbg1="%{pdbg1}" %s
>>>> +#
>>>> +# END.
>>>> +
>>>> +
>>>> +# CHECK: Testing: 4 tests
>>>> +
>>>> +
>>>> +# In the case of the external shell, we check for only RUN lines in
>>>> stderr in
>>>> +# case some shell implementations format "set -x" output differently.
>>>> +
>>>> +# CHECK-LABEL: FAIL: shtest-run-at-line :: external-shell/basic.txt
>>>> +
>>>> +# CHECK: Script:
>>>> +# CHECK: [[pdbg0]]RUN: at line 4[[pdbg1]] && true
>>>> +# CHECK-NEXT: [[pdbg0]]RUN: at line 5[[pdbg1]] && false
>>>> +# CHECK-NEXT: [[pdbg0]]RUN: at line 6[[pdbg1]] && true
>>>> +
>>>> +# CHECK: RUN: at line 4
>>>> +# CHECK: RUN: at line 5
>>>> +# CHECK-NOT: RUN
>>>> +
>>>> +# CHECK-LABEL: FAIL: shtest-run-at-line ::
>>>> external-shell/line-continuation.txt
>>>> +
>>>> +# CHECK: Script:
>>>> +# CHECK: [[pdbg0]]RUN: at line 4[[pdbg1]] && echo 'foo bar' |
>>>> FileCheck
>>>> +# CHECK-NEXT: [[pdbg0]]RUN: at line 6[[pdbg1]] && echo 'foo baz' |
>>>> FileCheck
>>>> +# CHECK-NEXT: [[pdbg0]]RUN: at line 9[[pdbg1]] && echo 'foo bar' |
>>>> FileCheck
>>>> +
>>>> +# CHECK: RUN: at line 4
>>>> +# CHECK: RUN: at line 6
>>>> +# CHECK-NOT: RUN
>>>> +
>>>> +
>>>> +# CHECK-LABEL: FAIL: shtest-run-at-line :: internal-shell/basic.txt
>>>> +
>>>> +# CHECK: Script:
>>>> +# CHECK: : 'RUN: at line 1' && true
>>>> +# CHECK-NEXT: : 'RUN: at line 2' && false
>>>> +# CHECK-NEXT: : 'RUN: at line 3' && true
>>>> +
>>>> +# CHECK: Command Output (stdout)
>>>> +# CHECK: $ ":" "RUN: at line 1"
>>>> +# CHECK-NEXT: $ "true"
>>>> +# CHECK-NEXT: $ ":" "RUN: at line 2"
>>>> +# CHECK-NEXT: $ "false"
>>>> +# CHECK-NOT: RUN
>>>> +
>>>> +# CHECK-LABEL: FAIL: shtest-run-at-line ::
>>>> internal-shell/line-continuation.txt
>>>> +
>>>> +# CHECK: Script:
>>>> +# CHECK: : 'RUN: at line 1' && : first line continued to second
>>>> line
>>>> +# CHECK-NEXT: : 'RUN: at line 3' && echo 'foo bar' | FileCheck
>>>> +# CHECK-NEXT: : 'RUN: at line 5' && echo 'foo baz' | FileCheck
>>>> +# CHECK-NEXT: : 'RUN: at line 8' && echo 'foo bar' | FileCheck
>>>> +
>>>> +# CHECK: Command Output (stdout)
>>>> +# CHECK: $ ":" "RUN: at line 1"
>>>> +# CHECK-NEXT: $ ":" "first" "line" "continued" "to" "second" "line"
>>>> +# CHECK-NEXT: $ ":" "RUN: at line 3"
>>>> +# CHECK-NEXT: $ "echo" "foo bar"
>>>> +# CHECK-NEXT: $ "FileCheck" "{{.*}}"
>>>> +# CHECK-NEXT: $ ":" "RUN: at line 5"
>>>> +# CHECK-NEXT: $ "echo" "foo baz"
>>>> +# CHECK-NEXT: $ "FileCheck" "{{.*}}"
>>>> +# CHECK-NOT: RUN
>>>>
>>>> Modified: llvm/trunk/utils/lit/tests/shtest-shell.py
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/shtest-shell.py?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/shtest-shell.py (original)
>>>> +++ llvm/trunk/utils/lit/tests/shtest-shell.py Wed May 30 17:55:32 2018
>>>> @@ -26,6 +26,14 @@
>>>> # CHECK: error: command failed with exit status: 1
>>>> # CHECK: ***
>>>>
>>>> +# CHECK: FAIL: shtest-shell :: colon-error.txt
>>>> +# CHECK: *** TEST 'shtest-shell :: colon-error.txt' FAILED ***
>>>> +# CHECK: $ ":"
>>>> +# CHECK: # command stderr:
>>>> +# CHECK: Unsupported: ':' cannot be part of a pipeline
>>>> +# CHECK: error: command failed with exit status: 127
>>>> +# CHECK: ***
>>>> +
>>>> # CHECK: FAIL: shtest-shell :: diff-error-0.txt
>>>> # CHECK: *** TEST 'shtest-shell :: diff-error-0.txt' FAILED ***
>>>> # CHECK: $ "diff" "diff-error-0.txt" "diff-error-0.txt"
>>>> @@ -153,7 +161,7 @@
>>>> #
>>>> # CHECK: FAIL: shtest-shell :: error-1.txt
>>>> # CHECK: *** TEST 'shtest-shell :: error-1.txt' FAILED ***
>>>> -# CHECK: shell parser error on: 'echo "missing quote'
>>>> +# CHECK: shell parser error on: ': \'RUN: at line 3\' && echo "missing
>>>> quote'
>>>> # CHECK: ***
>>>>
>>>> # CHECK: FAIL: shtest-shell :: error-2.txt
>>>> @@ -219,4 +227,4 @@
>>>> # CHECK: PASS: shtest-shell :: sequencing-0.txt
>>>> # CHECK: XFAIL: shtest-shell :: sequencing-1.txt
>>>> # CHECK: PASS: shtest-shell :: valid-shell.txt
>>>> -# CHECK: Failing Tests (26)
>>>> +# CHECK: Failing Tests (27)
>>>>
>>>> Modified: llvm/trunk/utils/lit/tests/unit/TestRunner.py
>>>> URL:
>>>> http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/lit/tests/unit/TestRunner.py?rev=333614&r1=333613&r2=333614&view=diff
>>>>
>>>> ==============================================================================
>>>> --- llvm/trunk/utils/lit/tests/unit/TestRunner.py (original)
>>>> +++ llvm/trunk/utils/lit/tests/unit/TestRunner.py Wed May 30 17:55:32
>>>> 2018
>>>> @@ -99,8 +99,8 @@ class TestIntegratedTestKeywordParser(un
>>>> cmd_parser = self.get_parser(parsers, 'MY_RUN:')
>>>> value = cmd_parser.getValue()
>>>> self.assertEqual(len(value), 2) # there are only two run lines
>>>> - self.assertEqual(value[0].strip(), 'baz')
>>>> - self.assertEqual(value[1].strip(), 'foo bar')
>>>> + self.assertEqual(value[0].strip(), "%dbg(MY_RUN: at line 4)
>>>> && baz")
>>>> + self.assertEqual(value[1].strip(), "%dbg(MY_RUN: at line 7)
>>>> && foo bar")
>>>>
>>>> def test_custom(self):
>>>> parsers = self.make_parsers()
>>>>
>>>>
>>>> _______________________________________________
>>>> llvm-commits mailing list
>>>> llvm-commits at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180530/ad65365d/attachment.html>
More information about the llvm-commits
mailing list