[test-suite] r266782 - Create a testsuite for litsupport

Matthias Braun via llvm-commits llvm-commits at lists.llvm.org
Fri Jan 13 10:48:52 PST 2017 

Oops, I had *.json in my .git/info/exclude.

I added the file now along with some other small fixes I had lying around and everything works for me in a fresh checkout of the test-suite.

- Matthias

> On Jan 13, 2017, at 2:02 AM, Kristof Beyls <kristof.beyls at arm.com> wrote:
> 
> Hi Matthias,
> 
> I just tried to run these litsupport tests for the first time, and I'm seeing the following error messages.
> Is it possible you forgot to add a previous.json file somewhere to this commit?
> 
> Thanks,
> 
> Kristof
> 
> ~/dev/llvm.org/test-suite/litsupport-tests$ PATH=$HOME/dev/llvm.org/build/bin:$PATH ./run.sh ~/dev/llvm.org/llvm/utils/lit/lit.py
> ===> hash/python2.7
>  python2.7 /home/kribey01/dev/llvm.org/llvm/utils/lit/lit.py . -j 1 -Dprevious=previous.json -o "/tmp/litoutput/hash/result.json" >& "/tmp/litoutput/hash/lit.out"
>  cat /tmp/litoutput/hash/lit.out | FileCheck "check/lit.out"
> check/lit.out:1:12: error: expected string not found in input
> CHECK-DAG: SKIPPED: test-suite :: tests/skip.test
>           ^
> <stdin>:1:1: note: scanning from here
> lit.py: /home/kribey01/dev/llvm.org/llvm/utils/lit/lit/TestingConfig.py:101: fatal: unable to parse config file '../../lit.cfg', traceback: Traceback (most recent call last):
> ^
> <stdin>:5:43: note: possible intended match here
> config.previous_results = json.load(open(previous_results_file))
>                                          ^
> 
>> On 19 Apr 2016, at 18:49, Matthias Braun via llvm-commits <llvm-commits at lists.llvm.org> wrote:
>> 
>> Author: matze
>> Date: Tue Apr 19 12:49:21 2016
>> New Revision: 266782
>> 
>> URL: http://llvm.org/viewvc/llvm-project?rev=266782&view=rev
>> Log:
>> Create a testsuite for litsupport
>> 
>> Added:
>>   test-suite/trunk/litsupport-tests/
>>   test-suite/trunk/litsupport-tests/hash/
>>   test-suite/trunk/litsupport-tests/hash/check/
>>   test-suite/trunk/litsupport-tests/hash/check/lit.out
>>   test-suite/trunk/litsupport-tests/hash/lit.site.cfg
>>   test-suite/trunk/litsupport-tests/hash/litflags.txt
>>   test-suite/trunk/litsupport-tests/hash/tests/
>>   test-suite/trunk/litsupport-tests/hash/tests/foo.sh   (with props)
>>   test-suite/trunk/litsupport-tests/hash/tests/normal.test
>>   test-suite/trunk/litsupport-tests/hash/tests/ok.sh   (with props)
>>   test-suite/trunk/litsupport-tests/hash/tests/prepare.test
>>   test-suite/trunk/litsupport-tests/hash/tests/skip.test
>>   test-suite/trunk/litsupport-tests/run/
>>   test-suite/trunk/litsupport-tests/run.sh   (with props)
>>   test-suite/trunk/litsupport-tests/run/check/
>>   test-suite/trunk/litsupport-tests/run/check/lit.out
>>   test-suite/trunk/litsupport-tests/run/check/test.log
>>   test-suite/trunk/litsupport-tests/run/lit.site.cfg
>>   test-suite/trunk/litsupport-tests/run/tests/
>>   test-suite/trunk/litsupport-tests/run/tests/a.sh   (with props)
>>   test-suite/trunk/litsupport-tests/run/tests/a.test
>>   test-suite/trunk/litsupport-tests/run/tests/fail.sh   (with props)
>>   test-suite/trunk/litsupport-tests/run/tests/noexe.test
>>   test-suite/trunk/litsupport-tests/run/tests/noverify.test
>>   test-suite/trunk/litsupport-tests/run/tests/run_fail.test
>>   test-suite/trunk/litsupport-tests/run/tests/verify_fail.test
>>   test-suite/trunk/litsupport-tests/run/tests/verify_ok.test
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/check/lit.out
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/check/lit.out?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/check/lit.out (added)
>> +++ test-suite/trunk/litsupport-tests/hash/check/lit.out Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,3 @@
>> +CHECK-DAG: SKIPPED: test-suite :: tests/skip.test
>> +CHECK-DAG: PASS: test-suite :: tests/normal.test
>> +CHECK: Expected Passes
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/lit.site.cfg
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/lit.site.cfg?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/lit.site.cfg (added)
>> +++ test-suite/trunk/litsupport-tests/hash/lit.site.cfg Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,7 @@
>> +import os
>> +mydir = os.path.dirname(__file__)
>> +config.test_source_root = mydir
>> +config.test_exec_root = "/tmp/litoutput/hash"
>> +config.test_modules = [ "run", "hash" ]
>> +
>> +lit_config.load_config(config, "../../lit.cfg")
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/litflags.txt
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/litflags.txt?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/litflags.txt (added)
>> +++ test-suite/trunk/litsupport-tests/hash/litflags.txt Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1 @@
>> +-Dprevious=previous.json
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/tests/foo.sh
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/tests/foo.sh?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/tests/foo.sh (added)
>> +++ test-suite/trunk/litsupport-tests/hash/tests/foo.sh Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,3 @@
>> +#!/bin/sh
>> +echo "foo"
>> +exit 0
>> 
>> Propchange: test-suite/trunk/litsupport-tests/hash/tests/foo.sh
>> ------------------------------------------------------------------------------
>>   svn:executable = *
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/tests/normal.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/tests/normal.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/tests/normal.test (added)
>> +++ test-suite/trunk/litsupport-tests/hash/tests/normal.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1 @@
>> +RUN: tests/ok.sh
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/tests/ok.sh
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/tests/ok.sh?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/tests/ok.sh (added)
>> +++ test-suite/trunk/litsupport-tests/hash/tests/ok.sh Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,2 @@
>> +#!/bin/sh
>> +exit 0
>> 
>> Propchange: test-suite/trunk/litsupport-tests/hash/tests/ok.sh
>> ------------------------------------------------------------------------------
>>   svn:executable = *
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/tests/prepare.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/tests/prepare.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/tests/prepare.test (added)
>> +++ test-suite/trunk/litsupport-tests/hash/tests/prepare.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,2 @@
>> +RUN: rm -f /tmp/litoutput/does_not_exist
>> +RUN: tests/foo.sh
>> 
>> Added: test-suite/trunk/litsupport-tests/hash/tests/skip.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/hash/tests/skip.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/hash/tests/skip.test (added)
>> +++ test-suite/trunk/litsupport-tests/hash/tests/skip.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1 @@
>> +RUN: tests/foo.sh
>> 
>> Added: test-suite/trunk/litsupport-tests/run.sh
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run.sh?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run.sh (added)
>> +++ test-suite/trunk/litsupport-tests/run.sh Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,37 @@
>> +#!/bin/bash
>> +TOP_OUTPUT="/tmp/litoutput"
>> +rm -rf "$TOP_OUTPUT"
>> +mkdir -p "$TOP_OUTPUT"
>> +
>> +function run_tests {
>> +	OUTPUT="$TOP_OUTPUT/$DIR"
>> +	rm -rf "$OUTPUT"
>> +	mkdir -p "$OUTPUT"
>> +
>> +	echo "Running lit"
>> +	LITFLAGS=""
>> +	if [ -e litflags.txt ]; then
>> +		LITFLAGS="$(cat litflags.txt)"
>> +	fi
>> +	$LIT . -j 1 $LITFLAGS -o "$OUTPUT/result.json" >& "$OUTPUT/lit.out"
>> +	for f in check/*; do
>> +		BASE="$(basename "$f")"
>> +		echo "Checking litoutput/$BASE ($f)"
>> +		cat $OUTPUT/$BASE | FileCheck "$f"
>> +	done
>> +}
>> +
>> +for i in */lit.site.cfg; do
>> +	DIR="$(dirname $i)"
>> +	pushd "$DIR" > /dev/null
>> +
>> +	echo "===> $DIR/python2.7"
>> +	LIT="python2.7 $(which pullvm-lit)"
>> +	run_tests
>> +
>> +	echo "===> $DIR/python3"
>> +	LIT="python3 $(which pullvm-lit)"
>> +	run_tests
>> +
>> +	popd > /dev/null
>> +done
>> 
>> Propchange: test-suite/trunk/litsupport-tests/run.sh
>> ------------------------------------------------------------------------------
>>   svn:executable = *
>> 
>> Added: test-suite/trunk/litsupport-tests/run/check/lit.out
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/check/lit.out?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/check/lit.out (added)
>> +++ test-suite/trunk/litsupport-tests/run/check/lit.out Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,8 @@
>> +CHECK-DAG: PASS: test-suite :: tests/a.test
>> +CHECK-DAG: FAIL: test-suite :: tests/noexe.test
>> +CHECK-DAG: FAIL: test-suite :: tests/noverify.test
>> +CHECK-DAG: FAIL: test-suite :: tests/run_fail.test
>> +CHECK-DAG: FAIL: test-suite :: tests/verify_fail.test
>> +CHECK-DAG: PASS: test-suite :: tests/verify_ok.test
>> +CHECK: Expected Passes
>> +CHECK: Unexpected Failures
>> 
>> Added: test-suite/trunk/litsupport-tests/run/check/test.log
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/check/test.log?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/check/test.log (added)
>> +++ test-suite/trunk/litsupport-tests/run/check/test.log Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1 @@
>> +CHECK: Loaded test module {{.*}}run.py
>> 
>> Added: test-suite/trunk/litsupport-tests/run/lit.site.cfg
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/lit.site.cfg?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/lit.site.cfg (added)
>> +++ test-suite/trunk/litsupport-tests/run/lit.site.cfg Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,7 @@
>> +import os
>> +mydir = os.path.dirname(__file__)
>> +config.test_source_root = mydir
>> +config.test_exec_root = "/tmp/litoutput/run"
>> +config.test_modules = [ "run" ]
>> +
>> +lit_config.load_config(config, "../../lit.cfg")
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/a.sh
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/a.sh?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/a.sh (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/a.sh Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,2 @@
>> +#!/bin/sh
>> +echo "Hello"
>> 
>> Propchange: test-suite/trunk/litsupport-tests/run/tests/a.sh
>> ------------------------------------------------------------------------------
>>   svn:executable = *
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/a.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/a.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/a.test (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/a.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1 @@
>> +RUN: tests/a.sh foo bar
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/fail.sh
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/fail.sh?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/fail.sh (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/fail.sh Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,2 @@
>> +#!/bin/sh
>> +exit 1
>> 
>> Propchange: test-suite/trunk/litsupport-tests/run/tests/fail.sh
>> ------------------------------------------------------------------------------
>>   svn:executable = *
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/noexe.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/noexe.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/noexe.test (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/noexe.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1 @@
>> +RUN: xxx_does_not_exist
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/noverify.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/noverify.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/noverify.test (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/noverify.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,2 @@
>> +RUN: tests/a.sh
>> +VERIFY: xx_does_not_exists
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/run_fail.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/run_fail.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/run_fail.test (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/run_fail.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1 @@
>> +RUN: tests/fail.sh
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/verify_fail.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/verify_fail.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/verify_fail.test (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/verify_fail.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,2 @@
>> +RUN: tests/a.sh
>> +VERIFY: tests/fail.sh
>> 
>> Added: test-suite/trunk/litsupport-tests/run/tests/verify_ok.test
>> URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/litsupport-tests/run/tests/verify_ok.test?rev=266782&view=auto
>> ==============================================================================
>> --- test-suite/trunk/litsupport-tests/run/tests/verify_ok.test (added)
>> +++ test-suite/trunk/litsupport-tests/run/tests/verify_ok.test Tue Apr 19 12:49:21 2016
>> @@ -0,0 +1,2 @@
>> +RUN: tests/a.sh
>> +VERIFY: tests/a.sh
>> 
>> 
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>

More information about the llvm-commits mailing list