[test-suite] r214118 - test commit for the ABI-Testsuite

[Sunil Srivastava]

Author: ssrivastava
Date: Mon Jul 28 16:01:10 2014
New Revision: 214118

URL: http://llvm.org/viewvc/llvm-project?rev=214118&view=rev
Log:
test commit for the ABI-Testsuite

Added:
    test-suite/trunk/ABI-Testsuite/
    test-suite/trunk/ABI-Testsuite/Design-of-ABI-Testsuite-7-2.pdf   (with props)
    test-suite/trunk/ABI-Testsuite/README.txt

Added: test-suite/trunk/ABI-Testsuite/Design-of-ABI-Testsuite-7-2.pdf
URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/ABI-Testsuite/Design-of-ABI-Testsuite-7-2.pdf?rev=214118&view=auto
==============================================================================
Binary file - no diff available.

Propchange: test-suite/trunk/ABI-Testsuite/Design-of-ABI-Testsuite-7-2.pdf
------------------------------------------------------------------------------
    svn:executable = *

Propchange: test-suite/trunk/ABI-Testsuite/Design-of-ABI-Testsuite-7-2.pdf
------------------------------------------------------------------------------
    svn:mime-type = application/octet-stream

Added: test-suite/trunk/ABI-Testsuite/README.txt
URL: http://llvm.org/viewvc/llvm-project/test-suite/trunk/ABI-Testsuite/README.txt?rev=214118&view=auto
==============================================================================
--- test-suite/trunk/ABI-Testsuite/README.txt (added)
+++ test-suite/trunk/ABI-Testsuite/README.txt Mon Jul 28 16:01:10 2014
@@ -0,0 +1,106 @@
+This directory contains the IA64-ABI testsuite.
+
+To run the testsuite on linux
+
+1) Make sure you have 'lit' in PYTHONOPATH, and FileCheck in PATH.
+2) Do 
+      python linux-x86.py <compiler+args> test [any lit arguments]
+
+   <compiler+args> is the compiler to be tested, with any args as one argument; such as
+
+      python linux-x86.py clang test                    # clang must be in PATH
+      python linux-x86.py /home/me/bin/my-clang test    # my own compiler, not in PATH
+      python linux-x86.py 'clang -m32' test             # test 32 bit mode
+      python linux-x86.py 'clang -O' test               # test with optimization on
+      python linux-x86.py 'gcc' test -v                 # test gcc, with -v passed to lit
+      python linux-x86.py clang test/basic              # test only 'basic' directory 
+      
+linux-x86.py runs a small executable to first figure out whether this is 32 bit or 64 bit 
+platform, and then chooses one of the two configurations for testing.
+
+Other, non-linux platforms will require edits to the python script to specify compilers,
+linker, runner (if any). We will provide a sample file that can be modified to other 
+'similar' platforms, however it should be noted that the testsuite, as released, supports
+only x86-linux-like platforms (both ILP32 and LP64).
+
+Tests are arranged in multiple diretories, starting from 'test'. The pyhthon script can be 
+given any subdirectory. The directory 'test/common' contains utility files used by the
+testsuite.
+
+The overview of files is as follows:
+
+README.text                         # This README
+linux-x86.py                        # top level python script
+
+test/basic/basic.x                  # A small directory to test that the basic configuration
+test/basic/T_st.x                   # is correct. If this fails, everything else will too.
+
+test/common/genselector.c           # Directory containing utilities used by the testsuite
+test/common/select2.h
+test/common/testsuite.c
+test/common/testsuite.h             
+
+test/lit.site.cfg                   # top config file called from lit
+
+test/mangling/arrays.xpp            # Mangling test. Using FileCheck
+test/mangling/c++11.xpp
+test/mangling/c++11s.xpp
+....
+
+test/misc/s2_8a.xpp                 # miscellaneous test directory
+test/misc/s2_8b.x
+test/misc/s2_9_5.x
+test/misc/s3_1.xpp
+...
+
+test/s2_4/T_novirt_1.x              # test targetted to particular section of the spec
+...
+
+test/s2_5/T_vf1.x
+test/s2_5/T_vf2.x
+...
+
+test/s2_6/T_isc.x
+test/s2_6/T_ksc0.x
+test/s2_6/T_ksc1.x
+...
+
+test/struct_layout_tests/CT_bf.x    #  tests for plain struct layout
+test/struct_layout_tests/CT_Snen_xaa.x
+test/struct_layout_tests/CT_Snen_xab.x
+...
+
+test/struct_layout_tests/PACKED/CT_Snen_xaa.x   # test for packed structs
+test/struct_layout_tests/PACKED/CT_Snen_xab.x
+...
+
+------------------------------------------------------------------
+
+Some points to note:
+
+a) This release consists of about 290 files. We plan to release more tests a little later.
+b) As one can see, test files are named with suffixes '.x' and '.xpp'. This is intentional.
+   There is a mechanism, in lit.site.cfg, which copies them to correspondig '.c' or '.cpp' 
+   files, looking into a 'skip_list' specified at the top level python file. This mechanism
+   is meant to allow users to mark certain tests 'XFAIL' (expected to fail), on per-file,
+   per-test-script basis. 
+c) Test files are all self-contained and independent. They can be removed or moved around to 
+   different directories.
+d) Most files are named T_*.x or CT_*.x and are 'combined' files having both C and C++ code,
+   separated by '#ifdef __cplusplus'. The '//RUN' header at the top of these files runs both the
+   C and C++ compilers on them and links the resulting object files to make the excutable test.
+
+------------------------------------------------------------------
+There are several ways to handle situations where some tests are failing for some understood 
+reasons.
+
+1) There is a mechanism of skip_list to mark certain tests XFAIL by adding them to the skip_list,
+   as shown in sample.py
+2) The results of test executablles can be saved as a 'golden master' to compare against future 
+   runs.
+3) Finally, 'test_params['checker'] can be changed from plain 'grep' to any user written program
+   that can match the test results to such a 'golden master'.
+
+   
+
+