[cfe-dev] FW: Proposal for an ABI testsuite for clang

[Srivastava, Sunil]

Hi,

> At the SN Systems division of Sony, we have developed an IA64 ABI 
> test-suite for clang/llvm based on the lit framework. We would like to 
> submit this to the LLVM community.

We are ready to submit first set of test files. I would like some feedback as to where in the tree it should go; how should the review process go.. etc.

I am quoting the README that gives more details giving the structure of the testsuite and instructions to run it.

=====================================================
This directory contains the IA64-ABI testsuite.

To run the testsuite on linux

1) Make sure you have 'lit' in PYTHONOPATH, and FileCheck in PATH.
2) Do 
      python linux-x86.py <compiler+args> test [any lit arguments]

   <compiler+args> is the compiler to be tested with any options,  as one argument; such as

      python linux-x86.py clang test                    # clang must be in PATH
      python linux-x86.py /home/me/bin/my-clang test    # my own compiler, not in PATH
      python linux-x86.py 'clang -m32' test             # test 32 bit mode
      python linux-x86.py 'clang -O' test               # test with optimization on
      python linux-x86.py 'gcc' test -v                 # test gcc, with -v passed to lit
      python linux-x86.py clang test/basic              # test only 'basic' directory 
      
linux-x86.py runs a small executable to first figure out whether this is 32 bit or 64 bit 
platform, and then chooses one of the two configurations for testing.

Other, non-linux platforms will require edits to the python script to specify compilers,
linker, runner (if any). We will provide a sample file that can be modified to other 
'similar' platforms, however it should be noted that the testsuite, as released, supports
only x86-linux-like platforms (both ILP32 and LP64).

Tests are arranged in multiple directory, starting from 'test'. The python script can be 
given any subdirectory. The directory 'test/common' contains utility files used by the
testsuite.

The overview of files is as follows:

README.text                         # This README
linux-x86.py                        # top level python script

test/basic/basic.x                  # A small directory to test that the basic configuration
test/basic/T_st.x                   # is correct. If this fails, everything else will too.

test/common/genselector.c           # Directory containing utilities used by the testsuite
test/common/select2.h
test/common/testsuite.c
test/common/testsuite.h             

test/lit.site.cfg                   # top config file called from lit

test/mangling/arrays.xpp            # Mangling test. Using FileCheck
test/mangling/c++11.xpp
test/mangling/c++11s.xpp
....

test/misc/s2_8a.xpp                 # miscellaneous test directory
test/misc/s2_8b.x
test/misc/s2_9_5.x
test/misc/s3_1.xpp
...

test/s2_4/T_novirt_1.x              # test targeted to particular section of the spec
...

test/s2_5/T_vf1.x
test/s2_5/T_vf2.x
...

test/s2_6/T_isc.x
test/s2_6/T_ksc0.x
test/s2_6/T_ksc1.x
...

test/struct_layout_tests/CT_bf.x    #  tests for plain struct layout
test/struct_layout_tests/CT_Snen_xaa.x
test/struct_layout_tests/CT_Snen_xab.x
...

test/struct_layout_tests/PACKED/CT_Snen_xaa.x   # test for packed structs
test/struct_layout_tests/PACKED/CT_Snen_xab.x
...

------------------------------------------------------------------

Some points to note:

a) This release consists of about 290 files. We plan to release more tests little later.
b) As one can see, test files are named with suffixes '.x' and '.xpp'. This is intentional.
   There is a mechanism, in lit.site.cfg, which copies them to corresponding '.c' or '.cpp' 
   files, looking into a 'skip_list' specified at the top level python file. This mechanism
   is meant to allow users to mark certain tests 'XFAIL' (expected to fail), on per-file,
   per-test-script basis. 
c) All test files are self-contained and independent. They can be removed or moved around to 
   different directories.
d) Most files names T_*.x or CT_*.x and 'combined' files having both C and C++ code, separated 
   by '#ifdef __cplusplus'. The '//RUN' header at the top of these files runs both C and C++
   compiler on them and links the resulting object files to make the executable test.


=====================================================
-----Original Message-----
From: Hal Finkel [mailto:hfinkel at anl.gov] 
Sent: Wednesday, July 02, 2014 8:33 PM
To: Srivastava, Sunil
Cc: cfe-dev at cs.uiuc.edu
Subject: Re: [cfe-dev] Proposal for an ABI testsuite for clang

----- Original Message -----
> From: "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>
> To: cfe-dev at cs.uiuc.edu
> Sent: Wednesday, July 2, 2014 5:51:20 PM
> Subject: [cfe-dev] Proposal for an ABI testsuite for clang
> 
> 
> 
> 
> 
> Hi,
> 
> At the SN Systems division of Sony, we have developed an IA64 ABI 
> test-suite for clang/llvm based on the lit framework. We would like to 
> submit this to the LLVM community.

I am strongly in favor of you submitting this for code review as soon as you can. A good ABI test suite is something we definitely should have.

 -Hal

> The test-suite currently
> supports clang in both LP64/x86-64 and ILP32/x86 targets, with the 
> ability of adding others. The tests perform target-side execution and 
> work with both cross and native targets.
> 
> Please find attached a pdf document that describes the design of this 
> test-suite. In summary, the test-suite covers:
> · struct layout rules, including bit-fields, · Object layout, base 
> classes, vtables, VTTs, construction vtables, · Name mangling, · 
> Contents of typeinfo variables, · Array cookies, · and a few other 
> items.
> 
> Please consider accepting this test-suite as an llvm project. We look 
> forward to addressing any comments and questions you have on this 
> topic.
> 
> Best regards,
> 
> 
> 
> Sunil Srivastava
> 
> SN Systems / Sony Computer Entertainment
> 
> 
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> 

--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory