[cfe-dev] FW: Proposal for an ABI testsuite for clang

Eric Christopher echristo at gmail.com
Fri Jul 18 10:37:45 PDT 2014 

On Fri, Jul 18, 2014 at 9:59 AM, Srivastava, Sunil
<sunil_srivastava at playstation.sony.com> wrote:
> Hi,
>
>> At the SN Systems division of Sony, we have developed an IA64 ABI
>> test-suite for clang/llvm based on the lit framework. We would like to
>> submit this to the LLVM community.
>
> We are ready to submit first set of test files. I would like some feedback as to where in the tree it should go; how should the review process go.. etc.
>

Personally I think a subdirectory of projects/test-suite would be
good. It would be nice if all of this could be abstracted around make
check in some way fitting in there nicely. I.e. when I run the
testsuite via make TEST=simple check it'll automagically run the ABI
testsuite as well. This isn't particularly ideal given that the rest
of the testsuite is more designed around performance etc.

Others may have other opinions here. I'll CC a couple of them.

-eric

> I am quoting the README that gives more details giving the structure of the testsuite and instructions to run it.
>
> =====================================================
> This directory contains the IA64-ABI testsuite.
>
> To run the testsuite on linux
>
> 1) Make sure you have 'lit' in PYTHONOPATH, and FileCheck in PATH.
> 2) Do
>       python linux-x86.py <compiler+args> test [any lit arguments]
>
>    <compiler+args> is the compiler to be tested with any options,  as one argument; such as
>
>       python linux-x86.py clang test                    # clang must be in PATH
>       python linux-x86.py /home/me/bin/my-clang test    # my own compiler, not in PATH
>       python linux-x86.py 'clang -m32' test             # test 32 bit mode
>       python linux-x86.py 'clang -O' test               # test with optimization on
>       python linux-x86.py 'gcc' test -v                 # test gcc, with -v passed to lit
>       python linux-x86.py clang test/basic              # test only 'basic' directory
>
> linux-x86.py runs a small executable to first figure out whether this is 32 bit or 64 bit
> platform, and then chooses one of the two configurations for testing.
>
> Other, non-linux platforms will require edits to the python script to specify compilers,
> linker, runner (if any). We will provide a sample file that can be modified to other
> 'similar' platforms, however it should be noted that the testsuite, as released, supports
> only x86-linux-like platforms (both ILP32 and LP64).
>
> Tests are arranged in multiple directory, starting from 'test'. The python script can be
> given any subdirectory. The directory 'test/common' contains utility files used by the
> testsuite.
>
> The overview of files is as follows:
>
> README.text                         # This README
> linux-x86.py                        # top level python script
>
> test/basic/basic.x                  # A small directory to test that the basic configuration
> test/basic/T_st.x                   # is correct. If this fails, everything else will too.
>
> test/common/genselector.c           # Directory containing utilities used by the testsuite
> test/common/select2.h
> test/common/testsuite.c
> test/common/testsuite.h
>
> test/lit.site.cfg                   # top config file called from lit
>
> test/mangling/arrays.xpp            # Mangling test. Using FileCheck
> test/mangling/c++11.xpp
> test/mangling/c++11s.xpp
> ....
>
> test/misc/s2_8a.xpp                 # miscellaneous test directory
> test/misc/s2_8b.x
> test/misc/s2_9_5.x
> test/misc/s3_1.xpp
> ...
>
> test/s2_4/T_novirt_1.x              # test targeted to particular section of the spec
> ...
>
> test/s2_5/T_vf1.x
> test/s2_5/T_vf2.x
> ...
>
> test/s2_6/T_isc.x
> test/s2_6/T_ksc0.x
> test/s2_6/T_ksc1.x
> ...
>
> test/struct_layout_tests/CT_bf.x    #  tests for plain struct layout
> test/struct_layout_tests/CT_Snen_xaa.x
> test/struct_layout_tests/CT_Snen_xab.x
> ...
>
> test/struct_layout_tests/PACKED/CT_Snen_xaa.x   # test for packed structs
> test/struct_layout_tests/PACKED/CT_Snen_xab.x
> ...
>
> ------------------------------------------------------------------
>
> Some points to note:
>
> a) This release consists of about 290 files. We plan to release more tests little later.
> b) As one can see, test files are named with suffixes '.x' and '.xpp'. This is intentional.
>    There is a mechanism, in lit.site.cfg, which copies them to corresponding '.c' or '.cpp'
>    files, looking into a 'skip_list' specified at the top level python file. This mechanism
>    is meant to allow users to mark certain tests 'XFAIL' (expected to fail), on per-file,
>    per-test-script basis.
> c) All test files are self-contained and independent. They can be removed or moved around to
>    different directories.
> d) Most files names T_*.x or CT_*.x and 'combined' files having both C and C++ code, separated
>    by '#ifdef __cplusplus'. The '//RUN' header at the top of these files runs both C and C++
>    compiler on them and links the resulting object files to make the executable test.
>
>
> =====================================================
> -----Original Message-----
> From: Hal Finkel [mailto:hfinkel at anl.gov]
> Sent: Wednesday, July 02, 2014 8:33 PM
> To: Srivastava, Sunil
> Cc: cfe-dev at cs.uiuc.edu
> Subject: Re: [cfe-dev] Proposal for an ABI testsuite for clang
>
> ----- Original Message -----
>> From: "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>
>> To: cfe-dev at cs.uiuc.edu
>> Sent: Wednesday, July 2, 2014 5:51:20 PM
>> Subject: [cfe-dev] Proposal for an ABI testsuite for clang
>>
>>
>>
>>
>>
>> Hi,
>>
>> At the SN Systems division of Sony, we have developed an IA64 ABI
>> test-suite for clang/llvm based on the lit framework. We would like to
>> submit this to the LLVM community.
>
> I am strongly in favor of you submitting this for code review as soon as you can. A good ABI test suite is something we definitely should have.
>
>  -Hal
>
>> The test-suite currently
>> supports clang in both LP64/x86-64 and ILP32/x86 targets, with the
>> ability of adding others. The tests perform target-side execution and
>> work with both cross and native targets.
>>
>> Please find attached a pdf document that describes the design of this
>> test-suite. In summary, the test-suite covers:
>> · struct layout rules, including bit-fields, · Object layout, base
>> classes, vtables, VTTs, construction vtables, · Name mangling, ·
>> Contents of typeinfo variables, · Array cookies, · and a few other
>> items.
>>
>> Please consider accepting this test-suite as an llvm project. We look
>> forward to addressing any comments and questions you have on this
>> topic.
>>
>> Best regards,
>>
>>
>>
>> Sunil Srivastava
>>
>> SN Systems / Sony Computer Entertainment
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev

More information about the cfe-dev mailing list