[cfe-dev] FW: Proposal for an ABI testsuite for clang

Eric Christopher echristo at gmail.com
Fri Jul 18 12:24:21 PDT 2014 

Was under the impression there were execution tests in there, if not,
then I agree that test/ABI would make the most sense. The ability to
do things via gcc for it would make a bit more sense to be in
projects/test-suite.

-eric

On Fri, Jul 18, 2014 at 12:18 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> ----- Original Message -----
>> From: "Eric Christopher" <echristo at gmail.com>
>> To: "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>, "Bob Wilson" <bob.wilson at apple.com>, "Daniel Dunbar"
>> <daniel at zuster.org>, "Chris Matthews" <chris.matthews at apple.com>
>> Cc: cfe-dev at cs.uiuc.edu
>> Sent: Friday, July 18, 2014 12:37:45 PM
>> Subject: Re: [cfe-dev] FW: Proposal for an ABI testsuite for clang
>>
>> On Fri, Jul 18, 2014 at 9:59 AM, Srivastava, Sunil
>> <sunil_srivastava at playstation.sony.com> wrote:
>> > Hi,
>> >
>> >> At the SN Systems division of Sony, we have developed an IA64 ABI
>> >> test-suite for clang/llvm based on the lit framework. We would
>> >> like to
>> >> submit this to the LLVM community.
>> >
>> > We are ready to submit first set of test files. I would like some
>> > feedback as to where in the tree it should go; how should the
>> > review process go.. etc.
>> >
>>
>> Personally I think a subdirectory of projects/test-suite would be
>> good. It would be nice if all of this could be abstracted around make
>> check in some way fitting in there nicely. I.e. when I run the
>> testsuite via make TEST=simple check it'll automagically run the ABI
>> testsuite as well. This isn't particularly ideal given that the rest
>> of the testsuite is more designed around performance etc.
>>
>> Others may have other opinions here. I'll CC a couple of them.
>
> Given that these are essentially regression tests, and use lit (just like all of our other regression tests), why would we not put them into test/ABI (or similar) along with Clang's other regression tests?
>
>  -Hal
>
>>
>> -eric
>>
>> > I am quoting the README that gives more details giving the
>> > structure of the testsuite and instructions to run it.
>> >
>> > =====================================================
>> > This directory contains the IA64-ABI testsuite.
>> >
>> > To run the testsuite on linux
>> >
>> > 1) Make sure you have 'lit' in PYTHONOPATH, and FileCheck in PATH.
>> > 2) Do
>> >       python linux-x86.py <compiler+args> test [any lit arguments]
>> >
>> >    <compiler+args> is the compiler to be tested with any options,
>> >     as one argument; such as
>> >
>> >       python linux-x86.py clang test                    # clang
>> >       must be in PATH
>> >       python linux-x86.py /home/me/bin/my-clang test    # my own
>> >       compiler, not in PATH
>> >       python linux-x86.py 'clang -m32' test             # test 32
>> >       bit mode
>> >       python linux-x86.py 'clang -O' test               # test with
>> >       optimization on
>> >       python linux-x86.py 'gcc' test -v                 # test gcc,
>> >       with -v passed to lit
>> >       python linux-x86.py clang test/basic              # test only
>> >       'basic' directory
>> >
>> > linux-x86.py runs a small executable to first figure out whether
>> > this is 32 bit or 64 bit
>> > platform, and then chooses one of the two configurations for
>> > testing.
>> >
>> > Other, non-linux platforms will require edits to the python script
>> > to specify compilers,
>> > linker, runner (if any). We will provide a sample file that can be
>> > modified to other
>> > 'similar' platforms, however it should be noted that the testsuite,
>> > as released, supports
>> > only x86-linux-like platforms (both ILP32 and LP64).
>> >
>> > Tests are arranged in multiple directory, starting from 'test'. The
>> > python script can be
>> > given any subdirectory. The directory 'test/common' contains
>> > utility files used by the
>> > testsuite.
>> >
>> > The overview of files is as follows:
>> >
>> > README.text                         # This README
>> > linux-x86.py                        # top level python script
>> >
>> > test/basic/basic.x                  # A small directory to test
>> > that the basic configuration
>> > test/basic/T_st.x                   # is correct. If this fails,
>> > everything else will too.
>> >
>> > test/common/genselector.c           # Directory containing
>> > utilities used by the testsuite
>> > test/common/select2.h
>> > test/common/testsuite.c
>> > test/common/testsuite.h
>> >
>> > test/lit.site.cfg                   # top config file called from
>> > lit
>> >
>> > test/mangling/arrays.xpp            # Mangling test. Using
>> > FileCheck
>> > test/mangling/c++11.xpp
>> > test/mangling/c++11s.xpp
>> > ....
>> >
>> > test/misc/s2_8a.xpp                 # miscellaneous test directory
>> > test/misc/s2_8b.x
>> > test/misc/s2_9_5.x
>> > test/misc/s3_1.xpp
>> > ...
>> >
>> > test/s2_4/T_novirt_1.x              # test targeted to particular
>> > section of the spec
>> > ...
>> >
>> > test/s2_5/T_vf1.x
>> > test/s2_5/T_vf2.x
>> > ...
>> >
>> > test/s2_6/T_isc.x
>> > test/s2_6/T_ksc0.x
>> > test/s2_6/T_ksc1.x
>> > ...
>> >
>> > test/struct_layout_tests/CT_bf.x    #  tests for plain struct
>> > layout
>> > test/struct_layout_tests/CT_Snen_xaa.x
>> > test/struct_layout_tests/CT_Snen_xab.x
>> > ...
>> >
>> > test/struct_layout_tests/PACKED/CT_Snen_xaa.x   # test for packed
>> > structs
>> > test/struct_layout_tests/PACKED/CT_Snen_xab.x
>> > ...
>> >
>> > ------------------------------------------------------------------
>> >
>> > Some points to note:
>> >
>> > a) This release consists of about 290 files. We plan to release
>> > more tests little later.
>> > b) As one can see, test files are named with suffixes '.x' and
>> > '.xpp'. This is intentional.
>> >    There is a mechanism, in lit.site.cfg, which copies them to
>> >    corresponding '.c' or '.cpp'
>> >    files, looking into a 'skip_list' specified at the top level
>> >    python file. This mechanism
>> >    is meant to allow users to mark certain tests 'XFAIL' (expected
>> >    to fail), on per-file,
>> >    per-test-script basis.
>> > c) All test files are self-contained and independent. They can be
>> > removed or moved around to
>> >    different directories.
>> > d) Most files names T_*.x or CT_*.x and 'combined' files having
>> > both C and C++ code, separated
>> >    by '#ifdef __cplusplus'. The '//RUN' header at the top of these
>> >    files runs both C and C++
>> >    compiler on them and links the resulting object files to make
>> >    the executable test.
>> >
>> >
>> > =====================================================
>> > -----Original Message-----
>> > From: Hal Finkel [mailto:hfinkel at anl.gov]
>> > Sent: Wednesday, July 02, 2014 8:33 PM
>> > To: Srivastava, Sunil
>> > Cc: cfe-dev at cs.uiuc.edu
>> > Subject: Re: [cfe-dev] Proposal for an ABI testsuite for clang
>> >
>> > ----- Original Message -----
>> >> From: "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>
>> >> To: cfe-dev at cs.uiuc.edu
>> >> Sent: Wednesday, July 2, 2014 5:51:20 PM
>> >> Subject: [cfe-dev] Proposal for an ABI testsuite for clang
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> Hi,
>> >>
>> >> At the SN Systems division of Sony, we have developed an IA64 ABI
>> >> test-suite for clang/llvm based on the lit framework. We would
>> >> like to
>> >> submit this to the LLVM community.
>> >
>> > I am strongly in favor of you submitting this for code review as
>> > soon as you can. A good ABI test suite is something we definitely
>> > should have.
>> >
>> >  -Hal
>> >
>> >> The test-suite currently
>> >> supports clang in both LP64/x86-64 and ILP32/x86 targets, with the
>> >> ability of adding others. The tests perform target-side execution
>> >> and
>> >> work with both cross and native targets.
>> >>
>> >> Please find attached a pdf document that describes the design of
>> >> this
>> >> test-suite. In summary, the test-suite covers:
>> >> · struct layout rules, including bit-fields, · Object layout, base
>> >> classes, vtables, VTTs, construction vtables, · Name mangling, ·
>> >> Contents of typeinfo variables, · Array cookies, · and a few other
>> >> items.
>> >>
>> >> Please consider accepting this test-suite as an llvm project. We
>> >> look
>> >> forward to addressing any comments and questions you have on this
>> >> topic.
>> >>
>> >> Best regards,
>> >>
>> >>
>> >>
>> >> Sunil Srivastava
>> >>
>> >> SN Systems / Sony Computer Entertainment
>> >>
>> >>
>> >> _______________________________________________
>> >> cfe-dev mailing list
>> >> cfe-dev at cs.uiuc.edu
>> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >>
>> >
>> > --
>> > Hal Finkel
>> > Assistant Computational Scientist
>> > Leadership Computing Facility
>> > Argonne National Laboratory
>> >
>> > _______________________________________________
>> > cfe-dev mailing list
>> > cfe-dev at cs.uiuc.edu
>> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory

More information about the cfe-dev mailing list