[cfe-dev] FW: Proposal for an ABI testsuite for clang
Eric Christopher
echristo at gmail.com
Fri Jul 18 12:35:23 PDT 2014
On Fri, Jul 18, 2014 at 12:26 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> ----- Original Message -----
>> From: "Eric Christopher" <echristo at gmail.com>
>> To: "Hal Finkel" <hfinkel at anl.gov>
>> Cc: cfe-dev at cs.uiuc.edu, "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>, "Bob Wilson"
>> <bob.wilson at apple.com>, "Daniel Dunbar" <daniel at zuster.org>, "Chris Matthews" <chris.matthews at apple.com>
>> Sent: Friday, July 18, 2014 2:24:21 PM
>> Subject: Re: [cfe-dev] FW: Proposal for an ABI testsuite for clang
>>
>> Was under the impression there were execution tests in there, if not,
>> then I agree that test/ABI would make the most sense. The ability to
>> do things via gcc for it would make a bit more sense to be in
>> projects/test-suite.
>
> I think they might be, but we have lit execution regression tests for the saniitizers in compiler-rt, so I don't see why we can't copy the relevant lit infrastructure and have them there as well?
>
We could, I suppose, but it's been a hard rule that execution tests
weren't in those testsuites up to now. Do we have a reason to put them
there? I don't really have a firm opinion here, but very few people
had replied otherwise.
-eric
> -Hal
>
>>
>> -eric
>>
>> On Fri, Jul 18, 2014 at 12:18 PM, Hal Finkel <hfinkel at anl.gov> wrote:
>> > ----- Original Message -----
>> >> From: "Eric Christopher" <echristo at gmail.com>
>> >> To: "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>,
>> >> "Bob Wilson" <bob.wilson at apple.com>, "Daniel Dunbar"
>> >> <daniel at zuster.org>, "Chris Matthews" <chris.matthews at apple.com>
>> >> Cc: cfe-dev at cs.uiuc.edu
>> >> Sent: Friday, July 18, 2014 12:37:45 PM
>> >> Subject: Re: [cfe-dev] FW: Proposal for an ABI testsuite for clang
>> >>
>> >> On Fri, Jul 18, 2014 at 9:59 AM, Srivastava, Sunil
>> >> <sunil_srivastava at playstation.sony.com> wrote:
>> >> > Hi,
>> >> >
>> >> >> At the SN Systems division of Sony, we have developed an IA64
>> >> >> ABI
>> >> >> test-suite for clang/llvm based on the lit framework. We would
>> >> >> like to
>> >> >> submit this to the LLVM community.
>> >> >
>> >> > We are ready to submit first set of test files. I would like
>> >> > some
>> >> > feedback as to where in the tree it should go; how should the
>> >> > review process go.. etc.
>> >> >
>> >>
>> >> Personally I think a subdirectory of projects/test-suite would be
>> >> good. It would be nice if all of this could be abstracted around
>> >> make
>> >> check in some way fitting in there nicely. I.e. when I run the
>> >> testsuite via make TEST=simple check it'll automagically run the
>> >> ABI
>> >> testsuite as well. This isn't particularly ideal given that the
>> >> rest
>> >> of the testsuite is more designed around performance etc.
>> >>
>> >> Others may have other opinions here. I'll CC a couple of them.
>> >
>> > Given that these are essentially regression tests, and use lit
>> > (just like all of our other regression tests), why would we not
>> > put them into test/ABI (or similar) along with Clang's other
>> > regression tests?
>> >
>> > -Hal
>> >
>> >>
>> >> -eric
>> >>
>> >> > I am quoting the README that gives more details giving the
>> >> > structure of the testsuite and instructions to run it.
>> >> >
>> >> > =====================================================
>> >> > This directory contains the IA64-ABI testsuite.
>> >> >
>> >> > To run the testsuite on linux
>> >> >
>> >> > 1) Make sure you have 'lit' in PYTHONOPATH, and FileCheck in
>> >> > PATH.
>> >> > 2) Do
>> >> > python linux-x86.py <compiler+args> test [any lit
>> >> > arguments]
>> >> >
>> >> > <compiler+args> is the compiler to be tested with any
>> >> > options,
>> >> > as one argument; such as
>> >> >
>> >> > python linux-x86.py clang test # clang
>> >> > must be in PATH
>> >> > python linux-x86.py /home/me/bin/my-clang test # my own
>> >> > compiler, not in PATH
>> >> > python linux-x86.py 'clang -m32' test # test
>> >> > 32
>> >> > bit mode
>> >> > python linux-x86.py 'clang -O' test # test
>> >> > with
>> >> > optimization on
>> >> > python linux-x86.py 'gcc' test -v # test
>> >> > gcc,
>> >> > with -v passed to lit
>> >> > python linux-x86.py clang test/basic # test
>> >> > only
>> >> > 'basic' directory
>> >> >
>> >> > linux-x86.py runs a small executable to first figure out whether
>> >> > this is 32 bit or 64 bit
>> >> > platform, and then chooses one of the two configurations for
>> >> > testing.
>> >> >
>> >> > Other, non-linux platforms will require edits to the python
>> >> > script
>> >> > to specify compilers,
>> >> > linker, runner (if any). We will provide a sample file that can
>> >> > be
>> >> > modified to other
>> >> > 'similar' platforms, however it should be noted that the
>> >> > testsuite,
>> >> > as released, supports
>> >> > only x86-linux-like platforms (both ILP32 and LP64).
>> >> >
>> >> > Tests are arranged in multiple directory, starting from 'test'.
>> >> > The
>> >> > python script can be
>> >> > given any subdirectory. The directory 'test/common' contains
>> >> > utility files used by the
>> >> > testsuite.
>> >> >
>> >> > The overview of files is as follows:
>> >> >
>> >> > README.text # This README
>> >> > linux-x86.py # top level python script
>> >> >
>> >> > test/basic/basic.x # A small directory to test
>> >> > that the basic configuration
>> >> > test/basic/T_st.x # is correct. If this fails,
>> >> > everything else will too.
>> >> >
>> >> > test/common/genselector.c # Directory containing
>> >> > utilities used by the testsuite
>> >> > test/common/select2.h
>> >> > test/common/testsuite.c
>> >> > test/common/testsuite.h
>> >> >
>> >> > test/lit.site.cfg # top config file called
>> >> > from
>> >> > lit
>> >> >
>> >> > test/mangling/arrays.xpp # Mangling test. Using
>> >> > FileCheck
>> >> > test/mangling/c++11.xpp
>> >> > test/mangling/c++11s.xpp
>> >> > ....
>> >> >
>> >> > test/misc/s2_8a.xpp # miscellaneous test
>> >> > directory
>> >> > test/misc/s2_8b.x
>> >> > test/misc/s2_9_5.x
>> >> > test/misc/s3_1.xpp
>> >> > ...
>> >> >
>> >> > test/s2_4/T_novirt_1.x # test targeted to
>> >> > particular
>> >> > section of the spec
>> >> > ...
>> >> >
>> >> > test/s2_5/T_vf1.x
>> >> > test/s2_5/T_vf2.x
>> >> > ...
>> >> >
>> >> > test/s2_6/T_isc.x
>> >> > test/s2_6/T_ksc0.x
>> >> > test/s2_6/T_ksc1.x
>> >> > ...
>> >> >
>> >> > test/struct_layout_tests/CT_bf.x # tests for plain struct
>> >> > layout
>> >> > test/struct_layout_tests/CT_Snen_xaa.x
>> >> > test/struct_layout_tests/CT_Snen_xab.x
>> >> > ...
>> >> >
>> >> > test/struct_layout_tests/PACKED/CT_Snen_xaa.x # test for
>> >> > packed
>> >> > structs
>> >> > test/struct_layout_tests/PACKED/CT_Snen_xab.x
>> >> > ...
>> >> >
>> >> > ------------------------------------------------------------------
>> >> >
>> >> > Some points to note:
>> >> >
>> >> > a) This release consists of about 290 files. We plan to release
>> >> > more tests little later.
>> >> > b) As one can see, test files are named with suffixes '.x' and
>> >> > '.xpp'. This is intentional.
>> >> > There is a mechanism, in lit.site.cfg, which copies them to
>> >> > corresponding '.c' or '.cpp'
>> >> > files, looking into a 'skip_list' specified at the top level
>> >> > python file. This mechanism
>> >> > is meant to allow users to mark certain tests 'XFAIL'
>> >> > (expected
>> >> > to fail), on per-file,
>> >> > per-test-script basis.
>> >> > c) All test files are self-contained and independent. They can
>> >> > be
>> >> > removed or moved around to
>> >> > different directories.
>> >> > d) Most files names T_*.x or CT_*.x and 'combined' files having
>> >> > both C and C++ code, separated
>> >> > by '#ifdef __cplusplus'. The '//RUN' header at the top of
>> >> > these
>> >> > files runs both C and C++
>> >> > compiler on them and links the resulting object files to make
>> >> > the executable test.
>> >> >
>> >> >
>> >> > =====================================================
>> >> > -----Original Message-----
>> >> > From: Hal Finkel [mailto:hfinkel at anl.gov]
>> >> > Sent: Wednesday, July 02, 2014 8:33 PM
>> >> > To: Srivastava, Sunil
>> >> > Cc: cfe-dev at cs.uiuc.edu
>> >> > Subject: Re: [cfe-dev] Proposal for an ABI testsuite for clang
>> >> >
>> >> > ----- Original Message -----
>> >> >> From: "Sunil Srivastava"
>> >> >> <sunil_srivastava at playstation.sony.com>
>> >> >> To: cfe-dev at cs.uiuc.edu
>> >> >> Sent: Wednesday, July 2, 2014 5:51:20 PM
>> >> >> Subject: [cfe-dev] Proposal for an ABI testsuite for clang
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >>
>> >> >> Hi,
>> >> >>
>> >> >> At the SN Systems division of Sony, we have developed an IA64
>> >> >> ABI
>> >> >> test-suite for clang/llvm based on the lit framework. We would
>> >> >> like to
>> >> >> submit this to the LLVM community.
>> >> >
>> >> > I am strongly in favor of you submitting this for code review as
>> >> > soon as you can. A good ABI test suite is something we
>> >> > definitely
>> >> > should have.
>> >> >
>> >> > -Hal
>> >> >
>> >> >> The test-suite currently
>> >> >> supports clang in both LP64/x86-64 and ILP32/x86 targets, with
>> >> >> the
>> >> >> ability of adding others. The tests perform target-side
>> >> >> execution
>> >> >> and
>> >> >> work with both cross and native targets.
>> >> >>
>> >> >> Please find attached a pdf document that describes the design
>> >> >> of
>> >> >> this
>> >> >> test-suite. In summary, the test-suite covers:
>> >> >> · struct layout rules, including bit-fields, · Object layout,
>> >> >> base
>> >> >> classes, vtables, VTTs, construction vtables, · Name mangling,
>> >> >> ·
>> >> >> Contents of typeinfo variables, · Array cookies, · and a few
>> >> >> other
>> >> >> items.
>> >> >>
>> >> >> Please consider accepting this test-suite as an llvm project.
>> >> >> We
>> >> >> look
>> >> >> forward to addressing any comments and questions you have on
>> >> >> this
>> >> >> topic.
>> >> >>
>> >> >> Best regards,
>> >> >>
>> >> >>
>> >> >>
>> >> >> Sunil Srivastava
>> >> >>
>> >> >> SN Systems / Sony Computer Entertainment
>> >> >>
>> >> >>
>> >> >> _______________________________________________
>> >> >> cfe-dev mailing list
>> >> >> cfe-dev at cs.uiuc.edu
>> >> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >> >>
>> >> >
>> >> > --
>> >> > Hal Finkel
>> >> > Assistant Computational Scientist
>> >> > Leadership Computing Facility
>> >> > Argonne National Laboratory
>> >> >
>> >> > _______________________________________________
>> >> > cfe-dev mailing list
>> >> > cfe-dev at cs.uiuc.edu
>> >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >>
>> >> _______________________________________________
>> >> cfe-dev mailing list
>> >> cfe-dev at cs.uiuc.edu
>> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>> >>
>> >
>> > --
>> > Hal Finkel
>> > Assistant Computational Scientist
>> > Leadership Computing Facility
>> > Argonne National Laboratory
>>
>
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
More information about the cfe-dev
mailing list