[cfe-dev] FW: Proposal for an ABI testsuite for clang
Hal Finkel
hfinkel at anl.gov
Fri Jul 18 12:44:32 PDT 2014
----- Original Message -----
> From: "Eric Christopher" <echristo at gmail.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: cfe-dev at cs.uiuc.edu, "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>, "Bob Wilson"
> <bob.wilson at apple.com>, "Daniel Dunbar" <daniel at zuster.org>, "Chris Matthews" <chris.matthews at apple.com>
> Sent: Friday, July 18, 2014 2:35:23 PM
> Subject: Re: [cfe-dev] FW: Proposal for an ABI testsuite for clang
>
> On Fri, Jul 18, 2014 at 12:26 PM, Hal Finkel <hfinkel at anl.gov> wrote:
> > ----- Original Message -----
> >> From: "Eric Christopher" <echristo at gmail.com>
> >> To: "Hal Finkel" <hfinkel at anl.gov>
> >> Cc: cfe-dev at cs.uiuc.edu, "Sunil Srivastava"
> >> <sunil_srivastava at playstation.sony.com>, "Bob Wilson"
> >> <bob.wilson at apple.com>, "Daniel Dunbar" <daniel at zuster.org>,
> >> "Chris Matthews" <chris.matthews at apple.com>
> >> Sent: Friday, July 18, 2014 2:24:21 PM
> >> Subject: Re: [cfe-dev] FW: Proposal for an ABI testsuite for clang
> >>
> >> Was under the impression there were execution tests in there, if
> >> not,
> >> then I agree that test/ABI would make the most sense. The ability
> >> to
> >> do things via gcc for it would make a bit more sense to be in
> >> projects/test-suite.
> >
> > I think they might be, but we have lit execution regression tests
> > for the saniitizers in compiler-rt, so I don't see why we can't
> > copy the relevant lit infrastructure and have them there as well?
> >
>
> We could, I suppose, but it's been a hard rule that execution tests
> weren't in those testsuites up to now.
True. (although I was under some impression this was more about execution time than anything else)
> Do we have a reason to put
> them
> there?
Only that, to me at least, they seem to logically fit (and fit in terms of infrastructure).
That having been said, maybe this can be the start of a lit conversion of the test-suite in general, and that might not be a bad thing either.
I also don't have a strong opinion either way.
-Hal
I don't really have a firm opinion here, but very few people
> had replied otherwise.
>
> -eric
>
> > -Hal
> >
> >>
> >> -eric
> >>
> >> On Fri, Jul 18, 2014 at 12:18 PM, Hal Finkel <hfinkel at anl.gov>
> >> wrote:
> >> > ----- Original Message -----
> >> >> From: "Eric Christopher" <echristo at gmail.com>
> >> >> To: "Sunil Srivastava" <sunil_srivastava at playstation.sony.com>,
> >> >> "Bob Wilson" <bob.wilson at apple.com>, "Daniel Dunbar"
> >> >> <daniel at zuster.org>, "Chris Matthews"
> >> >> <chris.matthews at apple.com>
> >> >> Cc: cfe-dev at cs.uiuc.edu
> >> >> Sent: Friday, July 18, 2014 12:37:45 PM
> >> >> Subject: Re: [cfe-dev] FW: Proposal for an ABI testsuite for
> >> >> clang
> >> >>
> >> >> On Fri, Jul 18, 2014 at 9:59 AM, Srivastava, Sunil
> >> >> <sunil_srivastava at playstation.sony.com> wrote:
> >> >> > Hi,
> >> >> >
> >> >> >> At the SN Systems division of Sony, we have developed an
> >> >> >> IA64
> >> >> >> ABI
> >> >> >> test-suite for clang/llvm based on the lit framework. We
> >> >> >> would
> >> >> >> like to
> >> >> >> submit this to the LLVM community.
> >> >> >
> >> >> > We are ready to submit first set of test files. I would like
> >> >> > some
> >> >> > feedback as to where in the tree it should go; how should the
> >> >> > review process go.. etc.
> >> >> >
> >> >>
> >> >> Personally I think a subdirectory of projects/test-suite would
> >> >> be
> >> >> good. It would be nice if all of this could be abstracted
> >> >> around
> >> >> make
> >> >> check in some way fitting in there nicely. I.e. when I run the
> >> >> testsuite via make TEST=simple check it'll automagically run
> >> >> the
> >> >> ABI
> >> >> testsuite as well. This isn't particularly ideal given that the
> >> >> rest
> >> >> of the testsuite is more designed around performance etc.
> >> >>
> >> >> Others may have other opinions here. I'll CC a couple of them.
> >> >
> >> > Given that these are essentially regression tests, and use lit
> >> > (just like all of our other regression tests), why would we not
> >> > put them into test/ABI (or similar) along with Clang's other
> >> > regression tests?
> >> >
> >> > -Hal
> >> >
> >> >>
> >> >> -eric
> >> >>
> >> >> > I am quoting the README that gives more details giving the
> >> >> > structure of the testsuite and instructions to run it.
> >> >> >
> >> >> > =====================================================
> >> >> > This directory contains the IA64-ABI testsuite.
> >> >> >
> >> >> > To run the testsuite on linux
> >> >> >
> >> >> > 1) Make sure you have 'lit' in PYTHONOPATH, and FileCheck in
> >> >> > PATH.
> >> >> > 2) Do
> >> >> > python linux-x86.py <compiler+args> test [any lit
> >> >> > arguments]
> >> >> >
> >> >> > <compiler+args> is the compiler to be tested with any
> >> >> > options,
> >> >> > as one argument; such as
> >> >> >
> >> >> > python linux-x86.py clang test #
> >> >> > clang
> >> >> > must be in PATH
> >> >> > python linux-x86.py /home/me/bin/my-clang test # my
> >> >> > own
> >> >> > compiler, not in PATH
> >> >> > python linux-x86.py 'clang -m32' test #
> >> >> > test
> >> >> > 32
> >> >> > bit mode
> >> >> > python linux-x86.py 'clang -O' test #
> >> >> > test
> >> >> > with
> >> >> > optimization on
> >> >> > python linux-x86.py 'gcc' test -v #
> >> >> > test
> >> >> > gcc,
> >> >> > with -v passed to lit
> >> >> > python linux-x86.py clang test/basic #
> >> >> > test
> >> >> > only
> >> >> > 'basic' directory
> >> >> >
> >> >> > linux-x86.py runs a small executable to first figure out
> >> >> > whether
> >> >> > this is 32 bit or 64 bit
> >> >> > platform, and then chooses one of the two configurations for
> >> >> > testing.
> >> >> >
> >> >> > Other, non-linux platforms will require edits to the python
> >> >> > script
> >> >> > to specify compilers,
> >> >> > linker, runner (if any). We will provide a sample file that
> >> >> > can
> >> >> > be
> >> >> > modified to other
> >> >> > 'similar' platforms, however it should be noted that the
> >> >> > testsuite,
> >> >> > as released, supports
> >> >> > only x86-linux-like platforms (both ILP32 and LP64).
> >> >> >
> >> >> > Tests are arranged in multiple directory, starting from
> >> >> > 'test'.
> >> >> > The
> >> >> > python script can be
> >> >> > given any subdirectory. The directory 'test/common' contains
> >> >> > utility files used by the
> >> >> > testsuite.
> >> >> >
> >> >> > The overview of files is as follows:
> >> >> >
> >> >> > README.text # This README
> >> >> > linux-x86.py # top level python script
> >> >> >
> >> >> > test/basic/basic.x # A small directory to
> >> >> > test
> >> >> > that the basic configuration
> >> >> > test/basic/T_st.x # is correct. If this
> >> >> > fails,
> >> >> > everything else will too.
> >> >> >
> >> >> > test/common/genselector.c # Directory containing
> >> >> > utilities used by the testsuite
> >> >> > test/common/select2.h
> >> >> > test/common/testsuite.c
> >> >> > test/common/testsuite.h
> >> >> >
> >> >> > test/lit.site.cfg # top config file called
> >> >> > from
> >> >> > lit
> >> >> >
> >> >> > test/mangling/arrays.xpp # Mangling test. Using
> >> >> > FileCheck
> >> >> > test/mangling/c++11.xpp
> >> >> > test/mangling/c++11s.xpp
> >> >> > ....
> >> >> >
> >> >> > test/misc/s2_8a.xpp # miscellaneous test
> >> >> > directory
> >> >> > test/misc/s2_8b.x
> >> >> > test/misc/s2_9_5.x
> >> >> > test/misc/s3_1.xpp
> >> >> > ...
> >> >> >
> >> >> > test/s2_4/T_novirt_1.x # test targeted to
> >> >> > particular
> >> >> > section of the spec
> >> >> > ...
> >> >> >
> >> >> > test/s2_5/T_vf1.x
> >> >> > test/s2_5/T_vf2.x
> >> >> > ...
> >> >> >
> >> >> > test/s2_6/T_isc.x
> >> >> > test/s2_6/T_ksc0.x
> >> >> > test/s2_6/T_ksc1.x
> >> >> > ...
> >> >> >
> >> >> > test/struct_layout_tests/CT_bf.x # tests for plain struct
> >> >> > layout
> >> >> > test/struct_layout_tests/CT_Snen_xaa.x
> >> >> > test/struct_layout_tests/CT_Snen_xab.x
> >> >> > ...
> >> >> >
> >> >> > test/struct_layout_tests/PACKED/CT_Snen_xaa.x # test for
> >> >> > packed
> >> >> > structs
> >> >> > test/struct_layout_tests/PACKED/CT_Snen_xab.x
> >> >> > ...
> >> >> >
> >> >> > ------------------------------------------------------------------
> >> >> >
> >> >> > Some points to note:
> >> >> >
> >> >> > a) This release consists of about 290 files. We plan to
> >> >> > release
> >> >> > more tests little later.
> >> >> > b) As one can see, test files are named with suffixes '.x'
> >> >> > and
> >> >> > '.xpp'. This is intentional.
> >> >> > There is a mechanism, in lit.site.cfg, which copies them
> >> >> > to
> >> >> > corresponding '.c' or '.cpp'
> >> >> > files, looking into a 'skip_list' specified at the top
> >> >> > level
> >> >> > python file. This mechanism
> >> >> > is meant to allow users to mark certain tests 'XFAIL'
> >> >> > (expected
> >> >> > to fail), on per-file,
> >> >> > per-test-script basis.
> >> >> > c) All test files are self-contained and independent. They
> >> >> > can
> >> >> > be
> >> >> > removed or moved around to
> >> >> > different directories.
> >> >> > d) Most files names T_*.x or CT_*.x and 'combined' files
> >> >> > having
> >> >> > both C and C++ code, separated
> >> >> > by '#ifdef __cplusplus'. The '//RUN' header at the top of
> >> >> > these
> >> >> > files runs both C and C++
> >> >> > compiler on them and links the resulting object files to
> >> >> > make
> >> >> > the executable test.
> >> >> >
> >> >> >
> >> >> > =====================================================
> >> >> > -----Original Message-----
> >> >> > From: Hal Finkel [mailto:hfinkel at anl.gov]
> >> >> > Sent: Wednesday, July 02, 2014 8:33 PM
> >> >> > To: Srivastava, Sunil
> >> >> > Cc: cfe-dev at cs.uiuc.edu
> >> >> > Subject: Re: [cfe-dev] Proposal for an ABI testsuite for
> >> >> > clang
> >> >> >
> >> >> > ----- Original Message -----
> >> >> >> From: "Sunil Srivastava"
> >> >> >> <sunil_srivastava at playstation.sony.com>
> >> >> >> To: cfe-dev at cs.uiuc.edu
> >> >> >> Sent: Wednesday, July 2, 2014 5:51:20 PM
> >> >> >> Subject: [cfe-dev] Proposal for an ABI testsuite for clang
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> At the SN Systems division of Sony, we have developed an
> >> >> >> IA64
> >> >> >> ABI
> >> >> >> test-suite for clang/llvm based on the lit framework. We
> >> >> >> would
> >> >> >> like to
> >> >> >> submit this to the LLVM community.
> >> >> >
> >> >> > I am strongly in favor of you submitting this for code review
> >> >> > as
> >> >> > soon as you can. A good ABI test suite is something we
> >> >> > definitely
> >> >> > should have.
> >> >> >
> >> >> > -Hal
> >> >> >
> >> >> >> The test-suite currently
> >> >> >> supports clang in both LP64/x86-64 and ILP32/x86 targets,
> >> >> >> with
> >> >> >> the
> >> >> >> ability of adding others. The tests perform target-side
> >> >> >> execution
> >> >> >> and
> >> >> >> work with both cross and native targets.
> >> >> >>
> >> >> >> Please find attached a pdf document that describes the
> >> >> >> design
> >> >> >> of
> >> >> >> this
> >> >> >> test-suite. In summary, the test-suite covers:
> >> >> >> · struct layout rules, including bit-fields, · Object
> >> >> >> layout,
> >> >> >> base
> >> >> >> classes, vtables, VTTs, construction vtables, · Name
> >> >> >> mangling,
> >> >> >> ·
> >> >> >> Contents of typeinfo variables, · Array cookies, · and a few
> >> >> >> other
> >> >> >> items.
> >> >> >>
> >> >> >> Please consider accepting this test-suite as an llvm
> >> >> >> project.
> >> >> >> We
> >> >> >> look
> >> >> >> forward to addressing any comments and questions you have on
> >> >> >> this
> >> >> >> topic.
> >> >> >>
> >> >> >> Best regards,
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> Sunil Srivastava
> >> >> >>
> >> >> >> SN Systems / Sony Computer Entertainment
> >> >> >>
> >> >> >>
> >> >> >> _______________________________________________
> >> >> >> cfe-dev mailing list
> >> >> >> cfe-dev at cs.uiuc.edu
> >> >> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> >> >> >>
> >> >> >
> >> >> > --
> >> >> > Hal Finkel
> >> >> > Assistant Computational Scientist
> >> >> > Leadership Computing Facility
> >> >> > Argonne National Laboratory
> >> >> >
> >> >> > _______________________________________________
> >> >> > cfe-dev mailing list
> >> >> > cfe-dev at cs.uiuc.edu
> >> >> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> >> >>
> >> >> _______________________________________________
> >> >> cfe-dev mailing list
> >> >> cfe-dev at cs.uiuc.edu
> >> >> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> >> >>
> >> >
> >> > --
> >> > Hal Finkel
> >> > Assistant Computational Scientist
> >> > Leadership Computing Facility
> >> > Argonne National Laboratory
> >>
> >
> > --
> > Hal Finkel
> > Assistant Computational Scientist
> > Leadership Computing Facility
> > Argonne National Laboratory
>
--
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory
More information about the cfe-dev
mailing list