<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">On Fri, Jul 18, 2014 at 12:18 PM, Hal Finkel <span dir="ltr"><<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="">----- Original Message -----<br>
> From: "Eric Christopher" <<a href="mailto:echristo@gmail.com">echristo@gmail.com</a>><br>
> To: "Sunil Srivastava" <<a href="mailto:sunil_srivastava@playstation.sony.com">sunil_srivastava@playstation.sony.com</a>>, "Bob Wilson" <<a href="mailto:bob.wilson@apple.com">bob.wilson@apple.com</a>>, "Daniel Dunbar"<br>
> <<a href="mailto:daniel@zuster.org">daniel@zuster.org</a>>, "Chris Matthews" <<a href="mailto:chris.matthews@apple.com">chris.matthews@apple.com</a>><br>
> Cc: <a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
> Sent: Friday, July 18, 2014 12:37:45 PM<br>
> Subject: Re: [cfe-dev] FW: Proposal for an ABI testsuite for clang<br>
><br>
> On Fri, Jul 18, 2014 at 9:59 AM, Srivastava, Sunil<br>
> <<a href="mailto:sunil_srivastava@playstation.sony.com">sunil_srivastava@playstation.sony.com</a>> wrote:<br>
> > Hi,<br>
> ><br>
> >> At the SN Systems division of Sony, we have developed an IA64 ABI<br>
> >> test-suite for clang/llvm based on the lit framework. We would<br>
> >> like to<br>
> >> submit this to the LLVM community.<br>
> ><br>
> > We are ready to submit first set of test files. I would like some<br>
> > feedback as to where in the tree it should go; how should the<br>
> > review process go.. etc.<br>
> ><br>
><br>
> Personally I think a subdirectory of projects/test-suite would be<br>
> good. It would be nice if all of this could be abstracted around make<br>
> check in some way fitting in there nicely. I.e. when I run the<br>
> testsuite via make TEST=simple check it'll automagically run the ABI<br>
> testsuite as well. This isn't particularly ideal given that the rest<br>
> of the testsuite is more designed around performance etc.<br>
><br>
> Others may have other opinions here. I'll CC a couple of them.<br>
<br>
</div>Given that these are essentially regression tests, and use lit (just like all of our other regression tests), why would we not put them into test/ABI (or similar) along with Clang's other regression tests?<br></blockquote>
<div><br></div><div>Since they add > 1h single core testing time and only find a small amount of bugs compared to that, I too would prefer to not have them in clang's regular test suite.</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="HOEnZb"><font color="#888888"><br>
-Hal<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
><br>
> -eric<br>
><br>
> > I am quoting the README that gives more details giving the<br>
> > structure of the testsuite and instructions to run it.<br>
> ><br>
> > =====================================================<br>
> > This directory contains the IA64-ABI testsuite.<br>
> ><br>
> > To run the testsuite on linux<br>
> ><br>
> > 1) Make sure you have 'lit' in PYTHONOPATH, and FileCheck in PATH.<br>
> > 2) Do<br>
> > python linux-x86.py <compiler+args> test [any lit arguments]<br>
> ><br>
> > <compiler+args> is the compiler to be tested with any options,<br>
> > as one argument; such as<br>
> ><br>
> > python linux-x86.py clang test # clang<br>
> > must be in PATH<br>
> > python linux-x86.py /home/me/bin/my-clang test # my own<br>
> > compiler, not in PATH<br>
> > python linux-x86.py 'clang -m32' test # test 32<br>
> > bit mode<br>
> > python linux-x86.py 'clang -O' test # test with<br>
> > optimization on<br>
> > python linux-x86.py 'gcc' test -v # test gcc,<br>
> > with -v passed to lit<br>
> > python linux-x86.py clang test/basic # test only<br>
> > 'basic' directory<br>
> ><br>
> > linux-x86.py runs a small executable to first figure out whether<br>
> > this is 32 bit or 64 bit<br>
> > platform, and then chooses one of the two configurations for<br>
> > testing.<br>
> ><br>
> > Other, non-linux platforms will require edits to the python script<br>
> > to specify compilers,<br>
> > linker, runner (if any). We will provide a sample file that can be<br>
> > modified to other<br>
> > 'similar' platforms, however it should be noted that the testsuite,<br>
> > as released, supports<br>
> > only x86-linux-like platforms (both ILP32 and LP64).<br>
> ><br>
> > Tests are arranged in multiple directory, starting from 'test'. The<br>
> > python script can be<br>
> > given any subdirectory. The directory 'test/common' contains<br>
> > utility files used by the<br>
> > testsuite.<br>
> ><br>
> > The overview of files is as follows:<br>
> ><br>
> > README.text # This README<br>
> > linux-x86.py # top level python script<br>
> ><br>
> > test/basic/basic.x # A small directory to test<br>
> > that the basic configuration<br>
> > test/basic/T_st.x # is correct. If this fails,<br>
> > everything else will too.<br>
> ><br>
> > test/common/genselector.c # Directory containing<br>
> > utilities used by the testsuite<br>
> > test/common/select2.h<br>
> > test/common/testsuite.c<br>
> > test/common/testsuite.h<br>
> ><br>
> > test/lit.site.cfg # top config file called from<br>
> > lit<br>
> ><br>
> > test/mangling/arrays.xpp # Mangling test. Using<br>
> > FileCheck<br>
> > test/mangling/c++11.xpp<br>
> > test/mangling/c++11s.xpp<br>
> > ....<br>
> ><br>
> > test/misc/s2_8a.xpp # miscellaneous test directory<br>
> > test/misc/s2_8b.x<br>
> > test/misc/s2_9_5.x<br>
> > test/misc/s3_1.xpp<br>
> > ...<br>
> ><br>
> > test/s2_4/T_novirt_1.x # test targeted to particular<br>
> > section of the spec<br>
> > ...<br>
> ><br>
> > test/s2_5/T_vf1.x<br>
> > test/s2_5/T_vf2.x<br>
> > ...<br>
> ><br>
> > test/s2_6/T_isc.x<br>
> > test/s2_6/T_ksc0.x<br>
> > test/s2_6/T_ksc1.x<br>
> > ...<br>
> ><br>
> > test/struct_layout_tests/CT_bf.x # tests for plain struct<br>
> > layout<br>
> > test/struct_layout_tests/CT_Snen_xaa.x<br>
> > test/struct_layout_tests/CT_Snen_xab.x<br>
> > ...<br>
> ><br>
> > test/struct_layout_tests/PACKED/CT_Snen_xaa.x # test for packed<br>
> > structs<br>
> > test/struct_layout_tests/PACKED/CT_Snen_xab.x<br>
> > ...<br>
> ><br>
> > ------------------------------------------------------------------<br>
> ><br>
> > Some points to note:<br>
> ><br>
> > a) This release consists of about 290 files. We plan to release<br>
> > more tests little later.<br>
> > b) As one can see, test files are named with suffixes '.x' and<br>
> > '.xpp'. This is intentional.<br>
> > There is a mechanism, in lit.site.cfg, which copies them to<br>
> > corresponding '.c' or '.cpp'<br>
> > files, looking into a 'skip_list' specified at the top level<br>
> > python file. This mechanism<br>
> > is meant to allow users to mark certain tests 'XFAIL' (expected<br>
> > to fail), on per-file,<br>
> > per-test-script basis.<br>
> > c) All test files are self-contained and independent. They can be<br>
> > removed or moved around to<br>
> > different directories.<br>
> > d) Most files names T_*.x or CT_*.x and 'combined' files having<br>
> > both C and C++ code, separated<br>
> > by '#ifdef __cplusplus'. The '//RUN' header at the top of these<br>
> > files runs both C and C++<br>
> > compiler on them and links the resulting object files to make<br>
> > the executable test.<br>
> ><br>
> ><br>
> > =====================================================<br>
> > -----Original Message-----<br>
> > From: Hal Finkel [mailto:<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>]<br>
> > Sent: Wednesday, July 02, 2014 8:33 PM<br>
> > To: Srivastava, Sunil<br>
> > Cc: <a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
> > Subject: Re: [cfe-dev] Proposal for an ABI testsuite for clang<br>
> ><br>
> > ----- Original Message -----<br>
> >> From: "Sunil Srivastava" <<a href="mailto:sunil_srivastava@playstation.sony.com">sunil_srivastava@playstation.sony.com</a>><br>
> >> To: <a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
> >> Sent: Wednesday, July 2, 2014 5:51:20 PM<br>
> >> Subject: [cfe-dev] Proposal for an ABI testsuite for clang<br>
> >><br>
> >><br>
> >><br>
> >><br>
> >><br>
> >> Hi,<br>
> >><br>
> >> At the SN Systems division of Sony, we have developed an IA64 ABI<br>
> >> test-suite for clang/llvm based on the lit framework. We would<br>
> >> like to<br>
> >> submit this to the LLVM community.<br>
> ><br>
> > I am strongly in favor of you submitting this for code review as<br>
> > soon as you can. A good ABI test suite is something we definitely<br>
> > should have.<br>
> ><br>
> > -Hal<br>
> ><br>
> >> The test-suite currently<br>
> >> supports clang in both LP64/x86-64 and ILP32/x86 targets, with the<br>
> >> ability of adding others. The tests perform target-side execution<br>
> >> and<br>
> >> work with both cross and native targets.<br>
> >><br>
> >> Please find attached a pdf document that describes the design of<br>
> >> this<br>
> >> test-suite. In summary, the test-suite covers:<br>
> >> · struct layout rules, including bit-fields, · Object layout, base<br>
> >> classes, vtables, VTTs, construction vtables, · Name mangling, ·<br>
> >> Contents of typeinfo variables, · Array cookies, · and a few other<br>
> >> items.<br>
> >><br>
> >> Please consider accepting this test-suite as an llvm project. We<br>
> >> look<br>
> >> forward to addressing any comments and questions you have on this<br>
> >> topic.<br>
> >><br>
> >> Best regards,<br>
> >><br>
> >><br>
> >><br>
> >> Sunil Srivastava<br>
> >><br>
> >> SN Systems / Sony Computer Entertainment<br>
> >><br>
> >><br>
> >> _______________________________________________<br>
> >> cfe-dev mailing list<br>
> >> <a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
> >> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br>
> >><br>
> ><br>
> > --<br>
> > Hal Finkel<br>
> > Assistant Computational Scientist<br>
> > Leadership Computing Facility<br>
> > Argonne National Laboratory<br>
> ><br>
> > _______________________________________________<br>
> > cfe-dev mailing list<br>
> > <a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
> > <a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br>
><br>
> _______________________________________________<br>
> cfe-dev mailing list<br>
> <a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br>
><br>
<br>
--<br>
Hal Finkel<br>
Assistant Computational Scientist<br>
Leadership Computing Facility<br>
Argonne National Laboratory<br>
<br>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br>
</div></div></blockquote></div><br></div></div>