<div dir="ltr"><div>Since the goal is compatibility with a previous version, why does the test suite itself contain the "right answers" that it checks against? Shouldn't the "right answers" be determined by a "golden master" version of the toolchain?<br>
</div><div><br></div><div>Roughly what I'm saying is instead of "check(..., expected)" instead do "print(...)". Then compare what is printed with the output produced by the program when compiled with the reference toolchain. This approach seems more useful, since:</div>
<div><br></div><div>a) it tests what we actually care about, which is that the code is compatible with a reference toolchain. At the end of the day, compatibility with existing compiled code is more important than actually sticking to the ABI specification.<br>
</div><div><br></div><div>b) it lets you use whatever baseline you want. E.g. possibly some internal toolchain that unfortunately shipped with a bug; or the MSVC toolchain, which would be immediately useful for the folks working on MS ABI compatibility. In fact, this test suite could easily be used to create an instant checklist of incompatibilities between *any* two compilers that claim to be compatible (not even from the same vendor).</div>
<div><br></div><div>c) Allows expansion to any other target or ABI by simply expanding the set of "every possible ..." to cover any extra details relevant to the new target or ABI. Hence the test suite is naturally target and ABI agnostic. Adding new checks for a different ABI automatically strengthens the checks for all ABI's.</div>
<div><br></div><div>As an example of c), consider the following: Suppose that to extend the test suite to MS ABI, you find that you need to generate a new code construct that exercises a particular aspect of the MS mangling which the existing Itanium cases didn't cover. If you compile this construct with two different Itanium ABI compilers and the mangled names differ, then that is an incompatibility.</div>
<div><br></div><div>In this sense, I think the goal of the ABI compatibility test suite should be to collect/generate source code constructs that exercise a particular dimension along which two C++ compilers could have an ABI incompatibility. This is mostly agnostic to the target or the compiler being used (a small amount of stuff would need some target hooks, like getting a mangled symbol name, but the bulk should be portable and platform agnostic).</div>
<div><br></div><div>I think that what you guys have already done (the steps you list under "A set of ‘interesting’ classes for the test-suite is generated by:") is excellent progress in this direction of collecting examples that exercise dimensions of potential incompatibility since by construction the Itanium ABI (or any ABI spec) tries to cover all the relevant dimensions which could cause an ABI incompatibility.</div>
<div><br></div><div>-- Sean Silva</div><div><br></div><div><br></div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Wed, Jul 2, 2014 at 4:51 PM, Srivastava, Sunil <span dir="ltr"><<a href="mailto:sunil_srivastava@playstation.sony.com" target="_blank">sunil_srivastava@playstation.sony.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="#0563C1" vlink="#954F72">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:black">Hi,</span><span style="color:black">
<br>
<br>
</span><span style="font-size:10.0pt;color:black">At the SN Systems division of Sony, we have developed an IA64 ABI test-suite for clang/llvm based on the lit framework. We would like to submit this to the LLVM community. The test-suite currently supports
clang in both LP64/x86-64 and ILP32/x86 targets, with the ability of adding others. The tests perform target-side execution and work with both cross and native targets.</span><span style="color:black">
<br>
<br>
</span><span style="font-size:10.0pt;color:black">Please find attached a pdf document that describes the design of this test-suite. In summary, the test-suite covers:</span><span style="color:black">
<br>
</span><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.5pt;color:black">
</span><span style="font-size:10.0pt;color:black">struct layout rules, including bit-fields,</span><span style="color:black">
<br>
</span><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.5pt;color:black">
</span><span style="font-size:10.0pt;color:black">Object layout, base classes, vtables, VTTs, construction vtables,</span><span style="color:black">
<br>
</span><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.5pt;color:black">
</span><span style="font-size:10.0pt;color:black">Name mangling,</span><span style="color:black">
<br>
</span><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.5pt;color:black">
</span><span style="font-size:10.0pt;color:black">Contents of typeinfo variables,</span><span style="color:black">
<br>
</span><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.5pt;color:black">
</span><span style="font-size:10.0pt;color:black">Array cookies,</span><span style="color:black">
<br>
</span><span style="font-size:10.0pt;font-family:Symbol;color:black">·</span><span style="font-size:7.5pt;color:black">
</span><span style="font-size:10.0pt;color:black">and a few other items.</span><span style="color:black">
<br>
<br>
</span><span style="font-size:10.0pt;color:black">Please consider accepting this test-suite as an llvm project. We look forward to addressing any comments and questions you have on this topic.</span><span style="color:black">
<br>
<br>
</span><span style="font-size:10.0pt">Best regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">Sunil Srivastava<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt">SN Systems / Sony Computer Entertainment<u></u><u></u></span></p>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
<br>_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@cs.uiuc.edu">cfe-dev@cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev</a><br>
<br></blockquote></div><br></div>