[cfe-dev] Proposal for an ABI testsuite for clang

[Sean Silva]

Since the goal is compatibility with a previous version, why does the test
suite itself contain the "right answers" that it checks against? Shouldn't
the "right answers" be determined by a "golden master" version of the
toolchain?

Roughly what I'm saying is instead of "check(..., expected)" instead do
"print(...)". Then compare what is printed with the output produced by the
program when compiled with the reference toolchain. This approach seems
more useful, since:

a) it tests what we actually care about, which is that the code is
compatible with a reference toolchain. At the end of the day, compatibility
with existing compiled code is more important than actually sticking to the
ABI specification.

b) it lets you use whatever baseline you want. E.g. possibly some internal
toolchain that unfortunately shipped with a bug; or the MSVC toolchain,
which would be immediately useful for the folks working on MS ABI
compatibility. In fact, this test suite could easily be used to create an
instant checklist of incompatibilities between *any* two compilers that
claim to be compatible (not even from the same vendor).

c) Allows expansion to any other target or ABI by simply expanding the set
of "every possible ..." to cover any extra details relevant to the new
target or ABI. Hence the test suite is naturally target and ABI agnostic.
Adding new checks for a different ABI automatically strengthens the checks
for all ABI's.

As an example of c), consider the following: Suppose that to extend the
test suite to MS ABI, you find that you need to generate a new code
construct that exercises a particular aspect of the MS mangling which the
existing Itanium cases didn't cover. If you compile this construct with two
different Itanium ABI compilers and the mangled names differ, then that is
an incompatibility.

In this sense, I think the goal of the ABI compatibility test suite should
be to collect/generate source code constructs that exercise a particular
dimension along which two C++ compilers could have an ABI incompatibility.
This is mostly agnostic to the target or the compiler being used (a small
amount of stuff would need some target hooks, like getting a mangled symbol
name, but the bulk should be portable and platform agnostic).

I think that what you guys have already done (the steps you list under "A
set of ‘interesting’ classes for the test-suite is generated by:") is
excellent progress in this direction of collecting examples that exercise
dimensions of potential incompatibility since by construction the Itanium
ABI (or any ABI spec) tries to cover all the relevant dimensions which
could cause an ABI incompatibility.

-- Sean Silva




On Wed, Jul 2, 2014 at 4:51 PM, Srivastava, Sunil <
sunil_srivastava at playstation.sony.com> wrote:

>  Hi,
>
> At the SN Systems division of Sony, we have developed an IA64 ABI
> test-suite for clang/llvm based on the lit framework.  We would like to
> submit this to the LLVM community. The test-suite currently supports clang
> in both LP64/x86-64 and ILP32/x86 targets, with the ability of adding
> others. The tests perform target-side execution and work with both cross
> and native targets.
>
> Please find attached a pdf document that describes the design of this
> test-suite.  In summary, the test-suite covers:
> ·         struct layout rules, including bit-fields,
> ·         Object layout, base classes, vtables, VTTs, construction
> vtables,
> ·         Name mangling,
> ·         Contents of typeinfo variables,
> ·         Array cookies,
> ·         and a few other items.
>
> Please consider accepting this test-suite as an llvm project. We look
> forward to addressing any comments and questions you have on this topic.
>
> Best regards,
>
>
>
> Sunil Srivastava
>
> SN Systems / Sony Computer Entertainment
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140702/fdb35034/attachment.html>