[cfe-dev] Proposal for an ABI testsuite for clang

[Nico Weber]

Hi Sunil,

looks cool. The included test file example looks "llvm-y" to me :-) A few
questions:

1.) How much of your 400 test files is from the manual abi method, how much
from your test case generator?
2.) Test runtime probably scales fairly well with the number of cores,
right? You say it takes one hour to run on one core, so an 8-core system
should take less than 10 minutes?
3.) How many bugs did you find with this so far?

(I tend to be somewhat cautious about programs that generate exhaustive
tests and then compare the results to golden files. In my experience, they
tend to take a long time to run since they generate many "not interesting"
test cases, and when changing something one has to update many golden files
and then it's easy to miss undesired changes. For an ABI test suite, the
golden files will hopefully not change often though.

I like the approach taken during clang/win bringup: They wrote fuzzers to
generate lots of test cases, and then checked in hand-reduced versions of
the test cases that turned out to be interesting, using the regular clang
testing architecture.)

Nico


On Wed, Jul 2, 2014 at 3:51 PM, Srivastava, Sunil <
sunil_srivastava at playstation.sony.com> wrote:

>  Hi,
>
> At the SN Systems division of Sony, we have developed an IA64 ABI
> test-suite for clang/llvm based on the lit framework.  We would like to
> submit this to the LLVM community. The test-suite currently supports clang
> in both LP64/x86-64 and ILP32/x86 targets, with the ability of adding
> others. The tests perform target-side execution and work with both cross
> and native targets.
>
> Please find attached a pdf document that describes the design of this
> test-suite.  In summary, the test-suite covers:
> ·         struct layout rules, including bit-fields,
> ·         Object layout, base classes, vtables, VTTs, construction
> vtables,
> ·         Name mangling,
> ·         Contents of typeinfo variables,
> ·         Array cookies,
> ·         and a few other items.
>
> Please consider accepting this test-suite as an llvm project. We look
> forward to addressing any comments and questions you have on this topic.
>
> Best regards,
>
>
>
> Sunil Srivastava
>
> SN Systems / Sony Computer Entertainment
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140703/097ea288/attachment.html>