[cfe-dev] A tool for generating reduced headers for test cases (delta debugging)
kremenek at apple.com
Wed Jul 9 09:46:42 PDT 2008
On Jul 9, 2008, at 9:16 AM, Eli Friedman wrote:
> On Wed, Jul 9, 2008 at 1:49 AM, Nikita Zhuk <nikita at zhuk.fi> wrote:
>> Several clang analysis test cases written in ObjC contain the
>> following comment line in their header: "These declarations were
>> reduced using Delta-Debugging from Foundation.h on Mac OS X.". Are
>> there some instructions about tools which should be used to generate
>> such reduced headers?
> If I'm not mistaken, the steps are just to generate a full file with
> clang -E, then reduce the file using Delta (http://delta.tigris.org/).
> But Ted should be able to comment with more complete steps.
That's exactly right.
The Delta tool requires a user-provided script that discerns a "test
case" between good and bad, and tries to remove pieces from the test
case (which is any text file) until a minimal test case is generated
that is still "good".
In the case of header file reduction, I'm working with two files:
t.i (the preprocessed header goop that I want to reduce)
t.c (the file that includes t.i)
The "test case" is t.i, and the my script (called "detect_error.pl")
classifies a version of t.i as good or bad by doing the following:
1) If "gcc -fsyntax-only t.c" fails, the test case is BAD.
2) if "clang -fsyntax-only t.c" fails, the case is BAD.
3) Otherwise, the test case is GOOD.
The trick is identifying what is failure: gcc emiting warnings, gcc or
clang crashing, etc.
Delta-debugging then whittles down t.i until it cannot make it any
smaller. Once it reduces t.i, I just include the contents of t.i
inside t.c (or t.m, or whatever).
>> Being able to do so would make it possible to
>> report bugs by using runnable test cases, and those test cases could
>> be eventually included with the clang itself.
I can certainly clean up the detect_error.pl script and post it the
website with directions.
> Reducing out Foundation.h only really helps for non-runnable
> testcases; the presence of Foundation.h is a pretty good indicator of
> the presence of the Foundation classes. :)
Exactly. In general, header reduction might be good for filing any
kind of test case against Clang that depends on system/library-
More information about the cfe-dev