[cfe-commits] r127525 - in /cfe/trunk: lib/StaticAnalyzer/Checkers/CMakeLists.txt lib/StaticAnalyzer/Checkers/Checkers.td lib/StaticAnalyzer/Checkers/IteratorsChecker.cpp test/Analysis/iterators.cpp

[Chandler Carruth]

On Tue, Mar 29, 2011 at 2:42 PM, Ted Kremenek <kremenek at apple.com> wrote:

> On Mar 29, 2011, at 1:30 PM, Chandler Carruth wrote:
>
> Sorry to dredge up an old thread, but I'm trying to get some policy
> clarification:
>
> On Fri, Mar 11, 2011 at 6:49 PM, Ted Kremenek <kremenek at apple.com> wrote:
>
>> Added: cfe/trunk/test/Analysis/iterators.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Analysis/
>> iterators.cpp?rev=127525&view=auto
>>
>> ==============================================================================
>> --- cfe/trunk/test/Analysis/iterators.cpp (added)
>> +++ cfe/trunk/test/Analysis/iterators.cpp Fri Mar 11 20:49:15 2011
>> @@ -0,0 +1,104 @@
>> +// RUN: %clang_cc1 -analyze -analyzer-checker=core,core.experimental.
>> Iterators -verify %s
>> +
>> +#include <vector>
>>
>
> I thought test cases were expected to be freestanding? Before this one,
> there were only two tests that violate that requirement. Thoughts?
>
> For reference, we build and run all Clang and LLVM tests in a freestanding
> environment out of necessity, so we end up catching these.
>
>
> The problem with making this freestanding is that it doesn't test that the
> checker works on a given platform.
>

I was under the impression that the test/... directory was for regression
and feature tests, not platform tests. In the past, those have typically
been reserved for the LLVM test-suite, although that has no real concept or
framework for testing the analyzer.

I feel like if we want to test how the checker works on a platform (or how
anything else in Clang works on a platform excepting generated code
performance), we should have a dedicated and separate group of tests for
that purpose. When we find bugs, we can then check in just enough code to
reproduce that problem into the regression suite in a freestanding form.

 The definitions of std::vector are allowed to vary, so having tests check
> against the local version actually helps us catch bugs in the checker.
>

Certainly. It seems like it would be good to check in several different
(hopefully greatly reduced) structures of code that represent ways of
implementing the iterator part of vector, although perhaps that is overkill.


> The other problem with making this freestanding is that simply copying the
> required parts from <vector> may not be an option if we want to keep the
> test case free of GPL code (in the case of using the stdc++ provided with
> GCC).  We of course could use the definition from LLVM's libc++, but then we
> wouldn't again be testing against what the user is always using when they
> compile.
>

libc++'s implementation would provide a good starting point for this, but I
would hope it could be reduced to a relatively small amount of code to keep
things focused on the needs of the checker.

I am genuinely interested in a specifically targeted platform tests area.
There are lots of things we don't really test well for that case already. It
would also make a good sanity test for folks packaging clang on a platform.
I suspect we want a test for the iterator checker that isn't dependent on
this though.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20110330/823a91e6/attachment.html>