[cfe-dev] Tests for Clang-tidy STL containers checks

[Eugene Zelenko via cfe-dev]

Hi, Alexander!

On Wed, Dec 16, 2015 at 5:53 AM, Alexander Kornienko <alexfh at google.com> wrote:
> We mock out STL containers for multiple reasons. One of them is to make the
> tests hermetic (as in "independent from the environment"). The developer has
> to make the mocks appropriately model the implementation of the standard
> library. If another implementation is sufficiently different, we need to add
> mocks for that implementation as well. Making the tests use actual library
> headers is problematic, since it will significantly increase testing time
> and complicate integration with different tests environments. So I'm
> strongly against this. We could make a mode where the actual headers are
> used instead of mock implementations, but we're not going to remove mocks
> completely.

Problem with hermetic tests that they are testing mocks, not real
implementations. I caught problems manually because I was lucky to
spot pattern which was supposed to be recognized by
readability-container-size-empty, but it was not. But this should be
caught by regular automated regressions to maintain reliability of
Clang-tidy checks.

I don't think that including STL headers will increase tests time significantly.

> Please file bugs for the specific issues you find. If you can isolate
> specific differences between library implementations that result in
> incorrect work of the checks (and maybe create the appropriate mocks for the
> library you use), it would help check developers to fix their checks.

I already filed bugs 25804, 25812 and 25813 and developers took care
about them, through problems were not fixed completely yet.

Eugene.