[LLVMdev] Any objections to my importing GoogleMock to go with GoogleTest in LLVM?

[Chris Lattner]

On Nov 14, 2013, at 3:16 AM, Chandler Carruth <chandlerc at google.com> wrote:
> Personally, I rather not do this, without very clear and compelling reasons.
> 
> I understand that this could be very useful for your bringup (and so could be very useful locally), but once the passmanager is the default, it will get lost of in-tree testing by just about everything in the compiler.
> 
> I would much rather have it in the tree than just use it locally. I think it will also make subsequent iterations much easier to test and show are correct. I think it would also allow significantly more precise regression testing in the future.
> 
> This also isn't the first time I've wanted it in LLVM and in Clang. It's just the first time I've been working on something large enoguh to feel like importing it would be worth the cost.

It is always worth reevaluating.

> My feeling is that both gtest and gmock suffer from the same flaw: they can easily be overused or misused in circumstances where there are clearly better ways to go about things. However, I feel like within LLVM we have been really good at pushing back against that and using integration tests with excellent tool support (how I love FileCheck) much more prevalently. As long as we continue to code review unittests with an eye toward skepticism, I think there is very little risk of things getting out of hand. I think adding gmock to gtest doesn't shift that risk in any significant way.
> 
> However, when we are adding interfaces or generic utilities to LLVM (admittedly, not the common case) I don't think we do ourselves any favors by using only half of the available tools to write unit tests for them.

I agree in principle, but it leads me to a different conclusion.  We have other great testing support, which means that the mocking *should* only be used sparingly.  Given that it will not be used much, the cost of carrying it around (and for people to learn how to use/maintain it) is high.

I’ve said this before, but I’m not a fan of our current use of gtest for unit testing.  I have never had the unit tests catch a bug, but I have had to update the tests countless times.  At least for my purposes, the unit tests cause significantly more harm than good - and it certainly isn’t because I write perfect code. :-)

There is definitely a culture/religion around testing and TDD, and I am well aware that many projects don’t have proper tests (which LLVM doesn’t suffer from).  However, there is a pragmatic balance to be struck here, and I personally think that adding gmock and pushing the unit tests stuff even further is a bad use of testing time (i.e., increases test cycles for make check) and maintenance time (updating tests given that we don’t have a stable API).

-Chris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20131114/15e5f4cb/attachment.html>