[LLVMdev] [Patch] Adding unit tests to LLVM

Sat Dec 27 19:58:05 PST 2008

On Dec 27, 2008, at 10:33 PM, Misha Brukman wrote:

> 2008/12/27 Mark Kromis <greybird at mac.com>
> So are you planning on maintaining whatever test system, or just  
> have them as a pre-requisite. For example are you going to have the  
> gtest incorporated, or have them install it separately first? I was  
> under the impression that the user would have to install gtest first.
>
> The current plan is to check in the unittest library into LLVM and  
> build it as part of the test process, so the user doesn't have to do  
> anything separately.
>
>
>> Boost also seems to want to use exceptions, and LLVM does not want  
>> to.  I'm not sure if there would be some difficulties in running a  
>> build where some libraries are compiled with no exceptions, some  
>> with, and the results are linked together.  At the best case, it  
>> would complicate our build system to be able to support different  
>> set of flags for building LLVM libraries vs. Boost.Test (and the  
>> rest of Boost that we import).
>
> http://www.boost.org/doc/libs/1_37_0/libs/utility/throw_exception.html
> #define BOOST_NO_EXCEPTIONS
>
> Thanks for the pointer.
>> Sample usage of Boost.Test: http://svn.boost.org/svn/boost/trunk/libs/test/example/unit_test_example_12.cpp
>> Note the code at the end setting up the test suite -- this is  
>> boilerplate code that I think shouldn't be necessary to setup and  
>> run tests.
>
> http://svn.boost.org/svn/boost/trunk/libs/test/example/unit_test_example_01.cpp
> My test cases are not that in-depth, I'm much closer to sample 1. I  
> haven't found a reason to go that crazy yet.
>
> You would need to use fixtures if you are testing classes and their  
> interactions, otherwise each of your test functions will re-do  
> everything that's in the fixture's SetUp() method -- why would you  
> want to repeat the same setup code for every test case?

They have something similar, non obvious though.
http://gamesfromwithin.com/?p=29#comment-47

>
>> Google Test, on the other hand, has no external dependencies, and  
>> is distributed as a dozen of .h/.cc files; supports Makefile,  
>> SCons, and Xcode; and doesn't use exceptions or RTTI.
>
> Gtest is much more lightweight, no comparison there. I know that  
> llvm is not very good with exceptions, but should a test case system  
> support that?
>
> GTest allows testing for exceptions, it just doesn't require them to  
> work properly:
> http://code.google.com/p/googletest/wiki/GoogleTestAdvancedGuide#Exception_Assertions

ah ok. I misunderstood it then.

>
>> Sample usage of GTest: http://code.google.com/p/googletest/source/browse/trunk/samples/sample5_unittest.cc
>> GTest-specific LOC besides the #include statement: 0.
>
> I think it links to a library as well.
>
> Yes, that's true -- just about any unittesting library will have  
> that requirement, that wasn't part of the comparison.  I was  
> pointing out how much C++ a user has to write in their test files  
> for a reasonable test with fixtures.
>
> Also for a note of reference, your links to the examples are the  
> most advanced samples. So boost can do more, thus has more weight/ 
> bloat behind it.
>
> Gordon Henriksen pointed out earlier in the thread that Boost.Test  
> will require pulling in 500 headers from Boost just to work, so  
> that's pretty heavy weight.  Also, I was comparing apples-to-apples,  
> not taking the simplest example of GTest and the most complex of  
> Boost -- the two examples I chose were doing the same thing, i.e.,  
> tests with fixtures.

ok. I havn't used fixtures for unit testing. Most of the classes don't  
really cross each other that much.

>
> What does Boost provide that GTest does not, that you think LLVM  
> needs, such that it's worth importing the entire Boost distribution  
> into the LLVM tree?
>
> Were the other test kits looked at? Is gtest the best solution for  
> the project.
>
> There are dozens of C++ unittesting libraries out there, I admit I  
> haven't looked at all of them.  I don't know which ones are still  
> under active development or use, but I (and a few other posters) are  
> familiar with and have used GTest enough to say that it's very well  
> suited for LLVM.
>
> Is this something your planning as putting in the tree, thus require  
> pulling in changes from google (license allowing), or does user need  
> to have the libraries/headers pre-installed?
>
> I already answered this question above (first option: in the tree).   
> GTest is under the BSD license, which is compatible with LLVM  
> (though I am not a copyright lawyer).
>
> For example gtest is very light weight test kit, that can do the  
> job, but will the tests outgrow what the test kit can do, and cause  
> a conversion to a more advanced one later?
>
> As soon as LLVM outgrows what GTest can do, we can easily write  
> wrapper macros to forward from GTest-style macros to the other, more  
> suitable framework; however, I think you're conflating the two  
> concepts of "light-weight" and "trivial" -- GTest is light-weight,  
> but not trivial.

I was just concerned that it was too light-weight. Not really dealing  
with gtest is why I asked the question.

>
> Misha
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev

Regards,
Mark Kromis

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081227/fb81ad27/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PGP.sig
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081227/fb81ad27/attachment.sig>