[lldb-dev] unit testing C++ code in LLDB

[Sean Callanan]

At risk of being burnt at the stake for practicing the the necromantic arts:
Jim’s point that internal APIs are somewhat fragile is well taken, and I don’t think there’s much value in using internal tests as “more convenient” substitutes for external tests.

That said, I think internal unit tests add several benefits that improve code:

Modular design.  If clients of an internal object have to do a lot of extra work to make the object work properly, this may indicate a design issue.
State encapsulation.  If an object changes behaviors depending on outside state, then that makes it much harder to use.
Code readability.  Test cases can demonstrate how an object is intended to be used, and act as compelling witnesses that that use case works.

I would be interested in using the expression parser as a guinea pig to introduce test-driven methods.
Todd/Zach, did you ever get a gtest-based unit test system working?

Sean

> On Jul 18, 2014, at 6:14 PM, Zachary Turner <zturner at google.com> wrote:
> 
> FWIW I'm kind of in favor of bringing in gtest with limited use.  When I first started digging into LLDB, probably the first 3-4 bugs I fixed were all in IRMemoryMap, and they would all have been caught if the class were unit tested properly.  
> 
> 
> On Fri, Jul 18, 2014 at 4:03 PM, Greg Clayton <gclayton at apple.com <mailto:gclayton at apple.com>> wrote:
> We could expose a new static function in SBDebugger:
> 
> class SBDebugger {
> 
> static void
> UnitTest (const char *args);
> 
> };
> 
> Then internally it can parse any arguments and run tests as needed? Each class that wants to register unit tests with the debugger during SBDebugger::Initialize() which must be called prior to doing anything with the LLDB API:
> 
> 
> SBDebugger::Initialize()
> {
>     Debugger::Intialize();
> }
> 
> Debugger::Initialize()
> {
>         ....
>         NamedPipe:: Initialize();
> }
> 
> 
> Then in the static NamedPipe::Initiazize() you could register your unit tests:
> 
> void
> NamedPipe::Initialiaze()
> {
>     Debugger::RegisterUnitTest(NamedPipe::InitTest1, "NamedPipe::InitTest1"));
> }
> 
> 
> Then you could just run:
> 
> SBDebugger::Initialize();
> SBDebugger::UnitTest(NULL); // Run all tests
> 
> Or run individual ones:
> 
> SBDebugger::Initialize();
> SBDebugger::UnitTest("--test NamedPipe::InitTest1"); // Run just "NamedPipe::InitTest1"
> 
> Of course then the LLDB test suite could run these unit tests first to make sure everything is good.
> 
> 
> 
> > On Jul 16, 2014, at 3:59 PM, Zachary Turner <zturner at google.com <mailto:zturner at google.com>> wrote:
> >
> > If it makes you feel any better LLVM is leery of it too, and it's only used, as you said, in specialized circumstances.  It's especially useful for testing abstract data types, where you just want to test the interface to a self-contained, reusable class.
> >
> >
> > On Wed, Jul 16, 2014 at 2:25 PM, <jingham at apple.com <mailto:jingham at apple.com>> wrote:
> > I'm a little leery about this.  We don't test at the lldb_private layer because those tests are likely to be fragile and easily broken.  For utility classes like NamedPipe I guess I don't so much mind, but I'm not sure its a great idea to do this more generally.
> >
> > Jim
> >
> > > On Jul 16, 2014, at 9:40 AM, Todd Fiala <tfiala at google.com <mailto:tfiala at google.com>> wrote:
> > >
> > > Hey guys,
> > >
> > > Sometimes I have smaller bits of code I'd like to test in LLDB as I'm developing them (i.e. TDD-style) that are C++ and won't be exposed directly via Python.  I'm not sure I've seen any facilities in the LLDB tests for adding such tests.  Essentially I'd want to do something like a gtest or cppunit test.
> > >
> > > Do we have any mechanism for doing that currently?  If we do, what is it?  If we don't, how about adding some mechanism to do it after we figure out how we'd like to approach it?  Or, if you have thoughts on a good, simple way to do it from Python that doesn't require extra Python bindings just to do it, that'd be fine by me as well.
> > >
> > > If we want to take a concrete example, here is one: I'm adding a NamedPipe class under the host dir.  I'd like to make some simple tests for it, and test it under Linux, Windows and MacOSX.  In the case of Windows, it would be the only real way for me to test that it's behaving exactly as I want at this point.  This isn't the only time I've wanted C++-level tests at a fairly fine granularity, but it's a good example of it.
> > >
> > > Any thoughts?
> > > --
> > > Todd Fiala |   Software Engineer |     tfiala at google.com <mailto:tfiala at google.com> |     650-943-3180 <tel:650-943-3180>
> > >
> > > _______________________________________________
> > > lldb-dev mailing list
> > > lldb-dev at cs.uiuc.edu <mailto:lldb-dev at cs.uiuc.edu>
> > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev <http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev>
> >
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at cs.uiuc.edu <mailto:lldb-dev at cs.uiuc.edu>
> > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev <http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev>
> >
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at cs.uiuc.edu <mailto:lldb-dev at cs.uiuc.edu>
> > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev <http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev>
> 
> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20141002/b81c1075/attachment.html>