[lldb-dev] unit testing C++ code in LLDB
Todd Fiala
tfiala at google.com
Thu Oct 2 21:27:41 PDT 2014
Hey Sean!
Yeah I've got a gtest subproject in lldb now. It's not wired into any main
test process (i.e. check-lldb or test targets don't run it, but if you run
the "gtest" project in the lldb workspace in Xcode, or go to the gtest dir
and run "do-gtest.py", you'll get it running).
I've got it rigged up to put Xcode error markers at the point in gtests
that fail at the failing assertion point after doing some output munging to
make it work.
The actual gtest build/runner works on MacOSX and Linux. It *should* work
on all platforms since I'm actually using the llvm-embedded gtest
framework, so technically if llvm builds, so should the gtest setup.
However, I do some weak (and need to be improved) inference of directory
structure/build structure to find some necessary files in the
gtest/make/Makefile.rules. They can be overridden by environment or make
variable settings, but they're definitely not going to work for everyone
and there are probably more robust methods for figuring out certain
directory locations.
I run it fine on MacOSX with the canonical Xcode directory layout and using
the Xcode Preferences | Locations set to put project output in a
project-relative DerivedData directory.
On Linux, I assume the build directory is a sibling directory to a llvm,
llvm/tools/clang, llvm/tools/lldb directory structure and (here's the
not-reasonable-default) I assume the build directory is called build-debug.
(So, I have a build-drectory alongside the top-level llvm directory).
Again, all these can be overridden by environment variable/make variable
settings, and I think I have the Makefile emit an error if it can't resolve
the directories it uses and tells what variable to override to fix it.
But, likely it needs some love.
For me, the source/Plugins/Process/Linux/ThreadStateCoordinator.{h/cpp} was
my guinea pig. I'm using that to replace some guts in the llgs signal
state transition logic that requires putting threads through some ptrace
state magic before it should tell lldb about some thread stops. And so I'm
testing that all the deferred callbacks (i.e. essentially GCD-like
constructs) fire off in the right circumstances. That's an example of a
small piece of code in C++ that I just need to know is working right and so
I introduced the gtest framework around it.
Hope that gets you started! Feel free to improve and/or offer suggestions.
For now, we had settled on using gtest/unittest as the root of 1-1
correspondence directory structure mirroring for testing single classes.
If we want to do collaboration tests (integration tests, etc.), we're
probably into the "should be in python category", but we might have a few
low-level multi-class testing scenarios where we might want a different
gtest/functional, gtest/integration or something similar directory
structure to handle those. Would be good to have discussion around that if
we find a valid use for it.
-Todd
On Thu, Oct 2, 2014 at 7:02 PM, Sean Callanan <scallanan at apple.com> wrote:
> Waaaaitasecond, I’m seeing the gtest subproject…
> it’s a bit embarrassing that I haven’t checked out top of tree in so long!
> I’m going to take a look around. Still interested in collaborating on
> this.
>
> Sean
>
> On Oct 2, 2014, at 6:11 PM, Zachary Turner <zturner at google.com> wrote:
>
> I gave it a brief look, but I didn't get it working. Clang and LLD both
> already do this, so I figured it would be as simple as copying their CMake
> for the gtest stuff and fixing it up with LLDB paths. Unfortunately, I was
> getting tons of linker errors, and I'm not really sure why, as there didn't
> appear to be much magic there. I didn't investigate it further after that,
> but it would definitely be a good thing to tackle.
>
> On Thu, Oct 2, 2014 at 2:01 PM, Sean Callanan <scallanan at apple.com> wrote:
>
>> At risk of being burnt at the stake for practicing the the necromantic
>> arts:
>> Jim’s point that internal APIs are somewhat fragile is well taken, and I
>> don’t think there’s much value in using internal tests as “more convenient”
>> substitutes for external tests.
>>
>> That said, I think internal unit tests add several benefits that improve
>> code:
>>
>>
>> - *Modular design*. If clients of an internal object have to do a
>> lot of extra work to make the object work properly, this may indicate a
>> design issue.
>> - *State encapsulation*. If an object changes behaviors depending on
>> outside state, then that makes it much harder to use.
>> - *Code readability*. Test cases can demonstrate how an object is
>> intended to be used, and act as compelling witnesses that that use case
>> works.
>>
>>
>> I would be interested in using the expression parser as a guinea pig to
>> introduce test-driven methods.
>> Todd/Zach, did you ever get a gtest-based unit test system working?
>>
>> Sean
>>
>> On Jul 18, 2014, at 6:14 PM, Zachary Turner <zturner at google.com> wrote:
>>
>> FWIW I'm kind of in favor of bringing in gtest with limited use. When I
>> first started digging into LLDB, probably the first 3-4 bugs I fixed were
>> all in IRMemoryMap, and they would all have been caught if the class were
>> unit tested properly.
>>
>>
>> On Fri, Jul 18, 2014 at 4:03 PM, Greg Clayton <gclayton at apple.com> wrote:
>>
>>> We could expose a new static function in SBDebugger:
>>>
>>> class SBDebugger {
>>>
>>> static void
>>> UnitTest (const char *args);
>>>
>>> };
>>>
>>> Then internally it can parse any arguments and run tests as needed? Each
>>> class that wants to register unit tests with the debugger during
>>> SBDebugger::Initialize() which must be called prior to doing anything with
>>> the LLDB API:
>>>
>>>
>>> SBDebugger::Initialize()
>>> {
>>> Debugger::Intialize();
>>> }
>>>
>>> Debugger::Initialize()
>>> {
>>> ....
>>> NamedPipe:: Initialize();
>>> }
>>>
>>>
>>> Then in the static NamedPipe::Initiazize() you could register your unit
>>> tests:
>>>
>>> void
>>> NamedPipe::Initialiaze()
>>> {
>>> Debugger::RegisterUnitTest(NamedPipe::InitTest1,
>>> "NamedPipe::InitTest1"));
>>> }
>>>
>>>
>>> Then you could just run:
>>>
>>> SBDebugger::Initialize();
>>> SBDebugger::UnitTest(NULL); // Run all tests
>>>
>>> Or run individual ones:
>>>
>>> SBDebugger::Initialize();
>>> SBDebugger::UnitTest("--test NamedPipe::InitTest1"); // Run just
>>> "NamedPipe::InitTest1"
>>>
>>> Of course then the LLDB test suite could run these unit tests first to
>>> make sure everything is good.
>>>
>>>
>>>
>>> > On Jul 16, 2014, at 3:59 PM, Zachary Turner <zturner at google.com>
>>> wrote:
>>> >
>>> > If it makes you feel any better LLVM is leery of it too, and it's only
>>> used, as you said, in specialized circumstances. It's especially useful
>>> for testing abstract data types, where you just want to test the interface
>>> to a self-contained, reusable class.
>>> >
>>> >
>>> > On Wed, Jul 16, 2014 at 2:25 PM, <jingham at apple.com> wrote:
>>> > I'm a little leery about this. We don't test at the lldb_private
>>> layer because those tests are likely to be fragile and easily broken. For
>>> utility classes like NamedPipe I guess I don't so much mind, but I'm not
>>> sure its a great idea to do this more generally.
>>> >
>>> > Jim
>>> >
>>> > > On Jul 16, 2014, at 9:40 AM, Todd Fiala <tfiala at google.com> wrote:
>>> > >
>>> > > Hey guys,
>>> > >
>>> > > Sometimes I have smaller bits of code I'd like to test in LLDB as
>>> I'm developing them (i.e. TDD-style) that are C++ and won't be exposed
>>> directly via Python. I'm not sure I've seen any facilities in the LLDB
>>> tests for adding such tests. Essentially I'd want to do something like a
>>> gtest or cppunit test.
>>> > >
>>> > > Do we have any mechanism for doing that currently? If we do, what
>>> is it? If we don't, how about adding some mechanism to do it after we
>>> figure out how we'd like to approach it? Or, if you have thoughts on a
>>> good, simple way to do it from Python that doesn't require extra Python
>>> bindings just to do it, that'd be fine by me as well.
>>> > >
>>> > > If we want to take a concrete example, here is one: I'm adding a
>>> NamedPipe class under the host dir. I'd like to make some simple tests for
>>> it, and test it under Linux, Windows and MacOSX. In the case of Windows,
>>> it would be the only real way for me to test that it's behaving exactly as
>>> I want at this point. This isn't the only time I've wanted C++-level tests
>>> at a fairly fine granularity, but it's a good example of it.
>>> > >
>>> > > Any thoughts?
>>> > > --
>>> > > Todd Fiala | Software Engineer | tfiala at google.com |
>>> 650-943-3180
>>> > >
>>> > > _______________________________________________
>>> > > lldb-dev mailing list
>>> > > lldb-dev at cs.uiuc.edu
>>> > > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>> >
>>> > _______________________________________________
>>> > lldb-dev mailing list
>>> > lldb-dev at cs.uiuc.edu
>>> > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>> >
>>> > _______________________________________________
>>> > lldb-dev mailing list
>>> > lldb-dev at cs.uiuc.edu
>>> > http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>>
>>>
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/lldb-dev
>>
>>
>>
>
>
--
Todd Fiala | Software Engineer | tfiala at google.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20141002/a85de6bd/attachment.html>
More information about the lldb-dev
mailing list