[lldb-dev] Is anyone using the gtest Xcode project?

Zachary Turner zturner at google.com
Thu Mar 12 17:06:14 PDT 2015

Wasn't really trying to get into an extended discussion about this, but
FWIW I definitely realize that lldb's tests are more complicated than what
lit currently supports.  But that's why I said "even if it meant extending
lit".  It was mostly just a general comment about how it's nice if everyone
is focused on making one thing better instead of everyone having different

As for specifics, my understanding is that lit parallelizes better (so
running tests is faster), understands how to build programs (so doesn't
require makefiles), and has a richer language for specifying how and under
what circumstances different tests should be run.  It's also familiar to
other LLVM developers (so encourages cross-collaboration), and allows one
to write self-contained tests with the program to test and the check in a
single file (less maintenance).

In any case, I'm really not an expert on lit, so +bogner and +chandlerc in
case they want to chime in.  I do think it's at least worth thinking about
whether lit *could* be extended to meet LLDB's needs -- if nothing else as
a thought exercise, and maybe learning more about how it works would give
us some ideas to make our own test suite better.

On Thu, Mar 12, 2015 at 4:39 PM <jingham at apple.com> wrote:

> > On Mar 12, 2015, at 4:08 PM, Zachary Turner <zturner at google.com> wrote:
> >
> > Oh I'm all for reusing as much of the existing mechanism as possible.
> Was just stating how the CMake worked as a discussion point.  Another
> possibility would be to just have the Xcode project build one executable
> that pulls in sources recursively from the entire subtree.  Is this as easy
> in Xcode as just adding all sources from a subfolder to a single target?
> >
> > One day far off in the future it would be nice if all of LLDB's tests
> were ported to lit (even if that meant extending lit to make it do what we
> needed it to do),
> Why would this be nice?  It looks like lit is a good test runner for tests
> that have some input, do something with the input, produce an output and
> check that output is matches some pattern.  That is not at all what the
> lldb tests look like.  They often have to do complex dances - for instance
> depending on how the line tables come out there are many "correct" ways to
> step through code.  If you are going to test this you've got to do "step,
> if I got to a close bracket, step again, if I got past it don't.  Etc...
> I see no benefit in extending a simple runner like lit to do the complex
> dances the lldb testsuite sometimes has to do.  I'm all for sharing, but it
> is also okay to have two implementations of some functionality if the two
> uses are sufficiently different, and this certainly seems like one of those
> cases.
> > so I can definitely see some value in hooking lit up to the Xcode build
> so it does everything the CMake build does.  I'll have to look into exactly
> what steps the CMake and/or autoconf build are taking, but I suspect it's
> going to involve running CMake from a script, so not very desirable.  I'm
> still learning a lot of this stuff though, so there may be a better way.
> Either way, I'll have to look into it a little bit.
> Jim
> >
> > In the meantime, if running unit tests from Xcode is not part of
> anyone's usual workflow, can I remove it for now?
> >
> > On Thu, Mar 12, 2015 at 4:01 PM <jingham at apple.com> wrote:
> > I'm not sure if this is what you meant, but I don't see a lot of value
> in making an Xcode project that has targets for each of the gtest binaries,
> and then tries to run the tests.  Seems to me it would be better if the
> gtest project just invokes whatever mechanism the cmake build would do to
> run the tests.  That's just another set of things to keep in sync.
> >
> > It is sufficient to have a target that just does whatever steps
> cmake/lit do to build the gtests & run them, if that is possible.  I guess
> if you can't do this without running cmake in the lldb top-level directory
> that would be a problem.  But it still seems better to me to wire that up,
> than to have to add tests to both Xcode & cmake.
> >
> > Jim
> >
> >
> >
> >
> >
> > > On Mar 12, 2015, at 3:46 PM, Zachary Turner <zturner at google.com>
> wrote:
> > >
> > > So I'm guessing the scheme runs do-gtest.py.  I'd like to delete that
> file as well as all the Makefiles in the directory if possible.  It seems
> like these files should be built using the normal Xcode build system the
> same way the rest of LLDB is built.
> > >
> > > The way the CMake does it is that each test folder generates a new
> executable.  So right now it will build HostTests.exe,
> ProcessLinuxTests.exe, and UtilityTests.exe.  And then CMake will invoke
> lit (the LLVM test runner) to run each of the executables one by one and
> print the output.
> > >
> > > I'm not sure if that's easy or feasible to do in the Xcode build.  I
> kind of don't want to leave this do-gtest.py and Makefiles in the build
> though, because the more of this stuff we have the more maintenance it is,
> and things tend to rot.
> > >
> > >
> > > On Thu, Mar 12, 2015 at 3:23 PM <jingham at apple.com> wrote:
> > > Xcode has "projects" and then "workspaces" and "schemes".  Workspaces
> aggregate projects.  Schemes exist in both workspaces and projects and are
> the way to say "do something with some of the stuff referred to by this
> project/workspace."  So the way to do this formally is to have the gtest
> scheme build & run the tests from the gtest project.
> > >
> > > The lldb.xcworkspace file does reference the gtest xcode project, and
> it has a scheme for the gtest.
> > >
> > > Not sure what the scheme does yet, I'll look in a few minutes if
> nobody beats me to it, I'm in the middle of things right now.
> > >
> > > Jim
> > >
> > >
> > >
> > > > On Mar 12, 2015, at 2:41 PM, Zachary Turner <zturner at google.com>
> wrote:
> > > >
> > > > In lldb/gtest there is a gtest.xcodeproj folder with what I guess is
> an Xcode project.  If I understand the way Xcode works, the way to use this
> is by opening this in another instance of Xcode separate from your normal
> LLDB project, and then building it.  Is this right?
> > > >
> > > > I have a patch that moves some files around, and if nobody is using
> this Xcode project, I would like to delete it.  Then, after I get the tests
> up and running in the CMake build, we can add it to the "real" Xcode
> project as a separate target similar to how you currently run the LLDB Test
> suite.
> > > >
> > > > Any objections to deleting the Xcode project?
> > >
> >
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20150313/6303d00e/attachment.html>

More information about the lldb-dev mailing list