[LLVMdev] 2.4 Pre-release (v1) Available for Testing

[Keir Mierle]

On Sat, Oct 11, 2008 at 11:58 PM, Talin <viridia at gmail.com> wrote:

> I've been using gtest (http://code.google.com/p/googletest/) for all of
> my frontend unit tests and I'm very happy with it. It does all of that
> automatic test discovery stuff pretty well. I haven't tried the XML test
> report generation stuff, but it does have that capability.


Justs to chime in: I'm one of the gtest devs. I'd like to add that gtest is
very portable, more so than llvm; it even works on WinCE and blackberry. If
there are specific features needed by LLVM, depending on what it is I may be
able to get it into gtest.

Keir


>
>
> I don't know much about DejaGNU, and from what little I know about it,
> I'm not sure its worth my time to learn about it. Just reading the intro
> doc scares me, it feels like autoconf/automake or sendmail, i.e.
> something bolted together out of several different scripting systems.
>
> So, I guess my question would be, what requirements are being satisfied
> by DejaGNU that wouldn't be satisfied by a C++-based testing framework?
>
> OvermindDL1 wrote:
> > On Sat, Oct 11, 2008 at 7:30 AM, Kenneth Boyd <zaimoni at zaimoni.com>
> wrote:
> >
> >> At the absolute minimum, simple counting of the success/failure/internal
> >> error results requires three different exit codes, and a test driver
> >> capable of tallying them up and reporting all failures (unexpected
> >> success, unexpected failure, and internal errors with a designated
> >> distinct exit code).
> >>
> >
> > This is what I am not understanding about how you are thinking about
> > this.  There are a great deal more responses and having a C++ test
> > program, say boost::test, it can handle a great deal of information,
> > spitting out exactly what caused various things to fail, it is not
> > just a matter of running a variety of programs and testing them, those
> > are more useful if you are testing an external app like llvm-gcc, not
> > llvm itself, which is a library, not an application, and if it was
> > treated as such then its testing framework for it and it alone could
> > become a great deal more powerful while being far easier to use.  It
> > could quite literally just be a single extra project, each test could
> > be in its cpp file, no header for the tests needed, and just have
> > something like boost::test run them all in succession.  If a test is
> > truly expected to fail in a spectacular way (say, by abort()ing the
> > application, those really need to be fixed and removed, there are ways
> > to hook the abort call) those could be made into a simple separate exe
> > and run by the main boost::test framework as well, nothing special
> > required.
> >
> > I use this style testing and it is very simple to use, very powerful,
> > plenty of information, and it tests llvm as a library, meaning it can
> > test *all* of it, not just what the mini-apps like llc and such
> > expose, since they just access specific parts of the library anyway.
> > No testing error codes, no watching the standard input/output (which
> > is a horrible way of testing regardless), etc...  And as stated, it is
> > also very simple for a C++ program to submit its own report, nothing
> > special there either.
> >
> > If any mini test apps needed to be used, then you could still have the
> > main test program watch the standard input/output, that is a simple to
> > do regardless of the OS you are, but even using something like
> > boost::interprocess to communicate would be even far more powerful,
> > and still very simple.  And it seems to support more compilers then
> > llvm anyway (these are just the tested compilers, it no doubt supports
> > more just fine, still more then llvm either way):
> >     * Visual 7.1 Windows XP
> >     * Visual 8.0 Windows XP
> >     * GCC 4.1.1 MinGW
> >     * GCC 3.4.4 Cygwin
> >     * Intel 9.1 Windows XP
> >     * GCC 4.1.2 Linux
> >     * GCC 3.4.3 Solaris 11
> >     * GCC 4.0 MacOs 10.4.1
> >
> > DejaGNU can send in reports, so can C++ itself.  It is simple for
> > boost::filesystem to parse through directories and, say, get all of
> > the .ll files, of which you can then run through various optimizers
> > and make sure the converted code is correct, or run through the
> > interpreter, or run through the JIT to test the JIT, etc...
> >
> > That does not really surprise me about CMake, but then mingw is not a
> > primary compiler on Windows, more so is VC++ or Intel, either way a
> > bug should be submitted to the CMake devs.  I thought you could
> > override things like that with some command line parameters anyway.
> > You cannot force it to output mingw files directly?
> > _______________________________________________
> > LLVM Developers mailing list
> > LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> >
> >
>
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20081012/af7a1aa6/attachment.html>