<div dir="ltr">On Sat, Oct 11, 2008 at 11:58 PM, Talin <span dir="ltr"><<a href="mailto:viridia@gmail.com">viridia@gmail.com</a>></span> wrote:<br><div class="gmail_quote"><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I've been using gtest (<a href="http://code.google.com/p/googletest/" target="_blank">http://code.google.com/p/googletest/</a>) for all of<br>
my frontend unit tests and I'm very happy with it. It does all of that<br>
automatic test discovery stuff pretty well. I haven't tried the XML test<br>
report generation stuff, but it does have that capability.</blockquote><div><br>Justs to chime in: I'm one of the gtest devs. I'd like to add that gtest is very portable, more so than llvm; it even works on WinCE and blackberry. If there are specific features needed by LLVM, depending on what it is I may be able to get it into gtest.<br>
<br>Keir<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br>
<br>
I don't know much about DejaGNU, and from what little I know about it,<br>
I'm not sure its worth my time to learn about it. Just reading the intro<br>
doc scares me, it feels like autoconf/automake or sendmail, i.e.<br>
something bolted together out of several different scripting systems.<br>
<br>
So, I guess my question would be, what requirements are being satisfied<br>
by DejaGNU that wouldn't be satisfied by a C++-based testing framework?<br>
<div><div></div><div class="Wj3C7c"><br>
OvermindDL1 wrote:<br>
> On Sat, Oct 11, 2008 at 7:30 AM, Kenneth Boyd <<a href="mailto:zaimoni@zaimoni.com">zaimoni@zaimoni.com</a>> wrote:<br>
><br>
>> At the absolute minimum, simple counting of the success/failure/internal<br>
>> error results requires three different exit codes, and a test driver<br>
>> capable of tallying them up and reporting all failures (unexpected<br>
>> success, unexpected failure, and internal errors with a designated<br>
>> distinct exit code).<br>
>><br>
><br>
> This is what I am not understanding about how you are thinking about<br>
> this. There are a great deal more responses and having a C++ test<br>
> program, say boost::test, it can handle a great deal of information,<br>
> spitting out exactly what caused various things to fail, it is not<br>
> just a matter of running a variety of programs and testing them, those<br>
> are more useful if you are testing an external app like llvm-gcc, not<br>
> llvm itself, which is a library, not an application, and if it was<br>
> treated as such then its testing framework for it and it alone could<br>
> become a great deal more powerful while being far easier to use. It<br>
> could quite literally just be a single extra project, each test could<br>
> be in its cpp file, no header for the tests needed, and just have<br>
> something like boost::test run them all in succession. If a test is<br>
> truly expected to fail in a spectacular way (say, by abort()ing the<br>
> application, those really need to be fixed and removed, there are ways<br>
> to hook the abort call) those could be made into a simple separate exe<br>
> and run by the main boost::test framework as well, nothing special<br>
> required.<br>
><br>
> I use this style testing and it is very simple to use, very powerful,<br>
> plenty of information, and it tests llvm as a library, meaning it can<br>
> test *all* of it, not just what the mini-apps like llc and such<br>
> expose, since they just access specific parts of the library anyway.<br>
> No testing error codes, no watching the standard input/output (which<br>
> is a horrible way of testing regardless), etc... And as stated, it is<br>
> also very simple for a C++ program to submit its own report, nothing<br>
> special there either.<br>
><br>
> If any mini test apps needed to be used, then you could still have the<br>
> main test program watch the standard input/output, that is a simple to<br>
> do regardless of the OS you are, but even using something like<br>
> boost::interprocess to communicate would be even far more powerful,<br>
> and still very simple. And it seems to support more compilers then<br>
> llvm anyway (these are just the tested compilers, it no doubt supports<br>
> more just fine, still more then llvm either way):<br>
> * Visual 7.1 Windows XP<br>
> * Visual 8.0 Windows XP<br>
> * GCC 4.1.1 MinGW<br>
> * GCC 3.4.4 Cygwin<br>
> * Intel 9.1 Windows XP<br>
> * GCC 4.1.2 Linux<br>
> * GCC 3.4.3 Solaris 11<br>
> * GCC 4.0 MacOs 10.4.1<br>
><br>
> DejaGNU can send in reports, so can C++ itself. It is simple for<br>
> boost::filesystem to parse through directories and, say, get all of<br>
> the .ll files, of which you can then run through various optimizers<br>
> and make sure the converted code is correct, or run through the<br>
> interpreter, or run through the JIT to test the JIT, etc...<br>
><br>
> That does not really surprise me about CMake, but then mingw is not a<br>
> primary compiler on Windows, more so is VC++ or Intel, either way a<br>
> bug should be submitted to the CMake devs. I thought you could<br>
> override things like that with some command line parameters anyway.<br>
> You cannot force it to output mingw files directly?<br>
> _______________________________________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
><br>
><br>
<br>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:LLVMdev@cs.uiuc.edu">LLVMdev@cs.uiuc.edu</a> <a href="http://llvm.cs.uiuc.edu" target="_blank">http://llvm.cs.uiuc.edu</a><br>
<a href="http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev" target="_blank">http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev</a><br>
</div></div></blockquote></div><br></div>