[LLVMdev] DejaGNU test fixes

[Gordon Henriksen]

Does merely sending output to stderr legitimately constitute a test  
failure? This doesn't seem entirely legitimate to me. I send  
checkpoints to stderr in the ocaml test case (because linking each  
individual test would be too slow); I was surprised and confused to  
see it failing this morning, when the exit code in fact correctly  
reports success.

— Gordon


On Jun 10, 2008, at 12:32, Matthijs Kooijman wrote:

> Hi all,
>
> while writing a testcase thate needed to do a grep containg {, I  
> found that the DejaGNU test framework didn't handle those very well.  
> It's a bit of a fuss to escape accolades properly, but most of all  
> the framework seemed to silently ignore errors in the escaping (and  
> just not run the command then). See [1].
>
> Fixing the framework resulted in 80 of the tests failing. I spent  
> the afternoon fixing 60 tests. Of these, I fixed 42 that have been  
> running fine so far, but generated output on stderr or were marked  
> as failing for some other reason. Additionally, I fixed 18 test  
> cases that were not run or only partly run before (though all of  
> those passed after fixing the test case).
>
> I leave the remaining 20 failing testcases, since I do not see a  
> clear resolution for them. In a lot of cases disabling or fixing a  
> warning might be enough, but since I can't always see what a test is  
> supposed to test, I'm afraid that that will actually render the  
> testcase useless.
>
> Of these final 20 tests, 3 are related to bugpoint producing stderr  
> output (but I can't see if that is expected output or not). Two are  
> related to an invalid -mcpu option, I think the corresponding RUN  
> lines should be removed or changed? One is related to the - 
> mattr=sse1 option that is not supported (anymore?). The remaining 14  
> are warnings produced by llvm-gcc.
>
> I've attached the error output for the remaining 20 tests, please  
> check it and fix them where possible.