[cfe-dev] tests broken
Daniel Dunbar
daniel at zuster.org
Mon Aug 10 22:37:33 PDT 2009
On Mon, Aug 10, 2009 at 2:45 PM, John
Thompson<john.thompson.jtsoftware at gmail.com> wrote:
> It looks like progress is being made on the Visual Studio-based tests. I'm
> not sure what the status of the work on the tests in general is, so excuse
> me if what I'm seeing is just an intermediate stage of the work.
Things are far enough along that, with patches, you can get it so that
(I think) the remaining failures are real problems.
> I just tried the latest clang-test project, which seems to work, as far as
> putting out messages. Currently it shows 342 failures, compared to 354 on
> MSYS/MinGW. Are you seeing this too? Or do I still have some problem with
> my set up?
The main thing I have modified in my setup is to add count.bat and
not.bat scripts in the llvm/test/Scripts directly, so that they work.
> Note that I got 3 or 4 abort dialogs, while they were running. It would be
> nice if these exceptions where caught instead.
>
> Also, is there a way to get the verbose output when running the tests from
> Visual Studio?
You can modify the clang-test build step to call MultiTestRunner.py
with the '-v' argument. That's probably a reasonable default.
> Running the script directly appears to be a work-around.
>
> I ran one of the scripts directly for a failing test, and here's the output:
In general running the script directly isn't going to work on Windows.
In fact, in --no-sh mode the script doesn't even get created
currently.
I attached the llvm and clang patches I have on my Windows box.
Currently I run the test runner by hand from a cmd prompt, like this:
--
c:\python25\python ..\utils\test\MultiTestRunner.py --clang
g:\public\llvm\build_msvc
9\bin\Debug\clang.exe --clang-cc
g:\public\llvm\build_msvc9\bin\Debug\clang-cc.exe --no-sh --path
g:\public\llvm\test\Scripts --path g:\public\llvm\buil
d_msvc9\bin\Debug -j 3 -v .
--
The last time I tried it, I had 98 failures.
- Daniel
> C:\Tools\llvm\tools\clang\test>C:/Tools/llvm/bin/Debug/clang-cc -emit
> llvm 0<C:\Tools\llvm\tools\clang\test\CodeGen\align-local.c | grep "align
> 16" | count 2
> 'C:' is not recognized as an internal or external command,
> operable program or batch file.
>
> I'm not sure what the problem exactly is here. Playing with the command
> line directly it seems the message is coming from grep. Maybe pipes work
> different in Windows scripts? Changing the source file from being
> redirected in to being on the command line didn't change anything.
>
> I'm guessing the pipe to the count script is also a problem. How will the
> scripts like count be handled on Windows?
>
> Anyway, thanks for doing this, We Windows-users appreciate it greatly.
>
> -John
>
> On Sat, Aug 8, 2009 at 5:06 AM, John Thompson
> <john.thompson.jtsoftware at gmail.com> wrote:
>>
>> Under MSYS/MinGW, I looked at a few of the failing tests:
>>
>> Analysis/cast.c: Apparently MinGW doesn't have sys/socket.h, which is
>> referenced in a #include. I searched for a package containing socket.h, but
>> didn't find one. A MinGW mailing list search yielded a hint that socket.h
>> is present in Cygwin, but not MinGW. Any MinGW users out there see this in
>> running the tests?
>>
>> CodeGenCXX/class-layout.cpp: The test does "grep '%.truct.A = type { i8
>> }' %t" in the class-layout.cpp.tmp file, but this file has "%struct.A = type
>> { i8 }".
>>
>> CodeGenCXX/copy-constructor-elim.cpp: Greps for a mangled symbol not in
>> the output file. Also has this message in stderr: "'count' is not
>> recognized as an internal or external command". I found a count script in
>> llvm/test/Scripts and added it to the path, but still get the same error. I
>> see that a lot of the failures are similar to this one, and for others of
>> those scripts.
>>
>> CodeGenCXX\reference-field.cpp: Here's the output from the log:
>>
>> ********************
>> FAIL: c:/Tools/llvm/tools/clang/test\CodeGenCXX\reference-field.cpp ( 166
>> of 1428)
>> ******************** TEST
>> 'c:/Tools/llvm/tools/clang/test\CodeGenCXX\reference-field.cpp' FAILED
>> ********************
>> Script:
>> --
>> c:/Tools/llvm/Debug/bin/clang-cc -emit-llvm -o -
>> c:\Tools\llvm\tools\clang\test\CodeGenCXX\reference-field.cpp -O2 | grep
>> "@_Z1bv"
>> --
>> Exit Code: 255
>> Command Output (stdout):
>> --
>> c:\Tools\llvm\tools\clang\test\CodeGenCXX>c:/Tools/llvm/Debug/bin/clang-cc
>> -emit-llvm -o -
>> c:\Tools\llvm\tools\clang\test\CodeGenCXX\reference-field.cpp -O2 | grep
>> "@_Z1bv"
>> --
>> Command Output (stderr):
>> --
>> 'c:' is not recognized as an internal or external command,
>> operable program or batch file.
>> --
>> ********************
>> I'm guessing that perhaps the Windows Python installer I used installed a
>> version of Python with Window'isms, kind of like the first Perl I used. I
>> didn't see a MinGW Python package on the MinGW or python.org sites, so I was
>> going to try downloading the source package for Unix-like systems and build
>> it, but the python.org site has been down for nearly 24 hours. Unless I
>> find another solution, or someone points me to the right package or
>> solution, I'll try this when python.org is back.
>>
>> On Ubuntu32 with gcc version 4.2.4 I ran the Analysis/casts.c test, which
>> had a segmentation fault. Running in GDB:
>> $ gdb /home/john/llvm/Debug/bin/clang-cc
>> GNU gdb 6.8-debian
>> Copyright (C) 2008 Free Software Foundation, Inc.
>> License GPLv3+: GNU GPL version 3 or later
>> <http://gnu.org/licenses/gpl.html>
>> This is free software: you are free to change and redistribute it.
>> There is NO WARRANTY, to the extent permitted by law. Type "show copying"
>> and "show warranty" for details.
>> This GDB was configured as "i486-linux-gnu"...
>> (gdb) run -analyze -checker-cfref -analyzer-store=region
>> /home/john/llvm/tools/clang/test/Analysis/casts.c
>> Starting program: /home/john/llvm/Debug/bin/clang-cc -analyze
>> -checker-cfref -analyzer-store=region
>> /home/john/llvm/tools/clang/test/Analysis/casts.c
>> [Thread debugging using libthread_db enabled]
>> [New Thread 0xb7c9e6c0 (LWP 2747)]
>> Program received signal SIGSEGV, Segmentation fault.
>> [Switching to Thread 0xb7c9e6c0 (LWP 2747)]
>> 0x08316a9c in clang::CXXScopeSpec::isSet (this=0x0)
>> at
>> /home/john/llvm/tools/clang/lib/Parse/../../include/clang/Parse/DeclSpec.h:454
>> 454 bool isSet() const { return ScopeRep != 0; }
>> (gdb) where
>> #0 0x08316a9c in clang::CXXScopeSpec::isSet (this=0x0)
>> at
>> /home/john/llvm/tools/clang/lib/Parse/../../include/clang/Parse/DeclSpec.h:454
>> #1 0x08316209 in clang::Sema::computeDeclContext (this=0xbf9a6940,
>> SS=@0x0,
>> EnteringContext=false) at SemaCXXScopeSpec.cpp:39
>> #2 0x083a57a8 in clang::Sema::LookupParsedName (this=0xbf9a6940,
>> S=0x959e9b0,
>> SS=0x0, Name={Ptr = 157096152},
>> NameKind=clang::Sema::LookupOrdinaryName,
>> RedeclarationOnly=false, AllowBuiltinCreation=false, Loc={ID = 0})
>> at SemaLookup.cpp:1165
>> #3 0x0832d978 in clang::Sema::getTypeName (this=0xbf9a6940,
>> II=@0x95d18d8,
>> NameLoc={ID = 2147532587}, S=0x959e9b0, SS=0x0) at SemaDecl.cpp:77
>> #4 0x084c277f in clang::Parser::ParseDeclarationSpecifiers
>> (this=0xbf9a6848,
>> DS=@0xbf9a62a0, TemplateInfo=@0xbf9a6300, AS=clang::AS_none,
>> DSContext=clang::Parser::DSC_normal) at ParseDecl.cpp:834
>> #5 0x084c72b7 in clang::Parser::ParseSimpleDeclaration (this=0xbf9a6848,
>> Context=0, DeclEnd=@0xbf9a66bc, RequireSemi=true) at ParseDecl.cpp:343
>> #6 0x084c7670 in clang::Parser::ParseDeclaration (this=0xbf9a6848,
>> Context=0,
>> DeclEnd=@0xbf9a66bc) at ParseDecl.cpp:323
>> #7 0x084b9f60 in clang::Parser::ParseExternalDeclaration
>> (this=0xbf9a6848)
>> at Parser.cpp:448
>> #8 0x084b9cec in clang::Parser::ParseExternalDeclaration
>> (this=0xbf9a6848)
>> at Parser.cpp:411
>> #9 0x084ba032 in clang::Parser::ParseTopLevelDecl (this=0xbf9a6848,
>> ---Type <return> to continue, or q <return> to quit---
>> Result=@0xbf9a6930) at Parser.cpp:354
>> #10 0x0830818f in clang::ParseAST (PP=@0x959e700, Consumer=0x959ebf0,
>> Ctx=@0x95b89a0, PrintStats=false, CompleteTranslationUnit=true)
>> at ParseAST.cpp:63
>> #11 0x08072598 in ProcessInputFile (PP=@0x959e700, PPF=@0xbf9a70d8,
>> InFile=@0x959ed80, PA=RunAnalysis, Features=@0xbf9a70f0,
>> Context=@0x95850d0) at clang-cc.cpp:2058
>> #12 0x080743b0 in main (argc=5, argv=0xbf9a7364) at clang-cc.cpp:2314
>> (gdb) list
>> 449
>> 450 /// isInvalid - An error occured during parsing of the scope
>> specifier.
>> 451 bool isInvalid() const { return isNotEmpty() && ScopeRep == 0; }
>> 452
>> 453 /// isSet - A scope specifier was resolved to a valid C++ scope.
>> 454 bool isSet() const { return ScopeRep != 0; }
>> 455
>> 456 void clear() {
>> 457 Range = SourceRange();
>> 458 ScopeRep = 0;
>> (gdb) p this
>> $1 = (const clang::CXXScopeSpec * const) 0x0
>> (gdb) up
>> #1 0x08316209 in clang::Sema::computeDeclContext (this=0xbf9a6940,
>> SS=@0x0,
>> EnteringContext=false) at SemaCXXScopeSpec.cpp:39
>> 39 if (!SS.isSet() || SS.isInvalid())
>> (gdb) list
>> 34 /// \returns the declaration context represented by the scope specifier
>> @p SS,
>> 35 /// or NULL if the declaration context cannot be computed (e.g.,
>> because it is
>> 36 /// dependent and not the current instantiation).
>> 37 DeclContext *Sema::computeDeclContext(const CXXScopeSpec &SS,
>> 38 bool EnteringContext) {
>> 39 if (!SS.isSet() || SS.isInvalid())
>> 40 return 0;
>> 41
>> 42 NestedNameSpecifier *NNS
>> 43 = static_cast<NestedNameSpecifier *>(SS.getScopeRep());
>> (gdb)
>> I see that this is the problem with most if not all the the failing tests
>> on Ubuntu. Before trying to figure out the reason for the crash in gdb, I
>> thought I'd ask if this has been seen. Also, I see there are over 300 mb of
>> updates for Ubuntu, so I'll install them (my internet connection is pretty
>> slow) and try again.
>> -John
>> On Thu, Aug 6, 2009 at 7:24 PM, Daniel Dunbar <daniel at zuster.org> wrote:
>>>
>>> On Thu, Aug 6, 2009 at 5:52 PM, John
>>> Thompson<john.thompson.jtsoftware at gmail.com> wrote:
>>> > On a VM running Ubuntu 32-bit I'm seeing 1032 failures, after a fresh
>>> > update, configure, make. My Python version is 2.5.2. A week and a
>>> > half
>>> > ago, there were no failures.
>>> >
>>> > Having gotten a build on MSYS/MinGW to suceed, in running the tests I'm
>>> > seeing 336 failures. Python is 2.6.1.
>>> >
>>> > Any suggestions on debugging this? I have VERBOSE logs, but they're
>>> > probably too big to post here.
>>>
>>> In general, my advice is (a) look at the buildbots to see whether
>>> failures are expected or not. If not, pick any failure which isn't
>>> failing on a build bot and trace it down (you can run individual tests
>>> using TestRunner.sh). In this case, currently all tests are passing on
>>> many platforms, so its likely a local configuration issue. If you
>>> can't figure it out, just file a bug with that one particular test
>>> case, and the details on why it fails.
>>>
>>> MSYS/MinGW is of course a separate issue from Ubuntu, but the same
>>> strategy applies.
>>>
>>> - Daniel
>>>
>>> > -John
>>> > On Fri, Jul 31, 2009 at 7:20 PM, Eli Friedman <eli.friedman at gmail.com>
>>> > wrote:
>>> >>
>>> >> On Fri, Jul 31, 2009 at 5:53 PM, John
>>> >> Thompson<john.thompson.jtsoftware at gmail.com> wrote:
>>> >> > After an update of both llvm and clang, configure, and rebuild on a
>>> >> > Linux
>>> >> > system in a virtual machine, hundreds of tests are now failing.
>>> >> >
>>> >> > Just in case you didn't already know...
>>> >> >
>>> >> > Or is my system messed up?
>>> >>
>>> >> Hundreds sounds a little high, but there are currently failing tests
>>> >> in trunk clang; see http://google1.osuosl.org:8011/waterfall . If
>>> >> you're still seeing failures after that gets cleared up, please send
>>> >> another email.
>>> >>
>>> >> -Eli
>>> >
>>> > --
>>> > John Thompson
>>> > John.Thompson.JTSoftware at gmail.com
>>> >
>>> >
>>> > _______________________________________________
>>> > cfe-dev mailing list
>>> > cfe-dev at cs.uiuc.edu
>>> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>> >
>>> >
>>
>>
>>
>> --
>> John Thompson
>> John.Thompson.JTSoftware at gmail.com
>>
>
>
>
> --
> John Thompson
> John.Thompson.JTSoftware at gmail.com
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: llvm.patch
Type: text/x-diff
Size: 1479 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20090810/6233efcc/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: clang.patch
Type: text/x-diff
Size: 1712 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20090810/6233efcc/attachment-0001.patch>
More information about the cfe-dev
mailing list