[LLVMdev] LLVM / CLANG Test Infrastructure Question
sundeepk at codeaurora.org
Fri Jul 29 14:35:18 PDT 2011
I finally got it to work. I think "LLVM Testing Infrastructure Guide" should
be updated to reflect the latest changes (use clang instead of llvm-gcc and
make command etc.).
I ran into problems with using system headers and startup files (crt*.o). So
I had to build and install llvm-gcc, next build clang using headers from
llvm-gcc install dir, and then use clang to run regressions under
projects/test-suite. It works beautifully.
Anyways, I wanted to thank you for all your help. I really appreciate it.
> -----Original Message-----
> From: Chad Rosier [mailto:mcrosier at apple.com]
> Sent: Thursday, July 28, 2011 12:37 PM
> To: Sundeep
> Cc: cfe-dev at cs.uiuc.edu; 'LLVM Dev'
> Subject: Re: [LLVMdev] LLVM / CLANG Test Infrastructure Question
> On Jul 28, 2011, at 8:18 AM, Sundeep wrote:
> >>> 1. I checked out and built llvm, clang, and test-suite from svn
> >> When I
> >>> run llvm/test *without* my changes on x86, I see 3 failures. Is it
> >> expected?
> >>> I was expecting svn tip on x86 to be clean. Is there a nightly
> >> regression
> >>> result available somewhere that I can check for broken tests. Here
> >> the
> >>> report from my log:
> >>> Failing Tests (3):
> >>> Clang :: Analysis/iterators.cpp
> >>> Clang :: Driver/hello.c
> >>> Clang :: Preprocessor/macro_paste_c_block_comment.c
> >>> Expected Passes : 8757
> >>> Expected Failures : 69
> >>> Unsupported Tests : 551
> >>> Unexpected Failures: 3
> >> Because you're living life on the bleeding edge (i.e., pulling code
> >> from the svn repository) it's possible for this to happen from time
> >> time. A fix is probably already in the works. One way to ensure
> >> don't introduce a new error is to maintain a clean (i.e., unmodified
> >> version of llvm/clang), which you can use as a baseline. Run
> >> regressions on both the clean and modified versions to see if you
> >> any additional failures.
> > Makes sense.
> >>> 2. I see clang has its own test suite under clang/test. I assume
> >> I run
> >>> llvm/test, the test infrastructure also runs clang/test.
> >> How are you invoking the tests? I typically do 'make check-all'
> >> my build directory, which runs both regression suites.
> > Yes, I am doing "make check-all".
> >>> 3. For running projects/test-suite, the instruction guide directs
> >> build
> >>> llvm-gcc. Why is llvm-gcc required? Why is clang not being used?
> >> Very possible the documentation needs updating.
> > I see. Do you run projects/test-suite with clang?
> From your build/project/test-suite directory, try running:
> make ARCH=x86_64 CC_UNDER_TEST_IS_CLANG=1
> CC_UNDER_TEST_TARGET_IS_X86_64=1 DISABLE_CBE=1 DISABLE_JIT=1
> ENABLE_HASHED_PROGRAM_OUTPUT=1 ENABLE_OPTIMIZED=1 LD_ENV_OVERRIDES="env
> LLC_OPTFLAGS=-O3 LLI_OPTFLAGS=-O3 OPTFLAGS=-O3 SMALL_PROBLEM_SIZE=1
> TARGET_CC="" TARGET_CXX="" TARGET_FLAGS="-arch x86_64"
> TEST=simple USE_REFERENCE_OUTPUT=1 report
> ***Note the ENABLE_OPTIMIZED=1 flag requires that you run configure
> with '--enable-optimized' at config time (i.e., ../llvm/configure --
> You may have to fiddle with the options, but I think this will get you
> going in the right direction.
> >>> 4. I see many compile errors when running projects/test-suite. Is
> >>> expected?
> >> No.
> > I will try to debug. Do you have a log file from your previous runs
> that I
> > can use?
> I don't have any on hand, nor do I think they would be of help. Try
> the above make command a see what happens. For the most part
> everything should PASS.
> > Thanks Chad. I appreciate it.
> You're welcome Sundeep.
> > -Sundeep
More information about the llvm-dev