[LLVMdev] LLVM / CLANG Test Infrastructure Question
mcrosier at apple.com
Thu Jul 28 10:36:42 PDT 2011
On Jul 28, 2011, at 8:18 AM, Sundeep wrote:
>>> 1. I checked out and built llvm, clang, and test-suite from svn tip.
>> When I
>>> run llvm/test *without* my changes on x86, I see 3 failures. Is it
>>> I was expecting svn tip on x86 to be clean. Is there a nightly
>>> result available somewhere that I can check for broken tests. Here is
>>> report from my log:
>>> Failing Tests (3):
>>> Clang :: Analysis/iterators.cpp
>>> Clang :: Driver/hello.c
>>> Clang :: Preprocessor/macro_paste_c_block_comment.c
>>> Expected Passes : 8757
>>> Expected Failures : 69
>>> Unsupported Tests : 551
>>> Unexpected Failures: 3
>> Because you're living life on the bleeding edge (i.e., pulling code
>> from the svn repository) it's possible for this to happen from time to
>> time. A fix is probably already in the works. One way to ensure you
>> don't introduce a new error is to maintain a clean (i.e., unmodified
>> version of llvm/clang), which you can use as a baseline. Run
>> regressions on both the clean and modified versions to see if you cause
>> any additional failures.
> Makes sense.
>>> 2. I see clang has its own test suite under clang/test. I assume when
>> I run
>>> llvm/test, the test infrastructure also runs clang/test.
>> How are you invoking the tests? I typically do 'make check-all' from
>> my build directory, which runs both regression suites.
> Yes, I am doing "make check-all".
>>> 3. For running projects/test-suite, the instruction guide directs to
>>> llvm-gcc. Why is llvm-gcc required? Why is clang not being used?
>> Very possible the documentation needs updating.
> I see. Do you run projects/test-suite with clang?
>From your build/project/test-suite directory, try running:
make ARCH=x86_64 CC_UNDER_TEST_IS_CLANG=1 CC_UNDER_TEST_TARGET_IS_X86_64=1 DISABLE_CBE=1 DISABLE_JIT=1 ENABLE_HASHED_PROGRAM_OUTPUT=1 ENABLE_OPTIMIZED=1 LD_ENV_OVERRIDES="env DYLD_LIBRARY_PATH=/path/to/your/build/Release+Asserts/lib" LLC_OPTFLAGS=-O3 LLI_OPTFLAGS=-O3 OPTFLAGS=-O3 SMALL_PROBLEM_SIZE=1 TARGET_CC="" TARGET_CXX="" TARGET_FLAGS="-arch x86_64" TARGET_LLVMGCC=/path/to/your/build/Release+Asserts/bin/clang TARGET_LLVMGXX=/path/to/your/build/Release+Asserts/bin/clang++ TEST=simple USE_REFERENCE_OUTPUT=1 report
***Note the ENABLE_OPTIMIZED=1 flag requires that you run configure with '--enable-optimized' at config time (i.e., ../llvm/configure --enable-optimized).
You may have to fiddle with the options, but I think this will get you going in the right direction.
>>> 4. I see many compile errors when running projects/test-suite. Is it
> I will try to debug. Do you have a log file from your previous runs that I
> can use?
I don't have any on hand, nor do I think they would be of help. Try the above make command a see what happens. For the most part everything should PASS.
> Thanks Chad. I appreciate it.
You're welcome Sundeep.
More information about the llvm-dev