[cfe-dev] Clang failing lots of FrontendC* tests

[Jeffrey Yasskin]

On Sat, Mar 6, 2010 at 9:09 PM, Douglas Gregor <dgregor at apple.com> wrote:
>
> On Mar 6, 2010, at 5:47 PM, Jeffrey Yasskin wrote:
>
> I managed to convince check-lit to run the FrontendC tests using clang
> by applying the following patch to my config.status (yes, it's a hack,
> and you'll have to change the paths to match your system):
>
> =========
> 661,664c661,664
> < s, at LLVMGCCCOMMAND@,|#_!!_#|llvm-gcc,g
> < s, at LLVMGXXCOMMAND@,|#_!!_#|llvm-g++,g
> < s, at LLVMGCC@,|#_!!_#|,g
> < s, at LLVMGXX@,|#_!!_#|,g
> ---
>
> s, at LLVMGCCCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
>
> s, at LLVMGXXCOMMAND@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
>
> s, at LLVMGCC@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang,g
>
> s, at LLVMGXX@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin/clang++,g
>
> 675,676c675,676
> < s, at LLVMGCCDIR@,|#_!!_#|,g
> < s, at LLVMGCC_LANGS@,|#_!!_#|,g
> ---
>
> s, at LLVMGCCDIR@,|#_!!_#|/Users/jyasskin/src/llvm/clang/install/bin,g
>
> s, at LLVMGCC_LANGS@,|#_!!_#|c\,c++\,objc\,obj-c++,g
>
> ========
>
> The following 110 tests failed, although the LLVMC ones don't look
> like real failures, and some of the ObjC ones are just warnings. I
> intend to just ignore these failures in my own testing, but I thought
> you guys might like to know. The full check-lit output is attached.
>
> Our view on the FrontendC* tests is that they're testing very specific
> behaviors of llvm-gcc that we don't necessarily care about for Clang. At
> some point, I'd like to look at the failures to convince myself that these
> tests aren't uncovering real bugs, but even when I do... I'd rather move the
> useful tests in Clang's regression test suite.

Thanks for looking. There are some interesting tests in FrontendC for
how llvm's backend handles debug information, for example, that aren't
covered by the rest of the test suite. (I know because they found a
bug in one of my recent changes that wasn't caught by the rest of the
suite.) Possibly we should just compile all of the FrontendC* tests to
IR and add them elsewhere in the test suite?