[llvm-dev] How to debug a test that fails only on some build bots?

[Friedman, Eli via llvm-dev]

On 1/4/2018 4:51 PM, Adrian McCarthy via llvm-dev wrote:
> I tried to land a small fix before going on vacation at the end of the 
> year, but I had to revert it because it broke on a few of the build 
> bots, specifically clang-cmake-thumbv7-a15, clang-cmake-armv7-a15, and 
> llvm-hexagon-elf.
>
> https://reviews.llvm.org/D41264

I think you meant to put the test into test/DebugInfo/X86/ ?  Your test 
specifies an x86 target, but those buildbots don't build the x86 target.

>
> It seemed to work as expected on the others.
>
> Unfortunately, the build logs don't give many clues.  The fix included 
> a new test, and that test failed with an exit code of 1 but with no 
> messages.  For example:
> FAIL: LLVM :: DebugInfo/void-typedef.ll (14416 of 23113) 
> ******************** TEST 'LLVM :: DebugInfo/void-typedef.ll' FAILED 
> ******************** Script: -- 
> /local/buildbot/slaves/hexagon-build-02/llvm-hexagon-elf/llvm.obj/bin/llc 
> /local/buildbot/slaves/hexagon-build-02/llvm-hexagon-elf/llvm.src/test/DebugInfo/void-typedef.ll 
> -o - 2>&1 | 
> /local/buildbot/slaves/hexagon-build-02/llvm-hexagon-elf/llvm.obj/bin/FileCheck 
> /local/buildbot/slaves/hexagon-build-02/llvm-hexagon-elf/llvm.src/test/DebugInfo/void-typedef.ll 
> -- Exit Code: 1 ********************

llc is probably producing some error message like "error: unable to get 
target for 'i686', see --version and --triple.", but you're redirecting 
stderr, so it isn't visible.  You should try to avoid this when possible.

Your test doesn't have any positive CHECK lines, so FileCheck passes.  
You should generally try to include some positive CHECK line in 
regression tests.  And you never need to check for the string "Assertion 
failed"; the test will fail if any command on the RUN line fails or crashes.

-Eli

-- 
Employee of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180104/3b54adc2/attachment.html>