[llvm-dev] Many LLDB test failures in 13.0.0-rc1 on macOS

[Tobias Hieta via llvm-dev]

Hello Stefan,

I haven't tried to rebuild the release branch recently. I will try
soon and report back.

-- Tobias

On Sun, Aug 22, 2021 at 4:11 PM Stefan Gränitz
<stefan.graenitz at gmail.com> wrote:
>
> Hi Tobias
>
> What's the latest status here? I just made a local build from the tagged
> version on macOS 11.5.2 and CommandLineTools 12.5.1. Running the entire
> test suite, I do see the typical flakiness but nothing surprising:
>
> Failed Tests (8):
>   lldb-api :: commands/expression/diagnostics/TestExprDiagnostics.py
>   lldb-api :: commands/platform/sdk/TestPlatformSDK.py
>   lldb-api :: lang/c/modules/TestCModules.py
>   lldb-api :: lang/objc/modules-auto-import/TestModulesAutoImport.py
>   lldb-api ::
> lang/objc/modules-non-objc-target/TestObjCModulesNonObjCTarget.py
>   lldb-api :: lang/objc/modules-objc-property/TestModulesObjCProperty.py
>   lldb-api :: lang/objc/modules/TestObjCModules.py
>   lldb-api :: macosx/posix_spawn/TestLaunchProcessPosixSpawn.py
>
> Unexpectedly Passed Tests (1):
>   lldb-api :: lang/objc/foundation/TestRuntimeTypes.py
>
> Am I missing anything here? My major concern is that the
> Shell/Breakpoint/jit-loader tests keep working in upcoming releases.
>
> Thanks
> Stefan
>
> On 16/08/2021 11:12, Tobias Hieta via llvm-dev wrote:
> > Hello,
> >
> > While testing LLVM 13.0.0-rc1 on macOS I noticed many lldb test
> > failures that are regressions from 12.0.1.
> >
> > I was going to post this to lldb-dev - but I was not subscribed and
> > the process seems manual still. So forgive me for posting it here. Let
> > me know if I can share anymore information about these test failures
> > with the team.
> >
> > List of failing tests:
> > FAIL: lldb-shell :: Commands/command-thread-select.test (94817 of 96071)
> > FAIL: lldb-shell :: Breakpoint/jit-loader_jitlink_elf.test (94818 of 96071)
> > FAIL: lldb-shell :: Breakpoint/jit-loader_rtdyld_elf.test (94819 of 96071)
> > FAIL: lldb-shell :: ExecControl/StopHook/stop-hook.test (94823 of 96071)
> > FAIL: lldb-shell :: Host/TestCustomShell.test (94842 of 96071)
> > FAIL: lldb-shell :: Expr/TestIRMemoryMap.test (94849 of 96071)
> > FAIL: lldb-shell :: Process/Optimization.test (94852 of 96071)
> > FAIL: lldb-shell :: Process/UnsupportedLanguage.test (94853 of 96071)
> > FAIL: lldb-shell :: Process/TestEnvironment.test (94854 of 96071)
> > FAIL: lldb-shell :: Register/x86-64-gp-read.test (94855 of 96071)
> > FAIL: lldb-shell :: Register/x86-64-read.test (94862 of 96071)
> > FAIL: lldb-shell :: Register/x86-64-ymm-read.test (94866 of 96071)
> > FAIL: lldb-shell :: Register/x86-multithread-read.test (94873 of 96071)
> > FAIL: lldb-shell :: Register/x86-multithread-write.test (94876 of 96071)
> > FAIL: lldb-shell :: Reproducer/Functionalities/TestImageList.test
> > (94879 of 96071)
> > FAIL: lldb-shell :: Reproducer/Functionalities/TestDataFormatter.test
> > (94880 of 96071)
> > FAIL: lldb-shell ::
> > Reproducer/Functionalities/TestExpressionEvaluation.test (94882 of
> > 96071)
> > FAIL: lldb-shell :: Reproducer/Functionalities/TestStepping.test
> > (94883 of 96071)
> > FAIL: lldb-shell :: Reproducer/Modules/TestModuleCXX.test (94884 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestDSYM.test (94902 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestDump.test (94922 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestFileRepro.test (94924 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestGDBRemoteRepro.test (94931 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestMultipleTargets.test (94932 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestRelativePath.test (94934 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestReuseDirectory.test (94946 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestVersionCheck.test (94947 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestVerify.test (94948 of 96071)
> > FAIL: lldb-shell :: Reproducer/TestWorkingDir.test (94950 of 96071)
> > FAIL: lldb-shell :: Settings/TestFrameFormatColor.test (94957 of 96071)
> > FAIL: lldb-shell :: Settings/TestFrameFormatMangling.test (94961 of 96071)
> > FAIL: lldb-shell :: Settings/TestFrameFormatNoColor.test (94969 of 96071)
> > FAIL: lldb-shell :: Settings/TestLineMarkerColor.test (94975 of 96071)
> > FAIL: lldb-shell :: Subprocess/fork-follow-parent-wp.test (95010 of 96071)
> > FAIL: lldb-shell :: Subprocess/fork-follow-parent.test (95032 of 96071)
> > FAIL: lldb-shell :: Subprocess/vfork-follow-parent.test (95045 of 96071)
> > FAIL: lldb-shell :: SymbolFile/DWARF/deterministic-build.cpp (95046 of 96071)
> > FAIL: lldb-shell :: Unwind/eh-frame-dwarf-unwind.test (95219 of 96071)
> > FAIL: lldb-shell :: Unwind/thread-step-out-ret-addr-check.test (96063 of 96071)
> > FAIL: lldb-shell :: Unwind/trap_frame_sym_ctx.test (96066 of 96071)
> > FAIL: lldb-shell :: Watchpoint/SetErrorCases.test (96067 of 96071)
> > FAIL: lldb-unit ::
> > tools/lldb-server/tests/./LLDBServerTests/StandardStartupTest.TestStopReplyContainsThreadPcs
> > (96068 of 96071)
> >
> > -- Tobias
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> --
> https://weliveindetail.github.io/blog/about/
>
>