[llvm-dev] [lit] check-all hanging
Kuba Mracek via llvm-dev
llvm-dev at lists.llvm.org
Thu Jan 3 07:53:25 PST 2019
+Fred, +Kostya
Sent from my iPhone
> On Jan 3, 2019, at 7:33 AM, Joel E. Denny via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>
> All,
>
> Thanks for the replies. Kuba: For LLDB, when were things expected to have improved? It's possible things improved for me at some point, but this isn't something I've found time to track carefully, and I still see problems.
>
> I ran check-all a couple of times last night at r350238, which I pulled yesterday. Here are the results:
>
> ```
> ********************
> Testing Time: 5043.24s
> ********************
> Unexpected Passing Tests (2):
> lldb-Suite :: functionalities/asan/TestMemoryHistory.py
> lldb-Suite :: functionalities/asan/TestReportData.py
>
> ********************
> Failing Tests (54):
> Clang :: CXX/modules-ts/basic/basic.link/p2/module.cpp
> Clang :: Modules/ExtDebugInfo.cpp
> Clang :: Modules/using-directive-redecl.cpp
> Clang :: Modules/using-directive.cpp
> Clang :: PCH/chain-late-anonymous-namespace.cpp
> Clang :: PCH/cxx-namespaces.cpp
> Clang :: PCH/namespaces.cpp
> LLDB :: ExecControl/StopHook/stop-hook-threads.test
> LeakSanitizer-AddressSanitizer-x86_64 :: TestCases/Linux/use_tls_dynamic.cc
> LeakSanitizer-Standalone-x86_64 :: TestCases/Linux/use_tls_dynamic.cc
> MemorySanitizer-X86_64 :: dtls_test.c
> MemorySanitizer-lld-X86_64 :: dtls_test.c
> lldb-Suite :: functionalities/register/register_command/TestRegisters.py
> lldb-Suite :: tools/lldb-server/TestGdbRemoteRegisterState.py
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestBorrowedReferences
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestDictionaryResolutionWithDot
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestExtractingUInt64ThroughStructuredData
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestGlobalNameResolutionNoDot
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestGlobalNameResolutionWithDot
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestInstanceNameResolutionNoDot
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestModuleNameResolutionNoDot
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestObjectAttributes
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestOwnedReferences
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonByteArray
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonBytes
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonCallableCheck
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonCallableInvoke
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonDictionaryManipulation
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonDictionaryToStructuredDictionary
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonDictionaryValueEquality
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonFile
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonInteger
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonIntegerToStr
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonIntegerToStructuredInteger
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonListManipulation
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonListToStructuredList
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonListValueEquality
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonString
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonStringToStr
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonStringToStructuredString
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonTupleInitializerList
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonTupleInitializerList2
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonTupleSize
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonTupleToStructuredList
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestPythonTupleValues
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestResetting
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonDataObjectsTest.TestTypeNameResolutionNoDot
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonExceptionStateTest.TestAcquisitionSemantics
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonExceptionStateTest.TestAutoRestoreChanged
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonExceptionStateTest.TestAutoRestoreSemantics
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonExceptionStateTest.TestDiscardSemantics
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonExceptionStateTest.TestExceptionStateChecking
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonExceptionStateTest.TestManualRestoreSemantics
> lldb-Unit :: ScriptInterpreter/Python/./ScriptInterpreterPythonTests/PythonExceptionStateTest.TestResetSemantics
>
> Expected Passes : 57489
> Expected Failures : 276
> Unsupported Tests : 1883
> Unexpected Passes : 2
> Unexpected Failures: 54
>
> 14 warning(s) in tests.
> FAILED: CMakeFiles/check-all
> ```
>
> I immediately ran it again and saw one new unexpected fail:
>
> ```
> lldb-Suite :: tools/lldb-mi/syntax/TestMiSyntax.py
> ```
>
> and one new unresolved test:
>
> ```
> lldb-Suite :: tools/lldb-vscode/breakpoint/TestVSCode_setBreakpoints.py
> ```
>
> On the second run but not the first, it hung all night long waiting for TestVSCode_setBreakpoints.py to terminate. I killed dotest.py to get the final results.
>
> I currently clone <https://github.com/llvm-project/llvm-project-20170507>. I configure with `BUILD_SHARED_LIBS=true` and `-DLLVM_ENABLE_PROJECTS='clang;openmp;libcxx;libcxxabi;lldb;compiler-rt;lld;polly'`, among other options. I have to run check-all with LD_LIBRARY_PATH pointing at my build's lib directory, or there are many more LLDB failures. I believe that's not true for most test suites. I'm building and testing under Ubuntu 18.04.1.
>
> Hope that helps. I'm happy to provide more details. Just tell me where you'd like to start.
>
> Thanks.
>
> Joel
>
>> On Wed, Jan 2, 2019 at 5:51 PM Kuba Mracek <mracek at apple.com> wrote:
>> +Fred, +me
>>
>> For LLDB tests: I believe this got much much better recently. Are you still seeing flaky LLDB tests? Any details you can share?
>> For sanitizer tests: I'm very much interesting in removing flakiness as well. Any specific tests you see as flaky?
>>
>> Kuba
>>
>>> On Jan 2, 2019, at 2:05 PM, Joel E. Denny via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>
>>> Hi David, Chandler,
>>>
>>> I see lldb tests hang often, and then I kill the dotest process.
>>>
>>> I'd like to stop running check-all too, but I feel it's important when I modify FileCheck. The flakiness that Chandler mentioned makes it time-consuming to verify test results.
>>>
>>> Joel
>>>
>>>> On Wed, Jan 2, 2019 at 4:41 PM Chandler Carruth via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>> What you're seeing is just the fact that lit is waiting on subprocesses (select is waiting on the pipes i suspect).
>>>>
>>>> Anyways, you'll need to dig into what it is waiting on, and what *that* process is doing that is stuck to make progress.
>>>>
>>>> I've not seen anything like this, but I basically never run `check-all` these days because LLDB and sanitizer tests are too flaky. =[ I've not been able to interest anyone in fixing this either sadly.
>>>>
>>>>> On Wed, Jan 2, 2019 at 10:09 AM David Greene via llvm-dev <llvm-dev at lists.llvm.org> wrote:
>>>>> Hi,
>>>>>
>>>>> From time to time, I see check-all hang during running of lit tests.
>>>>> The hang always happens at the > 90% completion stage and I'm pretty
>>>>> sure all tests have been run and check-all is just waiting for
>>>>> lit/python to exit. I see a single python processing running, taking
>>>>> very little CPU time. An strace of that process shows this:
>>>>>
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 32168}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 1000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 2000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 4000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 8000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 16000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 32000}) = 0 (Timeout)
>>>>> futex(0x3bcc8c0, FUTEX_WAKE_PRIVATE, 1) = 1
>>>>> futex(0x3bcc8c0, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff) = 0
>>>>> futex(0x3bcc8c0, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff) = -1 EAGAIN (Resourc
>>>>> e temporarily unavailable)
>>>>> futex(0x3bcc8c0, FUTEX_WAKE_PRIVATE, 1) = 1
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> futex(0x3bcc8c0, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 0, NULL, ffffffff) = -1 EAGAIN (Resourc
>>>>> e temporarily unavailable)
>>>>> futex(0x3bcc8c0, FUTEX_WAKE_PRIVATE, 1) = 1
>>>>> futex(0x3bcc8c0, FUTEX_WAKE_PRIVATE, 1) = 1
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>> select(0, NULL, NULL, NULL, {0, 50000}) = 0 (Timeout)
>>>>>
>>>>> It appears that python is waiting for some I/O or something which never
>>>>> appears.
>>>>>
>>>>> Has anyone else seen this before? Any ideas of what is going on or how
>>>>> to fix it?
>>>>>
>>>>> -David
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> llvm-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190103/c366c1b4/attachment.html>
More information about the llvm-dev
mailing list