[lldb-dev] Need help with failing LLDB tests on Windows
Adrian McCarthy via lldb-dev
lldb-dev at lists.llvm.org
Tue Dec 8 10:28:21 PST 2020
FYI: The problem was the SWIG bug that becomes critical when combined with
Python 3.7+. Updating SWIG to 4.0.2 resolved all of the test failures.
I'm not sure why this wasn't consistently a problem before, though, since I
was already using this combination of versions for a while before the tests
started failing.
There's actually a Cmake warning about the version incompatibility. I'm
going to propose that this warning be upgraded to an error.
Thanks everyone!
On Tue, Nov 10, 2020 at 3:06 PM Tatyana Krasnukha <
Tatyana.Krasnukha at synopsys.com> wrote:
> On Windows one should run debug version of Python (python_d.exe) to load
> debug version of liblldb.dll. I hope this will help you.
>
>
>
> *From:* lldb-dev <lldb-dev-bounces at lists.llvm.org> *On Behalf Of *Adrian
> McCarthy via lldb-dev
> *Sent:* Tuesday, November 10, 2020 4:00 AM
> *To:* Ted Woodward <tedwood at quicinc.com>
> *Cc:* LLDB <lldb-dev at lists.llvm.org>
> *Subject:* Re: [lldb-dev] Need help with failing LLDB tests on Windows
>
>
>
> Thanks for all the info and pointers. That's helping me zero in on the
> problem.
>
>
>
> This category of the failures appears to all be dotest.py tests, so it
> makes sense that it's the second import statement (per Pavel's explanation).
>
>
>
> The module is not being found because it's actually named _lldb_d.pyd.
> Apparently the `_d` suffix is because I'm building debug. That seems
> consistent with Stella's experience.
>
>
>
> However, I've been building debug since before these problems arose. (In
> fact, I've been working on fixes for a small number of tests that only fail
> in debug, because of an assertion that detects the problem.)
>
>
>
> Ted's got me thinking that it was working due to a symlink that somehow
> got blown away and/or isn't being recreated by the build. If I recall
> correctly, the symlinks on Windows are created using ln.exe, which may come
> from GnuWin32 or from git/usr/bin. In my case, it's git/usr/bin. There
> seem to have been many git updates in the past couple months, so perhaps
> one of those updates tweaked ln.exe. That could have been the trigger for
> me. Folks who didn't take the git update or who are configured to prefer
> GnuWin32 tools might not have been affected.
>
>
>
> I'll let you know what I eventually find.
>
>
>
> On Wed, Nov 4, 2020 at 12:05 PM Ted Woodward <tedwood at quicinc.com> wrote:
>
> To expand a bit on what Pavel has written, the lldb module should be in
> <install>\lib\site-packages\lldb . In that directory is a file, _lldb.pyd,
> that should be a copy of <install>\bin\liblldb.dll .
>
> Do both files exist? Is _lldb.pyd a copy of liblldb.dll?
> See function create_relative_symlink in llvm-project/lldb/CMakeLists.txt
> for the copy (on non-unix hosts) procedure.
>
> Did you recently change your version of swig? LLDB requires swig 2, but,
> as you pointed out last year, there are issues with some versions of swig.
> We use 4.0.1 on Windows.
>
> > -----Original Message-----
> > From: lldb-dev <lldb-dev-bounces at lists.llvm.org> On Behalf Of Pavel
> Labath
> > via lldb-dev
> > Sent: Wednesday, November 4, 2020 2:49 AM
> > To: Adrian McCarthy <amccarth at google.com>; LLDB <lldb-
> > dev at lists.llvm.org>
> > Subject: [EXT] Re: [lldb-dev] Need help with failing LLDB tests on
> Windows
> >
> > On 04/11/2020 01:53, Adrian McCarthy via lldb-dev wrote:
> > > For the past couple weeks, I've been trying to figure out why
> > > approximately 900+ LLDB tests have been failing for me on my local
> > > Windows builds. Bisect turned up nothing--the "good" version that was
> > > working for me no longer works. Since nobody else seems to be seeing
> > > these failures, I suspect it's something environmental.
> > >
> > > There are three categories of errors. I'm currently focused on
> > > failures that look like this:
> > >
> > > FAIL: lldb-api :: lang/objc/unicode-string/TestUnicodeString.py
> (732
> > > of 2180)
> > > ******************** TEST 'lldb-api ::
> > > lang/objc/unicode-string/TestUnicodeString.py' FAILED
> > > ********************
> > > Script:
> > > --
> > > C:/Program Files/Python38/python.exe
> > > D:/src/llvm/llvm-project/lldb\test\API\dotest.py -S nm -u CXXFLAGS
> > > -u CFLAGS --enable-crash-dialog --env
> > > LLVM_LIBS_DIR=D:/src/llvm/build/ninja_dbg/./lib --arch x86_64
> > > --build-dir D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex -s
> > > D:/src/llvm/build/ninja_dbg/lldb-test-traces
> --lldb-module-cache-dir
> > > D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex/module-cache-
> > lldb\lldb-api
> > > --clang-module-cache-dir
> > > D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex/module-cache-
> > clang\lldb-api
> > > --executable D:/src/llvm/build/ninja_dbg/./bin/lldb.exe --compiler
> > > D:/src/llvm/build/ninja_dbg/bin/clang.exe --dsymutil
> > > D:/src/llvm/build/ninja_dbg/./bin/dsymutil.exe --filecheck
> > > D:/src/llvm/build/ninja_dbg/./bin/FileCheck.exe --yaml2obj
> > > D:/src/llvm/build/ninja_dbg/./bin/yaml2obj.exe --lldb-libs-dir
> > > D:/src/llvm/build/ninja_dbg/./lib
> > > D:\src\llvm\llvm-project\lldb\test\API\lang\objc\unicode-string -p
> > > TestUnicodeString.py
> > > --
> > > Exit Code: 1
> > >
> > > Command Output (stdout):
> > > --
> > > lldb version 12.0.0 (https://github.com/llvm/llvm-project.git
> <https://urldefense.com/v3/__https:/github.com/llvm/llvm-project.git__;!!A4F2R9G_pg!Jabk6KDSuWsHly57fuTAqQs8uS9tkYAt92H-UgbVnuz-reHkfeimBjQbWAe1kxnspRrcm9D_$>
> > > <https://github.com/llvm/llvm-project.git
> <https://urldefense.com/v3/__https:/github.com/llvm/llvm-project.git__;!!A4F2R9G_pg!Jabk6KDSuWsHly57fuTAqQs8uS9tkYAt92H-UgbVnuz-reHkfeimBjQbWAe1kxnspRrcm9D_$>>
> revision
> > > 0fdcd1ae1c988fa19d0c97e99999e8678b93a0da)
> > > clang revision 0fdcd1ae1c988fa19d0c97e99999e8678b93a0da
> > > llvm revision 0fdcd1ae1c988fa19d0c97e99999e8678b93a0da
> > >
> > > --
> > > Command Output (stderr):
> > > --
> > > Traceback (most recent call last):
> > > File
> > > "D:\src\llvm\build\ninja_dbg\Lib\site-packages\lldb\__init__.py",
> > > line 35, in <module>
> > > import _lldb
> > > ModuleNotFoundError: No module named '_lldb'
> > >
> > > During handling of the above exception, another exception occurred:
> > >
> > > Traceback (most recent call last):
> > > File "D:/src/llvm/llvm-project/lldb\test\API\dotest.py", line 7,
> > > in <module>
> > > lldbsuite.test.run_suite()
> > > File
> > > "D:\src\llvm\llvm-
> > project\lldb\packages\Python\lldbsuite\test\dotest.py",
> > > line 874, in run_suite
> > > import lldb
> > > File
> > > "D:\src\llvm\build\ninja_dbg\Lib\site-packages\lldb\__init__.py",
> > > line 38, in <module>
> > > from . import _lldb
> > > ImportError: cannot import name '_lldb' from partially initialized
> > > module 'lldb' (most likely due to a circular import)
> > > (D:\src\llvm\build\ninja_dbg\Lib\site-packages\lldb\__init__.py)
> > >
> > >
> > > It looks like the code in question is generated by Swig (so perhaps it
> > > depends on the version of Swig?). The relevant bit seems to be:
> > >
> > > try:
> > > # Try an absolute import first. If we're being loaded from
> lldb,
> > > # _lldb should be a built-in module.
> > > import _lldb
> > > except ImportError:
> > > # Relative import should work if we are being loaded by
> Python.
> > > from . import _lldb
> > >
> > >
> > > I don't have much background in Python modules or how Swig produces
> > > the bindings. It seems suspicious to me that both import attempts are
> > > failing (and that we need two).
> >
> > The reason behind the two imports is that lldb+python have two ways of
> > loading each other, depending on who is "on top".
> >
> > If you're starting with a c++ program (e.g. lldb driver), then the
> (lib)lldb library
> > will be loaded first. It will register itself as a "builtin" python
> module so that
> > "import _lldb" loads _it_, instead of trying to load another copy of
> lldb.
> >
> > OTOH, if we are starting from python (like the dotests do), then there
> is no
> > builtin module, and we want to use the second import statement to load
> lldb
> > relative to the __init__.py location.
> >
> > The fact that the selection of the two methods is implemented by catching
> > the exceptions from the first attempt is not ideal. It's possible this
> could be
> > implemented differently (we'd need to find some other way to determine
> > which scenario are we in). However, I don't think that will fix the
> problem
> > you're running into.
> >
> > Regarding python3.8+windows, we also have this
> > <https://bugs.llvm.org/show_bug.cgi?id=46891
> <https://urldefense.com/v3/__https:/bugs.llvm.org/show_bug.cgi?id=46891__;!!A4F2R9G_pg!Jabk6KDSuWsHly57fuTAqQs8uS9tkYAt92H-UgbVnuz-reHkfeimBjQbWAe1kxnspUyzoU1N$>>
> bug open, but that also
> > doesn't sound like the same issue.
> >
> > BTW, this particular piece of code comes from
> > lldb/bindings/python/python.swig, so it is fairly easy to change that.
> >
> > > I'm hoping someone can offer some clues about what's going on here
> > >and how it's supposed to work. Is the hint about an import cycle
> > >relevant or a red herring?
> >
> > It sounds like a red herring. I get the same error (on linux+python3.8)
> if I
> > delete _lldb.so. So it sounds to me like python is having trouble
> finding the
> > native module (either it's not there or it has wrong debug-ness).
> >
> > It's also good to check whether you are able to use python scripting from
> > inside the lldb driver (e.g. lldb -o "script 1+1").
> >
> > pl
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
> <https://urldefense.com/v3/__https:/lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev__;!!A4F2R9G_pg!Jabk6KDSuWsHly57fuTAqQs8uS9tkYAt92H-UgbVnuz-reHkfeimBjQbWAe1kxnspd0OYR0R$>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20201208/4759d294/attachment-0001.html>
More information about the lldb-dev
mailing list