[lldb-dev] Need help with failing LLDB tests on Windows
Adrian McCarthy via lldb-dev
lldb-dev at lists.llvm.org
Mon Nov 9 17:00:13 PST 2020
Thanks for all the info and pointers. That's helping me zero in on the
problem.
This category of the failures appears to all be dotest.py tests, so it
makes sense that it's the second import statement (per Pavel's explanation).
The module is not being found because it's actually named _lldb_d.pyd.
Apparently the `_d` suffix is because I'm building debug. That seems
consistent with Stella's experience.
However, I've been building debug since before these problems arose. (In
fact, I've been working on fixes for a small number of tests that only fail
in debug, because of an assertion that detects the problem.)
Ted's got me thinking that it was working due to a symlink that somehow got
blown away and/or isn't being recreated by the build. If I recall
correctly, the symlinks on Windows are created using ln.exe, which may come
from GnuWin32 or from git/usr/bin. In my case, it's git/usr/bin. There
seem to have been many git updates in the past couple months, so perhaps
one of those updates tweaked ln.exe. That could have been the trigger for
me. Folks who didn't take the git update or who are configured to prefer
GnuWin32 tools might not have been affected.
I'll let you know what I eventually find.
On Wed, Nov 4, 2020 at 12:05 PM Ted Woodward <tedwood at quicinc.com> wrote:
> To expand a bit on what Pavel has written, the lldb module should be in
> <install>\lib\site-packages\lldb . In that directory is a file, _lldb.pyd,
> that should be a copy of <install>\bin\liblldb.dll .
>
> Do both files exist? Is _lldb.pyd a copy of liblldb.dll?
> See function create_relative_symlink in llvm-project/lldb/CMakeLists.txt
> for the copy (on non-unix hosts) procedure.
>
> Did you recently change your version of swig? LLDB requires swig 2, but,
> as you pointed out last year, there are issues with some versions of swig.
> We use 4.0.1 on Windows.
>
> > -----Original Message-----
> > From: lldb-dev <lldb-dev-bounces at lists.llvm.org> On Behalf Of Pavel
> Labath
> > via lldb-dev
> > Sent: Wednesday, November 4, 2020 2:49 AM
> > To: Adrian McCarthy <amccarth at google.com>; LLDB <lldb-
> > dev at lists.llvm.org>
> > Subject: [EXT] Re: [lldb-dev] Need help with failing LLDB tests on
> Windows
> >
> > On 04/11/2020 01:53, Adrian McCarthy via lldb-dev wrote:
> > > For the past couple weeks, I've been trying to figure out why
> > > approximately 900+ LLDB tests have been failing for me on my local
> > > Windows builds. Bisect turned up nothing--the "good" version that was
> > > working for me no longer works. Since nobody else seems to be seeing
> > > these failures, I suspect it's something environmental.
> > >
> > > There are three categories of errors. I'm currently focused on
> > > failures that look like this:
> > >
> > > FAIL: lldb-api :: lang/objc/unicode-string/TestUnicodeString.py
> (732
> > > of 2180)
> > > ******************** TEST 'lldb-api ::
> > > lang/objc/unicode-string/TestUnicodeString.py' FAILED
> > > ********************
> > > Script:
> > > --
> > > C:/Program Files/Python38/python.exe
> > > D:/src/llvm/llvm-project/lldb\test\API\dotest.py -S nm -u CXXFLAGS
> > > -u CFLAGS --enable-crash-dialog --env
> > > LLVM_LIBS_DIR=D:/src/llvm/build/ninja_dbg/./lib --arch x86_64
> > > --build-dir D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex -s
> > > D:/src/llvm/build/ninja_dbg/lldb-test-traces
> --lldb-module-cache-dir
> > > D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex/module-cache-
> > lldb\lldb-api
> > > --clang-module-cache-dir
> > > D:/src/llvm/build/ninja_dbg/lldb-test-build.noindex/module-cache-
> > clang\lldb-api
> > > --executable D:/src/llvm/build/ninja_dbg/./bin/lldb.exe --compiler
> > > D:/src/llvm/build/ninja_dbg/bin/clang.exe --dsymutil
> > > D:/src/llvm/build/ninja_dbg/./bin/dsymutil.exe --filecheck
> > > D:/src/llvm/build/ninja_dbg/./bin/FileCheck.exe --yaml2obj
> > > D:/src/llvm/build/ninja_dbg/./bin/yaml2obj.exe --lldb-libs-dir
> > > D:/src/llvm/build/ninja_dbg/./lib
> > > D:\src\llvm\llvm-project\lldb\test\API\lang\objc\unicode-string -p
> > > TestUnicodeString.py
> > > --
> > > Exit Code: 1
> > >
> > > Command Output (stdout):
> > > --
> > > lldb version 12.0.0 (https://github.com/llvm/llvm-project.git
> > > <https://github.com/llvm/llvm-project.git> revision
> > > 0fdcd1ae1c988fa19d0c97e99999e8678b93a0da)
> > > clang revision 0fdcd1ae1c988fa19d0c97e99999e8678b93a0da
> > > llvm revision 0fdcd1ae1c988fa19d0c97e99999e8678b93a0da
> > >
> > > --
> > > Command Output (stderr):
> > > --
> > > Traceback (most recent call last):
> > > File
> > > "D:\src\llvm\build\ninja_dbg\Lib\site-packages\lldb\__init__.py",
> > > line 35, in <module>
> > > import _lldb
> > > ModuleNotFoundError: No module named '_lldb'
> > >
> > > During handling of the above exception, another exception occurred:
> > >
> > > Traceback (most recent call last):
> > > File "D:/src/llvm/llvm-project/lldb\test\API\dotest.py", line 7,
> > > in <module>
> > > lldbsuite.test.run_suite()
> > > File
> > > "D:\src\llvm\llvm-
> > project\lldb\packages\Python\lldbsuite\test\dotest.py",
> > > line 874, in run_suite
> > > import lldb
> > > File
> > > "D:\src\llvm\build\ninja_dbg\Lib\site-packages\lldb\__init__.py",
> > > line 38, in <module>
> > > from . import _lldb
> > > ImportError: cannot import name '_lldb' from partially initialized
> > > module 'lldb' (most likely due to a circular import)
> > > (D:\src\llvm\build\ninja_dbg\Lib\site-packages\lldb\__init__.py)
> > >
> > >
> > > It looks like the code in question is generated by Swig (so perhaps it
> > > depends on the version of Swig?). The relevant bit seems to be:
> > >
> > > try:
> > > # Try an absolute import first. If we're being loaded from
> lldb,
> > > # _lldb should be a built-in module.
> > > import _lldb
> > > except ImportError:
> > > # Relative import should work if we are being loaded by
> Python.
> > > from . import _lldb
> > >
> > >
> > > I don't have much background in Python modules or how Swig produces
> > > the bindings. It seems suspicious to me that both import attempts are
> > > failing (and that we need two).
> >
> > The reason behind the two imports is that lldb+python have two ways of
> > loading each other, depending on who is "on top".
> >
> > If you're starting with a c++ program (e.g. lldb driver), then the
> (lib)lldb library
> > will be loaded first. It will register itself as a "builtin" python
> module so that
> > "import _lldb" loads _it_, instead of trying to load another copy of
> lldb.
> >
> > OTOH, if we are starting from python (like the dotests do), then there
> is no
> > builtin module, and we want to use the second import statement to load
> lldb
> > relative to the __init__.py location.
> >
> > The fact that the selection of the two methods is implemented by catching
> > the exceptions from the first attempt is not ideal. It's possible this
> could be
> > implemented differently (we'd need to find some other way to determine
> > which scenario are we in). However, I don't think that will fix the
> problem
> > you're running into.
> >
> > Regarding python3.8+windows, we also have this
> > <https://bugs.llvm.org/show_bug.cgi?id=46891> bug open, but that also
> > doesn't sound like the same issue.
> >
> > BTW, this particular piece of code comes from
> > lldb/bindings/python/python.swig, so it is fairly easy to change that.
> >
> > > I'm hoping someone can offer some clues about what's going on here
> > >and how it's supposed to work. Is the hint about an import cycle
> > >relevant or a red herring?
> >
> > It sounds like a red herring. I get the same error (on linux+python3.8)
> if I
> > delete _lldb.so. So it sounds to me like python is having trouble
> finding the
> > native module (either it's not there or it has wrong debug-ness).
> >
> > It's also good to check whether you are able to use python scripting from
> > inside the lldb driver (e.g. lldb -o "script 1+1").
> >
> > pl
> > _______________________________________________
> > lldb-dev mailing list
> > lldb-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20201109/b0a4de2b/attachment.html>
More information about the lldb-dev
mailing list