[lldb-dev] LLDB_PYTHON_HOME

Thu Feb 27 11:14:29 PST 2020

So to make my previous explanation more concrete:

On Thu, Feb 27, 2020 at 11:05 AM Jonas Devlieghere <jonas at devlieghere.com>
wrote:

>
>
> On Thu, Feb 27, 2020 at 10:53 AM Adrian McCarthy <amccarth at google.com>
> wrote:
>
>> Thanks for the info.  Setting Python3_ROOT_DIR solves the problem.
>>
>> Looking at the cmake output from before setting Python3_ROOT_DIR, cmake
>> looks for Python twice and finds it at the two different locations.
>>
>> Early on:
>>
>> -- Found PythonInterp: C:/Python36/python.exe (found version "3.6.8")
>>
>
^ This is using the "old" (CMake < 3.12) way of finding the Python
interpreter.

>
>> Which looks good (modulo the incorrect slash direction).  But later:
>>
>> -- Found Python3: C:/Program Files (x86)/Microsoft Visual
>> Studio/Shared/Python37_64/python.exe (found version "3.7.5") found
>> components:  Interpreter Development
>> -- Found PythonInterpAndLibs: C:/Program Files (x86)/Microsoft Visual
>> Studio/Shared/Python37_64/libs/python37.lib
>>
>
^ This is using the "new" (CMake > 3.12) way of finding the Python
interpreter and libraries.

>
>> Which is where the discrepancy comes in.  Note that only C:\Python36 is
>> in my PATH.
>>
>> It's frustrating that this keeps breaking.  Last time, I had to purge all
>> but one Python installation from my machine to get it to make a consistent
>> choice.  But I just upgraded to VS 2019, and it smuggled in its own version.
>>
>> So why are there two searches anyway?  And why do they have different
>> algorithms that lead to different results?  (I'm not sure _how_ it ever
>> found the Microsoft copy, since there's nothing in the process environment
>> that points that way.)
>>
>
> The reason there's two searches is because LLVM and LLDB have different
> requirements. LLVM just needs a python interpreter to run some scripts.
> LLDB on the other hand needs an interpreter and a matching Python library
> to link against. Before CMake 3.12, finding the interpreter and the
> libraries are two separate calls to find with no guarantees that they
> match. This lead to all kinds of issues, where you're linking against one
> version of Python and then trying to run the test suite with a totally
> different interpreter. There were other problems on Windows, which meant
> that we had our own hand-rolled implementation to find Python.
>
> This was all fixed in CMake 3.12. With FindPython{2,3} you know you'll
> have a matching interpreter and library. It also fixed all the problems we
> had to work around for Windows. Unfortunately, LLVM's minimum CMake version
> is 3.4, so we can't use it yet. For LLDB on Windows we agreed that the
> benefits of using FindPython3 are worth bumping the minimum required CMake
> version (see lldb/CMakeLists.txt, line 2-4). Once LLVM moves to CMake 3.12
> or later, all these problems should be fixed. We can then call FindPython3
> once and rely on everything being consistent.
>
>
>>
>> On Thu, Feb 27, 2020 at 10:23 AM Jonas Devlieghere <jonas at devlieghere.com>
>> wrote:
>>
>>> Hey Adrian,
>>>
>>> Config.h gets generated by expanding the corresponding CMake variables.
>>> If you look at LLDBConfig.cmake, you can see that LLDB_PYTHON_HOME is
>>> computed from PYTHON_EXECUTABLE. The problem appears that somehow CMake
>>> ignored your specified PYTHON_HOME and decided to pick a different Python.
>>> I'm not sure why though, because I use a similar CMake invocation on
>>> Windows.
>>>
>>> > cmake ..\llvm-project\llvm -G Ninja -DCMAKE_BUILD_TYPE=RelWithDebInfo
>>> -DLLVM_ENABLE_PROJECTS="llvm;clang;lldb;lld" -DLLVM_ENABLE_ASSERTIONS=OFF
>>> -DLLVM_ENABLE_ZLIB=FALSE -DLLDB_ENABLE_PYTHON=TRUE
>>> -DPYTHON_HOME="C:/Program Files/Python36/"
>>>
>>> According to FindPython3 (
>>> https://cmake.org/cmake/help/v3.12/module/FindPython3.html), you can
>>> set Python3_ROOT_DIR as a hint. Can you give that a try? If that works we
>>> should populate that variable from PYTHON_HOME in
>>> FindPythonInterpAndLibs.cmake.
>>>
>>> Cheers,
>>> Jonas
>>>
>>> On Thu, Feb 27, 2020 at 10:10 AM Adrian McCarthy via lldb-dev <
>>> lldb-dev at lists.llvm.org> wrote:
>>>
>>>> Is there documentation on how lldb\include\lldb\host\config.h is
>>>> generated?  I'm again having the problem of the config trying to point to
>>>> the wrong Python installation.
>>>>
>>>> When I run cmake, I explicitly point PYTHON_HOME to C:\Python36 like
>>>> this:
>>>>
>>>> cmake -GNinja -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON
>>>> -DCMAKE_BUILD_TYPE=Debug -DLLDB_TEST_DEBUG_TEST_CRASHES=1
>>>> -DPYTHON_HOME=C:\Python36
>>>> -DLLDB_TEST_COMPILER=D:\src\llvm\build\ninja\bin\clang.exe
>>>> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
>>>> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb"
>>>>
>>>> But the generated Config.h contains:
>>>>
>>>> #define LLDB_PYTHON_HOME "C:/Program Files (x86)/Microsoft Visual
>>>> Studio/Shared/Python37_64"
>>>>
>>>>
>>>> And the mismatch causes my build to fail because it goes looking for
>>>> python37_d.dll, which is apparently not part of the Microsoft distribution.
>>>> _______________________________________________
>>>> lldb-dev mailing list
>>>> lldb-dev at lists.llvm.org
>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>>>
I've put up a patch to use PYTHON_HOME as the hint:
https://reviews.llvm.org/D75275
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20200227/be965b85/attachment.html>