[cfe-dev] Cannot CMake self-hosted clang on Windows for lack of libxml2

Mon Aug 23 15:17:21 PDT 2021

Yikes!  Thanks for the heads up.

This is the third time I've bisected to find build configuration problems
that affect only certain Windows configurations only to learn that it was
(primarily) a change in our tools that broke things (retroactively).  Since
the bot configurations aren't updated very often, it's easy for these
problems to go unnoticed for a very long time.

It's one thing not to have hermetic builds, but it's quite another to allow
every developer to choose from a wide range of versions for each of the
myriad tools necessary to build the product.  The builds are so sensitive
to so many details of the environment, it's rather amazing to me how often
we manage to succeed.

On Mon, Aug 23, 2021 at 12:43 PM Alexandre Ganea <
alexandre.ganea at ubisoft.com> wrote:

> Adrian,
>
>
>
> Like you’re suspecting, I think this whole situation was caused by
> https://github.com/Kitware/CMake/commit/b12aec6c8daa3e087e6d0fa0441f59622251eb46
>
> Before cmake 3.20 it used to look for mt.exe, and now it looks for
> llvm-mt.exe
>
> In essence, if you roll back to cmake 3.19 you won’t have this issue.
>
>
>
> I’ve spend an afternoon trying to compile with llvm-mt & libXml2 on
> Windows to no avail. If you manage to make it work for both x64 & x86 on a
> plain windows shell (not cygwin or mingw), please post the steps to get
> there ;-) From what I remember, all the precompiled libXm2.lib available on
> internet are older versions that don’t support one of the APIs used by
> llvm-mt (something in xmlReader.h I think).
>
>
>
> *De :* cfe-dev <cfe-dev-bounces at lists.llvm.org> *De la part de* Adrian
> McCarthy via cfe-dev
> *Envoyé :* August 23, 2021 3:20 PM
> *À :* Shoaib Meenai <smeenai at fb.com>
> *Cc :* cfe-dev <cfe-dev at lists.llvm.org>
> *Objet :* Re: [cfe-dev] Cannot CMake self-hosted clang on Windows for
> lack of libxml2
>
>
>
> I bisected back to c4c3883b00d3a6aa657a5e3e515c90c9ea1f81c6 .  Before
> that, our CMake scripts still required Python 2, which I no longer have
> installed (because it confuses lldb builds to have both Python 2 and 3 on
> the machine).  Thus the dependency was probably introduced some time before
> April 2020.
>
>
>
> So even if it was inadvertent, the problem has existed long enough that we
> probably just have to accept that libxml2 is now a requirement for LLVM
> builds on Windows.  I'll propose a documentation change.
>
>
>
> Adrian.
>
>
>
> On Fri, Aug 20, 2021 at 5:31 PM Shoaib Meenai <smeenai at fb.com> wrote:
>
> This is a workaround and not a solution, but you should be able to add
> /manifest:no to your link flags (similar to
> https://github.com/llvm/llvm-project/blob/3187a4f3f1d4d68a75a21154569db1232a62a57d/llvm/cmake/platforms/WinMsvc.cmake#L286-L287)
> to prevent CMake from trying to invoke mt.
>
>
>
> *From: *cfe-dev <cfe-dev-bounces at lists.llvm.org> on behalf of cfe-dev <
> cfe-dev at lists.llvm.org>
> *Reply-To: *Adrian McCarthy <amccarth at google.com>
> *Date: *Friday, August 20, 2021 at 4:35 PM
> *To: *cfe-dev <cfe-dev at lists.llvm.org>
> *Subject: *[cfe-dev] Cannot CMake self-hosted clang on Windows for lack
> of libxml2
>
>
>
> I recently tried to build a self-hosted clang on Windows, but it fails
> when I use CMake to generate a Ninja file.
>
>
>
> CMake finds my MSVC-built clang and tries to test it.  This invokes
> llvm-mt (the LLVM implementation of the manifest tool).  This tool depends
> on libxml2, which isn't typically installed on Windows.
>
>
>
> I'm told the Windows build shouldn't need to invoke llvm-mt, so this is
> probably due to a CMake change.  Unfortunately, I don't know how far back I
> have to go to find a working build so that I can bisect the breaking
> change.  It's been a long time since I've tried a self-hosted build.
>
>
>
> Maybe somebody changed this intentionally, and we just need to update the
> documentation to include libxml2 on the (already very long) list of things
> Windows developers have to install in order to build llvm.
>
>
>
> If you have knowledge of this, let me know.  In the meantime, I'll attempt
> to bisect.
>
>
>
> See below for the gory details.
>
>
>
> Adrian.
>
>
>
> cmake -GNinja -DPYTHON_HOME="C:\Program Files\Python38"
> -DLLDB_PYTHON_HOME="C:\Program Files\Python38"
> -DPYTHON_EXECUTABLE="C:\Program Files\Python38\bin\python.exe"
> -DPython3_ROOT_DIR="C:\Program Files\Python38" -DPython3_FIND_REGISTRY=LAST
> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb" -DLLVM_TARGETS_TO_BUILD=X86
> -DCMAKE_BUILD_TYPE=Release
> -DCMAKE_C_COMPILER=D:/src/llvm/build/ninja/bin/clang-cl.exe
> -DCMAKE_CXX_COMPILER=D:/src/llvm/build/ninja/bin/clang-cl.exe
>
>
>
> -- The C compiler identification is Clang 14.0.0 with MSVC-like
> command-line
> -- The CXX compiler identification is Clang 14.0.0 with MSVC-like
> command-line
> -- The ASM compiler identification is Clang
> -- Found assembler: D:/src/llvm/build/ninja/bin/clang-cl.exe
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - failed
> -- Check for working C compiler: D:/src/llvm/build/ninja/bin/clang-cl.exe
> -- Check for working C compiler: D:/src/llvm/build/ninja/bin/clang-cl.exe
> - broken
> CMake Error at C:/Program Files (x86)/Microsoft Visual
> Studio/2019/Professional/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/CMakeTestCCompiler.cmake:66
> (message):
>
>   The C compiler
>
>
>     "D:/src/llvm/build/ninja/bin/clang-cl.exe"
>
>   is not able to compile a simple test program.
>
>   It fails with the following output:
>
>     Change Dir: D:/src/llvm/build/selfhost/CMakeFiles/CMakeTmp
>
>     Run Build
> Command(s):C:/PROGRA~2/MICROS~1/2019/PROFES~1/Common7/IDE/COMMON~1/MICROS~1/CMake/Ninja/ninja.exe
> cmTC_62af5 && [1/2] Building C object
> CMakeFiles\cmTC_62af5.dir\testCCompiler.c.obj
>     [2/2] Linking C executable cmTC_62af5.exe
>     FAILED: cmTC_62af5.exe
>     cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual
> Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"
> -E vs_link_exe --intdir=CMakeFiles\cmTC_62af5.dir
> --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
> --mt=D:\src\llvm\build\ninja\bin\llvm-mt.exe --manifests  --
> D:\src\llvm\build\ninja\bin\lld-link.exe /nologo
> CMakeFiles\cmTC_62af5.dir\testCCompiler.c.obj  /out:cmTC_62af5.exe
> /implib:cmTC_62af5.lib /pdb:cmTC_62af5.pdb /version:0.0 /machine:x64
>  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib
> winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib
> advapi32.lib && cd ."
>     MT: command "D:\src\llvm\build\ninja\bin\llvm-mt.exe /nologo /manifest
> CMakeFiles\cmTC_62af5.dir/intermediate.manifest
> /out:CMakeFiles\cmTC_62af5.dir/embed.manifest /notify_update" failed (exit
> code 0x1) with the following output:
>     llvm-mt: error: no libxml2
>     llvm-mt: ignoring unsupported 'notify_update' option
>
>     ninja: build stopped: subcommand failed.
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20210823/1afe2471/attachment-0001.html>