[cfe-dev] Cannot CMake self-hosted clang on Windows for lack of libxml2

Mon Aug 23 18:52:48 PDT 2021

We build llvm-mt and libxml2 on Windows for x64, you can see the CMake
invocation here:

https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket/8838067054724908241/+/u/libxml2/configure/execution_details
(this is libxml2)
https://logs.chromium.org/logs/fuchsia/buildbucket/cr-buildbucket/8838067054724908241/+/u/clang/configure/execution_details
(this is Clang)

What took me the longest to figure out is using -DLibXml2_DIR=... and
-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON. That's because we build libxml2 as a
static library (we want our toolchain to be self-contained) and we want
LLVM CMake build to use libxml2-config.cmake, not FindLibXml2.cmake, to
include the correct dependencies.

Let me know if you have any questions.

On Mon, Aug 23, 2021 at 12:43 PM Alexandre Ganea via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> Adrian,
>
>
>
> Like you’re suspecting, I think this whole situation was caused by
> https://github.com/Kitware/CMake/commit/b12aec6c8daa3e087e6d0fa0441f59622251eb46
>
> Before cmake 3.20 it used to look for mt.exe, and now it looks for
> llvm-mt.exe
>
> In essence, if you roll back to cmake 3.19 you won’t have this issue.
>
>
>
> I’ve spend an afternoon trying to compile with llvm-mt & libXml2 on
> Windows to no avail. If you manage to make it work for both x64 & x86 on a
> plain windows shell (not cygwin or mingw), please post the steps to get
> there ;-) From what I remember, all the precompiled libXm2.lib available on
> internet are older versions that don’t support one of the APIs used by
> llvm-mt (something in xmlReader.h I think).
>
>
>
> *De :* cfe-dev <cfe-dev-bounces at lists.llvm.org> *De la part de* Adrian
> McCarthy via cfe-dev
> *Envoyé :* August 23, 2021 3:20 PM
> *À :* Shoaib Meenai <smeenai at fb.com>
> *Cc :* cfe-dev <cfe-dev at lists.llvm.org>
> *Objet :* Re: [cfe-dev] Cannot CMake self-hosted clang on Windows for
> lack of libxml2
>
>
>
> I bisected back to c4c3883b00d3a6aa657a5e3e515c90c9ea1f81c6 .  Before
> that, our CMake scripts still required Python 2, which I no longer have
> installed (because it confuses lldb builds to have both Python 2 and 3 on
> the machine).  Thus the dependency was probably introduced some time before
> April 2020.
>
>
>
> So even if it was inadvertent, the problem has existed long enough that we
> probably just have to accept that libxml2 is now a requirement for LLVM
> builds on Windows.  I'll propose a documentation change.
>
>
>
> Adrian.
>
>
>
> On Fri, Aug 20, 2021 at 5:31 PM Shoaib Meenai <smeenai at fb.com> wrote:
>
> This is a workaround and not a solution, but you should be able to add
> /manifest:no to your link flags (similar to
> https://github.com/llvm/llvm-project/blob/3187a4f3f1d4d68a75a21154569db1232a62a57d/llvm/cmake/platforms/WinMsvc.cmake#L286-L287)
> to prevent CMake from trying to invoke mt.
>
>
>
> *From: *cfe-dev <cfe-dev-bounces at lists.llvm.org> on behalf of cfe-dev <
> cfe-dev at lists.llvm.org>
> *Reply-To: *Adrian McCarthy <amccarth at google.com>
> *Date: *Friday, August 20, 2021 at 4:35 PM
> *To: *cfe-dev <cfe-dev at lists.llvm.org>
> *Subject: *[cfe-dev] Cannot CMake self-hosted clang on Windows for lack
> of libxml2
>
>
>
> I recently tried to build a self-hosted clang on Windows, but it fails
> when I use CMake to generate a Ninja file.
>
>
>
> CMake finds my MSVC-built clang and tries to test it.  This invokes
> llvm-mt (the LLVM implementation of the manifest tool).  This tool depends
> on libxml2, which isn't typically installed on Windows.
>
>
>
> I'm told the Windows build shouldn't need to invoke llvm-mt, so this is
> probably due to a CMake change.  Unfortunately, I don't know how far back I
> have to go to find a working build so that I can bisect the breaking
> change.  It's been a long time since I've tried a self-hosted build.
>
>
>
> Maybe somebody changed this intentionally, and we just need to update the
> documentation to include libxml2 on the (already very long) list of things
> Windows developers have to install in order to build llvm.
>
>
>
> If you have knowledge of this, let me know.  In the meantime, I'll attempt
> to bisect.
>
>
>
> See below for the gory details.
>
>
>
> Adrian.
>
>
>
> cmake -GNinja -DPYTHON_HOME="C:\Program Files\Python38"
> -DLLDB_PYTHON_HOME="C:\Program Files\Python38"
> -DPYTHON_EXECUTABLE="C:\Program Files\Python38\bin\python.exe"
> -DPython3_ROOT_DIR="C:\Program Files\Python38" -DPython3_FIND_REGISTRY=LAST
> ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF
> -DLLVM_ENABLE_PROJECTS="clang;lld;lldb" -DLLVM_TARGETS_TO_BUILD=X86
> -DCMAKE_BUILD_TYPE=Release
> -DCMAKE_C_COMPILER=D:/src/llvm/build/ninja/bin/clang-cl.exe
> -DCMAKE_CXX_COMPILER=D:/src/llvm/build/ninja/bin/clang-cl.exe
>
>
>
> -- The C compiler identification is Clang 14.0.0 with MSVC-like
> command-line
> -- The CXX compiler identification is Clang 14.0.0 with MSVC-like
> command-line
> -- The ASM compiler identification is Clang
> -- Found assembler: D:/src/llvm/build/ninja/bin/clang-cl.exe
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - failed
> -- Check for working C compiler: D:/src/llvm/build/ninja/bin/clang-cl.exe
> -- Check for working C compiler: D:/src/llvm/build/ninja/bin/clang-cl.exe
> - broken
> CMake Error at C:/Program Files (x86)/Microsoft Visual
> Studio/2019/Professional/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/CMakeTestCCompiler.cmake:66
> (message):
>
>   The C compiler
>
>
>     "D:/src/llvm/build/ninja/bin/clang-cl.exe"
>
>   is not able to compile a simple test program.
>
>   It fails with the following output:
>
>     Change Dir: D:/src/llvm/build/selfhost/CMakeFiles/CMakeTmp
>
>     Run Build
> Command(s):C:/PROGRA~2/MICROS~1/2019/PROFES~1/Common7/IDE/COMMON~1/MICROS~1/CMake/Ninja/ninja.exe
> cmTC_62af5 && [1/2] Building C object
> CMakeFiles\cmTC_62af5.dir\testCCompiler.c.obj
>     [2/2] Linking C executable cmTC_62af5.exe
>     FAILED: cmTC_62af5.exe
>     cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual
> Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe"
> -E vs_link_exe --intdir=CMakeFiles\cmTC_62af5.dir
> --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
> --mt=D:\src\llvm\build\ninja\bin\llvm-mt.exe --manifests  --
> D:\src\llvm\build\ninja\bin\lld-link.exe /nologo
> CMakeFiles\cmTC_62af5.dir\testCCompiler.c.obj  /out:cmTC_62af5.exe
> /implib:cmTC_62af5.lib /pdb:cmTC_62af5.pdb /version:0.0 /machine:x64
>  /debug /INCREMENTAL /subsystem:console  kernel32.lib user32.lib gdi32.lib
> winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib
> advapi32.lib && cd ."
>     MT: command "D:\src\llvm\build\ninja\bin\llvm-mt.exe /nologo /manifest
> CMakeFiles\cmTC_62af5.dir/intermediate.manifest
> /out:CMakeFiles\cmTC_62af5.dir/embed.manifest /notify_update" failed (exit
> code 0x1) with the following output:
>     llvm-mt: error: no libxml2
>     llvm-mt: ignoring unsupported 'notify_update' option
>
>     ninja: build stopped: subcommand failed.
>
>
>
>
>
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20210823/166e18b4/attachment-0001.html>