<div dir="ltr">Yikes! Thanks for the heads up.<div><br></div><div>This is the third time I've bisected to find build configuration problems that affect only certain Windows configurations only to learn that it was (primarily) a change in our tools that broke things (retroactively). Since the bot configurations aren't updated very often, it's easy for these problems to go unnoticed for a very long time.</div><div><br></div><div>It's one thing not to have hermetic builds, but it's quite another to allow every developer to choose from a wide range of versions for each of the myriad tools necessary to build the product. The builds are so sensitive to so many details of the environment, it's rather amazing to me how often we manage to succeed.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Aug 23, 2021 at 12:43 PM Alexandre Ganea <<a href="mailto:alexandre.ganea@ubisoft.com">alexandre.ganea@ubisoft.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-CA" style="overflow-wrap: break-word;">
<div class="gmail-m_-5000797608211638665WordSection1">
<p class="MsoNormal"><span>Adrian,<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>Like you’re suspecting, I think this whole situation was caused by
<a href="https://github.com/Kitware/CMake/commit/b12aec6c8daa3e087e6d0fa0441f59622251eb46" target="_blank">
https://github.com/Kitware/CMake/commit/b12aec6c8daa3e087e6d0fa0441f59622251eb46</a><u></u><u></u></span></p>
<p class="MsoNormal"><span>Before cmake 3.20 it used to look for mt.exe, and now it looks for llvm-mt.exe<u></u><u></u></span></p>
<p class="MsoNormal"><span>In essence, if you roll back to cmake 3.19 you won’t have this issue.<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>I’ve spend an afternoon trying to compile with llvm-mt & libXml2 on Windows to no avail. If you manage to make it work for both x64 & x86 on a plain windows shell (not cygwin or mingw), please post
the steps to get there ;-) From what I remember, all the precompiled libXm2.lib available on internet are older versions that don’t support one of the APIs used by llvm-mt (something in xmlReader.h I think).<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(225,225,225);padding:3pt 0cm 0cm">
<p class="MsoNormal"><b><span lang="FR">De :</span></b><span lang="FR"> cfe-dev <<a href="mailto:cfe-dev-bounces@lists.llvm.org" target="_blank">cfe-dev-bounces@lists.llvm.org</a>>
<b>De la part de</b> Adrian McCarthy via cfe-dev<br>
<b>Envoyé :</b> August 23, 2021 3:20 PM<br>
<b>À :</b> Shoaib Meenai <<a href="mailto:smeenai@fb.com" target="_blank">smeenai@fb.com</a>><br>
<b>Cc :</b> cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>
<b>Objet :</b> Re: [cfe-dev] Cannot CMake self-hosted clang on Windows for lack of libxml2<u></u><u></u></span></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">I bisected back to c4c3883b00d3a6aa657a5e3e515c90c9ea1f81c6 . Before that, our CMake scripts still required Python 2, which I no longer have installed (because it confuses lldb builds to have both Python 2 and 3 on the machine). Thus
the dependency was probably introduced some time before April 2020.<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">So even if it was inadvertent, the problem has existed long enough that we probably just have to accept that libxml2 is now a requirement for LLVM builds on Windows. I'll propose a documentation change.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Adrian.<u></u><u></u></p>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Fri, Aug 20, 2021 at 5:31 PM Shoaib Meenai <<a href="mailto:smeenai@fb.com" target="_blank">smeenai@fb.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-top:none;border-right:none;border-bottom:none;border-left:1pt solid rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal"><span lang="EN-US">This is a workaround and not a solution, but you should be able to add /manifest:no to your link flags (similar to
<a href="https://github.com/llvm/llvm-project/blob/3187a4f3f1d4d68a75a21154569db1232a62a57d/llvm/cmake/platforms/WinMsvc.cmake#L286-L287" target="_blank">
https://github.com/llvm/llvm-project/blob/3187a4f3f1d4d68a75a21154569db1232a62a57d/llvm/cmake/platforms/WinMsvc.cmake#L286-L287</a>) to prevent CMake from trying to invoke mt.<u></u><u></u></span></p>
<p class="MsoNormal"><span lang="EN-US"> <u></u><u></u></span></p>
<div style="border-right:none;border-bottom:none;border-left:none;border-top:1pt solid rgb(181,196,223);padding:3pt 0cm 0cm">
<p class="MsoNormal" style="margin-left:36pt">
<b><span lang="EN-US" style="font-size:12pt;color:black">From: </span></b><span lang="EN-US" style="font-size:12pt;color:black">cfe-dev <<a href="mailto:cfe-dev-bounces@lists.llvm.org" target="_blank">cfe-dev-bounces@lists.llvm.org</a>> on behalf of cfe-dev
<<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>
<b>Reply-To: </b>Adrian McCarthy <<a href="mailto:amccarth@google.com" target="_blank">amccarth@google.com</a>><br>
<b>Date: </b>Friday, August 20, 2021 at 4:35 PM<br>
<b>To: </b>cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>><br>
<b>Subject: </b>[cfe-dev] Cannot CMake self-hosted clang on Windows for lack of libxml2</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US">I recently tried to build a self-hosted clang on Windows, but it fails when I use CMake to generate a Ninja file.<u></u><u></u></span></p>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US">CMake finds my MSVC-built clang and tries to test it. This invokes llvm-mt (the LLVM implementation of the manifest tool). This tool depends on libxml2, which isn't typically installed on Windows.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US">I'm told the Windows build shouldn't need to invoke llvm-mt, so this is probably due to a CMake change. Unfortunately, I don't know how far back I have to go to find a working build so that I can bisect the breaking change. It's been a
long time since I've tried a self-hosted build.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US">Maybe somebody changed this intentionally, and we just need to update the documentation to include libxml2 on the (already very long) list of things Windows developers have to install in order to build llvm.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US">If you have knowledge of this, let me know. In the meantime, I'll attempt to bisect.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US">See below for the gory details.<u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US">Adrian.<u></u><u></u></span></p>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US" style="font-size:7.5pt;font-family:"Courier New",serif">cmake -GNinja -DPYTHON_HOME="C:\Program Files\Python38" -DLLDB_PYTHON_HOME="C:\Program Files\Python38" -DPYTHON_EXECUTABLE="C:\Program Files\Python38\bin\python.exe" -DPython3_ROOT_DIR="C:\Program
Files\Python38" -DPython3_FIND_REGISTRY=LAST ..\..\llvm-project\llvm -DLLVM_ENABLE_ZLIB=OFF -DLLVM_ENABLE_PROJECTS="clang;lld;lldb" -DLLVM_TARGETS_TO_BUILD=X86 -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=D:/src/llvm/build/ninja/bin/clang-cl.exe -DCMAKE_CXX_COMPILER=D:/src/llvm/build/ninja/bin/clang-cl.exe</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
</div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- The C compiler identification is Clang 14.0.0 with MSVC-like command-line</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- The CXX compiler identification is Clang 14.0.0 with MSVC-like command-line</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- The ASM compiler identification is Clang</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- Found assembler: D:/src/llvm/build/ninja/bin/clang-cl.exe</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- Detecting C compiler ABI info</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- Detecting C compiler ABI info - failed</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- Check for working C compiler: D:/src/llvm/build/ninja/bin/clang-cl.exe</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">-- Check for working C compiler: D:/src/llvm/build/ninja/bin/clang-cl.exe - broken</span><span lang="EN-US"><br>
</span><span lang="EN-US" style="font-size:10pt;font-family:"Courier New",serif">CMake Error at C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/Common7/IDE/CommonExtensions/Microsoft/CMake/CMake/share/cmake-3.20/Modules/CMakeTestCCompiler.cmake:66
(message):</span><span lang="EN-US"><u></u><u></u></span></p>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US" style="font-size:7.5pt;font-family:"Courier New",serif"> The C compiler</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US" style="font-size:7.5pt;font-family:"Courier New",serif"><br>
"D:/src/llvm/build/ninja/bin/clang-cl.exe"<br>
<br>
is not able to compile a simple test program.<br>
<br>
It fails with the following output:<br>
<br>
Change Dir: D:/src/llvm/build/selfhost/CMakeFiles/CMakeTmp<br>
<br>
Run Build Command(s):C:/PROGRA~2/MICROS~1/2019/PROFES~1/Common7/IDE/COMMON~1/MICROS~1/CMake/Ninja/ninja.exe cmTC_62af5 && [1/2] Building C object CMakeFiles\cmTC_62af5.dir\testCCompiler.c.obj<br>
[2/2] Linking C executable cmTC_62af5.exe<br>
FAILED: cmTC_62af5.exe<br>
cmd.exe /C "cd . && "C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin\cmake.exe" -E vs_link_exe --intdir=CMakeFiles\cmTC_62af5.dir --rc=C:\PROGRA~2\WI3CF2~1\10\bin\100183~1.0\x64\rc.exe
--mt=D:\src\llvm\build\ninja\bin\llvm-mt.exe --manifests -- D:\src\llvm\build\ninja\bin\lld-link.exe /nologo CMakeFiles\cmTC_62af5.dir\testCCompiler.c.obj /out:cmTC_62af5.exe /implib:cmTC_62af5.lib /pdb:cmTC_62af5.pdb /version:0.0 /machine:x64 /debug /INCREMENTAL
/subsystem:console kernel32.lib user32.lib gdi32.lib winspool.lib shell32.lib ole32.lib oleaut32.lib uuid.lib comdlg32.lib advapi32.lib && cd ."<br>
MT: command "D:\src\llvm\build\ninja\bin\llvm-mt.exe /nologo /manifest CMakeFiles\cmTC_62af5.dir/intermediate.manifest /out:CMakeFiles\cmTC_62af5.dir/embed.manifest /notify_update" failed (exit code 0x1) with the following output:<br>
llvm-mt: error: no libxml2<br>
llvm-mt: ignoring unsupported 'notify_update' option</span><span lang="EN-US"><u></u><u></u></span></p>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US" style="font-size:7.5pt;font-family:"Courier New",serif"> ninja: build stopped: subcommand failed.</span><span lang="EN-US"><u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-left:36pt">
<span lang="EN-US"> <u></u><u></u></span></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>