<div dir="ltr"><div>The discussion in <a href="https://reviews.llvm.org/D76240">https://reviews.llvm.org/D76240</a></div><div>may be of interest to the participants of this thread. Specifically, Fangrui's planned work on a command line option to limit threads for LLD. Until that becomes a reality, you might want to use the CMake variable -DLLVM_ENABLE_THREADS=OFF when building the LLD that you are using to prevent parallelism.<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div lang="EN-CA">
<div class="gmail-m_3778330944022309505WordSection1">
<p class="MsoNormal"><span lang="FR-CA">Alex :<u></u><u></u></span></p>
<p class="MsoNormal"><span>Can you please try `numactl` or `taskset` after
<a href="https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74" target="_blank">
https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74</a> ?<u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><span>There was a tiny bug in how
</span><b><span style="color:rgb(68,68,68);background:white none repeat scroll 0% 0%">sched_getaffinity</span></b><span style="color:rgb(68,68,68);background:white none repeat scroll 0% 0%">() was used, see:
</span><a href="https://reviews.llvm.org/D75153#1942336" target="_blank">https://reviews.llvm.org/D75153#1942336</a><span style="font-size:10pt">
</span><span><u></u><u></u></span></p>
<p class="MsoNormal"><span><u></u> <u></u></span></p>
<p class="MsoNormal"><b>De :</b> Alex Brachet-Mialot <<a href="mailto:alexbrachetmialot@gmail.com" target="_blank">alexbrachetmialot@gmail.com</a>>
<br>
<b>Envoyé :</b> March 28, 2020 12:11 PM<br>
<b>À :</b> Itaru Kitayama <<a href="mailto:itaru.kitayama@gmail.com" target="_blank">itaru.kitayama@gmail.com</a>><br>
<b>Cc :</b> LLVM Dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>; Alexandre Ganea <<a href="mailto:alexandre.ganea@ubisoft.com" target="_blank">alexandre.ganea@ubisoft.com</a>>; Fāng-ruì Sòng <<a href="mailto:maskray@google.com" target="_blank">maskray@google.com</a>><br>
<b>Objet :</b> Re: [llvm-dev] LLD issue on a massively parallel build machine<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">I also run into problems with lld on machines with lots of cores (208), but I run into rlimits for number of processes I can create, something outside of my control.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Unfortunately things like numactl or taskset do not limit the concurrency of lld currently.<u></u><u></u></p>
</div>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<p class="MsoNormal">On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0cm 0cm 0cm 6pt;margin-left:4.8pt;margin-right:0cm">
<div>
<div>
<p class="MsoNormal">Hi,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">On a 1296-core Intel machine with 376 GB, setting <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">-DLLVM_PARALLEL_LINK_JOB=1<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">does not help (switching back to ld scales) see:<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal">[5085/5201] Linking CXX executable bin/clang-11<br>
FAILED: bin/clang-11<br>
: && /home/usr4/c74014i/opt/clang/current/bin/clang++ -stdlib=libc++ -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
-Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -fno-common -Woverloaded-virtual -Wno-nested-anon-types
-O3 -DNDEBUG -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics -Wl,-allow-shlib-undefined -Wl,--export-dynamic -Wl,-O3 tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o -o bin/clang-11 -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMAArch64CodeGen.a lib/libLLVMAArch64AsmParser.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Disassembler.a lib/libLLVMAArch64Info.a
lib/libLLVMAArch64Utils.a lib/libLLVMAMDGPUCodeGen.a lib/libLLVMAMDGPUAsmParser.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUDisassembler.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMCodeGen.a lib/libLLVMARMAsmParser.a lib/libLLVMARMDesc.a
lib/libLLVMARMDisassembler.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a lib/libLLVMAVRCodeGen.a lib/libLLVMAVRAsmParser.a lib/libLLVMAVRDesc.a lib/libLLVMAVRDisassembler.a lib/libLLVMAVRInfo.a lib/libLLVMBPFCodeGen.a lib/libLLVMBPFAsmParser.a lib/libLLVMBPFDesc.a
lib/libLLVMBPFDisassembler.a lib/libLLVMBPFInfo.a lib/libLLVMHexagonCodeGen.a lib/libLLVMHexagonAsmParser.a lib/libLLVMHexagonDesc.a lib/libLLVMHexagonDisassembler.a lib/libLLVMHexagonInfo.a lib/libLLVMLanaiCodeGen.a lib/libLLVMLanaiAsmParser.a lib/libLLVMLanaiDesc.a
lib/libLLVMLanaiDisassembler.a lib/libLLVMLanaiInfo.a lib/libLLVMMipsCodeGen.a lib/libLLVMMipsAsmParser.a lib/libLLVMMipsDesc.a lib/libLLVMMipsDisassembler.a lib/libLLVMMipsInfo.a lib/libLLVMMSP430CodeGen.a lib/libLLVMMSP430AsmParser.a lib/libLLVMMSP430Desc.a
lib/libLLVMMSP430Disassembler.a lib/libLLVMMSP430Info.a lib/libLLVMNVPTXCodeGen.a lib/libLLVMNVPTXDesc.a lib/libLLVMNVPTXInfo.a lib/libLLVMPowerPCCodeGen.a lib/libLLVMPowerPCAsmParser.a lib/libLLVMPowerPCDesc.a lib/libLLVMPowerPCDisassembler.a lib/libLLVMPowerPCInfo.a
lib/libLLVMRISCVCodeGen.a lib/libLLVMRISCVAsmParser.a lib/libLLVMRISCVDesc.a lib/libLLVMRISCVDisassembler.a lib/libLLVMRISCVInfo.a lib/libLLVMRISCVUtils.a lib/libLLVMSparcCodeGen.a lib/libLLVMSparcAsmParser.a lib/libLLVMSparcDesc.a lib/libLLVMSparcDisassembler.a
lib/libLLVMSparcInfo.a lib/libLLVMSystemZCodeGen.a lib/libLLVMSystemZAsmParser.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZDisassembler.a lib/libLLVMSystemZInfo.a lib/libLLVMWebAssemblyCodeGen.a lib/libLLVMWebAssemblyAsmParser.a lib/libLLVMWebAssemblyDesc.a
lib/libLLVMWebAssemblyDisassembler.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a lib/libLLVMX86Utils.a lib/libLLVMXCoreCodeGen.a lib/libLLVMXCoreDesc.a
lib/libLLVMXCoreDisassembler.a lib/libLLVMXCoreInfo.a lib/libLLVMAnalysis.a lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a lib/libLLVMMCParser.a
lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a lib/libLLVMScalarOpts.a lib/libLLVMSupport.a lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a -lpthread lib/libclangBasic.a lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a lib/libclangFrontendTool.a
lib/libclangSerialization.a lib/libLLVMAArch64Desc.a lib/libLLVMAArch64Info.a lib/libLLVMAArch64Utils.a lib/libLLVMMIRParser.a lib/libLLVMAMDGPUDesc.a lib/libLLVMAMDGPUInfo.a lib/libLLVMAMDGPUUtils.a lib/libLLVMARMDesc.a lib/libLLVMARMInfo.a lib/libLLVMARMUtils.a
lib/libLLVMHexagonDesc.a lib/libLLVMHexagonInfo.a lib/libLLVMLanaiDesc.a lib/libLLVMLanaiInfo.a lib/libLLVMSystemZDesc.a lib/libLLVMSystemZInfo.a lib/libLLVMWebAssemblyDesc.a lib/libLLVMWebAssemblyInfo.a lib/libLLVMCFGuard.a lib/libLLVMGlobalISel.a
lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a lib/libLLVMSelectionDAG.a lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMCoverage.a lib/libLLVMLTO.a lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMCodeGen.a lib/libLLVMTarget.a
lib/libLLVMCoroutines.a lib/libLLVMipo.a lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a lib/libLLVMBitWriter.a lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMScalarOpts.a lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a
lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a lib/libclangStaticAnalyzerFrontend.a lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a lib/libclangCrossTU.a lib/libclangIndex.a lib/libclangFrontend.a lib/libclangDriver.a
lib/libLLVMOption.a lib/libclangParse.a lib/libclangSerialization.a lib/libclangSema.a lib/libclangAnalysis.a lib/libclangASTMatchers.a lib/libclangEdit.a lib/libclangFormat.a lib/libclangToolingInclusions.a lib/libclangToolingCore.a lib/libclangAST.a
lib/libLLVMFrontendOpenMP.a lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMProfileData.a lib/libLLVMObject.a lib/libLLVMMCParser.a lib/libLLVMBitReader.a lib/libLLVMTextAPI.a lib/libclangRewrite.a lib/libclangLex.a lib/libclangBasic.a
lib/libLLVMCore.a lib/libLLVMRemarks.a lib/libLLVMBitstreamReader.a lib/libLLVMMC.a lib/libLLVMBinaryFormat.a lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a lib/libLLVMSupport.a -lz -lrt -ldl -ltinfo -lpthread -lm lib/libLLVMDemangle.a
&& :<br>
ld.lld: error: failed to open bin/clang-11: Cannot allocate memory<u></u><u></u></p>
</div>
<p class="MsoNormal">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>