[llvm-dev] LLD issue on a massively parallel build machine

Nemanja Ivanovic via llvm-dev llvm-dev at lists.llvm.org
Sat Mar 28 09:44:46 PDT 2020 

The discussion in https://reviews.llvm.org/D76240
may be of interest to the participants of this thread. Specifically,
Fangrui's planned work on a command line option to limit threads for LLD.
Until that becomes a reality, you might want to use the CMake variable
-DLLVM_ENABLE_THREADS=OFF when building the LLD that you are using to
prevent parallelism.

On Sat, Mar 28, 2020 at 12:32 PM Alexandre Ganea via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> Alex :
>
> Can you please try `numactl` or `taskset` after
> https://github.com/llvm/llvm-project/commit/09158252f777c2e2f06a86b154c44abcbcf9bb74
> ?
>
>
>
> There was a tiny bug in how *sched_getaffinity*() was used, see:
> https://reviews.llvm.org/D75153#1942336
>
>
>
> *De :* Alex Brachet-Mialot <alexbrachetmialot at gmail.com>
> *Envoyé :* March 28, 2020 12:11 PM
> *À :* Itaru Kitayama <itaru.kitayama at gmail.com>
> *Cc :* LLVM Dev <llvm-dev at lists.llvm.org>; Alexandre Ganea <
> alexandre.ganea at ubisoft.com>; Fāng-ruì Sòng <maskray at google.com>
> *Objet :* Re: [llvm-dev] LLD issue on a massively parallel build machine
>
>
>
> I also run into problems with lld on machines with lots of cores (208),
> but I run into rlimits for number of processes I can create, something
> outside of my control.
>
>
>
> Unfortunately things like numactl or taskset do not limit the
> concurrency of lld currently.
>
>
>
> On Sat, Mar 28, 2020 at 4:10 AM Itaru Kitayama via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi,
>
> On a 1296-core Intel machine with 376 GB, setting
>
>
>
> -DLLVM_PARALLEL_LINK_JOB=1
>
>
>
> does not help (switching back to ld scales) see:
>
>
>
> [5085/5201] Linking CXX executable bin/clang-11
> FAILED: bin/clang-11
> : && /home/usr4/c74014i/opt/clang/current/bin/clang++  -stdlib=libc++
> -fPIC -fvisibility-inlines-hidden -Werror=date-time
> -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter
> -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic
> -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default
> -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections
> -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3 -DNDEBUG
>  -stdlib=libc++ -fuse-ld=lld -Wl,--color-diagnostics
> -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
> tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1gen_reproducer_main.cpp.o
>  -o bin/clang-11  -Wl,-rpath,"\$ORIGIN/../lib"  lib/libLLVMAArch64CodeGen.a
>  lib/libLLVMAArch64AsmParser.a  lib/libLLVMAArch64Desc.a
>  lib/libLLVMAArch64Disassembler.a  lib/libLLVMAArch64Info.a
>  lib/libLLVMAArch64Utils.a  lib/libLLVMAMDGPUCodeGen.a
>  lib/libLLVMAMDGPUAsmParser.a  lib/libLLVMAMDGPUDesc.a
>  lib/libLLVMAMDGPUDisassembler.a  lib/libLLVMAMDGPUInfo.a
>  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMCodeGen.a
>  lib/libLLVMARMAsmParser.a  lib/libLLVMARMDesc.a
>  lib/libLLVMARMDisassembler.a  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a
>  lib/libLLVMAVRCodeGen.a  lib/libLLVMAVRAsmParser.a  lib/libLLVMAVRDesc.a
>  lib/libLLVMAVRDisassembler.a  lib/libLLVMAVRInfo.a
>  lib/libLLVMBPFCodeGen.a  lib/libLLVMBPFAsmParser.a  lib/libLLVMBPFDesc.a
>  lib/libLLVMBPFDisassembler.a  lib/libLLVMBPFInfo.a
>  lib/libLLVMHexagonCodeGen.a  lib/libLLVMHexagonAsmParser.a
>  lib/libLLVMHexagonDesc.a  lib/libLLVMHexagonDisassembler.a
>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiCodeGen.a
>  lib/libLLVMLanaiAsmParser.a  lib/libLLVMLanaiDesc.a
>  lib/libLLVMLanaiDisassembler.a  lib/libLLVMLanaiInfo.a
>  lib/libLLVMMipsCodeGen.a  lib/libLLVMMipsAsmParser.a
>  lib/libLLVMMipsDesc.a  lib/libLLVMMipsDisassembler.a
>  lib/libLLVMMipsInfo.a  lib/libLLVMMSP430CodeGen.a
>  lib/libLLVMMSP430AsmParser.a  lib/libLLVMMSP430Desc.a
>  lib/libLLVMMSP430Disassembler.a  lib/libLLVMMSP430Info.a
>  lib/libLLVMNVPTXCodeGen.a  lib/libLLVMNVPTXDesc.a  lib/libLLVMNVPTXInfo.a
>  lib/libLLVMPowerPCCodeGen.a  lib/libLLVMPowerPCAsmParser.a
>  lib/libLLVMPowerPCDesc.a  lib/libLLVMPowerPCDisassembler.a
>  lib/libLLVMPowerPCInfo.a  lib/libLLVMRISCVCodeGen.a
>  lib/libLLVMRISCVAsmParser.a  lib/libLLVMRISCVDesc.a
>  lib/libLLVMRISCVDisassembler.a  lib/libLLVMRISCVInfo.a
>  lib/libLLVMRISCVUtils.a  lib/libLLVMSparcCodeGen.a
>  lib/libLLVMSparcAsmParser.a  lib/libLLVMSparcDesc.a
>  lib/libLLVMSparcDisassembler.a  lib/libLLVMSparcInfo.a
>  lib/libLLVMSystemZCodeGen.a  lib/libLLVMSystemZAsmParser.a
>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZDisassembler.a
>  lib/libLLVMSystemZInfo.a  lib/libLLVMWebAssemblyCodeGen.a
>  lib/libLLVMWebAssemblyAsmParser.a  lib/libLLVMWebAssemblyDesc.a
>  lib/libLLVMWebAssemblyDisassembler.a  lib/libLLVMWebAssemblyInfo.a
>  lib/libLLVMX86CodeGen.a  lib/libLLVMX86AsmParser.a  lib/libLLVMX86Desc.a
>  lib/libLLVMX86Disassembler.a  lib/libLLVMX86Info.a  lib/libLLVMX86Utils.a
>  lib/libLLVMXCoreCodeGen.a  lib/libLLVMXCoreDesc.a
>  lib/libLLVMXCoreDisassembler.a  lib/libLLVMXCoreInfo.a
>  lib/libLLVMAnalysis.a  lib/libLLVMCodeGen.a  lib/libLLVMCore.a
>  lib/libLLVMipo.a  lib/libLLVMAggressiveInstCombine.a
>  lib/libLLVMInstCombine.a  lib/libLLVMInstrumentation.a  lib/libLLVMMC.a
>  lib/libLLVMMCParser.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMOption.a
>  lib/libLLVMScalarOpts.a  lib/libLLVMSupport.a  lib/libLLVMTransformUtils.a
>  lib/libLLVMVectorize.a  -lpthread  lib/libclangBasic.a
>  lib/libclangCodeGen.a  lib/libclangDriver.a  lib/libclangFrontend.a
>  lib/libclangFrontendTool.a  lib/libclangSerialization.a
>  lib/libLLVMAArch64Desc.a  lib/libLLVMAArch64Info.a
>  lib/libLLVMAArch64Utils.a  lib/libLLVMMIRParser.a  lib/libLLVMAMDGPUDesc.a
>  lib/libLLVMAMDGPUInfo.a  lib/libLLVMAMDGPUUtils.a  lib/libLLVMARMDesc.a
>  lib/libLLVMARMInfo.a  lib/libLLVMARMUtils.a  lib/libLLVMHexagonDesc.a
>  lib/libLLVMHexagonInfo.a  lib/libLLVMLanaiDesc.a  lib/libLLVMLanaiInfo.a
>  lib/libLLVMSystemZDesc.a  lib/libLLVMSystemZInfo.a
>  lib/libLLVMWebAssemblyDesc.a  lib/libLLVMWebAssemblyInfo.a
>  lib/libLLVMCFGuard.a  lib/libLLVMGlobalISel.a  lib/libLLVMAsmPrinter.a
>  lib/libLLVMDebugInfoDWARF.a  lib/libLLVMSelectionDAG.a
>  lib/libLLVMMCDisassembler.a  lib/libclangCodeGen.a  lib/libLLVMCoverage.a
>  lib/libLLVMLTO.a  lib/libLLVMObjCARCOpts.a  lib/libLLVMPasses.a
>  lib/libLLVMCodeGen.a  lib/libLLVMTarget.a  lib/libLLVMCoroutines.a
>  lib/libLLVMipo.a  lib/libLLVMInstrumentation.a  lib/libLLVMVectorize.a
>  lib/libLLVMBitWriter.a  lib/libLLVMIRReader.a  lib/libLLVMAsmParser.a
>  lib/libLLVMLinker.a  lib/libLLVMScalarOpts.a
>  lib/libLLVMAggressiveInstCombine.a  lib/libLLVMInstCombine.a
>  lib/libclangRewriteFrontend.a  lib/libclangARCMigrate.a
>  lib/libclangStaticAnalyzerFrontend.a  lib/libclangStaticAnalyzerCheckers.a
>  lib/libclangStaticAnalyzerCore.a  lib/libclangCrossTU.a
>  lib/libclangIndex.a  lib/libclangFrontend.a  lib/libclangDriver.a
>  lib/libLLVMOption.a  lib/libclangParse.a  lib/libclangSerialization.a
>  lib/libclangSema.a  lib/libclangAnalysis.a  lib/libclangASTMatchers.a
>  lib/libclangEdit.a  lib/libclangFormat.a  lib/libclangToolingInclusions.a
>  lib/libclangToolingCore.a  lib/libclangAST.a  lib/libLLVMFrontendOpenMP.a
>  lib/libLLVMTransformUtils.a  lib/libLLVMAnalysis.a
>  lib/libLLVMProfileData.a  lib/libLLVMObject.a  lib/libLLVMMCParser.a
>  lib/libLLVMBitReader.a  lib/libLLVMTextAPI.a  lib/libclangRewrite.a
>  lib/libclangLex.a  lib/libclangBasic.a  lib/libLLVMCore.a
>  lib/libLLVMRemarks.a  lib/libLLVMBitstreamReader.a  lib/libLLVMMC.a
>  lib/libLLVMBinaryFormat.a  lib/libLLVMDebugInfoCodeView.a
>  lib/libLLVMDebugInfoMSF.a  lib/libLLVMSupport.a  -lz  -lrt  -ldl  -ltinfo
>  -lpthread  -lm  lib/libLLVMDemangle.a && :
> ld.lld: error: failed to open bin/clang-11: Cannot allocate memory
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20200328/b9e626b6/attachment-0001.html>

More information about the llvm-dev mailing list