[llvm-dev] Memory overflow during cmake/ninja build

[Ron Brender via llvm-dev]

Peter, Steve, Alex, Neil,

Thanks for the many suggestions. I will try again this weekend...

Ron


On 6/21/2019 9:25 PM, Neil Nelson via llvm-dev wrote:
> It has also been suggested to try compiling with clang and linking 
> with lld. The gold linker should be already installed and is also 
> recommended to reduce memory use.
>
> apt install clang
>
> apt install lld
>
> 4G is very tight. You will want to run just the terminal emulator for 
> the compile and another running top to see when you go into swap and 
> what process is doing that. You will need to avoid running anything 
> else that is not absolutely necessary to make the most memory available.
>
> You can control-c out of the compile sequence and restart at any time. 
> The sequence will resume.
>
> If you have more than one core and it goes into swap, you might stop 
> the compile and use
>
> ninja -j 1
>
> to try to squeak through. You can stop the compile, go back to more 
> cores, and restart in more easy stretches.
>
> I used
>
> cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_USE_LINKER=lld 
> -DCMAKE_BUILD_TYPE="Release" ../llvm
>
> this morning and it appeared in general to stay below 4G and perhaps 
> with enough swap you will make it. But when it goes into heavy swap, 
> you are doing a number on your disk with not much forward gain.
>
> If you change compilers such as from GNU cpp to clang after running 
> the above cmake you will need to delete the build directory for the 
> new compiler to be used.
>
> Change the compiler (after apt above) by adding the following lines to 
> /etc/environment and reboot.
> export CC=clang
> export CXX=clang++
>
> On 6/21/19 8:00 AM, Ron Brender via llvm-dev wrote:
>> I'm trying to do a simple build from the git 8.0.0 sources. The 
>> sources seem to build OK but a link step fails from running out of 
>> memory. I need some clues how to figure out where the bottleneck 
>> might be.
>>
>> The cmake command is:
>>
>> cmake -G Ninja                                          \
>>     -DLLVM_TARGETS_TO_BUILD=X86                         \
>>     -DCMAKE_INSTALL_PREFIX=/home/ron/bin/llvm_project/  \
>>     ~/llvm-project/llvm
>>
>> All seems to go well up until this link step:
>>
>> [8/323] Linking CXX executable bin/llvm-lto
>> FAILED: bin/llvm-lto
>> : && /usr/bin/c++  -fPIC -fvisibility-inlines-hidden 
>> -Werror=date-time -std=c++11 -Wall -Wextra -Wno-unused-parameter 
>> -Wwrite-strings -Wcast-qual -Wno-missing-field-initializers -pedantic 
>> -Wno-long-long -Wimplicit-fallthrough -Wno-maybe-uninitialized 
>> -Wno-class-memaccess -Wno-noexcept-type -Wdelete-non-virtual-dtor 
>> -Wno-comment -fdiagnostics-color -g -Wl,-allow-shlib-undefined 
>> -Wl,-rpath-link,/home/ron/llvm-project/build-try3/./lib 
>> tools/llvm-lto/CMakeFiles/llvm-lto.dir/llvm-lto.cpp.o  -o 
>> bin/llvm-lto  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMX86CodeGen.a 
>> lib/libLLVMX86AsmParser.a lib/libLLVMX86AsmPrinter.a 
>> lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a 
>> lib/libLLVMX86Info.a lib/libLLVMX86Utils.a lib/libLLVMBitReader.a 
>> lib/libLLVMBitWriter.a lib/libLLVMCore.a lib/libLLVMIRReader.a 
>> lib/libLLVMLTO.a lib/libLLVMMC.a lib/libLLVMObject.a 
>> lib/libLLVMSupport.a lib/libLLVMTarget.a -lpthread 
>> lib/libLLVMAsmPrinter.a lib/libLLVMDebugInfoDWARF.a 
>> lib/libLLVMGlobalISel.a lib/libLLVMSelectionDAG.a 
>> lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a 
>> lib/libLLVMMCDisassembler.a lib/libLLVMObjCARCOpts.a 
>> lib/libLLVMPasses.a lib/libLLVMCodeGen.a lib/libLLVMTarget.a 
>> lib/libLLVMipo.a lib/libLLVMBitWriter.a lib/libLLVMIRReader.a 
>> lib/libLLVMAsmParser.a lib/libLLVMLinker.a lib/libLLVMScalarOpts.a 
>> lib/libLLVMAggressiveInstCombine.a lib/libLLVMInstCombine.a 
>> lib/libLLVMInstrumentation.a lib/libLLVMVectorize.a 
>> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a lib/libLLVMObject.a 
>> lib/libLLVMBitReader.a lib/libLLVMMCParser.a lib/libLLVMMC.a 
>> lib/libLLVMDebugInfoCodeView.a lib/libLLVMDebugInfoMSF.a 
>> lib/libLLVMProfileData.a lib/libLLVMCore.a lib/libLLVMBinaryFormat.a 
>> lib/libLLVMSupport.a -lrt -ldl -lpthread -lm lib/libLLVMDemangle.a && :
>> /usr/bin/ld: 
>> /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/librt.so: 
>> error adding symbols: memory exhausted
>> collect2: error: ld returned 1 exit status
>> [9/323] Building CXX object 
>> tools/bugpoint/CMakeFiles/bugpoint.dir/bugpoint.cpp.o
>> [10/323] Linking CXX shared library lib/libLTO.so.9svn
>> ninja: build stopped: subcommand failed.
>>
>> My hardware is a rather ancient HP a6745f (AMD) with 4GB memory, 200 
>> GB HDD. OS is Kubuntu 18.10.
>>
>> Is my system just too limited or is there something else I should 
>> look for? What other info can I provide?
>>
>> Ron
>>
>>
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
Ron Brender
Whose favorite airplane is N6119A, a 1979 Cessna T210.