[llvm-dev] Memory overflow during cmake/ninja build

[Neil Nelson via llvm-dev]

It has also been suggested to try compiling with clang and linking with 
lld. The gold linker should be already installed and is also recommended 
to reduce memory use.

apt install clang

apt install lld

4G is very tight. You will want to run just the terminal emulator for 
the compile and another running top to see when you go into swap and 
what process is doing that. You will need to avoid running anything else 
that is not absolutely necessary to make the most memory available.

You can control-c out of the compile sequence and restart at any time. 
The sequence will resume.

If you have more than one core and it goes into swap, you might stop the 
compile and use

ninja -j 1

to try to squeak through. You can stop the compile, go back to more 
cores, and restart in more easy stretches.

I used

cmake -G Ninja -DLLVM_ENABLE_PROJECTS="clang" -DLLVM_USE_LINKER=lld 
-DCMAKE_BUILD_TYPE="Release" ../llvm

this morning and it appeared in general to stay below 4G and perhaps 
with enough swap you will make it. But when it goes into heavy swap, you 
are doing a number on your disk with not much forward gain.

If you change compilers such as from GNU cpp to clang after running the 
above cmake you will need to delete the build directory for the new 
compiler to be used.

Change the compiler (after apt above) by adding the following lines to 
/etc/environment and reboot.
export CC=clang
export CXX=clang++

On 6/21/19 8:00 AM, Ron Brender via llvm-dev wrote:
> I'm trying to do a simple build from the git 8.0.0 sources. The 
> sources seem to build OK but a link step fails from running out of 
> memory. I need some clues how to figure out where the bottleneck might 
> be.
>
> The cmake command is:
>
> cmake -G Ninja                                          \
>     -DLLVM_TARGETS_TO_BUILD=X86                         \
>     -DCMAKE_INSTALL_PREFIX=/home/ron/bin/llvm_project/  \
>     ~/llvm-project/llvm
>
> All seems to go well up until this link step:
>
> [8/323] Linking CXX executable bin/llvm-lto
> FAILED: bin/llvm-lto
> : && /usr/bin/c++  -fPIC -fvisibility-inlines-hidden -Werror=date-time 
> -std=c++11 -Wall -Wextra -Wno-unused-parameter -Wwrite-strings 
> -Wcast-qual -Wno-missing-field-initializers -pedantic -Wno-long-long 
> -Wimplicit-fallthrough -Wno-maybe-uninitialized -Wno-class-memaccess 
> -Wno-noexcept-type -Wdelete-non-virtual-dtor -Wno-comment 
> -fdiagnostics-color -g -Wl,-allow-shlib-undefined 
> -Wl,-rpath-link,/home/ron/llvm-project/build-try3/./lib 
> tools/llvm-lto/CMakeFiles/llvm-lto.dir/llvm-lto.cpp.o  -o 
> bin/llvm-lto  -Wl,-rpath,"\$ORIGIN/../lib" lib/libLLVMX86CodeGen.a 
> lib/libLLVMX86AsmParser.a lib/libLLVMX86AsmPrinter.a 
> lib/libLLVMX86Desc.a lib/libLLVMX86Disassembler.a lib/libLLVMX86Info.a 
> lib/libLLVMX86Utils.a lib/libLLVMBitReader.a lib/libLLVMBitWriter.a 
> lib/libLLVMCore.a lib/libLLVMIRReader.a lib/libLLVMLTO.a 
> lib/libLLVMMC.a lib/libLLVMObject.a lib/libLLVMSupport.a 
> lib/libLLVMTarget.a -lpthread lib/libLLVMAsmPrinter.a 
> lib/libLLVMDebugInfoDWARF.a lib/libLLVMGlobalISel.a 
> lib/libLLVMSelectionDAG.a lib/libLLVMX86AsmPrinter.a 
> lib/libLLVMX86Utils.a lib/libLLVMMCDisassembler.a 
> lib/libLLVMObjCARCOpts.a lib/libLLVMPasses.a lib/libLLVMCodeGen.a 
> lib/libLLVMTarget.a lib/libLLVMipo.a lib/libLLVMBitWriter.a 
> lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a 
> lib/libLLVMScalarOpts.a lib/libLLVMAggressiveInstCombine.a 
> lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a 
> lib/libLLVMVectorize.a lib/libLLVMTransformUtils.a 
> lib/libLLVMAnalysis.a lib/libLLVMObject.a lib/libLLVMBitReader.a 
> lib/libLLVMMCParser.a lib/libLLVMMC.a lib/libLLVMDebugInfoCodeView.a 
> lib/libLLVMDebugInfoMSF.a lib/libLLVMProfileData.a lib/libLLVMCore.a 
> lib/libLLVMBinaryFormat.a lib/libLLVMSupport.a -lrt -ldl -lpthread -lm 
> lib/libLLVMDemangle.a && :
> /usr/bin/ld: 
> /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/librt.so: 
> error adding symbols: memory exhausted
> collect2: error: ld returned 1 exit status
> [9/323] Building CXX object 
> tools/bugpoint/CMakeFiles/bugpoint.dir/bugpoint.cpp.o
> [10/323] Linking CXX shared library lib/libLTO.so.9svn
> ninja: build stopped: subcommand failed.
>
> My hardware is a rather ancient HP a6745f (AMD) with 4GB memory, 200 
> GB HDD. OS is Kubuntu 18.10.
>
> Is my system just too limited or is there something else I should look 
> for? What other info can I provide?
>
> Ron
>
>
>