[llvm-dev] [llvm-toolchain v3.8.1] LTO: Linking clang hangs with ld.gold and LLVMgold.so plugin

Sat Jul 23 14:59:32 PDT 2016

> On Jul 23, 2016, at 1:53 PM, Sedat Dilek via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> 
> On Sat, Jul 23, 2016 at 7:48 PM, Piotr Padlewski <prazek at google.com <mailto:prazek at google.com>> wrote:
>> How big is your project?
>> LTO eats RAM even faster than chrome. For example linking clang with LTO
>> could take 16GB of ram.
>> 
>> Have you tried using LTO on your project on that machine, or is it your
>> first time?
>> 
> 
> Wow hu hu - 16GiB of RAM :-).

It is a bit less now: should be closer to 11GB with Full Debug Info.
You may want to try without debug info (or with -gline-tables-only) and it should get under 4GB.

— 
Mehdi

> 
> This is my first time dealing with LTO in general.
> 
> Not sure if it matters if I build with GCC and its LTO-plugin and
> ld.bfd as linker.
> Here I tried CLANG with LLVMgold-plugin and ld.gold (v2.26.1).
> 
> These are the last lines I see...
> 
> [2114/2157] Linking CXX executable bin/diagtool
> [2115/2157] Linking CXX executable bin/clang-format
> [2116/2157] Linking CXX executable bin/clang-3.8
> FAILED: bin/clang-3.8
> : && /opt/llvm/bin/clang++-3.8   -fPIC -fvisibility-inlines-hidden
> -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> -Wmissing-field-initializers -pedantic -Wno-long-long
> -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> -std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections
> -flto -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3
> -flto -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
> -Wl,--gc-sections
> tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o  -o
> bin/clang-3.8  lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmPrinter.a
> lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Info.a
> lib/libLLVMX86Disassembler.a lib/libLLVMAnalysis.a
> lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a
> lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a
> lib/libLLVMMCParser.a lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a
> lib/libLLVMScalarOpts.a lib/libLLVMSupport.a
> lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a lib/libclangBasic.a
> lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a
> lib/libclangFrontendTool.a lib/libLLVMAsmPrinter.a
> lib/libLLVMSelectionDAG.a lib/libLLVMCodeGen.a
> lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a lib/libLLVMX86Info.a
> lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMipo.a
> lib/libLLVMVectorize.a lib/libLLVMInstrumentation.a
> lib/libLLVMObjCARCOpts.a lib/libLLVMScalarOpts.a
> lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMBitWriter.a
> lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a
> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a
> lib/libLLVMProfileData.a lib/libLLVMObject.a
> lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a
> lib/libclangStaticAnalyzerFrontend.a lib/libclangFrontend.a
> lib/libclangDriver.a lib/libLLVMOption.a lib/libclangParse.a
> lib/libLLVMMCParser.a lib/libclangSerialization.a
> lib/libLLVMBitReader.a lib/libclangSema.a lib/libclangEdit.a
> lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a
> lib/libclangAnalysis.a lib/libclangAST.a lib/libclangRewrite.a
> lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a
> lib/libLLVMMC.a lib/libLLVMSupport.a -lrt -ldl -ltinfo -lpthread -lz
> -lm -Wl,-rpath,"\$ORIGIN/../lib" && :
> clang-3.8: error: unable to execute command: Killed
> clang-3.8: error: linker command failed due to signal (use -v to see invocation)
> ninja: build stopped: subcommand failed.
> 
> This time I just build and installed ld.gold from binutils v2.26.1 and
> symlinked system-wide with my ld of my /opt-installation.
> The generated LLVMgold.so plugin I copied into /usr/lib/bfd-plugins/
> directory (bfd-plugins subdir needed to be created manually).
> 
> - Sedat -
> 
> P.S.: Output of free and df commands
> 
> $ free -m
>             total       used       free     shared    buffers     cached
> Mem:          3846       2211       1635          0        729        818
> -/+ buffers/cache:        662       3183
> Swap:          255        143        112
> 
> $ df -T | grep ext4
> /dev/loop0     ext4      17753424  15997332    831216   96% /
> 
> - EOT -
> 
>> Piotr
>> 
>> 
>> On Sat, Jul 23, 2016 at 2:42 AM, Sedat Dilek via llvm-dev
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>> 
>>> On Thu, Jul 21, 2016 at 12:01 PM, Sedat Dilek <sedat.dilek at gmail.com <mailto:sedat.dilek at gmail.com>>
>>> wrote:
>>>> Hi,
>>>> 
>>>> unfortunately, my build somehow hangs when linking clang binary and my
>>>> system is in an unusable state.
>>>> 
>>>> My toolchain is clang-3.8, gold-1.11 and LLVMgold.so from binutils
>>>> v2.26.1 (both selfmade) and LTO-flag is enabled.
>>>> My buildsystem uses cmake-3.6.0 and ninja-1.7.1 (both prebuilt).
>>>> I have 52 last steps left in my 3rd build.
>>>> 
>>>> My Linux-kernel is v3.13.0-92 from official Ubuntu repositories.
>>>> 
>>>> On my Ubuntu/precise AMD64 I have ***4GiB RAM and 256MiB SWAP***.
>>>> Is this not enough RAM/SWAP for building/linking?
>>>> 
>>>> Need some additional informations?
>>>> 
>>>> Any help appreciated.
>>>> 
>>>> Regards,
>>>> - Sedat -
>>>> 
>>>> [1] http://llvm.org/docs/GoldPlugin.html <http://llvm.org/docs/GoldPlugin.html>
>>>> 
>>>> P.S.: More informations...
>>>> 
>>>> [ First toolchain build ]
>>>> 
>>>> I built a 1st llvm-toolchain v3.8.1 with GCC v4.9.2 and binutils v2.22.
>>>> No speedup settings, no extra-patches applied etc.
>>>> 
>>>> [ Selfmade binutils ]
>>>> 
>>>> As binutils v2.22 was somehow not able to generate a LLVMgold.so on
>>>> Ubuntu/precise AMD64, I have built binutils v2.26.1 manually and use
>>>> all its binaries for building/linking (GOLD for linking).
>>>> 
>>>> [ 2nd llvm-toolchain LLVMgold.so ]
>>>> 
>>>> In a 2nd llvm-toolchain v3.8.1 build a LLVMgold.so was generated.
>>>> I placed LLVMgold.so into my selfmade
>>>> /opt/binutils-2.26.1/lib/bfd-plugins/ directory.
>>>> ( I had to create a bfd-plugins subdir manually. )
>>>> 
>>>> [ 3rd llvm-toolchain LTO ]
>>>> 
>>>> Now, I am able with backported LTO-flag from upstream to build with
>>>> '-flto'.
>>>> 
>>>> - EOT -
>>> 
>>> Hi,
>>> 
>>> in the meantime I tried with Linux v4.4.y LTS and 2GiB swap-space.
>>> So I have 4 GiB RAM and 2GiB SWAP in total 6GiB.
>>> 
>>> Paul Rouschal recommended to reduce parallel-compile-jobs from 2 to 1...
>>> 
>>> LLVM_PARALLEL_COMPILE_JOBS=1
>>> LLVM_PARALLEL_LINK_JOBS=1
>>> 
>>> ...but that did not help.
>>> 
>>> My Ubuntu/precise hangs and looking at top shows MEM/SWAP to be eaten.
>>> 
>>> Anyone has experiences how much RAM or SWAP I need when building a
>>> LTO-optimized llvm-toolchain with Clang, GNU/gold and LLVMgold-Plugin?
>>> 
>>> My build-script is attached.
>>> 
>>> Thanks.
>>> 
>>> Regards,
>>> - Sedat -
>>> 
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>> 
>> 
> <build_llvm-toolchain.sh>_______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160723/ddafe538/attachment-0001.html>