[llvm-dev] [llvm-toolchain v3.8.1] LTO: Linking clang hangs with ld.gold and LLVMgold.so plugin

Sat Jul 23 13:53:59 PDT 2016

On Sat, Jul 23, 2016 at 7:48 PM, Piotr Padlewski <prazek at google.com> wrote:
> How big is your project?
> LTO eats RAM even faster than chrome. For example linking clang with LTO
> could take 16GB of ram.
>
> Have you tried using LTO on your project on that machine, or is it your
> first time?
>

Wow hu hu - 16GiB of RAM :-).

This is my first time dealing with LTO in general.

Not sure if it matters if I build with GCC and its LTO-plugin and
ld.bfd as linker.
Here I tried CLANG with LLVMgold-plugin and ld.gold (v2.26.1).

These are the last lines I see...

[2114/2157] Linking CXX executable bin/diagtool
[2115/2157] Linking CXX executable bin/clang-format
[2116/2157] Linking CXX executable bin/clang-3.8
FAILED: bin/clang-3.8
: && /opt/llvm/bin/clang++-3.8   -fPIC -fvisibility-inlines-hidden
-Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual
-Wmissing-field-initializers -pedantic -Wno-long-long
-Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
-std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections
-flto -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3
-flto -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
-Wl,--gc-sections
tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o  -o
bin/clang-3.8  lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmPrinter.a
lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Info.a
lib/libLLVMX86Disassembler.a lib/libLLVMAnalysis.a
lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a
lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a
lib/libLLVMMCParser.a lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a
lib/libLLVMScalarOpts.a lib/libLLVMSupport.a
lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a lib/libclangBasic.a
lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a
lib/libclangFrontendTool.a lib/libLLVMAsmPrinter.a
lib/libLLVMSelectionDAG.a lib/libLLVMCodeGen.a
lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a lib/libLLVMX86Info.a
lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMipo.a
lib/libLLVMVectorize.a lib/libLLVMInstrumentation.a
lib/libLLVMObjCARCOpts.a lib/libLLVMScalarOpts.a
lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMBitWriter.a
lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a
lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a
lib/libLLVMProfileData.a lib/libLLVMObject.a
lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a
lib/libclangStaticAnalyzerFrontend.a lib/libclangFrontend.a
lib/libclangDriver.a lib/libLLVMOption.a lib/libclangParse.a
lib/libLLVMMCParser.a lib/libclangSerialization.a
lib/libLLVMBitReader.a lib/libclangSema.a lib/libclangEdit.a
lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a
lib/libclangAnalysis.a lib/libclangAST.a lib/libclangRewrite.a
lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a
lib/libLLVMMC.a lib/libLLVMSupport.a -lrt -ldl -ltinfo -lpthread -lz
-lm -Wl,-rpath,"\$ORIGIN/../lib" && :
clang-3.8: error: unable to execute command: Killed
clang-3.8: error: linker command failed due to signal (use -v to see invocation)
ninja: build stopped: subcommand failed.

This time I just build and installed ld.gold from binutils v2.26.1 and
symlinked system-wide with my ld of my /opt-installation.
The generated LLVMgold.so plugin I copied into /usr/lib/bfd-plugins/
directory (bfd-plugins subdir needed to be created manually).

- Sedat -

P.S.: Output of free and df commands

$ free -m
             total       used       free     shared    buffers     cached
Mem:          3846       2211       1635          0        729        818
-/+ buffers/cache:        662       3183
Swap:          255        143        112

$ df -T | grep ext4
/dev/loop0     ext4      17753424  15997332    831216   96% /

- EOT -

> Piotr
>
>
> On Sat, Jul 23, 2016 at 2:42 AM, Sedat Dilek via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>>
>> On Thu, Jul 21, 2016 at 12:01 PM, Sedat Dilek <sedat.dilek at gmail.com>
>> wrote:
>> > Hi,
>> >
>> > unfortunately, my build somehow hangs when linking clang binary and my
>> > system is in an unusable state.
>> >
>> > My toolchain is clang-3.8, gold-1.11 and LLVMgold.so from binutils
>> > v2.26.1 (both selfmade) and LTO-flag is enabled.
>> > My buildsystem uses cmake-3.6.0 and ninja-1.7.1 (both prebuilt).
>> > I have 52 last steps left in my 3rd build.
>> >
>> > My Linux-kernel is v3.13.0-92 from official Ubuntu repositories.
>> >
>> > On my Ubuntu/precise AMD64 I have ***4GiB RAM and 256MiB SWAP***.
>> > Is this not enough RAM/SWAP for building/linking?
>> >
>> > Need some additional informations?
>> >
>> > Any help appreciated.
>> >
>> > Regards,
>> > - Sedat -
>> >
>> > [1] http://llvm.org/docs/GoldPlugin.html
>> >
>> > P.S.: More informations...
>> >
>> > [ First toolchain build ]
>> >
>> > I built a 1st llvm-toolchain v3.8.1 with GCC v4.9.2 and binutils v2.22.
>> > No speedup settings, no extra-patches applied etc.
>> >
>> > [ Selfmade binutils ]
>> >
>> > As binutils v2.22 was somehow not able to generate a LLVMgold.so on
>> > Ubuntu/precise AMD64, I have built binutils v2.26.1 manually and use
>> > all its binaries for building/linking (GOLD for linking).
>> >
>> > [ 2nd llvm-toolchain LLVMgold.so ]
>> >
>> > In a 2nd llvm-toolchain v3.8.1 build a LLVMgold.so was generated.
>> > I placed LLVMgold.so into my selfmade
>> > /opt/binutils-2.26.1/lib/bfd-plugins/ directory.
>> > ( I had to create a bfd-plugins subdir manually. )
>> >
>> > [ 3rd llvm-toolchain LTO ]
>> >
>> > Now, I am able with backported LTO-flag from upstream to build with
>> > '-flto'.
>> >
>> > - EOT -
>>
>> Hi,
>>
>> in the meantime I tried with Linux v4.4.y LTS and 2GiB swap-space.
>> So I have 4 GiB RAM and 2GiB SWAP in total 6GiB.
>>
>> Paul Rouschal recommended to reduce parallel-compile-jobs from 2 to 1...
>>
>> LLVM_PARALLEL_COMPILE_JOBS=1
>> LLVM_PARALLEL_LINK_JOBS=1
>>
>> ...but that did not help.
>>
>> My Ubuntu/precise hangs and looking at top shows MEM/SWAP to be eaten.
>>
>> Anyone has experiences how much RAM or SWAP I need when building a
>> LTO-optimized llvm-toolchain with Clang, GNU/gold and LLVMgold-Plugin?
>>
>> My build-script is attached.
>>
>> Thanks.
>>
>> Regards,
>> - Sedat -
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: build_llvm-toolchain.sh
Type: application/x-sh
Size: 6393 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160723/7de8f227/attachment.sh>