[llvm-dev] [llvm-toolchain v3.8.1] LTO: Linking clang hangs with ld.gold and LLVMgold.so plugin

Sun Jul 24 01:50:57 PDT 2016

On Sat, Jul 23, 2016 at 11:59 PM, Mehdi Amini <mehdi.amini at apple.com> wrote:
>
> On Jul 23, 2016, at 1:53 PM, Sedat Dilek via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
> On Sat, Jul 23, 2016 at 7:48 PM, Piotr Padlewski <prazek at google.com> wrote:
>
> How big is your project?
> LTO eats RAM even faster than chrome. For example linking clang with LTO
> could take 16GB of ram.
>
> Have you tried using LTO on your project on that machine, or is it your
> first time?
>
>
> Wow hu hu - 16GiB of RAM :-).
>
>
> It is a bit less now: should be closer to 11GB with Full Debug Info.
> You may want to try without debug info (or with -gline-tables-only) and it
> should get under 4GB.
>

Hmm, I have here a built-in small SanDisk-SSD with 16GiB and used that as SWAP.

I was able to build but when starting the install procedure I had 1916
steps again for building/linking.

When I looked while watching TV the max SWAP I have seen was 3,5GiB in
top (linking clang-3.8).

So, build-time increases rapidly here.

What is the benefit of "-gline-tables-only"?
Does it have any correlation/problem with the below cmake-options?

In a next run I wanted to enable Profile-Guided Optimizations (PGO),
optimized-TableGen and split-DWARF.

# CMake LTO (link-time optimizations) settings (WIP)
# NOTE: Not available for LLVM v3.8 (requires a backport of SVN r259766)
LTO_CMAKE_OPTS="-DLLVM_ENABLE_LTO=ON"

# CMake PGO (profile-guided optimizations) settings (WIP)
PGO_CMAKE_OPTS="-DLLVM_USE_OPROFILE=ON"

# CMake TABLEGEN settings (WIP)
TABLEGEN_CMAKE_OPTS="-DLLVM_OPTIMIZED_TABLEGEN=ON"

# CMake DWARF settings (WIP)
# NOTE: Move parts of the debug info into separate files: split DWARF
DWARF_CMAKE_OPTS="-DLLVM_USE_SPLIT_DWARF=ON"

SPEEDUP_CMAKE_OPTS="$LTO_CMAKE_OPTS $PGO_CMAKE_OPTS $TABLEGEN_CMAKE_OPTS

Thanks for the feedback.

- sed@ -

> —
> Mehdi
>
>
>
> This is my first time dealing with LTO in general.
>
> Not sure if it matters if I build with GCC and its LTO-plugin and
> ld.bfd as linker.
> Here I tried CLANG with LLVMgold-plugin and ld.gold (v2.26.1).
>
> These are the last lines I see...
>
> [2114/2157] Linking CXX executable bin/diagtool
> [2115/2157] Linking CXX executable bin/clang-format
> [2116/2157] Linking CXX executable bin/clang-3.8
> FAILED: bin/clang-3.8
> : && /opt/llvm/bin/clang++-3.8   -fPIC -fvisibility-inlines-hidden
> -Wall -W -Wno-unused-parameter -Wwrite-strings -Wcast-qual
> -Wmissing-field-initializers -pedantic -Wno-long-long
> -Wcovered-switch-default -Wnon-virtual-dtor -Wdelete-non-virtual-dtor
> -std=c++11 -fcolor-diagnostics -ffunction-sections -fdata-sections
> -flto -fno-common -Woverloaded-virtual -Wno-nested-anon-types -O3
> -flto -Wl,-allow-shlib-undefined   -Wl,--export-dynamic  -Wl,-O3
> -Wl,--gc-sections
> tools/clang/tools/driver/CMakeFiles/clang.dir/driver.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1_main.cpp.o
> tools/clang/tools/driver/CMakeFiles/clang.dir/cc1as_main.cpp.o  -o
> bin/clang-3.8  lib/libLLVMX86CodeGen.a lib/libLLVMX86AsmPrinter.a
> lib/libLLVMX86AsmParser.a lib/libLLVMX86Desc.a lib/libLLVMX86Info.a
> lib/libLLVMX86Disassembler.a lib/libLLVMAnalysis.a
> lib/libLLVMCodeGen.a lib/libLLVMCore.a lib/libLLVMipo.a
> lib/libLLVMInstCombine.a lib/libLLVMInstrumentation.a lib/libLLVMMC.a
> lib/libLLVMMCParser.a lib/libLLVMObjCARCOpts.a lib/libLLVMOption.a
> lib/libLLVMScalarOpts.a lib/libLLVMSupport.a
> lib/libLLVMTransformUtils.a lib/libLLVMVectorize.a lib/libclangBasic.a
> lib/libclangCodeGen.a lib/libclangDriver.a lib/libclangFrontend.a
> lib/libclangFrontendTool.a lib/libLLVMAsmPrinter.a
> lib/libLLVMSelectionDAG.a lib/libLLVMCodeGen.a
> lib/libLLVMX86AsmPrinter.a lib/libLLVMX86Utils.a lib/libLLVMX86Info.a
> lib/libLLVMMCDisassembler.a lib/libclangCodeGen.a lib/libLLVMipo.a
> lib/libLLVMVectorize.a lib/libLLVMInstrumentation.a
> lib/libLLVMObjCARCOpts.a lib/libLLVMScalarOpts.a
> lib/libLLVMInstCombine.a lib/libLLVMTarget.a lib/libLLVMBitWriter.a
> lib/libLLVMIRReader.a lib/libLLVMAsmParser.a lib/libLLVMLinker.a
> lib/libLLVMTransformUtils.a lib/libLLVMAnalysis.a
> lib/libLLVMProfileData.a lib/libLLVMObject.a
> lib/libclangRewriteFrontend.a lib/libclangARCMigrate.a
> lib/libclangStaticAnalyzerFrontend.a lib/libclangFrontend.a
> lib/libclangDriver.a lib/libLLVMOption.a lib/libclangParse.a
> lib/libLLVMMCParser.a lib/libclangSerialization.a
> lib/libLLVMBitReader.a lib/libclangSema.a lib/libclangEdit.a
> lib/libclangStaticAnalyzerCheckers.a lib/libclangStaticAnalyzerCore.a
> lib/libclangAnalysis.a lib/libclangAST.a lib/libclangRewrite.a
> lib/libclangLex.a lib/libclangBasic.a lib/libLLVMCore.a
> lib/libLLVMMC.a lib/libLLVMSupport.a -lrt -ldl -ltinfo -lpthread -lz
> -lm -Wl,-rpath,"\$ORIGIN/../lib" && :
> clang-3.8: error: unable to execute command: Killed
> clang-3.8: error: linker command failed due to signal (use -v to see
> invocation)
> ninja: build stopped: subcommand failed.
>
> This time I just build and installed ld.gold from binutils v2.26.1 and
> symlinked system-wide with my ld of my /opt-installation.
> The generated LLVMgold.so plugin I copied into /usr/lib/bfd-plugins/
> directory (bfd-plugins subdir needed to be created manually).
>
> - Sedat -
>
> P.S.: Output of free and df commands
>
> $ free -m
>             total       used       free     shared    buffers     cached
> Mem:          3846       2211       1635          0        729        818
> -/+ buffers/cache:        662       3183
> Swap:          255        143        112
>
> $ df -T | grep ext4
> /dev/loop0     ext4      17753424  15997332    831216   96% /
>
> - EOT -
>
> Piotr
>
>
> On Sat, Jul 23, 2016 at 2:42 AM, Sedat Dilek via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
>
> On Thu, Jul 21, 2016 at 12:01 PM, Sedat Dilek <sedat.dilek at gmail.com>
> wrote:
>
> Hi,
>
> unfortunately, my build somehow hangs when linking clang binary and my
> system is in an unusable state.
>
> My toolchain is clang-3.8, gold-1.11 and LLVMgold.so from binutils
> v2.26.1 (both selfmade) and LTO-flag is enabled.
> My buildsystem uses cmake-3.6.0 and ninja-1.7.1 (both prebuilt).
> I have 52 last steps left in my 3rd build.
>
> My Linux-kernel is v3.13.0-92 from official Ubuntu repositories.
>
> On my Ubuntu/precise AMD64 I have ***4GiB RAM and 256MiB SWAP***.
> Is this not enough RAM/SWAP for building/linking?
>
> Need some additional informations?
>
> Any help appreciated.
>
> Regards,
> - Sedat -
>
> [1] http://llvm.org/docs/GoldPlugin.html
>
> P.S.: More informations...
>
> [ First toolchain build ]
>
> I built a 1st llvm-toolchain v3.8.1 with GCC v4.9.2 and binutils v2.22.
> No speedup settings, no extra-patches applied etc.
>
> [ Selfmade binutils ]
>
> As binutils v2.22 was somehow not able to generate a LLVMgold.so on
> Ubuntu/precise AMD64, I have built binutils v2.26.1 manually and use
> all its binaries for building/linking (GOLD for linking).
>
> [ 2nd llvm-toolchain LLVMgold.so ]
>
> In a 2nd llvm-toolchain v3.8.1 build a LLVMgold.so was generated.
> I placed LLVMgold.so into my selfmade
> /opt/binutils-2.26.1/lib/bfd-plugins/ directory.
> ( I had to create a bfd-plugins subdir manually. )
>
> [ 3rd llvm-toolchain LTO ]
>
> Now, I am able with backported LTO-flag from upstream to build with
> '-flto'.
>
> - EOT -
>
>
> Hi,
>
> in the meantime I tried with Linux v4.4.y LTS and 2GiB swap-space.
> So I have 4 GiB RAM and 2GiB SWAP in total 6GiB.
>
> Paul Rouschal recommended to reduce parallel-compile-jobs from 2 to 1...
>
> LLVM_PARALLEL_COMPILE_JOBS=1
> LLVM_PARALLEL_LINK_JOBS=1
>
> ...but that did not help.
>
> My Ubuntu/precise hangs and looking at top shows MEM/SWAP to be eaten.
>
> Anyone has experiences how much RAM or SWAP I need when building a
> LTO-optimized llvm-toolchain with Clang, GNU/gold and LLVMgold-Plugin?
>
> My build-script is attached.
>
> Thanks.
>
> Regards,
> - Sedat -
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
> <build_llvm-toolchain.sh>_______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>