[llvm-dev] [llvm-toolchain v3.8.1] LTO: Linking clang hangs with ld.gold and LLVMgold.so plugin

[Sedat Dilek via llvm-dev]

On Sat, Jul 23, 2016 at 11:59 PM, Mehdi Amini <mehdi.amini at apple.com> wrote:
>
> On Jul 23, 2016, at 1:53 PM, Sedat Dilek via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
>
> On Sat, Jul 23, 2016 at 7:48 PM, Piotr Padlewski <prazek at google.com> wrote:
>
> How big is your project?
> LTO eats RAM even faster than chrome. For example linking clang with LTO
> could take 16GB of ram.
>
> Have you tried using LTO on your project on that machine, or is it your
> first time?
>
>
> Wow hu hu - 16GiB of RAM :-).
>
>
> It is a bit less now: should be closer to 11GB with Full Debug Info.
> You may want to try without debug info (or with -gline-tables-only) and it
> should get under 4GB.
>

What are you recommending?

I am normally doing RELEASE builds.

"Using Sampling Profilers" [2] is a sub-section of "Profile Guided
Optimization" which says...

#1: Build the code with source line table information. You can use all
the usual build flags that you always build your application with. The
only requirement is that you add -gline-tables-only or -g to the
command line. This is important for the profiler to be able to map
instructions back to source line locations.

$ clang++ -O2 -gline-tables-only code.cc -o code

...currently, I do not have PGO enabled via cmake-options.

How to pass "-gline-tables-only" to the *_FLAGS_* cmake-options?

$ cmake ../llvm -LA | egrep 'FLAGS_DEBUG:' | sort
CMAKE_ASM_FLAGS_DEBUG:STRING=-g
CMAKE_C_FLAGS_DEBUG:STRING=-g
CMAKE_CXX_FLAGS_DEBUG:STRING=-g
CMAKE_EXE_LINKER_FLAGS_DEBUG:STRING=
CMAKE_MODULE_LINKER_FLAGS_DEBUG:STRING=
CMAKE_SHARED_LINKER_FLAGS_DEBUG:STRING=
CMAKE_STATIC_LINKER_FLAGS_DEBUG:STRING=

$ cmake ../llvm -LA | egrep 'FLAGS_RELEASE:' | sort
CMAKE_ASM_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_C_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELEASE:STRING=-O3 -DNDEBUG
CMAKE_EXE_LINKER_FLAGS_RELEASE:STRING=
CMAKE_MODULE_LINKER_FLAGS_RELEASE:STRING=
CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=
CMAKE_STATIC_LINKER_FLAGS_RELEASE:STRING=

Not sure, is there a cmake-options available to set "-gline-tables-only"

$ cmake ../llvm -LA | egrep -i 'table|line|pgo|prof' | sort
CLANG_EXECUTABLE_VERSION:STRING=3.8
CLANG_TABLEGEN:STRING=clang-tblgen
GO_EXECUTABLE:FILEPATH=GO_EXECUTABLE-NOTFOUND
GOLD_EXECUTABLE:FILEPATH=/opt/binutils/bin/ld.gold
LIBXML2_XMLLINT_EXECUTABLE:FILEPATH=/usr/bin/xmllint
LLVM_OPTIMIZED_TABLEGEN:BOOL=OFF
LLVM_PROFDATA_FILE:FILEPATH=
LLVM_TABLEGEN:STRING=llvm-tblgen
LLVM_TOOL_LLVM_PROFDATA_BUILD:BOOL=ON
LLVM_USE_OPROFILE:BOOL=OFF
PKG_CONFIG_EXECUTABLE:FILEPATH=/usr/bin/pkg-config
PYTHON_EXECUTABLE:FILEPATH=/usr/bin/python2.7

Many questions, I know...

Thanks.

- sed@ -

[1] http://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization
[2] http://clang.llvm.org/docs/UsersManual.html#using-sampling-profilers