[llvm-bugs] [Bug 42030] New: lld doesn't generate DW_MACRO_import like ld.bfd does, leading to gdb cpu/memory hog

[via llvm-bugs]

https://bugs.llvm.org/show_bug.cgi?id=42030

            Bug ID: 42030
           Summary: lld doesn't generate DW_MACRO_import like ld.bfd does,
                    leading to gdb cpu/memory hog
           Product: lld
           Version: unspecified
          Hardware: PC
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P
         Component: ELF
          Assignee: unassignedbugs at nondot.org
          Reporter: romain.geissler at amadeus.com
                CC: llvm-bugs at lists.llvm.org, peter.smith at linaro.org

Hi,

While transitioning from ld.bfd to ld.lld, I found that on big binaries gdb was
*VERY* slow to display backtraces (up to 30 minutes for the full stack) and
consumed tens and tens of GBs of memory.

I was able to reproduce this, and found that the issue comes from the
.debug_macro section of the final binary, if I remove it with objcopy then gdb
happily displays stacks almost instantly. When checking the difference between
ld.bfd and ld.lld (using "readelf --debug-dump=macro") I saw these kind of
differences:

--- macro.exe.bfd       2019-05-26 20:45:11.174723997 +0000
+++ macro.exe.lld       2019-05-26 20:45:16.882756121 +0000
@@ -3433,38 +3433,38 @@
  DW_MACRO_import - offset : 0x4abb
  DW_MACRO_start_file - lineno: 0 filenum: 1 filename: object.cpp
  DW_MACRO_start_file - lineno: 0 filenum: 55 filename:
/opt/1A/toolchain/x86_64-v19.0.20/include/stdc-predef.h
- DW_MACRO_import - offset : 0x13cc
+ DW_MACRO_import - offset : 0x0
  DW_MACRO_end_file
  DW_MACRO_start_file - lineno: 1 filenum: 56 filename:
/remote/tools/Linux/2.6/1A/toolchain/x86_64-v19.0.20/include/c++/9.1.1/string
  DW_MACRO_define_strp - lineno : 34 macro : _GLIBCXX_STRING 1
  DW_MACRO_start_file - lineno: 38 filenum: 4 filename:
/remote/tools/Linux/2.6/1A/toolchain/x86_64-v19.0.20/include/c++/9.1.1/x86_64-1a-linux-gnu/bits/c++config.h
- DW_MACRO_import - offset : 0x13e8
+ DW_MACRO_import - offset : 0x0
  DW_MACRO_start_file - lineno: 524 filenum: 57 filename:
/remote/tools/Linux/2.6/1A/toolchain/x86_64-v19.0.20/include/c++/9.1.1/x86_64-1a-linux-gnu/bits/os_defines.h
- DW_MACRO_import - offset : 0x1564
+ DW_MACRO_import - offset : 0x0
  DW_MACRO_start_file - lineno: 39 filenum: 58 filename:
/opt/1A/toolchain/x86_64-v19.0.20/include/features.h
- DW_MACRO_import - offset : 0x1574
+ DW_MACRO_import - offset : 0x0
  DW_MACRO_start_file - lineno: 450 filenum: 59 filename:
/opt/1A/toolchain/x86_64-v19.0.20/include/sys/cdefs.h

always around the value of DW_MACRO_import which is always 0 when using ld.lld,
and always non zero when using ld.bfd.

I have applied a warkaround in gdb to ignore DW_MACRO_import when the value is
0, and indeed it "fixes" our issue (bug maybe we loose some macro data).

Here is a full reproducer of my issue:

> cat main.cpp 
#include <iostream>

extern int f1();

int main()
{
    std::cout << f1() << std::endl;
}

> cat object.cpp
#include <string>
#include <memory>
#include <algorithm>
#include <iostream>
#include <fstream>
#include <sstream>
#include <map>
#include <list>
#include <vector>
#include <unordered_map>

#define CONCAT_AUX(a,b) a##b
#define CONCAT(a,b) CONCAT_AUX(a,b)

extern int CONCAT(f,BUG_NEXT_ITERATION)();

int CONCAT(f,BUG_ITERATION)()
{   
    return CONCAT(f,BUG_NEXT_ITERATION)() + BUG_ITERATION;
}

#if BUG_ITERATION == BUG_LAST_ITERATION
int CONCAT(f,BUG_NEXT_ITERATION)()
{
    return 42;
}   
#endif

> cat Makefile
N=10000
OBJECT_FILES=$(shell echo object{1..${N}}.o)

all:exe.bfd exe.lld exe.lld.no-macro

exe.bfd exe.lld:${OBJECT_FILES} main.o
        g++ -fuse-ld=${LINKER} -o "$@" $^

exe.lld.no-macro:exe.lld
        objcopy --remove-section .debug_macro "$<" "$@"

exe.bfd:LINKER=bfd
exe.lld:LINKER=lld

main.o:main.cpp
        g++ -ggdb3 -O2 -c "$<" -o "$@"

${OBJECT_FILES}:object.cpp
        g++ -DBUG_ITERATION=$$(sed -e 's/[^[:digit:]]*//g'<<<"$@")
-DBUG_NEXT_ITERATION=$$(( $$(sed -e 's/[^[:digit:]]*//g'<<<"$@") + 1 ))
-DBUG_LAST_ITERATION=${N} -ggdb3 -O2 -c "$<" -o "$@"

clean:  
        ${RM} *.o exe.bfd exe.lld exe.lld.no-macro

.PHONY:all clean

Just type "make -j $CPU_COUNT" and wait for the 10000 object files to be
created. Then open "exe.lld" with gdb, add a breakpoint on "f10000", run the
program, and when the breakpoint is hit just display the backtrace: it will be
very slow. If you do the same with "exe.bfd", it will be done instantly.

Note: I am using gcc 9, if you don't use it then "-fuse-ld=lld" will not work.
I don't know if using clang exposes similar issues.

Cheers,
Romain

-- 
You are receiving this mail because:
You are on the CC list for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-bugs/attachments/20190527/c544e233/attachment.html>