<div dir="ltr"><span style="font-family:monospace">Hi<br><br>There appears to be some bugs in LLD's handling of the '--wrap' option that results in unexpected<br>"undefined symbols" in the output.<br><br>I've broken this report into { 1. Issues; 2. Reproduction; 3. Reproduction Tests; 4: Side Observations }.<br>Please can somebody take a look at {1,2,3} to confirm if these are genuine issues?<br><br>Many Thanks,<br>Martin<br><br>1) Issues:<br>==========<br>  1.1) Using '--wrap x' results in undefined references to 'x' even when '__real_x' isn't used in<br>     the source code.<br>    - i.e. The symbol table implies there is a dependency where one doesn't exist<br>    - Present in at least LLD-9, LLD-11<br>  1.2) Using '--wrap x' where 'x' is in "llvm-project/compiler-rt/lib/dfsan/done_abilist.txt"*<br>     results in undefined references even when 'x' is never used<br>    - i.e. The symbol table implies there is a dependency where one doesn't exist, and exposes<br>      internal details of the linker implementation<br>    - *There a strong correlation with the symbols on this list, but has NOT been proven as the<br>      root cause(!)<br>    - Present in at least LLD-9, LLD-11<br><br>Whilst investigating the above I also came across the following issue that appears to be resolved<br>in LLD-11, but am recording here in case it is relevant, or helps anybody else who encounters the<br>issue:<br>  1.3) Using '--wrap x' results in undefined references to '__real_x'<br>    - i.e. The symbol table implies there is a dependency where one doesn't exist<br>    - Present in at least LLD-9, and resolved via <a href="https://reviews.llvm.org/D34993">https://reviews.llvm.org/D34993</a><br><br>2) Reproduction<br>===============<br><br>2.1) Source setup<br>-----------------<br><br>Source file, foo.c, which has function calls to:<br>  a) bar_fn_not_wrapped<br>    - A regular function, no wrapping<br>    - Expect undefined symbols: bar_fn_not_wrapped<br>    - Unexpected undefined symbols: __wrap_bar_fn_not_wrapped, __real_bar_fn_not_wrapped<br>  b) bar_fn_wrapped<br>    – In the tests we’ll wrap this call<br>    - Expect undefined symbols: __wrap_bar_fn_wrapped<br>    - Unexpected undefined symbols: bar_fn_wrapped, __real_bar_fn_wrapped<br>  c) gettimeofday<br>    – In the tests we’ll wrap this call<br>    - Expect undefined symbols: __wrap_gettimeofday<br>    - Unexpected undefined symbols: gettimeofday, __real_gettimeofday<br>Also note foo.c does NOT call the following functions:<br>  d) sigaction<br>    – In the tests we’ll wrap this call<br>    - Expected undefined symbols: <none><br>    - Unexpected undefined symbols: sigation, __wrap_sigaction, __real_sigaction<br>  e) bar_fn_other<br>    – In the tests we’ll wrap this call<br>    - Expected undefined symbols: <none><br>    - Unexpected undefined symbols: bar_fn_other, __wrap_bar_fn_other, __real_bar_fn_other<br><br>This is all summarized in the table below, and we’ll use this to compare against in the tests:<br>+--------------------+-----------------+-------------------+-----------------------------------+<br>| Symbol x           | x used in foo.c | Wrapped in tests? | Expect undefined symbol to ... ?  |<br>|                    |                 |                   | x         | __wrap_x  | __real_x  |<br>+--------------------+-----------------+-------------------+-----------+-----------+-----------+<br>| bar_fn_not_wrapped | Y               | N                 | Y         | N         | N         |<br>| bar_fn_wrapped     | Y               | Y                 | N         | Y         | N         |<br>| gettimeofday       | Y               | Y                 | N         | Y         | N         |<br>| sigaction          | N               | Y                 | N         | N         | N         |<br>| bar_fn_other       | N               | Y                 | N         | N         | N         |<br>+--------------------+-----------------+-------------------+-----------+-----------+-----------+<br><br>2.3) foo.c source code<br>----------------------<br>  /* --- foo.c start --- */<br>  #include <sys/time.h><br><br>  void bar_fn_not_wrapped(void);<br>  void bar_fn_wrapped(void);<br><br>  void<br>  foo_fn (void)<br>  {<br>      struct timeval  tv;<br>      struct timezone tz;<br><br>      bar_fn_wrapped();<br>      bar_fn_not_wrapped();<br>      (void)gettimeofday(&tv, &tz);<br>  }<br>  /* --- foo.c end --- */<br><br><br>2.3) Test setup<br>---------------<br><br>For each compiler/linker combination below we’ll run the following command:<br>  $ <compiler> [ -fuse-ld=<optional-linker-choice ] -fPIC -shared foo.c -Wl,--wrap=sigaction \<br>    -Wl,--wrap=gettimeofday -Wl,--wrap=bar_fn_wrapped -Wl,--wrap=bar_fn_other -o libfoo.so<br>And search for the interesting symbols using:<br>  $ nm -D libfoo.so --undefined-only | grep -E "(sig|get|bar)" | tr -s ' ' | sed 's/^/  /'<br><br>The compiler/linkers used are:<br> a) gcc 4.7.0, gnu-ld<br> b) clang-9, gnu-ld<br> c) clang-9, llvm-lld-9<br> d) clang-11, llvm-lld-11<br> <br>3) Reproduction Tests:<br>======================<br>NB: Bad results are highlighted with *Y* or *N*<br><br>a) gcc 4.7.0, gnu-ld<br>  U bar_fn_not_wrapped<br>  U __wrap_bar_fn_wrapped<br>  U __wrap_gettimeofday<br><br>  +--------------------+-----------------+-------------------+-----------------------------------+<br>  | Symbol x           | x used in foo.c | Wrapped in tests? | Undefined symbol to ... ?         |<br>  |                    |                 |                   | x         | __wrap_x  | __real_x  |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br>  | bar_fn_not_wrapped | Y               | N                 | Y         | N         | N         |<br>  | bar_fn_wrapped     | Y               | Y                 | N         | Y         | N         |<br>  | gettimeofday       | Y               | Y                 | N         | Y         | N         |<br>  | sigaction          | N               | Y                 | N         | N         | N         |<br>  | bar_fn_other       | N               | Y                 | N         | N         | N         |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br><br>b) clang-9, gnu-ld<br>  U bar_fn_not_wrapped<br>  U __wrap_bar_fn_wrapped<br>  U __wrap_gettimeofday<br><br>  +--------------------+-----------------+-------------------+-----------------------------------+<br>  | Symbol x           | x used in foo.c | Wrapped in tests? | Undefined symbol to ... ?         |<br>  |                    |                 |                   | x         | __wrap_x  | __real_x  |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br>  | bar_fn_not_wrapped | Y               | N                 | Y         | N         | N         |<br>  | bar_fn_wrapped     | Y               | Y                 | N         | Y         | N         |<br>  | gettimeofday       | Y               | Y                 | N         | Y         | N         |<br>  | sigaction          | N               | Y                 | N         | N         | N         |<br>  | bar_fn_other       | N               | Y                 | N         | N         | N         |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br><br>c) clang-9, llvm-lld-9<br>  U bar_fn_not_wrapped<br>  U bar_fn_wrapped<br>  U gettimeofday<br>  U __real_bar_fn_wrapped<br>  U __real_gettimeofday<br>  w __real_sigaction<br>  w sigaction<br>  U __wrap_bar_fn_wrapped<br>  U __wrap_gettimeofday<br>  U __wrap_sigaction<br><br>  +--------------------+-----------------+-------------------+-----------------------------------+<br>  | Symbol x           | x used in foo.c | Wrapped in tests? | Undefined symbol to ... ?         |<br>  |                    |                 |                   | x         | __wrap_x  | __real_x  |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br>  | bar_fn_not_wrapped | Y               | N                 | Y         | N         | N         |<br>  | bar_fn_wrapped     | Y               | Y                 | *Y*       | Y         | *Y*       |<br>  | gettimeofday       | Y               | Y                 | *Y*       | Y         | *Y*       |<br>  | sigaction          | N               | Y                 | *Y*       | *Y*       | *Y*       |<br>  | bar_fn_other       | N               | Y                 | N         | N         | N         |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br><br>d) clang-11, llvm-lld-11<br>   U __wrap_bar_fn_wrapped<br>   U __wrap_gettimeofday<br>   U __wrap_sigaction<br>   U bar_fn_not_wrapped<br>   U bar_fn_wrapped<br>   U gettimeofday<br>   w sigaction<br><br>  +--------------------+-----------------+-------------------+-----------------------------------+<br>  | Symbol x           | x used in foo.c | Wrapped in tests? | Undefined symbol to ... ?         |<br>  |                    |                 |                   | x         | __wrap_x  | __real_x  |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br>  | bar_fn_not_wrapped | Y               | N                 | Y         | N         | N         |<br>  | bar_fn_wrapped     | Y               | Y                 | *Y*       | Y         | N         |<br>  | gettimeofday       | Y               | Y                 | *Y*       | Y         | N         |<br>  | sigaction          | N               | Y                 | *Y*       | *Y*       | N         |<br>  | bar_fn_other       | N               | Y                 | N         | N         | N         |<br>  +--------------------+-----------------+-------------------+-----------+-----------+-----------+<br> <br>4) Side observations:<br>=====================<br>A few observations made whilst investigating the main issues. It's likely that these won't be<br>critical to the main report, but are left here in case it aids discussion on this topic:<br><br>  a) I can’t find much online regarding this behaviour. One interesting reference, although<br>     doesn’t explain the above, is at <a href="http://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities">http://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities</a>:<br>    - """<br>      Semantics of --wrap:<br>      GNU ld hand LLD have slightly different --wrap semantics. I use "slightly" because in most<br>      use cases users will not observe a difference.<br>      In GNU ld, --wrap only applies to undefined symbols. In LLD, --wrap happens after all other<br>      symbol resolution steps. The implementation is to mangle the symbol table of each object<br>      file (foo -> __wrap_foo; __real_foo -> foo) so that all relocations to foo or __real_foo<br>      will be redirected.<br>      The LLD semantics have the advantage that non-LTO, LTO and relocatable link behaviors are<br>      consistent. I filed <a href="https://sourceware.org/bugzilla/show_bug.cgi?id=26358">https://sourceware.org/bugzilla/show_bug.cgi?id=26358</a> for GNU ld.<br>      """<br>    - Looking at the corresponding bug <a href="https://sourceware.org/bugzilla/show_bug.cgi?id=26358">https://sourceware.org/bugzilla/show_bug.cgi?id=26358</a>, the<br>      suggestion is that LLD is more consistent, but when I’ve tried the steps for their partial<br>      linking example it demonstrates different behaviour.<br>  b) I wonder if the run-time behaviour with these unexpected undefined symbols is<br>     impacted? e.g. what happens with -z,now? (Not investigated)<br>  c) I wonder if the build-link-time behaviour is impacted. e.g. what happens when linking with a<br>     library dependency that has one of these missing symbol dependencies? (Not investigated)</span></div>