<div dir="ltr"><p style="margin-top:0px;margin-bottom:0px"><span style="font-size:10pt">Hello,</span><br></p><div style="color:rgb(0,0,0)"><div><div id="gmail-divtagdefaultwrapper" dir="ltr" style="font-size:10pt"><p style="margin-top:0px;margin-bottom:0px;font-size:13.3333px"><br></p><p style="margin-top:0px;margin-bottom:0px;font-size:13.3333px">I've stumbled upon a linkage error when mixing static libraries built with and without CFI support. I haven't found any information related to the correctness of such mixing, but here are some insights on the LTO optimization process leading to the errors (ThinLTO version is used). Non-CFI library sources are compiled with flags: </p><p style="margin-top:0px;margin-bottom:0px;font-size:13.3333px"><br></p><p style="margin-top:0px;margin-bottom:0px;font-size:13.3333px"></p><div style="font-size:13.3333px">--param ssp-buffer-size=4 -DNDEBUG -D_FORTIFY_SOURCE=2 -O2 -fPIC -fPIE -fgnu-keywords -fstack-protector-all -fstack-protector-strong -fstack-protector-strong -fstrict-<span style="font-size:10pt">aliasing -g -std=c++11 -stdlib=libc++</span></div><div style="font-size:13.3333px"><span style="font-size:10pt"><br></span></div><div style="font-size:13.3333px"><span style="font-size:10pt">CFI library sources are compiled with flags:</span></div><div style="font-size:13.3333px"><span style="font-size:10pt"><br></span></div><div style="font-size:13.3333px"><span style="font-size:10pt"><div>--param ssp-buffer-size=4 -DNDEBUG -D_FORTIFY_SOURCE=2 -D_GLIBCXX_USE_CXX11_ABI=1 -D_LIBCPP_ENABLE_CXX17_REMOVED_AUTO_PTR=1 -D_LIBCPP_ENABLE_CXX17_REMOVED_BINDERS=1 -D<span style="font-size:10pt">_LIBCPP_ENABLE_CXX17_REMOVED_RANDOM_SHUFFLE=1 -O2 -fPIE -fgnu-keywords -flto=thin -fno-sanitize-trap=all -fsanitize-blacklist=/path/to/blacklist/blacklist.txt -fsaniti</span><span style="font-size:10pt">ze-cfi-cross-dso -fsanitize=cfi -fsanitize=safe-stack -fstack-protector-all -fstack-protector-strong -fstandalone-debug -fvisibility-inlines-hidden -fvisibility=hidden </span><span style="font-size:10pt">-g -std=c++17 -stdlib=libc++</span></div><div><span style="font-size:10pt"><br></span></div><div><span style="font-size:10pt">Resulting executable is linked against two static libraries: non-CFI libbenchmark.a, which is Google's Benchmark library, and CFI-enabled libdsl.a, which is mine. Depending on the order of those two libraries in the link command, either linking completes successfully, or multiple undefined hidden symbol errors are emitted:</span></div><div><span style="font-size:10pt"><br></span></div><div><span style="font-size:10pt"><div>ld.lld: error: undefined hidden symbol: std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> ><span style="font-size:10pt">(std::__1::basic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long) (.cfi)</span></div><div>>>> referenced by ld-temp.o</div><div>>>>               lto.tmp:(std::__1::basic_ostream<char, std::__1::char_traits<char> >& std::__1::__put_character_sequence<char, std::__1::char_traits<char> >(std::__1::ba<span style="font-size:10pt">sic_ostream<char, std::__1::char_traits<char> >&, char const*, unsigned long))</span></div><div><br></div><div>ld.lld: error: undefined hidden symbol: std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> > std::__1::__pad_and_output<char, std::__1::char_traits<char> >(st<span style="font-size:10pt">d::__1::ostreambuf_iterator<char, std::__1::char_traits<char> >, char const*, char const*, char const*, std::__1::ios_base&, char) (.cfi)</span></div><div>>>> referenced by ld-temp.o</div><div>>>>               lto.tmp:(std::__1::ostreambuf_iterator<char, std::__1::char_traits<char> > std::__1::__pad_and_output<char, std::__1::char_traits<char> >(std::__1::ostre<span style="font-size:10pt">ambuf_iterator<char, std::__1::char_traits<char> >, char const*, char const*, char const*, std::__1::ios_base&, char))</span></div><div><span style="font-size:10pt"><br></span></div><div><span style="font-size:10pt">An attempt to trace undefined symbol resolution using linker option -Wl,--trace-symbol gives the following result for the failing build:</span></div><div><span style="font-size:10pt"><br></span></div><div><span style="font-size:10pt"><div>.contribs/google_benchmark/lib/libbenchmark.a(benchmark.cc.o): definition of _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_<span style="font-size:10pt">m</span></div><div>.contribs/google_benchmark/lib/libbenchmark.a(benchmark_register.cc.o): reference to _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EE<span style="font-size:10pt">S7_PKS4_m</span></div><div>...more references from libbenchmark.a...</div><div></div><div>lib/libdsl.a(markup_translator.cpp.o): reference to _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><div></div><div>/usr/toolchain/x86_64-pc-linux-gnu_gcc10.2.0_glibc2.11_clang11.0.0/x86_64-pc-linux-gnu/lib/libc++.so.1: shared definition of _ZNSt3__124__put_character_sequenceIcNS_11char<span style="font-size:10pt">_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</span></div><div>lto.tmp: definition of _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><div>lto.tmp: reference to _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><br></span></div></span></div>Successfull build looks like this:</span></div><div style="font-size:13.3333px"><span style="font-size:10pt"><br></span></div><div style="font-size:13.3333px"><span style="font-size:10pt"><div>lib/libdsl.a(markup_translator.cpp.o): definition of _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><div>.contribs/google_benchmark/lib/libbenchmark.a(benchmark.cc.o): reference to _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><div><span id="gmail-ms-rterangepaste-start"></span><span style="font-family:Arial,Helvetica,sans-serif,EmojiFont,"Apple Color Emoji","Segoe UI Emoji",NotoColorEmoji,"Segoe UI Symbol","Android Emoji",EmojiSymbols;font-size:13.3333px">...more references from libbenchmark.a...</span></div><div>/usr/toolchain/x86_64-pc-linux-gnu_gcc10.2.0_glibc2.11_clang11.0.0/x86_64-pc-linux-gnu/lib/libc++.so.1: shared definition of _ZNSt3__124__put_character_sequenceIcNS_11char<span style="font-size:10pt">_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</span></div><div><internal>: reference to _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><div>lto.tmp: definition of _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><div>lto.tmp: reference to _ZNSt3__124__put_character_sequenceIcNS_11char_traitsIcEEEERNS_13basic_ostreamIT_T0_EES7_PKS4_m</div><br></span></div><div style="font-size:13.3333px">It looks like linker fails  to resolve symbol's corresponding .cfi definition if symbol definition is initially found in a library built without CFI support, and succeeds otherwise. Also, additional <internal> reference is present in the latter case. <span style="font-size:10pt">Rebuilding all static libraries with CFI support solves the problem, but it is unclear to me whether this behavior is correct or this is some kind of a bug. Unfortunately, </span><span style="font-size:10pt">I was unable to make</span><span style="font-size:10pt"> a minimal example reproducing the issue.</span></div><br class="gmail-Apple-interchange-newline"></div></div></div></div>