<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Jul 23, 2020, at 2:12 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" class="">dblaikie@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">(certainly seems like documenting this somewhere would be good, yes - perhaps a FAQ section or similar in: <a href="https://clang.llvm.org/docs/ThinLTO.html#usage" class="">https://clang.llvm.org/docs/ThinLTO.html#usage</a> ? (that's the first result for thinlto on google, so seems like a good place for more documentation))</div></div></blockquote><div><br class=""></div>This is more like how a static library is constructed for Darwin platform. The takeaway is that you need to know when you need to have the correct libLTO to read bitcode file. Not sure if the page you mentioned is the best place for this.</div><div><br class=""></div><div><blockquote type="cite" class=""><div class=""><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 23, 2020 at 1:10 PM Tobias Hieta <<a href="mailto:tobias@plexapp.com" class="">tobias@plexapp.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto" class="">Steven,<div dir="auto" class=""><br class=""></div><div dir="auto" class="">Thanks so much for your help! How do you check the symtab in an archive so that I can learn and diagnose this by myself in the future?</div></div></blockquote></div></div></blockquote><div><br class=""></div><div>I am not sure there is a tool to help you read the symtab in the static archive. It is the first file in the archive which encodes a symbol table for faster lookup.</div><div><br class=""></div><div>The way I do it is to use `ar x` to extract the archive and hexdump the "__.SYMDEF SORTED" file from the archive.</div><div><br class=""></div><div>Steven</div><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto" class=""><div dir="auto" class=""><br class=""></div><div dir="auto" class="">I had problems with libtool and llvm-ranlib before - but I will revisit and now I know the fallback is to set RANLIB=true</div><div dir="auto" class=""><br class=""></div><div dir="auto" class="">I wonder if edge cases like this could have a good place to be documented since it's not obvious at all and hard to diagnose.</div><div dir="auto" class=""><br class=""></div><div dir="auto" class="">Once again I am very grateful you took the time and looked at this! Have a great weekend!</div><div dir="auto" class=""><br class=""></div><div dir="auto" class="">Tobias. </div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Jul 23, 2020, 19:16 Steven Wu <<a href="mailto:stevenwu@apple.com" target="_blank" class="">stevenwu@apple.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="overflow-wrap: break-word;" class="">Hi Tobias<div class=""><br class=""></div><div class="">The problem is that your static archive has a SYMDEF that is empty, so linker thinks the static library provided doesn't contain any symbol. The reason for that is you are using the `ranlib` from Xcode, which is too old to understand the new bitcode object files produced by llvm 10.</div><div class=""><br class=""></div><div class="">There are lots of ways to fix that:</div><div class="">* The standard way to create static library on macOS is to use `libtool`. You can use DYLD_LIBRARY_PATH to point libtool to the libLTO from llvm 10. `ranlib` from Xcode is actually a symlink to `libtool` which provide a compatible legacy command line interface.</div><div class="">* Use llvm-ranlib from llvm 10 instead.</div><div class="">* Don't use `ranlib`. Then the static library won't have SYMDEF and linker will fallback to parse all the object file in the static library. That will sacrifice some link time but it will link correctly.</div><div class=""><br class=""></div><div class="">Steven</div><div class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Jul 22, 2020, at 11:41 PM, Tobias Hieta <<a href="mailto:tobias@plexapp.com" rel="noreferrer" target="_blank" class="">tobias@plexapp.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class="">Steven,<div class=""><br class=""></div><div class="">This is confusing. In the orig_command_line you can see that src/.libs/libogg.a is passed to the command line and even when running verbose clang I can see that clang invokes the linker with libogg.a on the line:</div><div class=""><br class=""></div><div class=""> "/Applications/Xcode-11.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld" -demangle -object_path_lto /var/folders/5c/85r7gp0909j5jbytzds1j7b40000gn/T/thinlto-74963a -lto_library /Applications/Xcode-11.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.15.0 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o test /var/folders/5c/85r7gp0909j5jbytzds1j7b40000gn/T/test-d5e7a1.o src/.libs/libogg.a -debug_snapshot -lc++ -lSystem /Applications/Xcode-11.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/11.0.0/lib/darwin/libclang_rt.osx.a<br class=""></div><div class=""><br class=""></div><div class="">When is the link_command file written and do you have any idea why this file is discarded at this point?</div><div class=""><br class=""></div><div class="">Using clang from xcode seems to work - it includes the file in archive_files.</div><div class=""><br class=""></div><div class="">I built libogg.a with both xcode/clang and <a href="http://llvm.org/" rel="noreferrer" target="_blank" class="">llvm.org</a> clang and then tried to link to them and it reproduces each time that it won't link to the <a href="http://llvm.org/" rel="noreferrer" target="_blank" class="">llvm.org</a> version while the xcode built one works fine. I have tarred the libogg files together and uploaded here:</div><div class=""><br class=""></div><div class=""><a href="https://drive.google.com/file/d/1_mgN1w4iPCB8vz0KBDOc-5ypaNBaiFJA/view?usp=sharing" rel="noreferrer" target="_blank" class="">https://drive.google.com/file/d/1_mgN1w4iPCB8vz0KBDOc-5ypaNBaiFJA/view?usp=sharing</a><br class=""></div><div class=""><br class=""></div><div class="">I tried to link to it with:</div><div class=""><br class=""></div><div class="">clang++ -o test -flto=thin test.cpp libogg_xcode.a -I include<br class=""></div><div class=""><br class=""></div><div class="">I am trying to think if I am doing something really bad here - but I have double checked everything I can think of and I am just downloading the <a href="http://llvm.org/" rel="noreferrer" target="_blank" class="">llvm.org</a> package and don't have any customizations and then just pass the -flto=thin flag and use llvm-ar as ar.</div><div class=""><br class=""></div><div class="">Thanks again,</div><div class="">Tobias</div></div></div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 22, 2020 at 10:10 PM Steven Wu <<a href="mailto:stevenwu@apple.com" rel="noreferrer" target="_blank" class="">stevenwu@apple.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="">Looks like your static library is not even pulled into the link command so the static library is not even in the snapshot. From the link command in the snapshot, the static library is not on the command line from snapshot:<div class=""><br class=""><div class="">/Applications/Xcode-11.3.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld -Z -demangle -object_path_lto /var/folders/5c/85r7gp0909j5jbytzds1j7b40000gn/T/thinlto-bf51d2 -lto_library data_files/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.15.0 -debug_snapshot -filelist objects/filelist dylibs/libSystem.tbd dylibs/libc++.tbd<br class=""><br class=""><div class=""><br class=""></div><div class="">Steven</div><div class=""><br class=""></div><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Jul 22, 2020, at 11:42 AM, Tobias Hieta <<a href="mailto:tobias@plexapp.com" rel="noreferrer" target="_blank" class="">tobias@plexapp.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class=""><div dir="ltr" class="">Hello,<div class=""><br class=""></div><div class="">Thanks for the reply Teresa and Steven,</div><div class=""><br class=""></div><div class="">I uploaded the snapshot to GDrive: <a href="https://drive.google.com/file/d/11Ngst9FOnVL4fWYlKalzoFXL--B1SaOb/view?usp=sharing" rel="noreferrer" target="_blank" class="">https://drive.google.com/file/d/11Ngst9FOnVL4fWYlKalzoFXL--B1SaOb/view?usp=sharing</a></div><div class=""><br class=""></div><div class="">My ld version is:</div><div class=""><br class=""></div><div class=""><div class="">➜ ld -v</div><div class="">@(#)PROGRAM:ld PROJECT:ld64-530</div><div class="">BUILD 18:57:17 Dec 13 2019</div><div class="">configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em</div><div class="">LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)</div><div class="">TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)</div></div><div class=""><br class=""></div><div class="">Since that said it was using LTO from Xcode I thought that might be the problem - but when I run clang with -v to see the linker invocation it seems to pass the correct libLTO.dylib:</div><div class=""> </div><div class="">"/usr/bin/ld" -demangle -object_path_lto /var/folders/5c/85r7gp0909j5jbytzds1j7b40000gn/T/thinlto-a8ae97 -lto_library /Users/tobias/tmp/clang+llvm-10.0.1-x86_64-apple-darwin/lib/libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -platform_version macos 10.15.0 10.15 -syslibroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o test /var/folders/5c/85r7gp0909j5jbytzds1j7b40000gn/T/test-d0eb14.o src/.libs/libogg.a -debug_snapshot -lc++ -lSystem /Users/tobias/tmp/clang+llvm-10.0.1-x86_64-apple-darwin/lib/clang/10.0.1/lib/darwin/libclang_rt.osx.a<br class=""></div><div class=""><br class=""></div><div class="">Thanks again,</div><div class="">Tobias</div></div></div></div></div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 22, 2020 at 7:20 PM Steven Wu <<a href="mailto:stevenwu@apple.com" rel="noreferrer" target="_blank" class="">stevenwu@apple.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="">This is usually a problem that is not using llvm-ar. I cannot reproduce this problem with either llvm 10.0 or TOT version. Which linker version are you using? You can also try pass "-Wl,-debug_snapshot" to the command where the error produces and then locate the "*.ld-snapshot" in /tmp directory and attach that as a reproducer.<div class=""><br class=""></div><div class="">Steven<br class=""><div class=""><br class=""><blockquote type="cite" class=""><div class="">On Jul 22, 2020, at 8:41 AM, Teresa Johnson <<a href="mailto:tejohnson@google.com" rel="noreferrer" target="_blank" class="">tejohnson@google.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class="">Adding Steven Wu who can hopefully help as this is MacOS. Unfortunately, I don't have a way to run the MacOS compiler myself, and it uses the old LTO API which I am less familiar with. Typically these issues happen if you don't use llvm-ar, but it looks like you are using that so I'm not sure.<div class=""><br class=""></div><div class="">Teresa</div></div><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none"><div dir="ltr" class="gmail_attr">On Wed, Jul 22, 2020 at 12:29 AM Tobias Hieta <<a href="mailto:tobias@plexapp.com" rel="noreferrer" target="_blank" class="">tobias@plexapp.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">David,<br class=""><br class="">Thanks for looking into this. I did a small reproduction on my machine<br class="">outside of my build system. So here is how to reproduce:<br class=""><br class="">Download<span class=""> </span><a href="https://downloads.xiph.org/releases/ogg/libogg-1.3.4.tar.xz" rel="noreferrer noreferrer" target="_blank" class="">https://downloads.xiph.org/releases/ogg/libogg-1.3.4.tar.xz</a><br class="">Download llvm-10.0.1 macOS binary<br class=""><br class="">export PATH=<path to llvm/bin>:$PATH<br class="">export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk<br class=""><br class="">untar libogg<br class="">AR=llvm-ar CC=clang CXX=clang++ CFLAGS="-flto=thin<br class="">-mmacosx-version-min=10.9" LDFLAGS=-flto=thin ./configure<br class="">--disable-shared --enable-static<br class="">make -j20<br class=""><br class="">Then try to link to the library in a small C++ program - I used this:<br class=""><br class="">#include <iostream><br class="">#include "ogg/ogg.h"<br class=""><br class="">using namespace std;<br class=""><br class="">int main()<br class="">{<br class="">ogg_stream_state os;<br class="">if (ogg_stream_init(&os, 123) == 0)<br class="">cout << "Initialized stream succesfully" << endl;<br class=""><br class="">return 0;<br class="">}<br class=""><br class="">And from the libogg directory I linked to it like this:<br class=""><br class="">clang++ -o test -flto=thin test.cpp src/.libs/libogg.a -I include<br class=""><br class="">undef: _ogg_stream_init<br class="">Undefined symbols for architecture x86_64:<br class="">"_ogg_stream_init", referenced from:<br class="">_main in 0.x86_64.thinlto.o<br class="">ld: symbol(s) not found for architecture x86_64<br class="">clang-10: error: linker command failed with exit code 1 (use -v to see<br class="">invocation)<br class=""><br class="">hope this helps - thanks!<br class=""><br class="">On Wed, Jul 22, 2020 at 9:11 AM David Blaikie <<a href="mailto:dblaikie@gmail.com" rel="noreferrer" target="_blank" class="">dblaikie@gmail.com</a>> wrote:<br class="">><br class="">> Got a link to the source/build instructions?<br class="">><br class="">> This sort of thing happens more often in C++ with templates where one<br class="">> object depends (incorrectly) on an implicit instantiation created in<br class="">> another object, rather than carrying its own instantiation.<br class="">><br class="">> Not sure what might cause it in C code.<br class="">><br class="">> On Tue, Jul 21, 2020 at 11:47 PM Tobias Hieta via llvm-dev<br class="">> <<a href="mailto:llvm-dev@lists.llvm.org" rel="noreferrer" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class="">> ><br class="">> > Hello,<br class="">> ><br class="">> > I am building libogg with clang (10.0.1) on macOS and if I pass<br class="">> > "-flto=thin" to C and LDFLAGS it will not link correctly claiming<br class="">> > missing symbols when linking to the archive (libogg.a).<br class="">> ><br class="">> > undef: _ogg_stream_init<br class="">> > Undefined symbols for architecture x86_64:<br class="">> > "_ogg_stream_init", referenced from:<br class="">> > _main in lto.o<br class="">> ><br class="">> > Removing lto=thin fixes the problem. Inspecting the AR libs with<br class="">> > llvm-nm I see the symbol there (but without address):<br class="">> ><br class="">> > not working archive:<br class="">> > ---------------- T _ogg_stream_init<br class="">> ><br class="">> > working archive:<br class="">> > 0000000000000200 T _ogg_stream_init<br class="">> ><br class="">> > My guess is that this output is correct since the archive contains<br class="">> > bitcode in the thin lto case and otherwise it's the finished object.<br class="">> ><br class="">> > It seems to me that the LTO decides to not include this symbol? It's<br class="">> > defined like this:<br class="">> ><br class="">> > extern int ogg_stream_init(ogg_stream_state *os,int serialno);<br class="">> ><br class="">> > llvm-ar is used to create the archive.<br class="">> ><br class="">> > Is there any good way to debug this?<br class="">> ><br class="">> > Thanks,<br class="">> > Tobias<br class="">> > _______________________________________________<br class="">> > LLVM Developers mailing list<br class="">> ><span class=""> </span><a href="mailto:llvm-dev@lists.llvm.org" rel="noreferrer" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class="">> ><span class=""> </span><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer noreferrer" target="_blank" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class=""></blockquote></div><br clear="all" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><br class=""></div><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none;float:none;display:inline" class="">--<span class=""> </span></span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><div dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;text-decoration:none" class=""><div dir="ltr" class=""><div class=""><span style="font-family:Times;font-size:inherit" class=""><table cellspacing="0" cellpadding="0" class=""><tbody class=""><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small" class=""><td nowrap="" style="border-top:2px solid rgb(213,15,37)" class="">Teresa Johnson |</td><td nowrap="" style="border-top:2px solid rgb(51,105,232)" class=""> Software Engineer |</td><td nowrap="" style="border-top:2px solid rgb(0,153,57)" class=""> <a href="mailto:tejohnson@google.com" rel="noreferrer" target="_blank" class="">tejohnson@google.com</a> |</td></tr></tbody></table></span></div></div></div></div></blockquote></div><br class=""></div></div></blockquote></div>
</div></blockquote></div><br class=""></div></div></div></blockquote></div>
</div></blockquote></div><br class=""></div></div></blockquote></div>
</blockquote></div>
</div></blockquote></div><br class=""></body></html>