<div dir="ltr"><div dir="ltr">Teresa,<div><br></div></div><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 9, 2020 at 9:10 AM Teresa Johnson <<a href="mailto:tejohnson@google.com">tejohnson@google.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Adding a couple of lld folks.</div><div><br></div><div>I helped Shishir debug this, the link line looked like:</div><div> /home/sjessu/build/bin/clang -O0 -flto -o jcstest jcstest.o ./.libs/libjpeg.a</div><div>and the issue was that libjpeg.a was created with the system ar instead of llvm-ar. It worked when recreating libjpeg.a with llvm-ar.</div><div><br></div><div>I noticed that the lld code has some special handling for the case when there is a missing symbol table, which often happens with system ar created archives containing bitcode. I noticed that the lld code will sometimes emit an error, but actually contains a special hack to handle archives containing *only* bitcode objects, so that they are handled correctly even when there is no symbol table because it was created with the system ar. Unfortunately, in this case it neither gave an error nor did the special handling, because libjpeg.a also contains some native objects and thus had a non-zero symbol table. I created a version of libjpeg.a using the system library and containing only the bitcode objects, and confirmed it links fine with lld (the native objects weren't needed in this case). BTW this is the code in ELF/Driver.cpp LinkerDriver::addFile.</div><div><br></div><div>Would it be possible to extend the hack in lld to handle cases like this with some bitcode objects and some non-bitcode objects, so that the bitcode objects are not simply ignored?</div></div></blockquote><div><br></div><div>Interesting suggestion. So, as you summarized, lld has a special hack for LTO in terms of archive file handling. That is, if an archive file's symbol table is completely empty, we consider it as a result that the system linker (which doesn't understand the LLVM bitcode file format) is wrongly used against bitcode files. However, if at least one member object file is in the native ELF format, the archive file will have some symbol in its symbol table, so the hack won't kick in.</div><div><br></div><div>I think one approach to fix the issue is to not trust the archive file symbol table for bitcode files at all. Instead, we can read directly from a symbol table of each archive member bitcode file. That shouldn't be technically difficult. I'm a bit worried about the performance penalty of doing that, though, because in order to read bitcode file symbol tables, we have to identify which file is bitcode file and which file is native ELF file. That means we have to read a file magic from all archive members. That might be noticeably slow, in particular, if thin archives are in use, but that's highly dependent on the filesystem where the input files are laid out.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><br></div><div>Thanks,</div><div>Teresa</div><br><div class="gmail_quote"></div><div dir="ltr" class="gmail_attr">On Wed, Apr 8, 2020 at 10:25 AM Shishir V Jessu via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">To correct a typo: I am using both clang 6.0.0, and a local build of clang 10.0.0, and each result in the same error.<br><br>Best, <br>Shishir Jessu</div><img style="width: 2px; max-height: 0px; overflow: hidden;"><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Apr 8, 2020 at 12:22 PM Shishir V Jessu <<a href="mailto:shishir.jessu@utexas.edu" target="_blank">shishir.jessu@utexas.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi, <br><br>I have tried to build <a href="https://github.com/libjpeg-turbo/libjpeg-turbo" target="_blank">libjpeg-turbo</a> with LTO in LLVM, using both clangbut get many errors in lld that look like the following: <br><br>ld: error: undefined symbol: jpeg_std_error<br>>>> referenced by jcstest.c:76<br>>>> lto.tmp:(main)<br><br>ld: error: undefined symbol: jpeg_CreateCompress<br>>>> referenced by jcstest.c:86<br>>>> lto.tmp:(main)<br><br>ld: error: undefined symbol: jpeg_set_defaults<br>>>> referenced by jcstest.c:88<br>>>> lto.tmp:(main)<br><br>ld: error: undefined symbol: jpeg_default_colorspace<br>>>> referenced by jcstest.c:90<br>>>> lto.tmp:(main)<br>>>> referenced by jcstest.c:114<br>>>> lto.tmp:(main)<br><br>This only occurs when compiling with the -flto flag. Has anyone been able to build libjpeg-turbo with LTO? Are there any modifications I need to make to the makefile or other configuration in order to do so? Thanks for your help!<br><br>Best, <br>Shishir Jessu</div>
</blockquote></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote><br clear="all"></div><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><span style="font-family:Times;font-size:medium"><table cellspacing="0" cellpadding="0"><tbody><tr style="color:rgb(85,85,85);font-family:sans-serif;font-size:small"><td nowrap style="border-top:2px solid rgb(213,15,37)">Teresa Johnson |</td><td nowrap style="border-top:2px solid rgb(51,105,232)"> Software Engineer |</td><td nowrap style="border-top:2px solid rgb(0,153,57)"> <a href="mailto:tejohnson@google.com" target="_blank">tejohnson@google.com</a> |</td><td nowrap style="border-top:2px solid rgb(238,178,17)"><br></td></tr></tbody></table></span></div></div></div>
</blockquote></div></div>