[LLVMbugs] [Bug 10240] New: Shell scripts generated by llvm-ld load redundant libraries

[bugzilla-daemon at llvm.org]

http://llvm.org/bugs/show_bug.cgi?id=10240

           Summary: Shell scripts generated by llvm-ld load redundant
                    libraries
           Product: new-bugs
           Version: 2.9
          Platform: PC
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P
         Component: new bugs
        AssignedTo: unassignedbugs at nondot.org
        ReportedBy: srk31 at srcf.ucam.org
                CC: llvmbugs at cs.uiuc.edu


When linking in bitcode libraries using -l, llvm-ld still emits "-load="
arguments to lli to load native versions. These are redundant and should not be
linked.

To reproduce: take any shared library installed on your system, them recompile
it as bitcode. Similarly, compile and link a client of that library using llvm,
making sure the bitcode library is found first in the library search path (-L).
In the output shell script, the native library will be loaded even though the
linker linked in the bitcode version. Here's a tarball that demonstrates this
with the command below, assuming you have native libssl and libz on your
system.
<http://www.cs.ox.ac.uk/people/stephen.kell/private/llvm-bug-20110701.tar.gz>
I'm doing this on a x86-64 FC13 system, with the 2.9 release of LLVM, compiled
Release+Debug+Asserts. 

$ llvm-ld -v -lpthread -Lout/host/linux-x86/pr/sim/obj/lib/llvm
-Lout/host/linux-x86/pr/sim/obj/lib/llvm -L\$ORIGIN/../lib/llvm 
-Lout/host/linux-x86/pr/sim/obj/lib/llvm
out/host/linux-x86/pr/sim/obj/EXECUTABLES/dalvikvm_intermediates/Main.o.bc  
-l=dvm -l=ssl -l=z -o
out/host/linux-x86/pr/sim/obj/EXECUTABLES/dalvikvm_intermediates/LINKED/dalvikvm.bc 
Adding library pthread to link items
Adding library dvm to link items
Adding library ssl to link items
Adding library z to link items
  Linking archive file '/usr/lib64/libpthread.a'
  Linking bitcode file
'out/host/linux-x86/pr/sim/obj/EXECUTABLES/dalvikvm_intermediates/Main.o.bc'
  Linked in file
'out/host/linux-x86/pr/sim/obj/EXECUTABLES/dalvikvm_intermediates/Main.o.bc'
  Linking bitcode file 'out/host/linux-x86/pr/sim/obj/lib/llvm/libdvm.so'
  Linked in file 'out/host/linux-x86/pr/sim/obj/lib/llvm/libdvm.so'
  Linking bitcode file 'out/host/linux-x86/pr/sim/obj/lib/llvm/libssl.so'
  Linked in file 'out/host/linux-x86/pr/sim/obj/lib/llvm/libssl.so'
  Linking bitcode file 'out/host/linux-x86/pr/sim/obj/lib/llvm/libz.so'
  Linked in file 'out/host/linux-x86/pr/sim/obj/lib/llvm/libz.so'
  Linking archive file '/usr/lib64/libpthread.a'
Generating Bitcode To
out/host/linux-x86/pr/sim/obj/EXECUTABLES/dalvikvm_intermediates/LINKED/dalvikvm.bc.bc
Emitting Shell Script

$ cat
out/host/linux-x86/pr/sim/obj/EXECUTABLES/dalvikvm_intermediates/LINKED/dalvikvm.bc
#!/bin/sh
lli=${LLVMINTERP-lli}
exec $lli \
    -load=/usr/lib64/libpthread.a \
    -load=/usr/lib/libssl.so \
    -load=/usr/lib/libz.so \
   
out/host/linux-x86/pr/sim/obj/EXECUTABLES/dalvikvm_intermediates/LINKED/dalvikvm.bc.bc
${1+"$@"}

The second and third -load= lines should not be there. (Also, neither should
the first, but that's a separate bug.)

This seems to be because the shell script generation code assumes that if an -l
request was satisfied by bitcode, then searching for native dynamic
libraries will yield nothing, so we can do it anyway. But sometimes both
bitcode and native versions can be found in the library path.

Instead, if a bitcode library was linked in, it should skip the native
equivalent. Here is a quick patch which does mostly the right thing.

--- tools/llvm-ld/llvm-ld.cpp
+++ tools/llvm-ld/llvm-ld.cpp.patched-1 2011-07-01 13:03:37.000000000 +0100
@@ -449,6 +449,8 @@
   // on the command line, so that we don't have to do this manually!
   for (std::vector<std::string>::iterator i = Libraries.begin(),
          e = Libraries.end(); i != e; ++i) {
+    // we might find a bitcode library; if so, it's already linked in
+    bool FoundBitcodeLibrary = false;
     // try explicit -L arguments first:
     sys::Path FullLibraryPath;
     for (cl::list<std::string>::const_iterator P = LibPaths.begin(),
@@ -457,16 +459,25 @@
       FullLibraryPath.appendComponent("lib" + *i);
       FullLibraryPath.appendSuffix(sys::Path::GetDLLSuffix());
       if (!FullLibraryPath.isEmpty()) {
-        if (!FullLibraryPath.isDynamicLibrary()) {
+        if (FullLibraryPath.isBitcodeFile()) {
+          FoundBitcodeLibrary = true;
+          break;
+        }
+        else if (!FullLibraryPath.isDynamicLibrary()) {
           // Not a native shared library; mark as invalid
           FullLibraryPath = sys::Path();
-        } else break;
+        } 
+        else break;
       }
     }
-    if (FullLibraryPath.isEmpty())
-      FullLibraryPath = sys::Path::FindLibrary(*i);
-    if (!FullLibraryPath.isEmpty())
-      Out2.os() << "    -load=" << FullLibraryPath.str() << " \\\n";
+    // only emit -load= lines for native *shared* libraries
+    if (!FoundBitcodeLibrary)
+    {
+      if (FullLibraryPath.isEmpty())
+        FullLibraryPath = sys::Path::FindLibrary(*i);
+      if (!FullLibraryPath.isEmpty())
+        Out2.os() << "    -load=" << FullLibraryPath.str() << " \\\n";
+    }
   }
   Out2.os() << "    "  << BitcodeOutputFilename << " ${1+\"$@\"}\n";
   Out2.keep();


Here we're assuming that our walk of the library paths precisely
replicates the Linker's, so that iff we found bitcode, so did the linker.
A more robust solution would check with the linker what files were
actually linked in, and then remove these from the list of Libraries to
handle in the shell script.

I'm also about to file a related bug triggered in the case where a call
to sys::Path::FindLibrary discovers an archive and not a shared object (i.e.
the first -load= in the shell script I showed).

-- 
Configure bugmail: http://llvm.org/bugs/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.