<div dir="ltr">yeah, something that people toss around from time to time - certainly if it's useful enough to you to motivate the work/time/effort, great!<br><br>Seconding your comments & Sean's: Implement features as needed (in a binutils compatible interface/behavior - but doesn't have to have all the features. Could start out with clear errors for all the features "this isn't supported" & fill out features as they're needed by folks with the motivation to implement them).<br><br>The LLVM use of objcopy for DWARF Fission's probably a marginal one - it'd be nice to remove that dependency and produce the separate files directly, at least when using the integrated assembler, but that's a fair bit more work. Having an LLVM objcopy would provide the opportunity to address a couple of the issues that exist already:<br>1) Windows support (I'm not sure what that really looks like - whether it would actually work with COFF files, etc, but I remember David Majnemer looking at this at one point)<br>2) Object file size regression (the LLVM object size optimization of having two of the object file string tables (strtab and shstrtab) in one section instead of two saves a bunch of file size - but binutils objcopy doesn't know that trick, so turning on fission undoes that improvement)<br><br><div class="gmail_quote"><div dir="ltr">On Thu, Jun 1, 2017 at 5:21 PM Jake Ehrlich via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>LLVM already implements its own version of almost all of binutils. The</div><div>exceptions to this rule are objcopy and strip. This is a proposal to implement</div><div>an llvm version of objcopy/strip to complete llvm’s binutils.</div><div><br></div><div>Several projects only use gnu binutils because of objcopy/strip. LLVM itself</div><div>uses objcopy in fact. Chromium and Fuchsia currently use objcopy as well. If you</div><div>want to distribute your build tools this is a problem due to licensing. It’s</div><div>also a bit of a blemish on LLVM because LLVM could be made more self sufficient</div><div>if there was an llvm version of objcopy. Additionally Chromium is one of the</div><div>popular benchmarks for LLVM so it would be nice if Chromium didn’t have to use</div><div>binutils. Using</div><div>[elftoolchain](<a href="https://sourceforge.net/p/elftoolchain/wiki/Home/" target="_blank">https://sourceforge.net/p/elftoolchain/wiki/Home/</a>)</div><div>solves the licensing issue for Fuchsia but is elf specific and only solves the</div><div>issue for Fuchsia. I propose implementing llvm-objcopy to be a minimum viable</div><div>replacement for objcopy.</div><div><br></div><div>I’ve gone though the sources of LLVM, Clang, Chromium, and Fuchsia to try and</div><div>find the major use cases of objcopy. Here is a list of use cases I have found</div><div>and which projects use them. This list includes some use cases not found in</div><div>these 4 projects.</div><div><br></div><div>1. Use Case: Stripping debug information of an executable to a file </div><div> Who uses it: LLVM, Fuchsia, Chromium</div><div><br></div><div> ```sh</div><div> objcopy --only-keep-debug foo foo.debug</div><div> objcopy --strip-debug foo foo</div><div> ```</div><div><br></div><div> [Example use](<a href="https://github.com/llvm-mirror/llvm/blob/cd789d8cfe12aa374e66eafc748f4fc06e149ca7/cmake/modules/AddLLVM.cmake" target="_blank">https://github.com/llvm-mirror/llvm/blob/cd789d8cfe12aa374e66eafc748f4fc06e149ca7/cmake/modules/AddLLVM.cmake</a>)</div><div> When it is useful: </div><div> This reduces the size of the file for distribution while maintaining the debug</div><div> information in a file for later use. Anyone distributing an executable in</div><div> anyway could benefit from this.</div><div><br></div><div>2. Use Case: Stripping debug information of a relocatable object to a file </div><div> Who uses it: None of the 4 projects considered</div><div><br></div><div> ```sh</div><div> objcopy --only-keep-debug foo.o foo.debug</div><div> objcopy --strip-debug foo.o foo.o</div><div> ```</div><div><br></div><div> When it is useful: </div><div> In distribution of an SDK in the form of an archive it would be nice to strip</div><div> this information. This allows debug information to be distributed separately.</div><div><br></div><div>3. Use Case: Stripping debug information of a shared library to a file </div><div> Who uses it: None of the 4 projects</div><div><br></div><div> ```sh</div><div> objcopy --only-keep-debug foo.so foo.debug</div><div> objcopy --strip-debug foo.so foo.so</div><div> ```</div><div><br></div><div> When is it Useful: </div><div> Same benefits as the previous case. If you want to distribute a library this</div><div> option allows you to distribute a smaller binary while maintaining the ability</div><div> to debug.</div><div><br></div><div>4. Use Case:<span class="m_3062673432079540799Apple-tab-span" style="white-space:pre-wrap"> </span>Stripping an executable</div><div> Who uses it:<span class="m_3062673432079540799Apple-tab-span" style="white-space:pre-wrap"> </span>None of the 4 projects</div><div><br></div><div> ```sh</div><div> objcopy --strip-all foo foo</div><div> ```</div><div><br></div><div> When is it useful: </div><div> Anytime an executable is being distributed and there is no reason to keep</div><div> debugging information. This makes the executable smaller than simply</div><div> stripping debug info and doesn't produce an extra file.</div><div><br></div><div>5. Use Case: “Complete stripping” an executable </div><div> Who uses it: None of the 4 projects</div><div> ```sh</div><div> eu-strip --strip-sections foo</div><div> ```</div><div> When is it useful: </div><div> This is an extreme form of stripping that even strips the section headers</div><div> since they are not needed for loading. This is useful in the same contexts as</div><div> stripping but some tools and dynamic linkers may be confused by it. This is</div><div> possibly only valid on ELF unlike general stripping which is a valid option on</div><div> multiple platforms.</div><div><br></div><div>6. Use Case: DWARF fission </div><div> Who uses it: Clang, Fuchsia, Chromium</div><div><br></div><div> ```sh</div><div> objcopy --extract-dwo foo foo.debug</div><div> objcopy --strip-dwo foo foo</div><div> ```</div><div><br></div><div> [Example use 1](<a href="https://github.com/llvm-mirror/clang/blob/3efd04e48004628cfaffead00ecb1c206b0b6cb2/lib/Driver/ToolChains/CommonArgs.cpp" target="_blank">https://github.com/llvm-mirror/clang/blob/3efd04e48004628cfaffead00ecb1c206b0b6cb2/lib/Driver/ToolChains/CommonArgs.cpp</a>)</div><div> [Example use 2](<a href="https://github.com/llvm-mirror/clang/blob/a0badfbffbee71c2c757d580fc852d2124dadc5a/test/Driver/split-debug.s" target="_blank">https://github.com/llvm-mirror/clang/blob/a0badfbffbee71c2c757d580fc852d2124dadc5a/test/Driver/split-debug.s</a>)</div><div><br></div><div> When is it useful: </div><div> DWARF fission can be used to speed up large builds. In some cases builds can</div><div> be too large to be handled and DWARF fission makes this manageable. DWARF</div><div> fission is useful in almost any project of sufficient size.</div><div><br></div><div>7. Use Case: Converting an executable to binary </div><div> Who uses it: Fuchsia</div><div><br></div><div> ```sh</div><div> objcopy -O binary magenta.elf magenta.bin</div><div> ```</div><div><br></div><div> [Example use](<a href="https://fuchsia.googlesource.com/magenta/+/master/make/build.mk#20" target="_blank">https://fuchsia.googlesource.com/magenta/+/master/make/build.mk#20</a>)</div><div><br></div><div> When is it useful: </div><div> For kernels and embedded applications that need just the raw segments.</div><div><br></div><div>8. Use Case: Adding a gdb index </div><div> Who uses it: Chromium</div><div><br></div><div> ```sh</div><div> gdb -batch foo -ex "save gdb-index dir" -ex quit</div><div> objcopy --add-section .gdb_index="dir/foo.gdb-index" \</div><div> --set-section-flags .gdb_index=readonly foo foo</div><div> ```</div><div><br></div><div> [Example use](<a href="https://cs.chromium.org/chromium/src/build/gdb-add-index?type=cs&q=objcopy&l=71" target="_blank">https://cs.chromium.org/chromium/src/build/gdb-add-index?type=cs&q=objcopy&l=71</a>)</div><div><br></div><div> When is it useful: </div><div> Adding a gdb index reduces startup time for debugging an application. Any</div><div> sufficiently large program with a sufficiently large amount of debug</div><div> information can potentially benefit from this.</div><div><br></div><div>9. Use Case: Converting between formats </div><div> Who uses it: Fuchsia (only in Magenta GCC build)</div><div><br></div><div> ```sh</div><div> objcopy --target=pei-x86-64 magenta.elf <a href="http://megenta.pe" target="_blank">megenta.pe</a></div><div> ```</div><div><br></div><div> [Example use](<a href="https://fuchsia.googlesource.com/magenta/+/master/bootloader/build.mk#97" target="_blank">https://fuchsia.googlesource.com/magenta/+/master/bootloader/build.mk#97</a>)</div><div><br></div><div> When is it useful: </div><div> This is primarily useful when you can’t directly target a needed format.</div><div><br></div><div>10. Use Case: Removing symbols not needed for relocation </div><div> Who uses it: Chromium</div><div><br></div><div> ```sh</div><div> objcopy --strip-unneeded foo foo</div><div> ```</div><div><br></div><div> [Example use](<a href="https://cs.chromium.org/chromium/src/third_party/libevdev/src/common.mk?type=cs&q=objcopy&l=397" target="_blank">https://cs.chromium.org/chromium/src/third_party/libevdev/src/common.mk?type=cs&q=objcopy&l=397</a>)</div><div><br></div><div> When is it useful: </div><div> This is useful when shipping an SDK or some relocatable binaries.</div><div><br></div><div>11. Use Case: Removing local symbols </div><div> Who uses it: LLVM</div><div><br></div><div> ```sh</div><div> objcopy --discard-all foo foo</div><div> ```</div><div><br></div><div> [Example use](<a href="https://github.com/llvm-mirror/llvm/blob/cd789d8cfe12aa374e66eafc748f4fc06e149ca7/cmake/modules/AddLLVM.cmake" target="_blank">https://github.com/llvm-mirror/llvm/blob/cd789d8cfe12aa374e66eafc748f4fc06e149ca7/cmake/modules/AddLLVM.cmake</a>)</div><div> (hidden in definition of “strip_command” using strip instead of objcopy and</div><div> using -x instead of --discard-all)</div><div><br></div><div> When is it useful: </div><div> Anytime you don’t need locals for debugging this can be useful.</div><div><br></div><div>12. Use Case: Removing a specific unwanted section </div><div> Who uses it: LLVM</div><div><br></div><div> ```sh</div><div> objcopy --remove-section=.debug_aranges foo foo</div><div> ```</div><div><br></div><div> [Example use](<a href="https://github.com/llvm-mirror/llvm/blob/93e6e5414ded14bcbb233baaaa5567132fee9a0c/test/DebugInfo/Inputs/fission-ranges.cc" target="_blank">https://github.com/llvm-mirror/llvm/blob/93e6e5414ded14bcbb233baaaa5567132fee9a0c/test/DebugInfo/Inputs/fission-ranges.cc</a>)</div><div><br></div><div> When is it useful:</div><div> This is useful when you know that you have an unwanted section that isn’t</div><div> removed by one of the other stripping options. This can also be used to</div><div> remove an existing section for replacement by a new section.</div><div><br></div><div>We would like to build this up incrementally by solving specific use cases</div><div>as they come up. To start with we would like to tackle the use cases</div><div>important to us. We primarily care about fully linked executables and not</div><div>relocatable files. I plan to implement conversion from ELF to binary first.</div><div>After that I plan on implementing stripping for ELF executables.</div><div><br></div></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div></div>