<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><div class=""><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""><br class=""></span></div><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">I tried to work on the Mach-O layer on LLD long-time ago (I got Dtrace User Probe working but as it was just before the LLD redesign and dev reboot, so never try to push the changes), and one thing I fond difficult was the complete lack of technical documentation on ld64 passes and flags.</span><div class=""><br class=""></div><div class="">I think that restarting a Mach-O parser from scratch (with ld64 feature parity as a goal) is a very good opportunity to also write a technical reference of the difference passes, especially the one that are Mach-O and Apple specific.</div><div class="">Do you plan something like that (as a page on <a href="http://lld.llvm.org" class="">lld.llvm.org</a> for example). It would also be very helpful for potential contributors.</div><div class=""><br class=""></div><div class=""><br class=""><div style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);"><br class=""><blockquote type="cite" class=""><div class="">Le 5 mars 2020 à 04:19, Shoaib Meenai via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> a écrit :</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1;"><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">We’ve discussed this with Jim Grosbach at Apple (CC’d). My understanding is that at this time, the officially supported linker for Apple’s platforms is ld64. We’d love to collaborate with Apple (and any other interested parties, for that matter) on LLD for Mach-O, and we’d be delighted if it were to become officially supported at some point, but there’s a lot of work to be done first on reaching feature parity with ld64 before that could even be considered :) Once we reach feature parity, I can envision several good reasons both for sticking with ld64 and for switching to LLD. On our end, we aim to create a feature-complete Mach-O linker. We also aim for the end product to be compelling enough that a switch could be considered, and we’d be happy to work with Apple on that front if it turns out to be.<o:p class=""></o:p></div><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">What sort of compatibility test suite did you have in mind? We’re adding lit-style unit tests as we add features (as is standard for LLVM), but we didn’t have anything in mind beyond that right now.<o:p class=""></o:p></div><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">We saw zld, and the speedups achieved by it make us hopeful of being able to achieve similar results with LLD (since we’ll also have better parallelization and better data structures in the form of the LLVM ones). (I’d also be curious if zld’s improvements could be contributed back to ld64 so that everyone can take advantage of them, but that’s completely tangential, of course.)<o:p class=""></o:p></div><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div><div class="" style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0in 0in;"><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><b class=""><span class="" style="font-size: 12pt;">From: </span></b><span class="" style="font-size: 12pt;">Dave Lee <<a href="mailto:davelee.com@gmail.com" class="" style="color: rgb(37, 37, 255);">davelee.com@gmail.com</a>><br class=""><b class="">Date: </b>Tuesday, March 3, 2020 at 7:37 AM<br class=""><b class="">To: </b>Shoaib Meenai <<a href="mailto:smeenai@fb.com" class="" style="color: rgb(37, 37, 255);">smeenai@fb.com</a>><br class=""><b class="">Cc: </b>"<a href="mailto:llvm-dev@lists.llvm.org" class="" style="color: rgb(37, 37, 255);">llvm-dev@lists.llvm.org</a>" <<a href="mailto:llvm-dev@lists.llvm.org" class="" style="color: rgb(37, 37, 255);">llvm-dev@lists.llvm.org</a>>, Jez Ng <<a href="mailto:jezng@fb.com" class="" style="color: rgb(37, 37, 255);">jezng@fb.com</a>><br class=""><b class="">Subject: </b>Re: [llvm-dev] Contributing LLD for Mach-O<o:p class=""></o:p></span></div></div><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div></div><div class=""><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Hi Shoaib,<o:p class=""></o:p></div></div><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div></div><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Has there been any recent discussion with anyone on Apple's side? Is there a way forward that results in a single unified open source linker? If not at the start, how would it work if they later want to take maintainership?<o:p class=""></o:p></div></div><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div></div><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Have you thought about a compatibility test suite? If so, I'm curious what the approach will be.<o:p class=""></o:p></div></div><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div></div><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Also for what it's worth, there's a recent fork of ld64's recent source drop, which is optimized for incremental compilation. It also uses non-STL data structures, parallelism, and a disk cache. See <a href="https://github.com/michaeleisel/zld" class="" style="color: rgb(37, 37, 255);">https://github.com/michaeleisel/zld</a><o:p class=""></o:p></div></div></div><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;"><o:p class=""> </o:p></div><div class=""><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">On Fri, Feb 28, 2020 at 1:32 PM Shoaib Meenai via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="" style="color: rgb(37, 37, 255);">llvm-dev@lists.llvm.org</a>> wrote:<o:p class=""></o:p></div></div><blockquote class="" style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;"><div class=""><div class=""><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">Hi all,<br class=""><br class="">We’re planning to contribute a new implementation of LLD for Mach-O, using the same design as the COFF and ELF ports. This design has proven to work very well for those ports, and we’re keen to explore it for Mach-O as well. Our work is based on an initial prototype created by Peter Collingbourne and Rui Ueyama.<br class=""><br class="">Our initial commit is up for review at <a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D75382&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=Du_g8hPBNR5iofm4w6TR-gFx_aFK-4RNxpEmlx-FMNc&s=z1H0UW2F3uSaikSViTE-eRlQmyNhhrkHNq0aj2VxbfE&e=" target="_blank" class="" style="color: rgb(37, 37, 255);">https://reviews.llvm.org/D75382</a>. We’ve intentionally stripped down this initial commit as much as possible to ease reviewing; we’ve kept it to the absolute minimum needed to produce and test a working macOS x86-64 executable for that prints “Hello World” via a syscall. We have several short-term follow-ups planned to add important functionality, such as linking against archives, universal binaries, dylibs, and tbd files, performing subsection splitting (atomization), and producing dylibs. The follow-ups should give a good sense of the overall design while still keeping each piece easily reviewable and testable individually. Our end goal is to create a full-featured Mach-O linker, and we’ll be working toward that goal over the next several months (and years, in all likelihood). We’d appreciate feedback and reviews.<o:p class=""></o:p></div></div></div><div class="" style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="" style="color: rgb(37, 37, 255);">llvm-dev@lists.llvm.org</a><br class=""><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=Du_g8hPBNR5iofm4w6TR-gFx_aFK-4RNxpEmlx-FMNc&s=WEM8UzKRSd0-QQZOXRcgL_Rkt5cgC1soudjdbkMnbgA&e=" target="_blank" class="" style="color: rgb(37, 37, 255);">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p class=""></o:p></div></blockquote></div></div><span class="" style="float: none; display: inline !important;">_______________________________________________</span><br class=""><span class="" style="float: none; display: inline !important;">LLVM Developers mailing list</span><br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="" style="color: rgb(37, 37, 255);">llvm-dev@lists.llvm.org</a><br class=""><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" class="" style="color: rgb(37, 37, 255);">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote><div class=""><br class=""></div></div><div><br class=""><blockquote type="cite" class=""><div class="">Le 5 mars 2020 à 04:19, Shoaib Meenai via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> a écrit :</div><br class="Apple-interchange-newline"><div class=""><div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;"><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">We’ve discussed this with Jim Grosbach at Apple (CC’d). My understanding is that at this time, the officially supported linker for Apple’s platforms is ld64. We’d love to collaborate with Apple (and any other interested parties, for that matter) on LLD for Mach-O, and we’d be delighted if it were to become officially supported at some point, but there’s a lot of work to be done first on reaching feature parity with ld64 before that could even be considered :) Once we reach feature parity, I can envision several good reasons both for sticking with ld64 and for switching to LLD. On our end, we aim to create a feature-complete Mach-O linker. We also aim for the end product to be compelling enough that a switch could be considered, and we’d be happy to work with Apple on that front if it turns out to be.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">What sort of compatibility test suite did you have in mind? We’re adding lit-style unit tests as we add features (as is standard for LLVM), but we didn’t have anything in mind beyond that right now.<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">We saw zld, and the speedups achieved by it make us hopeful of being able to achieve similar results with LLD (since we’ll also have better parallelization and better data structures in the form of the LLVM ones). (I’d also be curious if zld’s improvements could be contributed back to ld64 so that everyone can take advantage of them, but that’s completely tangential, of course.)<o:p class=""></o:p></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div style="border-style: solid none none; border-top-width: 1pt; border-top-color: rgb(181, 196, 223); padding: 3pt 0in 0in;" class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><b class=""><span style="font-size: 12pt;" class="">From:<span class="Apple-converted-space"> </span></span></b><span style="font-size: 12pt;" class="">Dave Lee <<a href="mailto:davelee.com@gmail.com" style="color: blue; text-decoration: underline;" class="">davelee.com@gmail.com</a>><br class=""><b class="">Date:<span class="Apple-converted-space"> </span></b>Tuesday, March 3, 2020 at 7:37 AM<br class=""><b class="">To:<span class="Apple-converted-space"> </span></b>Shoaib Meenai <<a href="mailto:smeenai@fb.com" style="color: blue; text-decoration: underline;" class="">smeenai@fb.com</a>><br class=""><b class="">Cc:<span class="Apple-converted-space"> </span></b>"<a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a>" <<a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a>>, Jez Ng <<a href="mailto:jezng@fb.com" style="color: blue; text-decoration: underline;" class="">jezng@fb.com</a>><br class=""><b class="">Subject:<span class="Apple-converted-space"> </span></b>Re: [llvm-dev] Contributing LLD for Mach-O<o:p class=""></o:p></span></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hi Shoaib,<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Has there been any recent discussion with anyone on Apple's side? Is there a way forward that results in a single unified open source linker? If not at the start, how would it work if they later want to take maintainership?<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Have you thought about a compatibility test suite? If so, I'm curious what the approach will be.<o:p class=""></o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div></div><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Also for what it's worth, there's a recent fork of ld64's recent source drop, which is optimized for incremental compilation. It also uses non-STL data structures, parallelism, and a disk cache. See<span class="Apple-converted-space"> </span><a href="https://github.com/michaeleisel/zld" style="color: blue; text-decoration: underline;" class="">https://github.com/michaeleisel/zld</a><o:p class=""></o:p></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class=""><o:p class=""> </o:p></div><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">On Fri, Feb 28, 2020 at 1:32 PM Shoaib Meenai via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a>> wrote:<o:p class=""></o:p></div></div><blockquote style="border-style: none none none solid; border-left-width: 1pt; border-left-color: rgb(204, 204, 204); padding: 0in 0in 0in 6pt; margin-left: 4.8pt; margin-right: 0in;" class=""><div class=""><div class=""><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">Hi all,<br class=""><br class="">We’re planning to contribute a new implementation of LLD for Mach-O, using the same design as the COFF and ELF ports. This design has proven to work very well for those ports, and we’re keen to explore it for Mach-O as well. Our work is based on an initial prototype created by Peter Collingbourne and Rui Ueyama.<br class=""><br class="">Our initial commit is up for review at<span class="Apple-converted-space"> </span><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__reviews.llvm.org_D75382&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=Du_g8hPBNR5iofm4w6TR-gFx_aFK-4RNxpEmlx-FMNc&s=z1H0UW2F3uSaikSViTE-eRlQmyNhhrkHNq0aj2VxbfE&e=" target="_blank" style="color: blue; text-decoration: underline;" class="">https://reviews.llvm.org/D75382</a>. We’ve intentionally stripped down this initial commit as much as possible to ease reviewing; we’ve kept it to the absolute minimum needed to produce and test a working macOS x86-64 executable for that prints “Hello World” via a syscall. We have several short-term follow-ups planned to add important functionality, such as linking against archives, universal binaries, dylibs, and tbd files, performing subsection splitting (atomization), and producing dylibs. The follow-ups should give a good sense of the overall design while still keeping each piece easily reviewable and testable individually. Our end goal is to create a full-featured Mach-O linker, and we’ll be working toward that goal over the next several months (and years, in all likelihood). We’d appreciate feedback and reviews.<o:p class=""></o:p></div></div></div><div style="margin: 0in 0in 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" style="color: blue; text-decoration: underline;" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.llvm.org_cgi-2Dbin_mailman_listinfo_llvm-2Ddev&d=DwMFaQ&c=5VD0RTtNlTh3ycd41b3MUw&r=o3kDXzdBUE3ljQXKeTWOMw&m=Du_g8hPBNR5iofm4w6TR-gFx_aFK-4RNxpEmlx-FMNc&s=WEM8UzKRSd0-QQZOXRcgL_Rkt5cgC1soudjdbkMnbgA&e=" target="_blank" style="color: blue; text-decoration: underline;" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p class=""></o:p></div></blockquote></div></div><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">LLVM Developers mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="color: blue; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">llvm-dev@lists.llvm.org</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="color: blue; text-decoration: underline; font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></div></body></html>