<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class=""><br class=""></div><br class=""><div><blockquote type="cite" class=""><div class="">On Jul 17, 2017, at 2:06 PM, Daniel Berlin <<a href="mailto:dannyb@google.com" class="">dannyb@google.com</a>> wrote:</div><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">This process works legally, but i can't speak to whether the foundation would be okay with it, as it may result in bad press, etc, if you rip code out.</div></div></div></div></div></blockquote><div><br class=""></div><div>Thank you for your reply!</div><div>I have two additional questions:</div><div><br class=""></div><div>1) Why is it ripping the code out if compiler-rt is still part of LLVM?</div><div>2) Does it mean that we should reach out to <a href="mailto:board@llvm.org" class="">board@llvm.org</a> to okay the issue?</div><div><br class=""></div><div>Thanks,</div><div>George</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""> <br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, Jul 12, 2017 at 12:41 PM, George Karpenkov <span dir="ltr" class=""><<a href="mailto:ekarpenkov@apple.com" target="_blank" class="">ekarpenkov@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><div class=""><div class="m_5191769219634074052h5"><blockquote type="cite" class=""><div class="">On Jul 12, 2017, at 11:58 AM, Kostya Serebryany <<a href="mailto:kcc@google.com" target="_blank" class="">kcc@google.com</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, Jul 12, 2017 at 11:54 AM, George Karpenkov <span dir="ltr" class=""><<a href="mailto:ekarpenkov@apple.com" target="_blank" class="">ekarpenkov@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On Jul 12, 2017, at 11:34 AM, Kostya Serebryany <<a href="mailto:kcc@google.com" target="_blank" class="">kcc@google.com</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059Apple-interchange-newline"><div class=""><div dir="ltr" class=""><br class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Wed, Jul 12, 2017 at 11:30 AM, George Karpenkov <span dir="ltr" class=""><<a href="mailto:ekarpenkov@apple.com" target="_blank" class="">ekarpenkov@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On Jul 12, 2017, at 11:01 AM, Kostya Serebryany <<a href="mailto:kcc@google.com" target="_blank" class="">kcc@google.com</a>> wrote:</div><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="">One question: will it make sense to *copy* the code to the new location, work on it, then delete the code from the old location, </div><div class="">instead of doing a move in a single commit? </div><div class="">I don't expect any dramatic changes in the code structure during a few weeks, so 'copy' might be much simpler than 'move’. <br class=""></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">I am not sure how we would handle CMake targets collision (I think we would have one: check-fuzzer vs. check-fuzzer)</div><div class="">Of course, I can give it a different name for now, </div></div></div></blockquote><div class=""><br class=""></div><div class="">Yep. 'check-fuzzer-temporary' or some such. </div><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="">but that would potentially add even more confusion and complexity.</div><div class=""><br class=""></div><div class="">Though it seems like a good idea to at least see what would happen to buildbots.</div><span class=""><br class=""><blockquote type="cite" class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="">I already forgot why we decided not to move the code to compiler-rt. </div></div></blockquote></span></div><div class=""><span class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="">This would solve at least this problem. </div><div class="">Since we now have -fsanitize=fuzzer it will actually be pretty natural. </div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Licensing concerns, compiler-rt has a different license.</div></div></div></blockquote><div class=""><br class=""></div><div class="">@%##%)*%</div><div class=""><br class=""></div><div class="">But wait a sec, the sanitizers are ok with this license, why libFuzzer isn't? </div><div class="">(Sorry, my memory has been flushed over the last month)</div></div></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Apparently because the code was already contributed under a different license.</div><div class="">IANAL but apparently to do the change in a fully correct way one would need to chase</div><div class="">down every single contributor and ask them to agree to a license change.</div></div></div></blockquote><div class=""><br class=""></div><div class="">This is a PITA, but we should do it. </div><div class="">licensing issues like this should not prevent us from making our own code better. </div></div></div></div></div></blockquote><div class=""><br class=""></div><div class=""><br class=""></div></div></div><div class=""><div class="">Should not be that hard:</div><div class=""><br class=""></div><div class=""><div class="">george@/Volumes/Transcend/code<wbr class="">/llvm (master)≻ git log --pretty=format:"%ae" lib/Fuzzer | egrep -v "chromium|apple|google" | sort | uniq</div><div class=""><a href="mailto:aaron@aaronballman.com" target="_blank" class="">aaron@aaronballman.com</a></div><div class=""><a href="mailto:ahmed.bougacha@gmail.com" target="_blank" class="">ahmed.bougacha@gmail.com</a></div><div class=""><a href="mailto:chandlerc@gmail.com" target="_blank" class="">chandlerc@gmail.com</a></div><div class=""><a href="mailto:craig.topper@gmail.com" target="_blank" class="">craig.topper@gmail.com</a></div><div class=""><a href="mailto:dan@su-root.co.uk" target="_blank" class="">dan@su-root.co.uk</a></div><div class=""><a href="mailto:eric@efcs.ca" target="_blank" class="">eric@efcs.ca</a></div><div class=""><a href="mailto:hans@hanshq.net" target="_blank" class="">hans@hanshq.net</a></div><div class=""><a href="mailto:juergen@ributzka.de" target="_blank" class="">juergen@ributzka.de</a></div><div class=""><a href="mailto:lenny@colorado.edu" target="_blank" class="">lenny@colorado.edu</a></div><div class=""><a href="mailto:mail@justinbogner.com" target="_blank" class="">mail@justinbogner.com</a></div><div class=""><a href="mailto:matze@braunis.de" target="_blank" class="">matze@braunis.de</a></div><div class=""><a href="mailto:mgrang@codeaurora.org" target="_blank" class="">mgrang@codeaurora.org</a></div><div class=""><a href="mailto:nicholas@mxc.ca" target="_blank" class="">nicholas@mxc.ca</a></div><div class=""><a href="mailto:peter@pcc.me.uk" target="_blank" class="">peter@pcc.me.uk</a></div><div class=""><a href="mailto:rafael.espindola@gmail.com" target="_blank" class="">rafael.espindola@gmail.com</a></div><div class=""><a href="mailto:richard-llvm@metafoo.co.uk" target="_blank" class="">richard-llvm@metafoo.co.uk</a></div><div class=""><a href="mailto:sanjoy@playingwithpointers.com" target="_blank" class="">sanjoy@playingwithpointers.com</a></div><div class=""><a href="mailto:tzuhsiang.chien@gmail.com" target="_blank" class="">tzuhsiang.chien@gmail.com</a></div><div class=""><a href="mailto:vonosmas@gmail.com" target="_blank" class="">vonosmas@gmail.com</a></div><div class=""><a href="mailto:yaron.keren@gmail.com" target="_blank" class="">yaron.keren@gmail.com</a></div><div class=""><br class=""></div><div class="">Would you know the formal process for a license change?</div><div class=""><br class=""></div></div></div><div class=""><div class="m_5191769219634074052h5"><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class=""><div class="m_5191769219634074052m_7285851062250931345h5"><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class=""><br class=""></div><div class="">BTW libFuzzer CMake has a crazy amount of hacks to work under Windows, where logic in many parts</div><div class="">is entirely different, so any help on testing and fixing arising issues would be much appreciated.</div></div></div></blockquote><div class=""><br class=""></div><div class="">It's totally fine for me if we *copy* the code to the new location and ditch the windows support completely. </div><div class="">Then, whoever cares about windows, will reinstate it in the new location once the dust settles. </div></div></div></div></div></blockquote><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="">All I can do is to make a commit and then try to understand the response from a bots,</div><div class="">which is not very efficient.</div><div class=""><div class="m_5191769219634074052m_7285851062250931345m_4542425844967384059h5"><br class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><br class=""></div><div class="">3) CMake files in LLVM root repository do a rather large amount of trickery:</div><div class="">more warning flags are introduced, default linking commands for different platforms are changed,</div><div class="">etc.</div><div class="">None of this is picked up when running from the “runtimes” directory, which is often undesirable,</div><div class="">as the runtime would be built very differently from the rest of LLVM, missing those desired warnings/optimizations.</div><div class=""><br class=""></div><div class="">Additionally, working with “runtimes” introduces some inherent restrictions.</div><div class="">Namely:</div><div class=""><br class=""></div><div class="">4) Extra arguments to CMake will not be propagated.</div><div class="">E.g. invoking `cmake … -DLLVM_USE_SANITIZE_COVERAGE=1<wbr class="">` will not have any effect on the compilation of any project</div><div class="">in “runtimes”.</div><div class="">It’s not clear how those flags can be explicitly propagated.</div><div class=""><br class=""></div><div class="">5) In a similar vein to (4), additional flags to `ninja` will not be propagated.</div><div class="">E.g. running “ninja -v check-blah” will actually not show the commands required to execute the tests,</div><div class="">as it goes through a CMake invocation.</div><div class="">This issue has a workaround: ninja can be launched directly from “runtimes/runtime-bins”, but that is counterintuitive.</div><div class=""><br class=""></div><div class="">6) Recursive invocation required for “runtimes” breaks `compile_commands.json` construction, which is used</div><div class="">by many editors and tools (e.g. Vim+Ale or rtags) for go-to-defintion and error highlight functionality.</div><div class="">There is a workaround: a separate `compile_commands.json` is generated for the `runtimes` directory,</div><div class="">and it might be possible to write some magic to inject it back into the parent one, but that would be yet-another-piece-of-build-inf<wbr class="">rastructure</div><div class="">which would have to be maintained.</div><div class=""><br class=""></div><div class="">7) Similarly to (6), recursive invocation breaks XCode tooling: in a generated XCode project,</div><div class="">no libfuzzer files are accessible. I would assume same would hold for other IDEs.</div><div class=""><br class=""></div><div class="">Any thoughts or comments on these?</div><div class="">While (1) is not really a problem, and I can probably find a workaround for (2), the issues</div><div class="">listed in (3)-(7) seem inherent to recursive CMake invocation.</div><div class="">I can think of a number of alternative suggestions, which would solve the problem of requiring a 2-stage build:</div><div class=""><br class=""></div><div class="">a) We can move the compilation commands for libFuzzer tests from CMake into lit.</div><div class="">This would have an added benefit of each lit test being self-contained: it would be sufficient to just run</div><div class="">“lit” to reproduce everything, and it would pick up all changes to compiler/coverage instrumentation/sanitizers/etc<wbr class="">.</div><div class="">The first run would generate the tested binaries, and further tinkering could be done with binaries directly if desired.</div><div class=""><br class=""></div><div class="">b) We can use the “tools” directory instead.</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">George</div><div class=""><div class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055h5"><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On May 11, 2017, at 10:31 AM, Kostya Serebryany <<a href="mailto:kcc@google.com" target="_blank" class="">kcc@google.com</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711Apple-interchange-newline"><div class=""><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711Apple-interchange-newline"><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">On Wed, May 10, 2017 at 6:09 PM, Chris Bieneman via llvm-dev<span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.<wbr class="">org</a>></span><span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 10, 2017, at 4:43 PM, George Karpenkov via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031Apple-interchange-newline"><div class=""><div style="word-wrap:break-word" class="">Actually, there’s another problem we have missed: libraries under `build/lib` are not installed into toolchain<div class="">on mac os (and neither on linux, I would suppose).</div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Actually that isn't accurate. By default we don't install the LLVM libraries, but that is completely configurable in the build system. It doesn't work for libFuzzer because the CMake build for libFuzzer is not built using any of the LLVM CMake modules or following any of LLVM's conventions.</div><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><div class="">Thus installations of Clang would not contain libLLVMFuzzer, but we would like them to, so that users would not have</div><div class="">to compile anything, and could just call `clang -fsanitize=fuzzer`.</div><div class=""><br class=""></div><div class="">That could probably be done with another CMake change, but I have no idea how to do that.</div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Yea, libFuzzer's CMake really needs a big overhaul, and probably an almost complete rewrite.</div></div></div></blockquote><div class=""><br class=""></div><div class="">no objections.</div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class=""><br class=""></div><div class="">-Chris</div><div class=""><div class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711h5"><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><div class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On May 9, 2017, at 4:04 PM, George Karpenkov via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031Apple-interchange-newline"><div class=""><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote type="cite" class=""><div class=""><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031Apple-interchange-newline">On May 9, 2017, at 3:00 PM, Kostya Serebryany <<a href="mailto:kcc@google.com" target="_blank" class="">kcc@google.com</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031Apple-interchange-newline"><div class=""><div dir="ltr" class="">Thanks for the explanations! (it was worth asking) <div class=""><br class=""></div><div class="">I do want to build libFuzzer itself (and its tests) using the <span style="font-size:12.8px" class="">just-built clang. So, </span><span style="font-size:12.8px" class="">llvm/runtimes then. </span></div><div class=""><span style="font-size:12.8px" class="">I'd name the directory </span><span style="font-size:12.8px" class="">llvm/runtimes/libFuz<wbr class="">zer, if possible (the old path was lib/Fuzzer which is how the tool got it's name, actually)</span></div><div class=""><span style="font-size:12.8px" class="">George, would you like to send the change for review? </span></div></div></div></blockquote>OK<br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div><div class=""><span style="font-size:12.8px" class="">--kcc</span></div><div class=""><br class=""></div><div class=""><span style="font-size:12.8px" class=""><br class=""></span></div></div><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, May 9, 2017 at 2:37 PM, Chris Bieneman<span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:cbieneman@apple.com" target="_blank" class="">cbieneman@apple.com</a>></span><span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031Apple-converted-space"><wbr class=""> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><br class=""><div class=""><span class=""><blockquote type="cite" class=""><div class="">On May 9, 2017, at 2:19 PM, George Karpenkov <<a href="mailto:ekarpenkov@apple.com" target="_blank" class="">ekarpenkov@apple.com</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-interchange-newline"><div class=""><div style="word-wrap:break-word" class=""><div class="">+Chris.</div><div class=""><br class=""></div><div class="">My understanding was that it is technically impossible for things in “lib”, as they are built first, and there’s no way to tell them to do that before “clang”.</div><div class="">I’m not a CMake expert, and I might be wrong.</div></div></div></blockquote><div class=""><br class=""></div></span><div class="">It is not impossible, it would just involve excessive hacks. Since it seems like this isn't a short-term solution we're talking about I am very opposed to throwing hacks into the build system. I'd rather we actually fix the problem(s). More below.</div><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><br class=""><div class=""><blockquote type="cite" class=""><div class="">On May 9, 2017, at 2:15 PM, Kostya Serebryany <<a href="mailto:kcc@google.com" target="_blank" class="">kcc@google.com</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-interchange-newline"><div class=""><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-interchange-newline"><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px">On Tue, May 9, 2017 at 1:56 PM, George Karpenkov<span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-converted-space"> </span><span dir="ltr" class=""><<a href="mailto:ekarpenkov@apple.com" target="_blank" class="">ekarpenkov@apple.co<wbr class="">m</a>></span><span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-converted-space"> </span>wrote:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class="">Again, after offline conversation with Chris Bieneman:</div><div class=""><br class=""></div><div class=""> - move to compiler-rt would be too complicated due to change in licenses</div><div class=""> - it would make much more sense to move to “tools” folder instead, for the following reasons:</div><div class="">   <span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-converted-space"> </span>* conceptually, it’s a tool, not a library</div><div class="">   <span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-converted-space"> </span>* all other projects in “lib” depend on LLVM and can not build without LLVM, libFuzzer does not</div><div class="">   <span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-converted-space"> </span>* practically speaking, CMake has no way of knowing whether Clang is being built when</div><div class="">     <span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790Apple-converted-space"> </span>“lib” is compiled, yet it does know for projects in tools.</div><div class=""><br class=""></div><div class="">Using a freshly built clang for projects in “tools” is embarrassingly easy and only requires a couple of lines</div><div class="">of configuration change.</div><div class=""><br class=""></div><div class="">Kostya, what about moving to “tools” then?</div></div></blockquote><div class=""><br class=""></div><div class="">Well, ok, this sounds cool. </div><div class="">But can we make one more step and try to preserve the code where it is, for the sake of compatibility? </div></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Please no. This code doesn't actually belong in lib, it has never fit the model of an LLVM library, we really need to pull it out of there. </div><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><div class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="">E.g. can we have the CMake in tools while still keeping the code in lib? </div></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div></span>Could we contrive a hack in the build system to do it? Yes, but I will fight violently against allowing that change into the build system because the right answer here is to move the code.</div><div class=""><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><div class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class="">Or a link of some kind?</div></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Links are incredibly fragile on Windows, and they trip up a lot of SCM tools. We have one in LLDB's repo that causes me nothing but problems, so I am also strongly opposed to that.</div><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><div class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class=""><br class=""></div><div class="">My worry is that there are already quite a few places that know where libFuzzer code is, </div><div class="">and I don't control all of them. </div></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Downstream clients will have to update. That is kinda how these things work, I can't imagine re-pointing an SCM checkout being a huge burden.</div><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><div class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class=""><br class=""></div><div class="">And, finally, I really don't get why we can do something in tools and can't do the same in lib. </div><div class="">Or we simply don't want to do it to keep things simple? </div></div></div></blockquote></div></div></div></blockquote><div class=""><br class=""></div></span><div class="">Not all functionality in CMake is order-independent. Specifically the detection of targets is not. In order to support what you're trying to do you are going to change behavior based on the presence of the clang target. Which means the clang target must be added before your CMake is processed.</div><div class=""><br class=""></div><div class="">To support this our build system has strict ordering requirements such that things in lib cannot depend on things in tools. If you need to depend on clang, you need to not be in lib.</div><div class=""><br class=""></div><div class="">Also, generally speaking Fuzzer is a library under lib that also has nested tests, which is *not* how the lib directory is supposed to be structured. It never should have been allowed to be structured like that. If you want the tests next to the library, it is a tool or a runtime, but not a lib.</div><div class=""><br class=""></div><div class="">As I see there are two options to move forward with, and it really depends on how you intend to use the just-built clang.</div><div class=""><br class=""></div><div class="">(1) If you want to use just-built clang to build libFuzzer and its tests, it should be a runtime.</div><div class="">(2) If you want to use just-built clang to only build libFuzzer's tests, it should be a tool.</div><div class=""><br class=""></div><div class="">I think that since it is a runtime library, it should be a runtime, and I expect it would mostly work to just copy the Fuzzer directory into llvm/runtimes.</div><span class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031HOEnZb"><font color="#888888" class=""><div class=""><br class=""></div><div class="">-Chris</div></font></span><span class=""><br class=""><blockquote type="cite" class=""><div class=""><div style="word-wrap:break-word" class=""><div class=""><blockquote type="cite" class=""><div class=""><div class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px"><div class=""><br class=""></div><div class="">--kcc </div><div class=""><br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div style="word-wrap:break-word" class=""><div class=""><div class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790h5"><br class=""><div class=""><blockquote type="cite" class=""><div class="">On May 9, 2017, at 11:07 AM, Dan Liew <<a href="mailto:dan@su-root.co.uk" target="_blank" class="">dan@su-root.co.uk</a>> wrote:</div><br class="m_5191769219634074052m_7285851062250931345m_4542425844967384059m_6099913875395367055m_-2065048435345201711m_3257897739411595031m_-5671233790260223790m_877734713401612435Apple-interchange-newline"><div class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">On 9 May 2017 at 18:55, Kostya Serebryany <</span><a href="mailto:kcc@google.com" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">kcc@google.com</a><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">> wrote:</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote type="cite" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br class=""><br class="">On Tue, May 9, 2017 at 10:23 AM, Dan Liew <<a href="mailto:dan@su-root.co.uk" target="_blank" class="">dan@su-root.co.uk</a>> wrote:<br class=""><blockquote type="cite" class=""><br class=""><blockquote type="cite" class="">Does anyone see good reasons why libFuzzer should remain in llvm repo<br class="">(as<br class="">opposed to moving it to compiler-rt)?<br class=""></blockquote><br class="">Does moving LibFuzzer to compiler-rt imply that it is compiled as part<br class="">of compiler-rt and shipped with it?<br class=""><br class="">How does that fit with LibFuzzer's model of allowing the user to<br class="">provide their own `main()`.<br class=""></blockquote><br class=""><br class="">libFuzzer doesn't allow users to use their own main (not any more).<br class="">Although I am not sure how that's related to moving libFuzzer somewhere.<br class=""></blockquote><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">Oops. That shows how long it's been since I looked at the source code.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">It was related in that if LibFuzzer was shipped as part of compiler-rt</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">I presumed we would need to supply both libraries to end users.</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">Given that this feature was removed it is a non-issue.</span></div></blockquote></div></div></div></div></blockquote></div></div></blockquote></div><br class=""></div></div></blockquote></span></div><br class=""></div></blockquote></div><br class=""></div></div></blockquote></div><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">______________________________<wbr class="">_________________</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">LLVM Developers mailing list</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><a href="mailto:llvm-dev@lists.llvm.org" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">llvm-dev@lists.llvm.org</a><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant-caps:normal;font-weight:normal;letter-spacing:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-dev</a></div></blockquote></div><br class=""></div></div>______________________________<wbr class="">_________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-dev</a><br class=""></div></blockquote></div></div></div><br class=""></div><br class="">______________________________<wbr class="">_________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/<wbr class="">mailman/listinfo/llvm-dev</a></blockquote></div></div></blockquote></div></div></div></div></div></blockquote></div></div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div></div>
</div></blockquote></div></div></div><br class=""></div></blockquote></div><br class=""></div>
</blockquote></div><br class=""></div></div>
</div></blockquote></div><br class=""></body></html>