<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><span style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif;" class=""><b class="">From: </b></span><span style="font-family: -webkit-system-font, 'Helvetica Neue', Helvetica, sans-serif;" class="">Alex Wang <<a href="mailto:aw1621107@gmail.com" class="">aw1621107@gmail.com</a>></span><br class=""><div><blockquote type="cite" class=""><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Date: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">September 28, 2015 at 4:48:29 PM EDT<br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">To: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class="">Chris Bieneman <<a href="mailto:beanz@apple.com" class="">beanz@apple.com</a>><br class=""></span></div><div style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0px;" class=""><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif; color:rgba(0, 0, 0, 1.0);" class=""><b class="">Subject: </b></span><span style="font-family: -webkit-system-font, Helvetica Neue, Helvetica, sans-serif;" class=""><b class="">Re: [llvm-dev] Cmake-gen'd parallel make breaks on native tablegen</b><br class=""></span></div><br class=""><div class=""><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><br class="Apple-interchange-newline">On Sep 28, 2015, at 3:56 PM, Chris Bieneman <<a href="mailto:beanz@apple.com" class="">beanz@apple.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="Apple-interchange-newline">On Sep 28, 2015, at 12:07 PM, Alex Wang via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""><br class="">Hello backend devs,<br class=""><br class="">Been working on a parallelization bug in the cmake configs, where parallel makefile builds of llvm with clang and native tablegen would usually break during linking in either either the clang_tblgen or llvm_tblgen targets. Looking at the cmake command that generates the tablegen makefile commands (I think) (cmake/modules/TableGen.cmake:113-117):<br class=""><br class=""> add_custom_command(OUTPUT ${${project}_TABLEGEN_EXE}<br class=""> COMMAND ${CMAKE_COMMAND} --build ../.. --target ${target} --config Release<br class=""> DEPENDS CONFIGURE_LLVM_NATIVE ${target}<br class=""> WORKING_DIRECTORY ${LLVM_NATIVE_BUILD}<br class=""> COMMENT "Building native TableGen...")<br class=""><br class="">My guess is that the clang and llvm tablegen builds are trying to place their output into the same folder, so depending on when the resulting library gets linked/moved one of the threads could be left dangling with a partially-completed library or no library.<br class=""></blockquote><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">If you build with LLVM_OPTIMIZED_TABLEGEN the native tablegen is a fully generated build subtree. Make seems to have some problems with thread scheduling with these builds. I haven’t diagnosed the issue, but it isn’t that the builds are writing to the same directory. It happens regardless of whether you’re building llvm with or without clang, and the llvm & clang tablegen executables are named differently so they don’t conflict.</span></div></blockquote><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Hmmm, that's odd. I never saw the build break like this when building only llvm. Seems I just got really (un)lucky...</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Probably could have explained this better... It wasn't that I thought the output executables would clash; I thought that both the llvm and clang tablegen builds were working on libLLVMSupport at the same time in the same place, so when one finished and moved the library to its proper place the other thread is left without anything to work with. Seems like what you said would still apply though.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class="">Another thing to note is that when building with clang the cmake percentages for that particular library would always go above 100%. I thought that meant that they could be writing to the same progress file, because that never happened with llvm-only builds.</div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><br class=""></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><blockquote type="cite" class=""><div class=""><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;">The error was either "Couldn't stat output file: ../LLVMSupport.a No such file or directory" or something about an archive member extending past the end of the file, and the cmake progress messages before linking were printed twice. Once for each thread, I'd guess? Sort of surprised that they seem to share so much code (or at least filenames)<br class=""><br class="">First thing I could think of to fix this is adding another layer or two of CMakeLists, so the tablegen creation command would be invoked in different folders. I'm not convinced that that is the best fix, and I couldn't figure out how to move the builds into different folders with only minor changes to the existing cmake files.<br class=""></blockquote><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><span class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;">I don’t think that will work. As I said above, they are building to different directories. You can note that in the custom command setting WORKING_DIRECTORY to ${LLVM_NATIVE_BUILD}, which is a nested fully-independent cmake build tree.</span><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class=""></div></blockquote><div class=""><br class=""></div>At least on my machine, ${LLVM_NATIVE_BUILD} is the same for both llvm and clang tablegen commands. <blockquote type="cite" class=""><div class=""><br class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><blockquote type="cite" class="" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px;"><br class="">Thoughts?<br class=""><br class="">-Alex<br class=""><br class="">(CrossCompile.cmake also has a typo in the first line: "toochain" --> "toolchain", if that is something to worry about)<br class="">_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class=""><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a></blockquote></div></blockquote></div></div></blockquote></div><br class=""></body></html>