<html><head><meta http-equiv="Content-Type" content="text/html charset=us-ascii"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">If the end result is "Base", "Misc", "Broad", "Narrow", "Util", "Support" or other non-descriptive names then I'm against it!</div><div class=""><br class=""></div><div class="">If you can find pieces that can be broken out as a logical units such as "TargetTriple"(for Triple.h and all the ARM parsers) or "object file formats" then I don't think anyone would object patches to do so. Motivating this with a better project structure sounds sensible to me, I am not convinced your buildtimes will change all that much: The next change to something like ADT/StringRef.h will come and you have to do the tablegen rebuild.</div><div class=""><br class=""></div><div class="">- Matthias</div><div class=""><br class=""></div><div><blockquote type="cite" class=""><div class="">On May 29, 2017, at 10:52 AM, Zachary Turner via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">Well you have to start somewhere right? Even if it doesn't completely solve the issue with TableGen, it is a step towards that, and it does help the issue of Support being a mess with everything under the kitchen sink. After doing that split, I'm going to wager that everything in ADT would depend only on Base (or be very close to depending only on Base), at which point LLVMBase.a can be ac ombination of ADT and Base instead of a combination of ADT and Support, and then you only have to look at the remaining headers in Support to see what tablegen depends on. The original list I posted that tablegen depended on from Support was<div class=""><br class=""></div><div class=""><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/Casting.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/CommandLine.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/Compiler.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/DataTypes.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/Debug.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/Error.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/ErrorHandling.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/Format.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/FormattedStream.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/LEB128.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/LowLevelTypeImpl.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/ManagedStatic.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/MathExtras.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/MemoryBuffer.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/PrettyStackTrace.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/Regex.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/SMLoc.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/ScopedPrinter.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/Signals.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/SourceMgr.h"</div><div style="color:rgb(33,33,33)" class="">#include "llvm/Support/raw_ostream.h"</div></div><div style="color:rgb(33,33,33)" class=""><br class=""></div><div style="color:rgb(33,33,33)" class="">And there's only 2 or 3 of these that I put on the "narrowly useful" list, so it doesn't seem that difficult to address from here. (And I came up with that separation without even looking at TableGen's dependencies, so it seems TableGen already only depends on most of the actually broadly useful stuff).</div><div style="color:rgb(33,33,33)" class=""><br class=""></div><div style="color:rgb(33,33,33)" class="">Of course, TableGen is only part of the build process (albeit the slowest part), I would be rather surprised if a split such as the one proposed (with some inevitable alternations based on user feedback) did not allow various other projects to also remove a dependency on Support in favor of one on Base, which could have a positive trickle down effect on build times for other projects not related to tablegen.</div></div><br class=""><div class="gmail_quote"><div dir="ltr" class="">On Mon, May 29, 2017 at 10:35 AM David Blaikie <<a href="mailto:dblaikie@gmail.com" class="">dblaikie@gmail.com</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class=""><div class="gmail_quote"><div dir="ltr" class="">On Mon, May 29, 2017 at 9:25 AM Zachary Turner via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class=""><div class=""><div class="gmail_quote"><div class="">On Sun, May 28, 2017 at 8:54 PM Mehdi AMINI via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=""><div class="">2017-05-26 17:47 GMT-07:00 Zachary Turner via llvm-dev <span class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>></span>:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><div class="">Changing a header file somewhere and having to spend 10 minutes waiting for a build leads to a lot of wasted developer time.<div class=""><br class=""></div><div class="">The real culprit here is tablegen. Can we split support and ADT into two - the parts that tablegen depends on and the parts that it doesn't?</div></div></blockquote><div class=""></div></div><div class=""><br class=""></div></div><div class=""><div class="">Splitting ADT just based on tablegen usage seems dubious to me. If we need to go this route, I'd replace as many uses of ADT data structure with STL ones to begin with to reduce the surface.</div></div></blockquote><div class=""><br class=""></div></div></div></div><div dir="ltr" class=""><div class=""><div class="gmail_quote"><div class="">Tablegen would not need to determine WHERE to split, it would just motivate the why. It's obvious just from looking at Support's include directory though that a lot of stuff in there doesn't belong together. A quick look over the include directory already suggests a split into "broadly useful stuff" and "narrowly useful stuff"</div><div class=""><br class=""></div><div class="">Broadly useful stuff:</div><div class="">AlignOf</div><div class="">Allocator</div><div class="">ArrayRecycler</div><div class="">Atomic</div><div class="">AtomicOrdering</div><div class="">Capacity</div><div class="">Casting</div><div class="">Chrono</div><div class="">circular_raw_ostream</div><div class="">COM.h</div><div class="">CommandLine.h</div><div class="">Compiler.h</div><div class="">ConvertUTF.h</div><div class="">CrashRecoveryContext.h</div><div class="">DataExtractor.h</div><div class="">Debug.h</div><div class="">Endian.h</div><div class="">EndianStream.h</div><div class="">Errc.h</div><div class="">Errno.h</div><div class="">Error.h</div><div class="">ErrorHandling.h</div><div class="">ErrorOr.h</div><div class="">FileOutputBuffer.h</div><div class="">FileSystem.h</div><div class="">FileUtilities.h</div><div class="">Format*.h</div><div class="">GlobPattern.h</div><div class="">Host.h</div><div class="">JamCRC.h</div><div class="">KnownBits.h</div><div class="">LineIterator.h</div><div class="">Locale.h</div><div class="">ManagedStatic.h</div><div class="">MathExtras.h</div><div class="">MD5.h</div><div class="">Memory.h</div><div class="">MemoryBuffer.h</div><div class="">Mutex.h</div><div class="">MutexGuard.h</div><div class="">NativeFormatting.h</div><div class="">Options.h</div><div class="">Parallel.h</div><div class="">Path.h</div><div class="">PointerLikeTypeTraits.h</div><div class="">PrettyStackTrace.h</div><div class="">Printable.h</div><div class="">Process.h</div><div class="">Program.h</div><div class="">RandomNumberGenerator.h</div><div class="">raw_os_ostream.h</div><div class="">raw_ostream.h</div><div class="">raw_sha1_ostream.h</div><div class="">Recycler.h</div><div class="">RecyclingAllocator.h</div><div class="">Regex.h</div><div class="">RWMutex.h</div><div class="">SaveAndRestore.h</div><div class="">ScaledNumber.h</div><div class="">SHA1.h</div><div class="">Signals.h</div><div class="">StringPool.h</div><div class="">StringSaver.h</div><div class="">SwapByteOrder.h</div><div class="">SystemUtils.h</div><div class="">thread.h</div><div class="">Threading.h</div><div class="">ThreadLocal.h</div><div class="">ThreadPool.h</div><div class="">Timer.h</div><div class="">TrailingObjects.h</div><div class="">Unicode.h</div><div class="">UnicodeCharRanges.h</div><div class="">UniqueLock.h</div><div class="">Watchdog.h</div><div class="">Win64EH.h</div><div class="">WindowsError.h</div><div class="">xxhash.h</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">Narrowly useful stuff:</div><div class="">AArch64TargetParser.def</div><div class="">ARMAttributeParser.h</div><div class="">ARMBuildAttriubtes.h</div><div class="">ARMEHABI.h</div><div class="">ARMTargetParser.def</div><div class="">ARMWinEH.h</div><div class="">Binary*Stream*.h</div><div class="">BlockFrequency.h</div><div class="">BranchProbability.h</div><div class="">CachePruning.h</div><div class="">CBindingWrapping.h</div><div class="">CodeGen.h</div><div class="">CodeGenCWrappers.h</div><div class="">COFF.h</div><div class="">Compression.h</div></div></div><div class=""><div class="gmail_quote"><div class=""><div class="">DebugCounter.h</div>DotGraphTraits.h</div><div class="">Dwarf.def</div><div class="">Dwarf.h</div><div class="">DynamicLibrary.h</div><div class="">ELF.h</div><div class="">GCOV.h</div><div class="">GenericDomTree.h</div><div class="">GenericDomTreeConstruction.h</div><div class="">GraphWriter.h</div><div class="">LEB128.h</div><div class="">LockFileManager.h</div><div class="">LowLevelTypeImpl.h</div><div class="">MachO.def</div><div class="">MachO.h</div><div class="">MipsABIFlags.h</div><div class="">OnDiskHashTable.h</div><div class="">PluginLoader.h</div><div class=""><div class="">Registry.h</div><div class="">ScopedPrinter.h</div>SMLoc.h</div><div class="">Solaris.h</div><div class="">SourceMgr.h</div><div class="">SpecialCaseList.h</div><div class="">TargetParser.h</div><div class="">TargetRegistry.h</div><div class="">TargetSelect.h</div><div class="">TarWriter.h</div><div class="">ToolOutputFile.h</div><div class="">TrigramIndex.h</div><div class="">TypeName.h</div><div class="">Valgrind.h</div><div class="">Wasm.h</div><div class="">YAMLParser.h</div><div class="">YAMLTraits.h</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">So, as a very crude first attempt, you call the first group of stuff "Base", the second group "Support", and add a dependency from Support to Base. This has nothing to do with tablegen, btw, and tablegen would still probably depend on Support even after this separation, but it makes sense even from a high level layering perspective (IMO)</div></div></div></div></blockquote></div></div><div dir="ltr" class=""><div class="gmail_quote"><div class=""><br class="">To what end, though? If most things will depend on both anyway - it doesn't sound like the split would help/improve the situation re: changes to base or support leading to full rebuilds of LLVM... no?<br class=""> </div></div></div><div dir="ltr" class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class=""><div class=""><div class="gmail_quote"><div class=""><br class=""></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class=""><div class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="gmail_extra"><div class="gmail_quote"></div></div></div><div class=""><div class="gmail_extra"><div class="gmail_quote">2017-05-28 8:25 GMT-07:00 Krzysztof Parzyszek via llvm-dev <span class=""><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a>></span>:<br class=""><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex"><span class="m_4608838139654627950m_-5899375805380188482m_1774889128856250233m_-2104759218787413549gmail-">On 5/26/2017 7:59 PM, Zachary Turner via llvm-dev wrote:<br class="">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
It's that TableGen depends on Support, so if you change one file in<br class="">
support, support gets recompiled into a new static archive, which<br class="">
triggers a rerun of tablegen on all the tablegen inputs, which is<br class="">
extremely slow.<br class="">
</blockquote>
<br class=""></span>
What exactly is extremely slow? In my experience TableGen itself takes a negligible amount of time compared to the rest of the build. This is particularly true in cases when something in Support or ADT is modified, as this usually triggers recompilation of large parts of LLVM.<br class=""></blockquote><div class=""><br class=""></div></div></div></div><div class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">Tablegen built in debug is really slow though, I remember an out-of-tree backend where running llvm-tblgen was taking up to 5 min per file!<div class="">The CMake option LLVM_OPTIMIZED_TABLEGEN helps a lot with this. Otherwise LLVM_TABLEGEN is even more efficient, but it's a double-edged sword.</div><div class=""><br class=""></div><div class="">But we could also use the diff-and-copy approach not on the tablegen output but on the llvm-tblgen binary itself, that way we wouldn't re-run it when it does not change itself (I'm not sure why CMake does not use this strategy by default for any file including .o and .a?).</div></div></div></div></div><div class=""><div class="gmail_extra"><div class="gmail_quote"><div class=""><br class=""></div><div class="">-- </div><div class="">Mehdi</div><div class=""><br class=""></div></div></div></div>
_______________________________________________<br class="">
LLVM Developers mailing list<br class="">
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class="">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">
</blockquote></div></div></div>
_______________________________________________<br class="">
LLVM Developers mailing list<br class="">
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank" class="">llvm-dev@lists.llvm.org</a><br class="">
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank" class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br class="">
</blockquote></div></div></blockquote></div>
_______________________________________________<br class="">LLVM Developers mailing list<br class=""><a href="mailto:llvm-dev@lists.llvm.org" class="">llvm-dev@lists.llvm.org</a><br class="">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev<br class=""></div></blockquote></div><br class=""></body></html>