<div>Let me try starting over without mentioning Tablegen :)</div><div><br></div><div>Is Support too big, and if so should we consider breaking it up?</div><div><br></div><div>So far it sounds like the general consensus is "yes", with some quibbles about precisely how to split it. </div><div><br></div><div>I'm not necessarily opposed to smaller libraries like Triple as you suggest, but i think that only addresses what to with largely disparate set of stuff which I'm calling "narrowly useful". It doesn't really address the fact that even after doing it at every logical division point, there will still be a set of stuff that doesn't belong with other stuff. </div><div><br></div><div>To address that, I'm proposing something like "Base" which is similar in spirit to Support as it exists today, but more restrictive, and must satisfy the "would this be useful outside of LLVM" bar.</div><div><br></div><div><br></div><div><br></div><div><br><div class="gmail_quote"><div>On Tue, May 30, 2017 at 11:05 AM Matthias Braun via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>In my experience the buildsystem works fine in combination with tablegen (at least this aspect of it). The real problem here is that tablegen is just slow. Some of the X86 tables take indeed </div><div><br></div><div>Last time I looked at it tablegen had still room to optimize the way it resolves class hierarchies and the variables within which it did basically one at a time, so it needed to traverse the hierarchies again for each variables.</div></div><div style="word-wrap:break-word"><br><div><blockquote type="cite"><div>On May 27, 2017, at 12:54 PM, David Blaikie via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:</div><br class="m_5838877945136994382Apple-interchange-newline"><div><div>Maybe we do and build systems aren't respecting/noticing this? I'm not sure.</div><br><div class="gmail_quote"><div>On Sat, May 27, 2017 at 12:50 PM Craig Topper <<a href="mailto:craig.topper@gmail.com" target="_blank">craig.topper@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div>I thought we already did write tablegen output to temporary files like X86GenAsmWriter1.inc.tmp first and then diffed them with the real .inc file and conditionally copied.</div><div class="gmail_extra"></div><div class="gmail_extra"><br clear="all"><div><div class="m_5838877945136994382m_-495019445101160162gmail_signature" data-smartmail="gmail_signature">~Craig</div></div></div><div class="gmail_extra">
<br><div class="gmail_quote">On Sat, May 27, 2017 at 11:02 AM, David Blaikie via llvm-dev <span><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><br><br><div class="gmail_quote"><div>On Fri, May 26, 2017 at 8:06 PM Zachary Turner via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It would be better, because a debug tablegen is slower than an optimized tablegen, but it's still slow and it doesn't address the problem that tablegen runs *at all* when it doesn't really need to. I think if tablegen wasn't running all the time we could incremental builds down from 15 minutes (and that's on my really powerful machine) to under 5, which seemed like a big productivity boost averaged out over all users x time<br></blockquote><div><br>Is that 10 minutes of running tablegen, though? I think this touches on Hal's comment:<br>"<span style="color:rgb(33,33,33);font-size:13px">Maybe we should use a diff-and-update approach (I thought, however, that we already did that)."<br></span><br>In that the 10 minutes is presumably from timestamp-based invalidation chaining down from running tablegen, to then rebuilding everything that depends on tablegen's output.<br><br>That could be addressed by either the build system or tablegen writing output to a temporary file, comparing the temp to the current file, and not modifying the current file if they're the same - thus not invalidating everything down the chain that depends on the tablegen output (not sure if that's actually enough to convince a build system not to rebuild other things - but worth a shot). This is a bit of a hackaround what the build system itself could/should be doing, but in the absence of a content based invalidation scheme - addressing this one particularly critical point in LLVM's build seems like it could be worth such a workaround. (this would speed up the build for even the times when tablegen's real dependencies change - add a new function to ArrayRef, or STLExtras, etc... and tablegen runs, but it doesn't cascade to everything else). <br><br>(plus, opt build of tablegen, so that conclusion can be reached more quickly)<br><br>As Hal also mentioned, I'd be OK with this if there's a good logical division that can be taken more than "whatever is/isn't used in tablegen".<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>We've got a lot of random stuff in Support, not all really related and not all relevant to every project. It seems like we should be able to do better.<br><div class="gmail_quote"><div>On Fri, May 26, 2017 at 6:48 PM Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><p><br>
</p>
<div class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137moz-cite-prefix">On 05/26/2017 07:59 PM, Zachary Turner
wrote:<br>
</div>
<blockquote type="cite">
<div>It's that TableGen depends on Support, so if you
change one file in support, support gets recompiled into a new
static archive, which triggers a rerun of tablegen on all the
tablegen inputs, which is extremely slow.</div>
</blockquote>
<br></div><div bgcolor="#FFFFFF" text="#000000">
I thought that we had a "build an optimized tablegen even in debug
mode" setting to work around this problem. Would that avoid this
problem?</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
-Hal</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
<blockquote type="cite"><br>
<div class="gmail_quote">
<div>On Fri, May 26, 2017 at 5:56 PM Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><p><br>
</p>
<br>
<div class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-cite-prefix">On
05/26/2017 07:47 PM, Zachary Turner via llvm-dev wrote:<br>
</div>
<blockquote type="cite">
<div>Changing a header file somewhere and having
to spend 10 minutes waiting for a build leads to a lot
of wasted developer time.
<div><br>
</div>
<div>The real culprit here is tablegen. Can we split
support and ADT into two - the parts that tablegen
depends on and the parts that it doesn't?</div>
</div>
</blockquote>
<br>
</div>
<div bgcolor="#FFFFFF" text="#000000"> What's the actual
problem here? Is it that TableGen regenerates different
files and so we then need to rebuild all dependencies of
those files? Maybe we should use a diff-and-update approach
(I thought, however, that we already did that).<br>
<br>
-Hal<br>
<br>
</div>
<div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<div>
<div><br>
</div>
<div>From what I can gather, Tablegen currently depends
on these headers and all of their transitive
dependencies.</div>
<div><br>
</div>
<div>
<div>#include "llvm/Support/Casting.h"</div>
<div>#include "llvm/Support/CommandLine.h"</div>
<div>#include "llvm/Support/Compiler.h"</div>
<div>#include "llvm/Support/DataTypes.h"</div>
<div>#include "llvm/Support/Debug.h"</div>
<div>#include "llvm/Support/Error.h"</div>
<div>#include "llvm/Support/ErrorHandling.h"</div>
<div>#include "llvm/Support/Format.h"</div>
<div>#include "llvm/Support/FormattedStream.h"</div>
<div>#include "llvm/Support/LEB128.h"</div>
<div>#include "llvm/Support/LowLevelTypeImpl.h"</div>
<div>#include "llvm/Support/ManagedStatic.h"</div>
<div>#include "llvm/Support/MathExtras.h"</div>
<div>#include "llvm/Support/MemoryBuffer.h"</div>
<div>#include "llvm/Support/PrettyStackTrace.h"</div>
<div>#include "llvm/Support/Regex.h"</div>
<div>#include "llvm/Support/SMLoc.h"</div>
<div>#include "llvm/Support/ScopedPrinter.h"</div>
<div>#include "llvm/Support/Signals.h"</div>
<div>#include "llvm/Support/SourceMgr.h"</div>
<div>#include "llvm/Support/raw_ostream.h"</div>
<div><br>
</div>
<div>#include "llvm/ADT/APInt.h"</div>
<div>#include "llvm/ADT/ArrayRef.h"</div>
<div>#include "llvm/ADT/BitVector.h"</div>
<div>#include "llvm/ADT/CachedHashString.h"</div>
<div>#include "llvm/ADT/DenseSet.h"</div>
<div>#include "llvm/ADT/IndexedMap.h"</div>
<div>#include "llvm/ADT/IntEqClasses.h"</div>
<div>#include "llvm/ADT/MapVector.h"</div>
<div>#include "llvm/ADT/Optional.h"</div>
<div>#include "llvm/ADT/PointerUnion.h"</div>
<div>#include "llvm/ADT/STLExtras.h"</div>
<div>#include "llvm/ADT/SetVector.h"</div>
<div>#include "llvm/ADT/SmallPtrSet.h"</div>
<div>#include "llvm/ADT/SmallSet.h"</div>
<div>#include "llvm/ADT/SmallVector.h"</div>
<div>#include "llvm/ADT/SparseBitVector.h"</div>
<div>#include "llvm/ADT/Statistic.h"</div>
<div>#include "llvm/ADT/StringExtras.h"</div>
<div>#include "llvm/ADT/StringMap.h"</div>
<div>#include "llvm/ADT/StringRef.h"</div>
<div>#include "llvm/ADT/StringSet.h"</div>
<div>#include "llvm/ADT/StringSwitch.h"</div>
<div>#include "llvm/ADT/TinyPtrVector.h"</div>
<div>#include "llvm/ADT/Twine.h"</div>
</div>
<div><br>
</div>
<div>
<div><br class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649inbox-inbox-Apple-interchange-newline">
Is this something worth putting effort into? If so,
I volunteer.</div>
<br class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649inbox-inbox-Apple-interchange-newline">
</div>
</div>
<br>
<fieldset class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
</div>
<div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<pre>_______________________________________________
LLVM Developers mailing list
<a class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>
<a class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
</div>
<div bgcolor="#FFFFFF" text="#000000"> <br><span class="m_5838877945136994382m_-495019445101160162HOEnZb"><font color="#888888">
<pre class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</font></span></div><span class="m_5838877945136994382m_-495019445101160162HOEnZb"><font color="#888888">
</font></span></blockquote><span class="m_5838877945136994382m_-495019445101160162HOEnZb"><font color="#888888">
</font></span></div><span class="m_5838877945136994382m_-495019445101160162HOEnZb"><font color="#888888">
</font></span></blockquote><span class="m_5838877945136994382m_-495019445101160162HOEnZb"><font color="#888888">
<br>
<pre class="m_5838877945136994382m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</font></span></div></blockquote></div><span class="m_5838877945136994382m_-495019445101160162HOEnZb"><font color="#888888">
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</font></span></blockquote></div></div>
<br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
<br></blockquote></div><br></div></blockquote></div>
_______________________________________________<br>LLVM Developers mailing list<br><a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br><a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br></div></blockquote></div><br></div>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div></div>