I think there are logical ways of splitting, but taken individually they would be rather small. For example, threading support, i/o related support, file system support.<br><br>At the same time, when you look at support there's kind of some glaringly obvious differences in how "low level" the various things are. Compiler.h is pretty low level. Dwarf.h, not so much. It's hard to really quantify this, but I could certainly imagine introducing something even more low level than Support, perhaps with a name like Base<br><div class="gmail_quote"><div dir="ltr">On Sat, May 27, 2017 at 12:55 PM David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Maybe we do and build systems aren't respecting/noticing this? I'm not sure.</div><br><div class="gmail_quote"><div dir="ltr">On Sat, May 27, 2017 at 12:50 PM Craig Topper <<a href="mailto:craig.topper@gmail.com" target="_blank">craig.topper@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I thought we already did write tablegen output to temporary files like X86GenAsmWriter1.inc.tmp first and then diffed them with the real .inc file and conditionally copied.</div><div class="gmail_extra"></div><div class="gmail_extra"><br clear="all"><div><div class="m_-2628403855158432989m_-495019445101160162gmail_signature" data-smartmail="gmail_signature">~Craig</div></div></div><div class="gmail_extra">
<br><div class="gmail_quote">On Sat, May 27, 2017 at 11:02 AM, David Blaikie via llvm-dev <span dir="ltr"><<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Fri, May 26, 2017 at 8:06 PM Zachary Turner via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It would be better, because a debug tablegen is slower than an optimized tablegen, but it's still slow and it doesn't address the problem that tablegen runs *at all* when it doesn't really need to. I think if tablegen wasn't running all the time we could incremental builds down from 15 minutes (and that's on my really powerful machine) to under 5, which seemed like a big productivity boost averaged out over all users x time<br></blockquote><div><br>Is that 10 minutes of running tablegen, though? I think this touches on Hal's comment:<br>"<span style="color:rgb(33,33,33);font-size:13px">Maybe we should use a diff-and-update approach (I thought, however, that we already did that)."<br></span><br>In that the 10 minutes is presumably from timestamp-based invalidation chaining down from running tablegen, to then rebuilding everything that depends on tablegen's output.<br><br>That could be addressed by either the build system or tablegen writing output to a temporary file, comparing the temp to the current file, and not modifying the current file if they're the same - thus not invalidating everything down the chain that depends on the tablegen output (not sure if that's actually enough to convince a build system not to rebuild other things - but worth a shot). This is a bit of a hackaround what the build system itself could/should be doing, but in the absence of a content based invalidation scheme - addressing this one particularly critical point in LLVM's build seems like it could be worth such a workaround. (this would speed up the build for even the times when tablegen's real dependencies change - add a new function to ArrayRef, or STLExtras, etc... and tablegen runs, but it doesn't cascade to everything else). <br><br>(plus, opt build of tablegen, so that conclusion can be reached more quickly)<br><br>As Hal also mentioned, I'd be OK with this if there's a good logical division that can be taken more than "whatever is/isn't used in tablegen".<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>We've got a lot of random stuff in Support, not all really related and not all relevant to every project. It seems like we should be able to do better.<br><div class="gmail_quote"><div dir="ltr">On Fri, May 26, 2017 at 6:48 PM Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<div class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137moz-cite-prefix">On 05/26/2017 07:59 PM, Zachary Turner
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">It's that TableGen depends on Support, so if you
change one file in support, support gets recompiled into a new
static archive, which triggers a rerun of tablegen on all the
tablegen inputs, which is extremely slow.</div>
</blockquote>
<br></div><div bgcolor="#FFFFFF" text="#000000">
I thought that we had a "build an optimized tablegen even in debug
mode" setting to work around this problem. Would that avoid this
problem?</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
-Hal</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
<blockquote type="cite"><br>
<div class="gmail_quote">
<div dir="ltr">On Fri, May 26, 2017 at 5:56 PM Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<br>
<div class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-cite-prefix">On
05/26/2017 07:47 PM, Zachary Turner via llvm-dev wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Changing a header file somewhere and having
to spend 10 minutes waiting for a build leads to a lot
of wasted developer time.
<div><br>
</div>
<div>The real culprit here is tablegen. Can we split
support and ADT into two - the parts that tablegen
depends on and the parts that it doesn't?</div>
</div>
</blockquote>
<br>
</div>
<div bgcolor="#FFFFFF" text="#000000"> What's the actual
problem here? Is it that TableGen regenerates different
files and so we then need to rebuild all dependencies of
those files? Maybe we should use a diff-and-update approach
(I thought, however, that we already did that).<br>
<br>
-Hal<br>
<br>
</div>
<div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>From what I can gather, Tablegen currently depends
on these headers and all of their transitive
dependencies.</div>
<div><br>
</div>
<div>
<div>#include "llvm/Support/Casting.h"</div>
<div>#include "llvm/Support/CommandLine.h"</div>
<div>#include "llvm/Support/Compiler.h"</div>
<div>#include "llvm/Support/DataTypes.h"</div>
<div>#include "llvm/Support/Debug.h"</div>
<div>#include "llvm/Support/Error.h"</div>
<div>#include "llvm/Support/ErrorHandling.h"</div>
<div>#include "llvm/Support/Format.h"</div>
<div>#include "llvm/Support/FormattedStream.h"</div>
<div>#include "llvm/Support/LEB128.h"</div>
<div>#include "llvm/Support/LowLevelTypeImpl.h"</div>
<div>#include "llvm/Support/ManagedStatic.h"</div>
<div>#include "llvm/Support/MathExtras.h"</div>
<div>#include "llvm/Support/MemoryBuffer.h"</div>
<div>#include "llvm/Support/PrettyStackTrace.h"</div>
<div>#include "llvm/Support/Regex.h"</div>
<div>#include "llvm/Support/SMLoc.h"</div>
<div>#include "llvm/Support/ScopedPrinter.h"</div>
<div>#include "llvm/Support/Signals.h"</div>
<div>#include "llvm/Support/SourceMgr.h"</div>
<div>#include "llvm/Support/raw_ostream.h"</div>
<div><br>
</div>
<div>#include "llvm/ADT/APInt.h"</div>
<div>#include "llvm/ADT/ArrayRef.h"</div>
<div>#include "llvm/ADT/BitVector.h"</div>
<div>#include "llvm/ADT/CachedHashString.h"</div>
<div>#include "llvm/ADT/DenseSet.h"</div>
<div>#include "llvm/ADT/IndexedMap.h"</div>
<div>#include "llvm/ADT/IntEqClasses.h"</div>
<div>#include "llvm/ADT/MapVector.h"</div>
<div>#include "llvm/ADT/Optional.h"</div>
<div>#include "llvm/ADT/PointerUnion.h"</div>
<div>#include "llvm/ADT/STLExtras.h"</div>
<div>#include "llvm/ADT/SetVector.h"</div>
<div>#include "llvm/ADT/SmallPtrSet.h"</div>
<div>#include "llvm/ADT/SmallSet.h"</div>
<div>#include "llvm/ADT/SmallVector.h"</div>
<div>#include "llvm/ADT/SparseBitVector.h"</div>
<div>#include "llvm/ADT/Statistic.h"</div>
<div>#include "llvm/ADT/StringExtras.h"</div>
<div>#include "llvm/ADT/StringMap.h"</div>
<div>#include "llvm/ADT/StringRef.h"</div>
<div>#include "llvm/ADT/StringSet.h"</div>
<div>#include "llvm/ADT/StringSwitch.h"</div>
<div>#include "llvm/ADT/TinyPtrVector.h"</div>
<div>#include "llvm/ADT/Twine.h"</div>
</div>
<div><br>
</div>
<div>
<div><br class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649inbox-inbox-Apple-interchange-newline">
Is this something worth putting effort into? If so,
I volunteer.</div>
<br class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649inbox-inbox-Apple-interchange-newline">
</div>
</div>
<br>
<fieldset class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
</div>
<div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<pre>_______________________________________________
LLVM Developers mailing list
<a class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>
<a class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
</div>
<div bgcolor="#FFFFFF" text="#000000"> <br><span class="m_-2628403855158432989m_-495019445101160162HOEnZb"><font color="#888888">
<pre class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</font></span></div><span class="m_-2628403855158432989m_-495019445101160162HOEnZb"><font color="#888888">
</font></span></blockquote><span class="m_-2628403855158432989m_-495019445101160162HOEnZb"><font color="#888888">
</font></span></div><span class="m_-2628403855158432989m_-495019445101160162HOEnZb"><font color="#888888">
</font></span></blockquote><span class="m_-2628403855158432989m_-495019445101160162HOEnZb"><font color="#888888">
<br>
<pre class="m_-2628403855158432989m_-495019445101160162m_-8452059004670669254m_3422182665239432404m_-6805375126177304137moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</font></span></div></blockquote></div><span class="m_-2628403855158432989m_-495019445101160162HOEnZb"><font color="#888888">
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</font></span></blockquote></div></div>
<br>_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
<br></blockquote></div><br></div></blockquote></div>
</blockquote></div>