<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Fri, May 26, 2017 at 8:06 PM Zachary Turner via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It would be better, because a debug tablegen is slower than an optimized tablegen, but it's still slow and it doesn't address the problem that tablegen runs *at all* when it doesn't really need to. I think if tablegen wasn't running all the time we could incremental builds down from 15 minutes (and that's on my really powerful machine) to under 5, which seemed like a big productivity boost averaged out over all users x time<br></blockquote><div><br>Is that 10 minutes of running tablegen, though? I think this touches on Hal's comment:<br>"<span style="color:rgb(33,33,33);font-size:13px">Maybe we should use a diff-and-update approach (I thought, however, that we already did that)."<br></span><br>In that the 10 minutes is presumably from timestamp-based invalidation chaining down from running tablegen, to then rebuilding everything that depends on tablegen's output.<br><br>That could be addressed by either the build system or tablegen writing output to a temporary file, comparing the temp to the current file, and not modifying the current file if they're the same - thus not invalidating everything down the chain that depends on the tablegen output (not sure if that's actually enough to convince a build system not to rebuild other things - but worth a shot). This is a bit of a hackaround what the build system itself could/should be doing, but in the absence of a content based invalidation scheme - addressing this one particularly critical point in LLVM's build seems like it could be worth such a workaround. (this would speed up the build for even the times when tablegen's real dependencies change - add a new function to ArrayRef, or STLExtras, etc... and tablegen runs, but it doesn't cascade to everything else). <br><br>(plus, opt build of tablegen, so that conclusion can be reached more quickly)<br><br>As Hal also mentioned, I'd be OK with this if there's a good logical division that can be taken more than "whatever is/isn't used in tablegen".<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br>We've got a lot of random stuff in Support, not all really related and not all relevant to every project. It seems like we should be able to do better.<br><div class="gmail_quote"><div dir="ltr">On Fri, May 26, 2017 at 6:48 PM Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<div class="m_3422182665239432404m_-6805375126177304137moz-cite-prefix">On 05/26/2017 07:59 PM, Zachary Turner
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">It's that TableGen depends on Support, so if you
change one file in support, support gets recompiled into a new
static archive, which triggers a rerun of tablegen on all the
tablegen inputs, which is extremely slow.</div>
</blockquote>
<br></div><div bgcolor="#FFFFFF" text="#000000">
I thought that we had a "build an optimized tablegen even in debug
mode" setting to work around this problem. Would that avoid this
problem?</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
-Hal</div><div bgcolor="#FFFFFF" text="#000000"><br>
<br>
<blockquote type="cite"><br>
<div class="gmail_quote">
<div dir="ltr">On Fri, May 26, 2017 at 5:56 PM Hal Finkel <<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p><br>
</p>
<br>
<div class="m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-cite-prefix">On
05/26/2017 07:47 PM, Zachary Turner via llvm-dev wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Changing a header file somewhere and having
to spend 10 minutes waiting for a build leads to a lot
of wasted developer time.
<div><br>
</div>
<div>The real culprit here is tablegen. Can we split
support and ADT into two - the parts that tablegen
depends on and the parts that it doesn't?</div>
</div>
</blockquote>
<br>
</div>
<div bgcolor="#FFFFFF" text="#000000"> What's the actual
problem here? Is it that TableGen regenerates different
files and so we then need to rebuild all dependencies of
those files? Maybe we should use a diff-and-update approach
(I thought, however, that we already did that).<br>
<br>
-Hal<br>
<br>
</div>
<div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<div dir="ltr">
<div><br>
</div>
<div>From what I can gather, Tablegen currently depends
on these headers and all of their transitive
dependencies.</div>
<div><br>
</div>
<div>
<div>#include "llvm/Support/Casting.h"</div>
<div>#include "llvm/Support/CommandLine.h"</div>
<div>#include "llvm/Support/Compiler.h"</div>
<div>#include "llvm/Support/DataTypes.h"</div>
<div>#include "llvm/Support/Debug.h"</div>
<div>#include "llvm/Support/Error.h"</div>
<div>#include "llvm/Support/ErrorHandling.h"</div>
<div>#include "llvm/Support/Format.h"</div>
<div>#include "llvm/Support/FormattedStream.h"</div>
<div>#include "llvm/Support/LEB128.h"</div>
<div>#include "llvm/Support/LowLevelTypeImpl.h"</div>
<div>#include "llvm/Support/ManagedStatic.h"</div>
<div>#include "llvm/Support/MathExtras.h"</div>
<div>#include "llvm/Support/MemoryBuffer.h"</div>
<div>#include "llvm/Support/PrettyStackTrace.h"</div>
<div>#include "llvm/Support/Regex.h"</div>
<div>#include "llvm/Support/SMLoc.h"</div>
<div>#include "llvm/Support/ScopedPrinter.h"</div>
<div>#include "llvm/Support/Signals.h"</div>
<div>#include "llvm/Support/SourceMgr.h"</div>
<div>#include "llvm/Support/raw_ostream.h"</div>
<div><br>
</div>
<div>#include "llvm/ADT/APInt.h"</div>
<div>#include "llvm/ADT/ArrayRef.h"</div>
<div>#include "llvm/ADT/BitVector.h"</div>
<div>#include "llvm/ADT/CachedHashString.h"</div>
<div>#include "llvm/ADT/DenseSet.h"</div>
<div>#include "llvm/ADT/IndexedMap.h"</div>
<div>#include "llvm/ADT/IntEqClasses.h"</div>
<div>#include "llvm/ADT/MapVector.h"</div>
<div>#include "llvm/ADT/Optional.h"</div>
<div>#include "llvm/ADT/PointerUnion.h"</div>
<div>#include "llvm/ADT/STLExtras.h"</div>
<div>#include "llvm/ADT/SetVector.h"</div>
<div>#include "llvm/ADT/SmallPtrSet.h"</div>
<div>#include "llvm/ADT/SmallSet.h"</div>
<div>#include "llvm/ADT/SmallVector.h"</div>
<div>#include "llvm/ADT/SparseBitVector.h"</div>
<div>#include "llvm/ADT/Statistic.h"</div>
<div>#include "llvm/ADT/StringExtras.h"</div>
<div>#include "llvm/ADT/StringMap.h"</div>
<div>#include "llvm/ADT/StringRef.h"</div>
<div>#include "llvm/ADT/StringSet.h"</div>
<div>#include "llvm/ADT/StringSwitch.h"</div>
<div>#include "llvm/ADT/TinyPtrVector.h"</div>
<div>#include "llvm/ADT/Twine.h"</div>
</div>
<div><br>
</div>
<div>
<div><br class="m_3422182665239432404m_-6805375126177304137m_-895338599256642649inbox-inbox-Apple-interchange-newline">
Is this something worth putting effort into? If so,
I volunteer.</div>
<br class="m_3422182665239432404m_-6805375126177304137m_-895338599256642649inbox-inbox-Apple-interchange-newline">
</div>
</div>
<br>
<fieldset class="m_3422182665239432404m_-6805375126177304137m_-895338599256642649mimeAttachmentHeader"></fieldset>
<br>
</blockquote>
</div>
<div bgcolor="#FFFFFF" text="#000000">
<blockquote type="cite">
<pre>_______________________________________________
LLVM Developers mailing list
<a class="m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-txt-link-abbreviated" href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>
<a class="m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a>
</pre>
</blockquote>
</div>
<div bgcolor="#FFFFFF" text="#000000"> <br>
<pre class="m_3422182665239432404m_-6805375126177304137m_-895338599256642649moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</div>
</blockquote>
</div>
</blockquote>
<br>
<pre class="m_3422182665239432404m_-6805375126177304137moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</div></blockquote></div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div></div>