[llvm-dev] Should we split llvm Support and ADT?

Sat May 27 12:50:49 PDT 2017

I thought we already did write tablegen output to temporary files like
X86GenAsmWriter1.inc.tmp first and then diffed them with the real .inc file
and conditionally copied.

~Craig

On Sat, May 27, 2017 at 11:02 AM, David Blaikie via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

>
>
> On Fri, May 26, 2017 at 8:06 PM Zachary Turner via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>> It would be better, because a debug tablegen is slower than an optimized
>> tablegen, but it's still slow and it doesn't address the problem that
>> tablegen runs *at all* when it doesn't really need to. I think if tablegen
>> wasn't running all the time we could incremental builds down from 15
>> minutes (and that's on my really powerful machine) to under 5, which seemed
>> like a big productivity boost averaged out over all users x time
>>
>
> Is that 10 minutes of running tablegen, though? I think this touches on
> Hal's comment:
> "Maybe we should use a diff-and-update approach (I thought, however, that
> we already did that)."
>
> In that the 10 minutes is presumably from timestamp-based invalidation
> chaining down from running tablegen, to then rebuilding everything that
> depends on tablegen's output.
>
> That could be addressed by either the build system or tablegen writing
> output to a temporary file, comparing the temp to the current file, and not
> modifying the current file if they're the same - thus not invalidating
> everything down the chain that depends on the tablegen output (not sure if
> that's actually enough to convince a build system not to rebuild other
> things - but worth a shot). This is a bit of a hackaround what the build
> system itself could/should be doing, but in the absence of a content based
> invalidation scheme - addressing this one particularly critical point in
> LLVM's build seems like it could be worth such a workaround. (this would
> speed up the build for even the times when tablegen's real dependencies
> change - add a new function to ArrayRef, or STLExtras, etc... and tablegen
> runs, but it doesn't cascade to everything else).
>
> (plus, opt build of tablegen, so that conclusion can be reached more
> quickly)
>
> As Hal also mentioned, I'd be OK with this if there's a good logical
> division that can be taken more than "whatever is/isn't used in tablegen".
>
>
>>
>> We've got a lot of random stuff in Support, not all really related and
>> not all relevant to every project. It seems like we should be able to do
>> better.
>> On Fri, May 26, 2017 at 6:48 PM Hal Finkel <hfinkel at anl.gov> wrote:
>>
>>>
>>> On 05/26/2017 07:59 PM, Zachary Turner wrote:
>>>
>>> It's that TableGen depends on Support, so if you change one file in
>>> support, support gets recompiled into a new static archive, which triggers
>>> a rerun of tablegen on all the tablegen inputs, which is extremely slow.
>>>
>>>
>>> I thought that we had a "build an optimized tablegen even in debug mode"
>>> setting to work around this problem. Would that avoid this problem?
>>>
>>>
>>>  -Hal
>>>
>>>
>>>
>>> On Fri, May 26, 2017 at 5:56 PM Hal Finkel <hfinkel at anl.gov> wrote:
>>>
>>>>
>>>>
>>>> On 05/26/2017 07:47 PM, Zachary Turner via llvm-dev wrote:
>>>>
>>>> Changing a header file somewhere and having to spend 10 minutes waiting
>>>> for a build leads to a lot of wasted developer time.
>>>>
>>>> The real culprit here is tablegen.  Can we split support and ADT into
>>>> two - the parts that tablegen depends on and the parts that it doesn't?
>>>>
>>>>
>>>> What's the actual problem here? Is it that TableGen regenerates
>>>> different files and so we then need to rebuild all dependencies of those
>>>> files? Maybe we should use a diff-and-update approach (I thought, however,
>>>> that we already did that).
>>>>
>>>>  -Hal
>>>>
>>>>
>>>> From what I can gather, Tablegen currently depends on these headers and
>>>> all of their transitive dependencies.
>>>>
>>>> #include "llvm/Support/Casting.h"
>>>> #include "llvm/Support/CommandLine.h"
>>>> #include "llvm/Support/Compiler.h"
>>>> #include "llvm/Support/DataTypes.h"
>>>> #include "llvm/Support/Debug.h"
>>>> #include "llvm/Support/Error.h"
>>>> #include "llvm/Support/ErrorHandling.h"
>>>> #include "llvm/Support/Format.h"
>>>> #include "llvm/Support/FormattedStream.h"
>>>> #include "llvm/Support/LEB128.h"
>>>> #include "llvm/Support/LowLevelTypeImpl.h"
>>>> #include "llvm/Support/ManagedStatic.h"
>>>> #include "llvm/Support/MathExtras.h"
>>>> #include "llvm/Support/MemoryBuffer.h"
>>>> #include "llvm/Support/PrettyStackTrace.h"
>>>> #include "llvm/Support/Regex.h"
>>>> #include "llvm/Support/SMLoc.h"
>>>> #include "llvm/Support/ScopedPrinter.h"
>>>> #include "llvm/Support/Signals.h"
>>>> #include "llvm/Support/SourceMgr.h"
>>>> #include "llvm/Support/raw_ostream.h"
>>>>
>>>> #include "llvm/ADT/APInt.h"
>>>> #include "llvm/ADT/ArrayRef.h"
>>>> #include "llvm/ADT/BitVector.h"
>>>> #include "llvm/ADT/CachedHashString.h"
>>>> #include "llvm/ADT/DenseSet.h"
>>>> #include "llvm/ADT/IndexedMap.h"
>>>> #include "llvm/ADT/IntEqClasses.h"
>>>> #include "llvm/ADT/MapVector.h"
>>>> #include "llvm/ADT/Optional.h"
>>>> #include "llvm/ADT/PointerUnion.h"
>>>> #include "llvm/ADT/STLExtras.h"
>>>> #include "llvm/ADT/SetVector.h"
>>>> #include "llvm/ADT/SmallPtrSet.h"
>>>> #include "llvm/ADT/SmallSet.h"
>>>> #include "llvm/ADT/SmallVector.h"
>>>> #include "llvm/ADT/SparseBitVector.h"
>>>> #include "llvm/ADT/Statistic.h"
>>>> #include "llvm/ADT/StringExtras.h"
>>>> #include "llvm/ADT/StringMap.h"
>>>> #include "llvm/ADT/StringRef.h"
>>>> #include "llvm/ADT/StringSet.h"
>>>> #include "llvm/ADT/StringSwitch.h"
>>>> #include "llvm/ADT/TinyPtrVector.h"
>>>> #include "llvm/ADT/Twine.h"
>>>>
>>>>
>>>> Is this something worth putting effort into?  If so, I volunteer.
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing listllvm-dev at lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>
>>>>
>>>> --
>>>> Hal Finkel
>>>> Lead, Compiler Technology and Programming Languages
>>>> Leadership Computing Facility
>>>> Argonne National Laboratory
>>>>
>>>>
>>> --
>>> Hal Finkel
>>> Lead, Compiler Technology and Programming Languages
>>> Leadership Computing Facility
>>> Argonne National Laboratory
>>>
>>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170527/2b4c7383/attachment.html>