[llvm-dev] Should we split llvm Support and ADT?

Sat May 27 12:54:58 PDT 2017

Maybe we do and build systems aren't respecting/noticing this? I'm not sure.

On Sat, May 27, 2017 at 12:50 PM Craig Topper <craig.topper at gmail.com>
wrote:

> I thought we already did write tablegen output to temporary files like
> X86GenAsmWriter1.inc.tmp first and then diffed them with the real .inc file
> and conditionally copied.
>
> ~Craig
>
> On Sat, May 27, 2017 at 11:02 AM, David Blaikie via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
>>
>>
>> On Fri, May 26, 2017 at 8:06 PM Zachary Turner via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> It would be better, because a debug tablegen is slower than an optimized
>>> tablegen, but it's still slow and it doesn't address the problem that
>>> tablegen runs *at all* when it doesn't really need to. I think if tablegen
>>> wasn't running all the time we could incremental builds down from 15
>>> minutes (and that's on my really powerful machine) to under 5, which seemed
>>> like a big productivity boost averaged out over all users x time
>>>
>>
>> Is that 10 minutes of running tablegen, though? I think this touches on
>> Hal's comment:
>> "Maybe we should use a diff-and-update approach (I thought, however,
>> that we already did that)."
>>
>> In that the 10 minutes is presumably from timestamp-based invalidation
>> chaining down from running tablegen, to then rebuilding everything that
>> depends on tablegen's output.
>>
>> That could be addressed by either the build system or tablegen writing
>> output to a temporary file, comparing the temp to the current file, and not
>> modifying the current file if they're the same - thus not invalidating
>> everything down the chain that depends on the tablegen output (not sure if
>> that's actually enough to convince a build system not to rebuild other
>> things - but worth a shot). This is a bit of a hackaround what the build
>> system itself could/should be doing, but in the absence of a content based
>> invalidation scheme - addressing this one particularly critical point in
>> LLVM's build seems like it could be worth such a workaround. (this would
>> speed up the build for even the times when tablegen's real dependencies
>> change - add a new function to ArrayRef, or STLExtras, etc... and tablegen
>> runs, but it doesn't cascade to everything else).
>>
>> (plus, opt build of tablegen, so that conclusion can be reached more
>> quickly)
>>
>> As Hal also mentioned, I'd be OK with this if there's a good logical
>> division that can be taken more than "whatever is/isn't used in tablegen".
>>
>>
>>>
>>> We've got a lot of random stuff in Support, not all really related and
>>> not all relevant to every project. It seems like we should be able to do
>>> better.
>>> On Fri, May 26, 2017 at 6:48 PM Hal Finkel <hfinkel at anl.gov> wrote:
>>>
>>>>
>>>> On 05/26/2017 07:59 PM, Zachary Turner wrote:
>>>>
>>>> It's that TableGen depends on Support, so if you change one file in
>>>> support, support gets recompiled into a new static archive, which triggers
>>>> a rerun of tablegen on all the tablegen inputs, which is extremely slow.
>>>>
>>>>
>>>> I thought that we had a "build an optimized tablegen even in debug
>>>> mode" setting to work around this problem. Would that avoid this problem?
>>>>
>>>>
>>>>  -Hal
>>>>
>>>>
>>>>
>>>> On Fri, May 26, 2017 at 5:56 PM Hal Finkel <hfinkel at anl.gov> wrote:
>>>>
>>>>>
>>>>>
>>>>> On 05/26/2017 07:47 PM, Zachary Turner via llvm-dev wrote:
>>>>>
>>>>> Changing a header file somewhere and having to spend 10 minutes
>>>>> waiting for a build leads to a lot of wasted developer time.
>>>>>
>>>>> The real culprit here is tablegen.  Can we split support and ADT into
>>>>> two - the parts that tablegen depends on and the parts that it doesn't?
>>>>>
>>>>>
>>>>> What's the actual problem here? Is it that TableGen regenerates
>>>>> different files and so we then need to rebuild all dependencies of those
>>>>> files? Maybe we should use a diff-and-update approach (I thought, however,
>>>>> that we already did that).
>>>>>
>>>>>  -Hal
>>>>>
>>>>>
>>>>> From what I can gather, Tablegen currently depends on these headers
>>>>> and all of their transitive dependencies.
>>>>>
>>>>> #include "llvm/Support/Casting.h"
>>>>> #include "llvm/Support/CommandLine.h"
>>>>> #include "llvm/Support/Compiler.h"
>>>>> #include "llvm/Support/DataTypes.h"
>>>>> #include "llvm/Support/Debug.h"
>>>>> #include "llvm/Support/Error.h"
>>>>> #include "llvm/Support/ErrorHandling.h"
>>>>> #include "llvm/Support/Format.h"
>>>>> #include "llvm/Support/FormattedStream.h"
>>>>> #include "llvm/Support/LEB128.h"
>>>>> #include "llvm/Support/LowLevelTypeImpl.h"
>>>>> #include "llvm/Support/ManagedStatic.h"
>>>>> #include "llvm/Support/MathExtras.h"
>>>>> #include "llvm/Support/MemoryBuffer.h"
>>>>> #include "llvm/Support/PrettyStackTrace.h"
>>>>> #include "llvm/Support/Regex.h"
>>>>> #include "llvm/Support/SMLoc.h"
>>>>> #include "llvm/Support/ScopedPrinter.h"
>>>>> #include "llvm/Support/Signals.h"
>>>>> #include "llvm/Support/SourceMgr.h"
>>>>> #include "llvm/Support/raw_ostream.h"
>>>>>
>>>>> #include "llvm/ADT/APInt.h"
>>>>> #include "llvm/ADT/ArrayRef.h"
>>>>> #include "llvm/ADT/BitVector.h"
>>>>> #include "llvm/ADT/CachedHashString.h"
>>>>> #include "llvm/ADT/DenseSet.h"
>>>>> #include "llvm/ADT/IndexedMap.h"
>>>>> #include "llvm/ADT/IntEqClasses.h"
>>>>> #include "llvm/ADT/MapVector.h"
>>>>> #include "llvm/ADT/Optional.h"
>>>>> #include "llvm/ADT/PointerUnion.h"
>>>>> #include "llvm/ADT/STLExtras.h"
>>>>> #include "llvm/ADT/SetVector.h"
>>>>> #include "llvm/ADT/SmallPtrSet.h"
>>>>> #include "llvm/ADT/SmallSet.h"
>>>>> #include "llvm/ADT/SmallVector.h"
>>>>> #include "llvm/ADT/SparseBitVector.h"
>>>>> #include "llvm/ADT/Statistic.h"
>>>>> #include "llvm/ADT/StringExtras.h"
>>>>> #include "llvm/ADT/StringMap.h"
>>>>> #include "llvm/ADT/StringRef.h"
>>>>> #include "llvm/ADT/StringSet.h"
>>>>> #include "llvm/ADT/StringSwitch.h"
>>>>> #include "llvm/ADT/TinyPtrVector.h"
>>>>> #include "llvm/ADT/Twine.h"
>>>>>
>>>>>
>>>>> Is this something worth putting effort into?  If so, I volunteer.
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing listllvm-dev at lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>
>>>>>
>>>>> --
>>>>> Hal Finkel
>>>>> Lead, Compiler Technology and Programming Languages
>>>>> Leadership Computing Facility
>>>>> Argonne National Laboratory
>>>>>
>>>>>
>>>> --
>>>> Hal Finkel
>>>> Lead, Compiler Technology and Programming Languages
>>>> Leadership Computing Facility
>>>> Argonne National Laboratory
>>>>
>>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170527/88042490/attachment.html>