[llvm-dev] Should we split llvm Support and ADT?

[Bob Haarman via llvm-dev]

---- On Tue, 30 May 2017 13:50:13 -0700 Zachary Turner <zturner at google.com> wrote ----



On Tue, May 30, 2017 at 12:52 PM Bob Haarman <llvm at inglorion.net> wrote:



I would like to better understand how you came to conclude that the tablegen re-runs based on changes in Support are what's causing your build to be slow and what part specifically is taking all that time. I can do a clean release + assertions build of LLVM, Clang, compiler-rt and lld in about 5 minutes, plus 40 seconds to run cmake, on what I think is similar hardware to what Zach is using. If I swap Ptr += ret and Size -= Ret in raw_fd_ostream::write_impl so that Support changes, I can do an incremental build in about 10 seconds, including the regeneration of various .inc files. How do we get from there to 10 minutes for an incremental build?





For the sake of comparison, I made the same change and it took  1:58.39.  A slightly different but more intrusive change to Format.h in format_object_base::print() took 5:6.54.  A clean build on the same machine takes about 12 minutes.






Zach already mentioned that he really wants to focus on Support rather than tablegen, but I gathered some numbers so I figured I might as well share them. I did a few builds on Windows 10 with Clang as the compiler and link.exe as the linker (so not the exact same configuration as Zach, who was using the MSVC compiler). Here are the numbers:

Debug build with LLVM_OPTIMIZED_TABLEGEN=off:

clean build: about 10 minutes
rebuild after raw_ostream change: about 100 seconds

Debug build with LLVM_OPTIMIZED_TABLEGEN=on:

clean build: about 10 minutes
rebuild after raw_ostream change: about 50 seconds

RelWithDebInfo + asserts build with LLVM_OPTIMIZED_TABLEGEN=off:

clean build: about 9 minutes
rebuild after raw_ostream change: about 20 seconds

We can see that using a Debug version of tablegen really slows things down a lot. We go from 50 seconds to 100 seconds. And the 50 seconds already includes building tablegen's dependencies twice - one optimized version for the optimized tablegen, and one debug version for the rest of the build.

My understanding is that we rebuild and re-run tablegen in response to changes in Support, and so this would re-run tablegen for everything, conclude that the generated files are the same as the previously generated files, and not overwrite the generated files. In other words, a build on the same hardware and OS that takes 5 or 10 minutes isn't spending more time in tablegen than the 100-second build I did, and so spends most of its time doing things that are not tablegen.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170530/79833d8a/attachment.html>