[llvm-dev] Tablegen ridiculously slow when compiling for Debug

David Blaikie via llvm-dev llvm-dev at lists.llvm.org
Mon Jul 1 12:57:55 PDT 2019 

If someone can manage it, it wouldn't be a bad thing - obviously open up
more parallelism (I don't know how much of LLVM can be built before you hit
everything that needs tblgen run - I guess libSupport and some other bits)

On Mon, Jul 1, 2019 at 12:54 PM Zakharin, Vyacheslav P via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> [resending to the whole list]
>
>
>
> I wonder if we can stop rebuilding TD files unconditionally, i.e. generate
> dependencies for TD files based on include directives and just allow the
> build system do its job?  Would that solve most of the build time issues?
>
>
>
> Thanks,
>
> Slava
>
>
>
>
>
> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of *Chris
> Bieneman via llvm-dev
> *Sent:* Monday, July 1, 2019 10:35 AM
> *To:* Joan Lluch <joan.lluch at icloud.com>
> *Cc:* llvm-dev <llvm-dev at lists.llvm.org>
> *Subject:* Re: [llvm-dev] Tablegen ridiculously slow when compiling for
> Debug
>
>
>
> Hey Joan,
>
>
>
> When looking for build support it is really useful to include a bunch of
> information about your build up front. Knowing that you are on macOS, and
> using the Xcode generator are really useful.
>
>
>
> On macOS, BUILD_SHARED_LIBS won’t really help much because the default
> linker (ld64) is pretty good. Using an IDE generator and setting
> LLVM_USE_OPTIMIZED_TABLEGEN will kill your release builds.
>
>
>
> In general Xcode takes 2x-3x longer than Ninja for incremental builds, and
> 1.5x-2x as long for clean builds. Lots of people use the Xcode generator to
> create a project file for navigation and editing, but most people I know
> doing LLVM development on macOS use Ninja for their builds.
>
>
>
> -Chris
>
>
> On Jun 30, 2019, at 12:18 PM, Joan Lluch via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Hi Greg,
>
>
>
> I tried to setup Ninja before on my mac but I mush have done something
> wrong and I didn’t manage to get it work. I’m not familiarised at all with
> the procedures involved. I may try that again to see If I have some luck
> though. It’s a pity that LLVM is not particularly friendly with familiar
> IDEs such as xCode on macs and Visual Studio on windows.
>
>
>
> John
>
>
>
> On 30 Jun 2019, at 17:43, Greg Bedwell <gregbedwell at gmail.com> wrote:
>
>
>
> This is also the case with the Visual Studio generators. Custom commands
> in a single cmake file essentially get written out line by line into a
> single batch file that gets processed as a custom build step. In the VS
> case this means that it can, for example, run X86 and Aarch64 tablegen
> steps in parallel with each other but all of the individual X86 invocations
> get processed serially. I can well imagine that the Xcode situation is
> similar although I've no experience with it myself to know for sure.
>
>
>
> As previously mentioned, the best solution is probably to try to adjust
> your workflow to use the Ninja (
>
> https://ninja-build.org
>
>  ) CMake generator if at all possible.  It's a bit of an adjustment but it
> does work very nicely with the LLVM build system.
>
>
>
> -Greg
>
>
>
>
>
> On Sun, 30 Jun 2019 at 12:08, Nicolai Hähnle via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Are you saying that the TableGen execution isn't parallelized? That
> seems like an obvious Xcode-specific problem...
>
> The TableGen executions are parallelized with cmake/ninja just fine.
>
> Cheers,
> Nicolai
>
> On 30.06.19 11:28, Joan Lluch via llvm-dev wrote:
> > Hi Praveen,
> >
> > Thanks for the tip, but Xcode seems to spend all the time running
> > tablegen "custom shell scripts", one by one at a time, not linking.
> > Linking is actually very fast, possibly less than a second. The
> > “scripts” that take longer are  “AArch64CommonTableGen" and
> > “AMDGPUCommonTableGen”.  As said this is on LLVM 9.0.
> >
> > However, on LLVM 7.0.1, the same process takes just 5-6 seconds in
> > total, with individual “scripts” taking significantly less than 1 second
> > each. There must be some difference between LLVM 9.0 and LLVM 7.0 that
> > might cause this (?)
> >
> > John
> >
> >
> >> On 30 Jun 2019, at 11:17, Praveen Velliengiri
> >> <praveenvelliengiri at gmail.com <mailto:praveenvelliengiri at gmail.com>>
> >> wrote:
> >>
> >> *
> >> *
> >> cmake *BUILD_SHARED_LIBS* option, it builds llvm as .so not as .a. It
> >> will use less memory during linking so you can increase the link
> >> threads and your build time will be lesser.
> >> Check this in : https://llvm.org/docs/CMake.html
> >>
> >> **
> >> **
> >>
> >> On Sun, 30 Jun 2019 at 14:42, Joan Lluch <joan.lluch at icloud.com
> >> <mailto:joan.lluch at icloud.com>> wrote:
> >>
> >>     Hi Praveen,
> >>
> >>     Please, can you elaborate on this?. What do do mean by “building
> >>     as shared objects”.
> >>
> >>     Thanks,
> >>
> >>     John
> >>
> >>
> >>
> >>>     On 30 Jun 2019, at 07:32, Praveen Velliengiri
> >>>     <praveenvelliengiri at gmail.com
> >>>     <mailto:praveenvelliengiri at gmail.com>> wrote:
> >>>
> >>>     Maybe try building llvm as a shared objects..
> >>>
> >>>     On Jun 30, 2019 1:30 AM, "Joan Lluch via llvm-dev"
> >>>     <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
> >>>
> >>>         Hi Florian,
> >>>
> >>>         Ok, I ran this:
> >>>
> >>>         cmake -S LLVM -DCMAKE_INSTALL_PREFIX=INSTALL
> >>>         -DLLVM_OPTIMIZED_TABLEGEN=On -G Xcode
> >>>
> >>>         Compiled it again from clean, and the situation is worse than
> >>>         before. Incremental builds take an incredible amount of time
> >>>         stuck in running Tablegen scripts for all targets. Now this
> >>>         happens both in Release and Debug configurations. Just before
> >>>         this, at least Release compiled fine, but that’s no longer
> >>>         the case.
> >>>
> >>>         Any other suggestions?  What could actually cause this?
> >>>
> >>>         Thanks
> >>>         John
> >>>
> >>>
> >>>
> >>>>         On 29 Jun 2019, at 19:37, Florian Hahn
> >>>>         <florian_hahn at apple.com <mailto:florian_hahn at apple.com>>
> wrote:
> >>>>
> >>>>         Hi,
> >>>>
> >>>>>         On Jun 29, 2019, at 18:26, Joan Lluch via llvm-dev
> >>>>>         <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>
> >>>>>         wrote:
> >>>>>
> >>>>>         Hi all,
> >>>>>
> >>>>>         On LLVM version 7.0.1, incremental builds are very fast for
> >>>>>         both Release and Debug. I’m compiling with Xcode
> >>>>>
> >>>>>         I recently downloaded LLVM 9.0 from the LLVM-mirror Github
> >>>>>         repository and found that Incremental "Debug” builds take a
> >>>>>         ridiculously long time due to Tablegen taking ages
> >>>>>         (literally more than 10 minutes) to generate files. This
> >>>>>         makes it totally unusable for debug purposes. However,
> >>>>>         incremental ‘Release’ builds only take a few seconds.
> >>>>>
> >>>>>         Why is that?. Any suggestions?.
> >>>>
> >>>>
> >>>>
> >>>>         You could give setting LLVM_OPTIMIZED_TABLEGEN a try
> >>>>         (https://llvm.org/docs/CMake.html).
> >>>>
> >>>>         Cheers,
> >>>>         Florian
> >>>
> >>>
> >>>         _______________________________________________
> >>>         LLVM Developers mailing list
> >>>         llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
> >>>         https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >>>
> >>
> >
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> >
>
> --
> Lerne, wie die Welt wirklich ist,
> Aber vergiss niemals, wie sie sein sollte.
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190701/9a3825b6/attachment.html>

More information about the llvm-dev mailing list