[cfe-dev] yet another question about modules

[Richard Smith]

On Thu, Jan 22, 2015 at 5:45 PM, mobi phil <mobi at mobiphil.com> wrote:
> Hi,
>
> sorry if it is annoying, yet another email about the topic, but I am working
> hard to understand that whatabouts of the modules. Promise hereby to give
> further the wisdom once gathered ;)
>
> Tried to joggle with modules again today, to see if it can improve the build
> times of a given project. Before going to that project I just tried to build
> simple C++ files with few template instantiations. The first conclusion is
> that using the modules, build times increase instead of decreasing.
>
> I took a very trivial use case.
>
> #include <map>
> #include <string>
> using namespace std;
> typedef map<string, string> map0;
> typedef map<map0, map0> map1;
> ......
> .......
> //typedef map<map"i-1", map"i-1"> map"i";
>
> int main () {
>
> map0 m0;
> map0 n0;
> m0=n0;
> map1 m1;
> map1 n1;
> m1=n1;
> map2 m2;
> map2 n2;
> m2=n2;
> ......
> ......
> .....
> }

I'm not sure what you're trying to investigate here. You've forced the
majority of the compile time to be used while processing the .cpp file
rather than its includes, so modules is not going to help you much
here.

> for "one level" instantiation, without modules, clang seems to be a little
> faster than gcc, but for i > 3. With modules compilation is much longer.

Including or excluding the one-time cost of building the module for
your standard library? I'm seeing a speedup in this testcase by
enabling modules (with an appropriately-modularized standard library);
the proportion speedup decreases as i increases, as expected, since
more of the time is outside the headers.

> Can anybody please help to understand if modules really help to speed up
> compilation and give an idea of how?
>
> Though unrelated an even more discouraging result is that for i > 4 clang
> takes an eternity. For i=9, I stopped the process after 13 minutes
> I have 16GB on my machine (8 cores AMD), and the compiler does not seem to
> use more than 2GB of memory (from /proc/.../status: VmPeak:  2033548 kB).

This is more interesting. I'd imagine that GCC is avoiding some of the
exponential-time costs here; in particular, its overload resolution
short-circuits template argument deduction if it gets an exact match
from a non-template candidate, and that's probably happening in the
map assignments.

> for i=6 I get
>
> g++: 0,05s system 99% cpu 0,375 total
> clang++ (both from dev branch and 3.5) 11,81s user 0,06s system 99% cpu
> 11,908 total
> this means factor 30? Is clang scaling so badly?
> Please do not feel offended about this report.

Not at all, thanks for reporting your findings.

> Anyway, the reason of this email is about understanding how much modules can
> help. Let's say that I have the situation with this (i=6) and assuming
> -std=c++11 can I assume that modules will bring compilation times to
> something reasonable?

No; modules is intended to reduce the costs of including headers. It
does not reduce costs associated with template instantiations that are
performed locally to your .cpp file. (If you instantiated map0, map1,
and so on in a header, then it would help.)

In principle, we could extend the modules system with a template
instantiation repository to cache the results of instantiating
templates from modules, but I don't think anyone is working on, or
planning, such a system for Clang at the moment.