<div dir="ltr">Did you use CMake? If no, ctai requres clang 4.0 and to be compiled with flags: `<span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:12px;white-space:pre">-std=c++1z -ftemplate-backtrace-limit=0 -ftemplate-depth=8096`</span><div><span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:12px;white-space:pre"><br></span></div><div><span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:12px;white-space:pre"><br></span></div><div>stryku<span style="color:rgb(36,41,46);font-family:sfmono-regular,consolas,"liberation mono",menlo,courier,monospace;font-size:12px;white-space:pre">
</span><div class="gmail_extra"><br><div class="gmail_quote">2017-04-28 9:53 GMT+02:00 mats petersson <span dir="ltr"><<a href="mailto:mats@planetcatfish.com" target="_blank">mats@planetcatfish.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I can't (after about 10 minutes of trying) get this to compile at all - most likely because I have a too old version of C++ library installed... <br><br>--<br></div>Mats<br></div><div class="gmail-HOEnZb"><div class="gmail-h5"><div class="gmail_extra"><br><div class="gmail_quote">On 27 April 2017 at 22:33, mateusz janek <span dir="ltr"><<a href="mailto:stryku2393@gmail.com" target="_blank">stryku2393@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">My bad, I was sure that I'm "replying all", thanks for that.<br><br>Of course I don't want you to feel any kind of pressure from my side. I'm just curious if I can somehow adjust llvm/clang to help me with such problem.<div><br></div><div>I'll try to provide MCVE, but unfortunately probably tomorrow.</div></div><div class="gmail-m_3339437415993011794HOEnZb"><div class="gmail-m_3339437415993011794h5"><div class="gmail_extra"><br><div class="gmail_quote">2017-04-27 21:56 GMT+02:00 mats petersson <span dir="ltr"><<a href="mailto:mats@planetcatfish.com" target="_blank">mats@planetcatfish.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><div><div>It would be good if you did "reply all" so that those other people that have read and is interested in this discussion can see the answers. I've added Brian and the mailing list back into this reply.<br><br></div>That is an impressive amount of template-code. I've never really done much TMP, but I will see if I can at least somewhat analyse where the memory/time goes in this.<br><br></div>It would probably have been better to have a SMALL example, that compiles in a few seconds to a few minutes, and a "plain" program that achieves roughly the same thing, but I'll see what I can do. <br><br></div>When I get home (it's about 9pm, and I'm still at work), and I make no promises of ANY kind...<br><br>--<br></div>Mats<div><div class="gmail-m_3339437415993011794m_-9038998629006721406h5"><br><div class="gmail_extra"><br><div class="gmail_quote">On 27 April 2017 at 18:51, mateusz janek <span dir="ltr"><<a href="mailto:stryku2393@gmail.com" target="_blank">stryku2393@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div>Thanks for the replies (:</div><div><br></div><div>@Brian Cain</div><span><div>>Do other compilers behave similarly in terms if memory consumption for your example code?</div></span><div>Good point. I'll check that today with the g++.</div><div><br></div><div><br></div><div>@Mats Petersson</div><span><div>>You will need ALL types (that are used at all) to be present at semantic analysis, since types are used to determine for example implicit casts (float to int, int to float, short to int, int to short - and even if your type can't be converted, it still needs to be present during that phase). Since Semantic analysis is done at the end of the compilation of a translation unit. And of course, types are used during translation from AST to IR, so need to be (at least somewhat) present during the IR generation phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to find `Bar<FooParam>::BarResult`, even if that is then translated to `size_t` or `int`.</div></span><div>Thanks a lot for an explanation (:</div><span><div><br></div><div>>Also, are you sure that it's the number of types that is the real cause of the problem?</div></span><div>Yes, I'm pretty sure. Almost everything in this project is a template type.</div><span><div><br></div><div>>Can you provide a more complete example (preferrably one that compiles, but does not take "over night", in a few minutes would be good, that illustrates that it uses a lot more memory than a comparable piece of code with less types introduced in it).</div></span><div>Of course. Check the repo: <a href="https://github.com/stryku/ctai" target="_blank">https://github.com/stryku/ctai</a> (Not sure if you've asked for this or for a complete new code which illustrates the problem, if the new code - sorry and let me know I'll try to prepare something)</div><div>There is still v1.0 version and it compiles just fine in about 1-2s. To increase the time and memory usage you can just add a couple of dummy asm instructions in main.cpp, e.g.:</div><div><br></div><div>```</div><div>using code = decltype(</div><div> "mov ebp , esp "</div><div> "push eax " // declare fourusing code = decltype(</div><div> "mov ebp , esp "</div><div> "push eax " // declare four variables</div><div> "push eax "</div><div> "push eax "</div><div> "push eax "</div><div> "mov DWORD PTR [ ebp + 8 ] , 0 "</div><div> "mov DWORD PTR [ ebp + 12 ] , 1 "</div><div> "mov DWORD PTR [ ebp + 16 ] , 1 "</div><div> "mov DWORD PTR [ ebp + 4 ] , 1 "</div><div>":loop_label "</div><div> "mov eax , DWORD PTR [ ebp + 4 ] "</div><div> "mov ebx , 15 " //we want to get 15th fibonacci element</div><div> "cmp eax , ebx "</div><div> "jge .end_label "</div><div> "mov edx , DWORD PTR [ ebp + 8 ] "</div><div> "mov eax , DWORD PTR [ ebp + 12 ] "</div><div> "add eax , edx "</div><div> "mov DWORD PTR [ ebp + 16 ] , eax "</div><div> "mov eax , DWORD PTR [ ebp + 12 ] "</div><div> "mov DWORD PTR [ ebp + 8 ] , eax "</div><div> "mov eax , DWORD PTR [ ebp + 16 ] "</div><div> "mov DWORD PTR [ ebp + 12 ] , eax "</div><div> "mov eax , DWORD PTR [ ebp + 4 ] "</div><div> "mov ebx , 1 "</div><div> "add eax , ebx "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "mov DWORD PTR [ ebp + 4 ] , eax "</div><div> "jmp .loop_label "</div><div>":end_label "</div><div> "mov eax , DWORD PTR [ ebp + 16 ] "</div><div> "exit"_s);</div><div>```</div><div><br></div><div>Version v1.0, from the github is simpler and works in a more naive way (e.g. every asm instruction is recursively expanded) but basically, it has same problems as the current version - there is a lot of types.</div><div><br></div><div>I know that I can optimize the template code, but I'm sure that at some point I'll hit the barrier and the only way will be to modify llvm/clang, which I'm trying to understand and eventually do.</div><div><br></div><div>Thanks,</div><div>Stryku</div></div><div><div class="gmail-m_3339437415993011794m_-9038998629006721406m_1785383655602821771m_7714440731079299381h5"><div class="gmail_extra"><br></div><div class="gmail_extra"><br></div><div class="gmail_extra"><br><div class="gmail_quote">2017-04-27 10:54 GMT+02:00 mats petersson <span dir="ltr"><<a href="mailto:mats@planetcatfish.com" target="_blank">mats@planetcatfish.com</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><span class="gmail-m_3339437415993011794m_-9038998629006721406m_1785383655602821771m_7714440731079299381m_-7329981449535212249gmail-">On 27 April 2017 at 03:13, Brian Cain via cfe-dev <span dir="ltr"><<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Do other compilers behave similarly in terms if memory consumption for your example code?</div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-m_3339437415993011794m_-9038998629006721406m_1785383655602821771m_7714440731079299381m_-7329981449535212249gmail-m_-4670047485408668149h5">On Apr 26, 2017 4:20 PM, "mateusz janek via cfe-dev" <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br type="attribution"></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-m_3339437415993011794m_-9038998629006721406m_1785383655602821771m_7714440731079299381m_-7329981449535212249gmail-m_-4670047485408668149h5"><div dir="ltr"><div>Hi,</div><div><br></div><div>Before I'll begin, I want to say that I don't want to implement below 'feature' in a clang production code. I want to create my own custom build for my own needs.</div><div><br></div><div>Abstract:</div><div>I'm developing a hobby project which creates a lot of types in a compile time, in one .cpp file. Really a lot. I tried to compile next version, but it was compiling whole night and took ~30G of memory (16G RAM + swap) and wasn't able to finish. I know that almost all of these types are 'used' only once, they're kind of helpers.</div><div>Basically what I want to achieve is to decrease memory usage to not need to use the swap, because it increases compilation time a lot.</div><div><br></div><div>Question:</div><div>Is there a way to implement a removing no longer used types? E.g.</div><div><br></div><div>template <typename FooParam></div><div>struct Foo</div><div>{</div><div> using FooResult = typename Bar<FooParam>::BarResult;</div><div>};</div><div><br></div><div>And I'd want to remove the 'Bar<FooParam>' type from a 'ASTContext' because I know that I won't need it anymore. (then, if there will again occur an initialization of very same 'Bar<FooParam>', I'm ok with that I'll need to create it from the scratch).</div><div>So basically is there any way to detect if type is still referenced somewhere, so I can remove it if it isn't?</div></div></div></div></blockquote></div></div></blockquote><div><br></div></span><div>You will need ALL types (that are used at all) to be present at semantic analysis, since types are used to determine for example implicit casts (float to int, int to float, short to int, int to short - and even if your type can't be converted, it still needs to be present during that phase). Since Semantic analysis is done at the end of the compilation of a translation unit. And of course, types are used during translation from AST to IR, so need to be (at least somewhat) present during the IR generation phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to find `Bar<FooParam>::BarResult`, even if that is then translated to `size_t` or `int`.<br><br></div><div>Also, are you sure that it's the number of types that is the real cause of the problem?<br><br></div><div>Can you provide a more complete example (preferrably one that compiles, but does not take "over night", in a few minutes would be good, that illustrates that it uses a lot more memory than a comparable piece of code with less types introduced in it).<br><br>--<br></div><div>Mats<br></div><span class="gmail-m_3339437415993011794m_-9038998629006721406m_1785383655602821771m_7714440731079299381m_-7329981449535212249gmail-"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-m_3339437415993011794m_-9038998629006721406m_1785383655602821771m_7714440731079299381m_-7329981449535212249gmail-m_-4670047485408668149h5"><div dir="ltr"><div><br></div><div>I hope I've described my problem well. I know that in fact I'm asking for a explanation how types in clang are related to each other - that's a lot, but I'd really like to learn and understand that (:</div><div><br></div><div>Thanks in advance for help!</div><div><br></div><div>Stryku</div></div>
<br></div></div>______________________________<wbr>_________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
<br></blockquote></div></div>
<br>______________________________<wbr>_________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/cfe-dev</a><br>
<br></blockquote></span></div><br></div></div>
</blockquote></div><br></div></div></div></div>
</blockquote></div><br></div></div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br></div></div></div>