[cfe-dev] Remove unused types from ASTContext
mats petersson via cfe-dev
cfe-dev at lists.llvm.org
Fri Apr 28 07:57:32 PDT 2017
Yes, using cmake. using clang 4.0 (pre-release). But I don't have a
particularly new C++ library installed on my machine (what comes with gcc
4.9.2, to be precise), and that's complaining about "has_facet" (from
memory, I'm not at home right now) somewhere inside the iostream (again,
from memory) header.
--
Mats
On 28 April 2017 at 12:11, mateusz janek <stryku2393 at gmail.com> wrote:
> Did you use CMake? If no, ctai requres clang 4.0 and to be compiled with
> flags: `-std=c++1z -ftemplate-backtrace-limit=0 -ftemplate-depth=8096`
>
>
> stryku
>
> 2017-04-28 9:53 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>
>> I can't (after about 10 minutes of trying) get this to compile at all -
>> most likely because I have a too old version of C++ library installed...
>>
>> --
>> Mats
>>
>> On 27 April 2017 at 22:33, mateusz janek <stryku2393 at gmail.com> wrote:
>>
>>> My bad, I was sure that I'm "replying all", thanks for that.
>>>
>>> Of course I don't want you to feel any kind of pressure from my side.
>>> I'm just curious if I can somehow adjust llvm/clang to help me with such
>>> problem.
>>>
>>> I'll try to provide MCVE, but unfortunately probably tomorrow.
>>>
>>> 2017-04-27 21:56 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>>>
>>>> It would be good if you did "reply all" so that those other people that
>>>> have read and is interested in this discussion can see the answers. I've
>>>> added Brian and the mailing list back into this reply.
>>>>
>>>> That is an impressive amount of template-code. I've never really done
>>>> much TMP, but I will see if I can at least somewhat analyse where the
>>>> memory/time goes in this.
>>>>
>>>> It would probably have been better to have a SMALL example, that
>>>> compiles in a few seconds to a few minutes, and a "plain" program that
>>>> achieves roughly the same thing, but I'll see what I can do.
>>>>
>>>> When I get home (it's about 9pm, and I'm still at work), and I make no
>>>> promises of ANY kind...
>>>>
>>>> --
>>>> Mats
>>>>
>>>>
>>>> On 27 April 2017 at 18:51, mateusz janek <stryku2393 at gmail.com> wrote:
>>>>
>>>>> Thanks for the replies (:
>>>>>
>>>>> @Brian Cain
>>>>> >Do other compilers behave similarly in terms if memory consumption
>>>>> for your example code?
>>>>> Good point. I'll check that today with the g++.
>>>>>
>>>>>
>>>>> @Mats Petersson
>>>>> >You will need ALL types (that are used at all) to be present at
>>>>> semantic analysis, since types are used to determine for example implicit
>>>>> casts (float to int, int to float, short to int, int to short - and even if
>>>>> your type can't be converted, it still needs to be present during that
>>>>> phase). Since Semantic analysis is done at the end of the compilation of a
>>>>> translation unit. And of course, types are used during translation from AST
>>>>> to IR, so need to be (at least somewhat) present during the IR generation
>>>>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>>>>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>>>>> `size_t` or `int`.
>>>>> Thanks a lot for an explanation (:
>>>>>
>>>>> >Also, are you sure that it's the number of types that is the real
>>>>> cause of the problem?
>>>>> Yes, I'm pretty sure. Almost everything in this project is a template
>>>>> type.
>>>>>
>>>>> >Can you provide a more complete example (preferrably one that
>>>>> compiles, but does not take "over night", in a few minutes would be good,
>>>>> that illustrates that it uses a lot more memory than a comparable piece of
>>>>> code with less types introduced in it).
>>>>> Of course. Check the repo: https://github.com/stryku/ctai (Not sure
>>>>> if you've asked for this or for a complete new code which illustrates the
>>>>> problem, if the new code - sorry and let me know I'll try to prepare
>>>>> something)
>>>>> There is still v1.0 version and it compiles just fine in about 1-2s.
>>>>> To increase the time and memory usage you can just add a couple of dummy
>>>>> asm instructions in main.cpp, e.g.:
>>>>>
>>>>> ```
>>>>> using code = decltype(
>>>>> "mov ebp , esp "
>>>>> "push eax " // declare fourusing code = decltype(
>>>>> "mov ebp , esp "
>>>>> "push eax " // declare four variables
>>>>> "push eax "
>>>>> "push eax "
>>>>> "push eax "
>>>>> "mov DWORD PTR [ ebp + 8 ] , 0 "
>>>>> "mov DWORD PTR [ ebp + 12 ] , 1 "
>>>>> "mov DWORD PTR [ ebp + 16 ] , 1 "
>>>>> "mov DWORD PTR [ ebp + 4 ] , 1 "
>>>>> ":loop_label "
>>>>> "mov eax , DWORD PTR [ ebp + 4 ] "
>>>>> "mov ebx , 15 " //we want to get 15th fibonacci element
>>>>> "cmp eax , ebx "
>>>>> "jge .end_label "
>>>>> "mov edx , DWORD PTR [ ebp + 8 ] "
>>>>> "mov eax , DWORD PTR [ ebp + 12 ] "
>>>>> "add eax , edx "
>>>>> "mov DWORD PTR [ ebp + 16 ] , eax "
>>>>> "mov eax , DWORD PTR [ ebp + 12 ] "
>>>>> "mov DWORD PTR [ ebp + 8 ] , eax "
>>>>> "mov eax , DWORD PTR [ ebp + 16 ] "
>>>>> "mov DWORD PTR [ ebp + 12 ] , eax "
>>>>> "mov eax , DWORD PTR [ ebp + 4 ] "
>>>>> "mov ebx , 1 "
>>>>> "add eax , ebx "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>>> "jmp .loop_label "
>>>>> ":end_label "
>>>>> "mov eax , DWORD PTR [ ebp + 16 ] "
>>>>> "exit"_s);
>>>>> ```
>>>>>
>>>>> Version v1.0, from the github is simpler and works in a more naive way
>>>>> (e.g. every asm instruction is recursively expanded) but basically, it has
>>>>> same problems as the current version - there is a lot of types.
>>>>>
>>>>> I know that I can optimize the template code, but I'm sure that at
>>>>> some point I'll hit the barrier and the only way will be to modify
>>>>> llvm/clang, which I'm trying to understand and eventually do.
>>>>>
>>>>> Thanks,
>>>>> Stryku
>>>>>
>>>>>
>>>>>
>>>>> 2017-04-27 10:54 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 27 April 2017 at 03:13, Brian Cain via cfe-dev <
>>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>> Do other compilers behave similarly in terms if memory consumption
>>>>>>> for your example code?
>>>>>>>
>>>>>>> On Apr 26, 2017 4:20 PM, "mateusz janek via cfe-dev" <
>>>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> Before I'll begin, I want to say that I don't want to implement
>>>>>>>> below 'feature' in a clang production code. I want to create my own custom
>>>>>>>> build for my own needs.
>>>>>>>>
>>>>>>>> Abstract:
>>>>>>>> I'm developing a hobby project which creates a lot of types in a
>>>>>>>> compile time, in one .cpp file. Really a lot. I tried to compile next
>>>>>>>> version, but it was compiling whole night and took ~30G of memory (16G RAM
>>>>>>>> + swap) and wasn't able to finish. I know that almost all of these types
>>>>>>>> are 'used' only once, they're kind of helpers.
>>>>>>>> Basically what I want to achieve is to decrease memory usage to not
>>>>>>>> need to use the swap, because it increases compilation time a lot.
>>>>>>>>
>>>>>>>> Question:
>>>>>>>> Is there a way to implement a removing no longer used types? E.g.
>>>>>>>>
>>>>>>>> template <typename FooParam>
>>>>>>>> struct Foo
>>>>>>>> {
>>>>>>>> using FooResult = typename Bar<FooParam>::BarResult;
>>>>>>>> };
>>>>>>>>
>>>>>>>> And I'd want to remove the 'Bar<FooParam>' type from a 'ASTContext'
>>>>>>>> because I know that I won't need it anymore. (then, if there will again
>>>>>>>> occur an initialization of very same 'Bar<FooParam>', I'm ok with that I'll
>>>>>>>> need to create it from the scratch).
>>>>>>>> So basically is there any way to detect if type is still referenced
>>>>>>>> somewhere, so I can remove it if it isn't?
>>>>>>>>
>>>>>>>
>>>>>> You will need ALL types (that are used at all) to be present at
>>>>>> semantic analysis, since types are used to determine for example implicit
>>>>>> casts (float to int, int to float, short to int, int to short - and even if
>>>>>> your type can't be converted, it still needs to be present during that
>>>>>> phase). Since Semantic analysis is done at the end of the compilation of a
>>>>>> translation unit. And of course, types are used during translation from AST
>>>>>> to IR, so need to be (at least somewhat) present during the IR generation
>>>>>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>>>>>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>>>>>> `size_t` or `int`.
>>>>>>
>>>>>> Also, are you sure that it's the number of types that is the real
>>>>>> cause of the problem?
>>>>>>
>>>>>> Can you provide a more complete example (preferrably one that
>>>>>> compiles, but does not take "over night", in a few minutes would be good,
>>>>>> that illustrates that it uses a lot more memory than a comparable piece of
>>>>>> code with less types introduced in it).
>>>>>>
>>>>>> --
>>>>>> Mats
>>>>>>
>>>>>>>
>>>>>>>> I hope I've described my problem well. I know that in fact I'm
>>>>>>>> asking for a explanation how types in clang are related to each other -
>>>>>>>> that's a lot, but I'd really like to learn and understand that (:
>>>>>>>>
>>>>>>>> Thanks in advance for help!
>>>>>>>>
>>>>>>>> Stryku
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> cfe-dev mailing list
>>>>>>>> cfe-dev at lists.llvm.org
>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> cfe-dev mailing list
>>>>>>> cfe-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170428/6c967313/attachment.html>
More information about the cfe-dev
mailing list