[cfe-dev] Remove unused types from ASTContext
mateusz janek via cfe-dev
cfe-dev at lists.llvm.org
Fri Apr 28 04:11:04 PDT 2017
Did you use CMake? If no, ctai requres clang 4.0 and to be compiled with
flags: `-std=c++1z -ftemplate-backtrace-limit=0 -ftemplate-depth=8096`
stryku
2017-04-28 9:53 GMT+02:00 mats petersson <mats at planetcatfish.com>:
> I can't (after about 10 minutes of trying) get this to compile at all -
> most likely because I have a too old version of C++ library installed...
>
> --
> Mats
>
> On 27 April 2017 at 22:33, mateusz janek <stryku2393 at gmail.com> wrote:
>
>> My bad, I was sure that I'm "replying all", thanks for that.
>>
>> Of course I don't want you to feel any kind of pressure from my side. I'm
>> just curious if I can somehow adjust llvm/clang to help me with such
>> problem.
>>
>> I'll try to provide MCVE, but unfortunately probably tomorrow.
>>
>> 2017-04-27 21:56 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>>
>>> It would be good if you did "reply all" so that those other people that
>>> have read and is interested in this discussion can see the answers. I've
>>> added Brian and the mailing list back into this reply.
>>>
>>> That is an impressive amount of template-code. I've never really done
>>> much TMP, but I will see if I can at least somewhat analyse where the
>>> memory/time goes in this.
>>>
>>> It would probably have been better to have a SMALL example, that
>>> compiles in a few seconds to a few minutes, and a "plain" program that
>>> achieves roughly the same thing, but I'll see what I can do.
>>>
>>> When I get home (it's about 9pm, and I'm still at work), and I make no
>>> promises of ANY kind...
>>>
>>> --
>>> Mats
>>>
>>>
>>> On 27 April 2017 at 18:51, mateusz janek <stryku2393 at gmail.com> wrote:
>>>
>>>> Thanks for the replies (:
>>>>
>>>> @Brian Cain
>>>> >Do other compilers behave similarly in terms if memory consumption for
>>>> your example code?
>>>> Good point. I'll check that today with the g++.
>>>>
>>>>
>>>> @Mats Petersson
>>>> >You will need ALL types (that are used at all) to be present at
>>>> semantic analysis, since types are used to determine for example implicit
>>>> casts (float to int, int to float, short to int, int to short - and even if
>>>> your type can't be converted, it still needs to be present during that
>>>> phase). Since Semantic analysis is done at the end of the compilation of a
>>>> translation unit. And of course, types are used during translation from AST
>>>> to IR, so need to be (at least somewhat) present during the IR generation
>>>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>>>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>>>> `size_t` or `int`.
>>>> Thanks a lot for an explanation (:
>>>>
>>>> >Also, are you sure that it's the number of types that is the real
>>>> cause of the problem?
>>>> Yes, I'm pretty sure. Almost everything in this project is a template
>>>> type.
>>>>
>>>> >Can you provide a more complete example (preferrably one that
>>>> compiles, but does not take "over night", in a few minutes would be good,
>>>> that illustrates that it uses a lot more memory than a comparable piece of
>>>> code with less types introduced in it).
>>>> Of course. Check the repo: https://github.com/stryku/ctai (Not sure if
>>>> you've asked for this or for a complete new code which illustrates the
>>>> problem, if the new code - sorry and let me know I'll try to prepare
>>>> something)
>>>> There is still v1.0 version and it compiles just fine in about 1-2s. To
>>>> increase the time and memory usage you can just add a couple of dummy asm
>>>> instructions in main.cpp, e.g.:
>>>>
>>>> ```
>>>> using code = decltype(
>>>> "mov ebp , esp "
>>>> "push eax " // declare fourusing code = decltype(
>>>> "mov ebp , esp "
>>>> "push eax " // declare four variables
>>>> "push eax "
>>>> "push eax "
>>>> "push eax "
>>>> "mov DWORD PTR [ ebp + 8 ] , 0 "
>>>> "mov DWORD PTR [ ebp + 12 ] , 1 "
>>>> "mov DWORD PTR [ ebp + 16 ] , 1 "
>>>> "mov DWORD PTR [ ebp + 4 ] , 1 "
>>>> ":loop_label "
>>>> "mov eax , DWORD PTR [ ebp + 4 ] "
>>>> "mov ebx , 15 " //we want to get 15th fibonacci element
>>>> "cmp eax , ebx "
>>>> "jge .end_label "
>>>> "mov edx , DWORD PTR [ ebp + 8 ] "
>>>> "mov eax , DWORD PTR [ ebp + 12 ] "
>>>> "add eax , edx "
>>>> "mov DWORD PTR [ ebp + 16 ] , eax "
>>>> "mov eax , DWORD PTR [ ebp + 12 ] "
>>>> "mov DWORD PTR [ ebp + 8 ] , eax "
>>>> "mov eax , DWORD PTR [ ebp + 16 ] "
>>>> "mov DWORD PTR [ ebp + 12 ] , eax "
>>>> "mov eax , DWORD PTR [ ebp + 4 ] "
>>>> "mov ebx , 1 "
>>>> "add eax , ebx "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>>> "jmp .loop_label "
>>>> ":end_label "
>>>> "mov eax , DWORD PTR [ ebp + 16 ] "
>>>> "exit"_s);
>>>> ```
>>>>
>>>> Version v1.0, from the github is simpler and works in a more naive way
>>>> (e.g. every asm instruction is recursively expanded) but basically, it has
>>>> same problems as the current version - there is a lot of types.
>>>>
>>>> I know that I can optimize the template code, but I'm sure that at some
>>>> point I'll hit the barrier and the only way will be to modify llvm/clang,
>>>> which I'm trying to understand and eventually do.
>>>>
>>>> Thanks,
>>>> Stryku
>>>>
>>>>
>>>>
>>>> 2017-04-27 10:54 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>>>>
>>>>>
>>>>>
>>>>> On 27 April 2017 at 03:13, Brian Cain via cfe-dev <
>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> Do other compilers behave similarly in terms if memory consumption
>>>>>> for your example code?
>>>>>>
>>>>>> On Apr 26, 2017 4:20 PM, "mateusz janek via cfe-dev" <
>>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Before I'll begin, I want to say that I don't want to implement
>>>>>>> below 'feature' in a clang production code. I want to create my own custom
>>>>>>> build for my own needs.
>>>>>>>
>>>>>>> Abstract:
>>>>>>> I'm developing a hobby project which creates a lot of types in a
>>>>>>> compile time, in one .cpp file. Really a lot. I tried to compile next
>>>>>>> version, but it was compiling whole night and took ~30G of memory (16G RAM
>>>>>>> + swap) and wasn't able to finish. I know that almost all of these types
>>>>>>> are 'used' only once, they're kind of helpers.
>>>>>>> Basically what I want to achieve is to decrease memory usage to not
>>>>>>> need to use the swap, because it increases compilation time a lot.
>>>>>>>
>>>>>>> Question:
>>>>>>> Is there a way to implement a removing no longer used types? E.g.
>>>>>>>
>>>>>>> template <typename FooParam>
>>>>>>> struct Foo
>>>>>>> {
>>>>>>> using FooResult = typename Bar<FooParam>::BarResult;
>>>>>>> };
>>>>>>>
>>>>>>> And I'd want to remove the 'Bar<FooParam>' type from a 'ASTContext'
>>>>>>> because I know that I won't need it anymore. (then, if there will again
>>>>>>> occur an initialization of very same 'Bar<FooParam>', I'm ok with that I'll
>>>>>>> need to create it from the scratch).
>>>>>>> So basically is there any way to detect if type is still referenced
>>>>>>> somewhere, so I can remove it if it isn't?
>>>>>>>
>>>>>>
>>>>> You will need ALL types (that are used at all) to be present at
>>>>> semantic analysis, since types are used to determine for example implicit
>>>>> casts (float to int, int to float, short to int, int to short - and even if
>>>>> your type can't be converted, it still needs to be present during that
>>>>> phase). Since Semantic analysis is done at the end of the compilation of a
>>>>> translation unit. And of course, types are used during translation from AST
>>>>> to IR, so need to be (at least somewhat) present during the IR generation
>>>>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>>>>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>>>>> `size_t` or `int`.
>>>>>
>>>>> Also, are you sure that it's the number of types that is the real
>>>>> cause of the problem?
>>>>>
>>>>> Can you provide a more complete example (preferrably one that
>>>>> compiles, but does not take "over night", in a few minutes would be good,
>>>>> that illustrates that it uses a lot more memory than a comparable piece of
>>>>> code with less types introduced in it).
>>>>>
>>>>> --
>>>>> Mats
>>>>>
>>>>>>
>>>>>>> I hope I've described my problem well. I know that in fact I'm
>>>>>>> asking for a explanation how types in clang are related to each other -
>>>>>>> that's a lot, but I'd really like to learn and understand that (:
>>>>>>>
>>>>>>> Thanks in advance for help!
>>>>>>>
>>>>>>> Stryku
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> cfe-dev mailing list
>>>>>>> cfe-dev at lists.llvm.org
>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> cfe-dev mailing list
>>>>>> cfe-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170428/75d5682a/attachment.html>
More information about the cfe-dev
mailing list