[cfe-dev] Remove unused types from ASTContext
mats petersson via cfe-dev
cfe-dev at lists.llvm.org
Fri Apr 28 00:53:26 PDT 2017
I can't (after about 10 minutes of trying) get this to compile at all -
most likely because I have a too old version of C++ library installed...
--
Mats
On 27 April 2017 at 22:33, mateusz janek <stryku2393 at gmail.com> wrote:
> My bad, I was sure that I'm "replying all", thanks for that.
>
> Of course I don't want you to feel any kind of pressure from my side. I'm
> just curious if I can somehow adjust llvm/clang to help me with such
> problem.
>
> I'll try to provide MCVE, but unfortunately probably tomorrow.
>
> 2017-04-27 21:56 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>
>> It would be good if you did "reply all" so that those other people that
>> have read and is interested in this discussion can see the answers. I've
>> added Brian and the mailing list back into this reply.
>>
>> That is an impressive amount of template-code. I've never really done
>> much TMP, but I will see if I can at least somewhat analyse where the
>> memory/time goes in this.
>>
>> It would probably have been better to have a SMALL example, that compiles
>> in a few seconds to a few minutes, and a "plain" program that achieves
>> roughly the same thing, but I'll see what I can do.
>>
>> When I get home (it's about 9pm, and I'm still at work), and I make no
>> promises of ANY kind...
>>
>> --
>> Mats
>>
>>
>> On 27 April 2017 at 18:51, mateusz janek <stryku2393 at gmail.com> wrote:
>>
>>> Thanks for the replies (:
>>>
>>> @Brian Cain
>>> >Do other compilers behave similarly in terms if memory consumption for
>>> your example code?
>>> Good point. I'll check that today with the g++.
>>>
>>>
>>> @Mats Petersson
>>> >You will need ALL types (that are used at all) to be present at
>>> semantic analysis, since types are used to determine for example implicit
>>> casts (float to int, int to float, short to int, int to short - and even if
>>> your type can't be converted, it still needs to be present during that
>>> phase). Since Semantic analysis is done at the end of the compilation of a
>>> translation unit. And of course, types are used during translation from AST
>>> to IR, so need to be (at least somewhat) present during the IR generation
>>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>>> `size_t` or `int`.
>>> Thanks a lot for an explanation (:
>>>
>>> >Also, are you sure that it's the number of types that is the real cause
>>> of the problem?
>>> Yes, I'm pretty sure. Almost everything in this project is a template
>>> type.
>>>
>>> >Can you provide a more complete example (preferrably one that compiles,
>>> but does not take "over night", in a few minutes would be good, that
>>> illustrates that it uses a lot more memory than a comparable piece of code
>>> with less types introduced in it).
>>> Of course. Check the repo: https://github.com/stryku/ctai (Not sure if
>>> you've asked for this or for a complete new code which illustrates the
>>> problem, if the new code - sorry and let me know I'll try to prepare
>>> something)
>>> There is still v1.0 version and it compiles just fine in about 1-2s. To
>>> increase the time and memory usage you can just add a couple of dummy asm
>>> instructions in main.cpp, e.g.:
>>>
>>> ```
>>> using code = decltype(
>>> "mov ebp , esp "
>>> "push eax " // declare fourusing code = decltype(
>>> "mov ebp , esp "
>>> "push eax " // declare four variables
>>> "push eax "
>>> "push eax "
>>> "push eax "
>>> "mov DWORD PTR [ ebp + 8 ] , 0 "
>>> "mov DWORD PTR [ ebp + 12 ] , 1 "
>>> "mov DWORD PTR [ ebp + 16 ] , 1 "
>>> "mov DWORD PTR [ ebp + 4 ] , 1 "
>>> ":loop_label "
>>> "mov eax , DWORD PTR [ ebp + 4 ] "
>>> "mov ebx , 15 " //we want to get 15th fibonacci element
>>> "cmp eax , ebx "
>>> "jge .end_label "
>>> "mov edx , DWORD PTR [ ebp + 8 ] "
>>> "mov eax , DWORD PTR [ ebp + 12 ] "
>>> "add eax , edx "
>>> "mov DWORD PTR [ ebp + 16 ] , eax "
>>> "mov eax , DWORD PTR [ ebp + 12 ] "
>>> "mov DWORD PTR [ ebp + 8 ] , eax "
>>> "mov eax , DWORD PTR [ ebp + 16 ] "
>>> "mov DWORD PTR [ ebp + 12 ] , eax "
>>> "mov eax , DWORD PTR [ ebp + 4 ] "
>>> "mov ebx , 1 "
>>> "add eax , ebx "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "mov DWORD PTR [ ebp + 4 ] , eax "
>>> "jmp .loop_label "
>>> ":end_label "
>>> "mov eax , DWORD PTR [ ebp + 16 ] "
>>> "exit"_s);
>>> ```
>>>
>>> Version v1.0, from the github is simpler and works in a more naive way
>>> (e.g. every asm instruction is recursively expanded) but basically, it has
>>> same problems as the current version - there is a lot of types.
>>>
>>> I know that I can optimize the template code, but I'm sure that at some
>>> point I'll hit the barrier and the only way will be to modify llvm/clang,
>>> which I'm trying to understand and eventually do.
>>>
>>> Thanks,
>>> Stryku
>>>
>>>
>>>
>>> 2017-04-27 10:54 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>>>
>>>>
>>>>
>>>> On 27 April 2017 at 03:13, Brian Cain via cfe-dev <
>>>> cfe-dev at lists.llvm.org> wrote:
>>>>
>>>>> Do other compilers behave similarly in terms if memory consumption for
>>>>> your example code?
>>>>>
>>>>> On Apr 26, 2017 4:20 PM, "mateusz janek via cfe-dev" <
>>>>> cfe-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Before I'll begin, I want to say that I don't want to implement below
>>>>>> 'feature' in a clang production code. I want to create my own custom build
>>>>>> for my own needs.
>>>>>>
>>>>>> Abstract:
>>>>>> I'm developing a hobby project which creates a lot of types in a
>>>>>> compile time, in one .cpp file. Really a lot. I tried to compile next
>>>>>> version, but it was compiling whole night and took ~30G of memory (16G RAM
>>>>>> + swap) and wasn't able to finish. I know that almost all of these types
>>>>>> are 'used' only once, they're kind of helpers.
>>>>>> Basically what I want to achieve is to decrease memory usage to not
>>>>>> need to use the swap, because it increases compilation time a lot.
>>>>>>
>>>>>> Question:
>>>>>> Is there a way to implement a removing no longer used types? E.g.
>>>>>>
>>>>>> template <typename FooParam>
>>>>>> struct Foo
>>>>>> {
>>>>>> using FooResult = typename Bar<FooParam>::BarResult;
>>>>>> };
>>>>>>
>>>>>> And I'd want to remove the 'Bar<FooParam>' type from a 'ASTContext'
>>>>>> because I know that I won't need it anymore. (then, if there will again
>>>>>> occur an initialization of very same 'Bar<FooParam>', I'm ok with that I'll
>>>>>> need to create it from the scratch).
>>>>>> So basically is there any way to detect if type is still referenced
>>>>>> somewhere, so I can remove it if it isn't?
>>>>>>
>>>>>
>>>> You will need ALL types (that are used at all) to be present at
>>>> semantic analysis, since types are used to determine for example implicit
>>>> casts (float to int, int to float, short to int, int to short - and even if
>>>> your type can't be converted, it still needs to be present during that
>>>> phase). Since Semantic analysis is done at the end of the compilation of a
>>>> translation unit. And of course, types are used during translation from AST
>>>> to IR, so need to be (at least somewhat) present during the IR generation
>>>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>>>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>>>> `size_t` or `int`.
>>>>
>>>> Also, are you sure that it's the number of types that is the real cause
>>>> of the problem?
>>>>
>>>> Can you provide a more complete example (preferrably one that compiles,
>>>> but does not take "over night", in a few minutes would be good, that
>>>> illustrates that it uses a lot more memory than a comparable piece of code
>>>> with less types introduced in it).
>>>>
>>>> --
>>>> Mats
>>>>
>>>>>
>>>>>> I hope I've described my problem well. I know that in fact I'm asking
>>>>>> for a explanation how types in clang are related to each other - that's a
>>>>>> lot, but I'd really like to learn and understand that (:
>>>>>>
>>>>>> Thanks in advance for help!
>>>>>>
>>>>>> Stryku
>>>>>>
>>>>>> _______________________________________________
>>>>>> cfe-dev mailing list
>>>>>> cfe-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> cfe-dev mailing list
>>>>> cfe-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20170428/a58631e7/attachment.html>
More information about the cfe-dev
mailing list