[llvm-dev] [cfe-dev] Remove unused types from ASTContext
mats petersson via llvm-dev
llvm-dev at lists.llvm.org
Thu Apr 27 12:56:42 PDT 2017
It would be good if you did "reply all" so that those other people that
have read and is interested in this discussion can see the answers. I've
added Brian and the mailing list back into this reply.
That is an impressive amount of template-code. I've never really done much
TMP, but I will see if I can at least somewhat analyse where the
memory/time goes in this.
It would probably have been better to have a SMALL example, that compiles
in a few seconds to a few minutes, and a "plain" program that achieves
roughly the same thing, but I'll see what I can do.
When I get home (it's about 9pm, and I'm still at work), and I make no
promises of ANY kind...
--
Mats
On 27 April 2017 at 18:51, mateusz janek <stryku2393 at gmail.com> wrote:
> Thanks for the replies (:
>
> @Brian Cain
> >Do other compilers behave similarly in terms if memory consumption for
> your example code?
> Good point. I'll check that today with the g++.
>
>
> @Mats Petersson
> >You will need ALL types (that are used at all) to be present at semantic
> analysis, since types are used to determine for example implicit casts
> (float to int, int to float, short to int, int to short - and even if your
> type can't be converted, it still needs to be present during that phase).
> Since Semantic analysis is done at the end of the compilation of a
> translation unit. And of course, types are used during translation from AST
> to IR, so need to be (at least somewhat) present during the IR generation
> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
> find `Bar<FooParam>::BarResult`, even if that is then translated to
> `size_t` or `int`.
> Thanks a lot for an explanation (:
>
> >Also, are you sure that it's the number of types that is the real cause
> of the problem?
> Yes, I'm pretty sure. Almost everything in this project is a template type.
>
> >Can you provide a more complete example (preferrably one that compiles,
> but does not take "over night", in a few minutes would be good, that
> illustrates that it uses a lot more memory than a comparable piece of code
> with less types introduced in it).
> Of course. Check the repo: https://github.com/stryku/ctai (Not sure if
> you've asked for this or for a complete new code which illustrates the
> problem, if the new code - sorry and let me know I'll try to prepare
> something)
> There is still v1.0 version and it compiles just fine in about 1-2s. To
> increase the time and memory usage you can just add a couple of dummy asm
> instructions in main.cpp, e.g.:
>
> ```
> using code = decltype(
> "mov ebp , esp "
> "push eax " // declare fourusing code = decltype(
> "mov ebp , esp "
> "push eax " // declare four variables
> "push eax "
> "push eax "
> "push eax "
> "mov DWORD PTR [ ebp + 8 ] , 0 "
> "mov DWORD PTR [ ebp + 12 ] , 1 "
> "mov DWORD PTR [ ebp + 16 ] , 1 "
> "mov DWORD PTR [ ebp + 4 ] , 1 "
> ":loop_label "
> "mov eax , DWORD PTR [ ebp + 4 ] "
> "mov ebx , 15 " //we want to get 15th fibonacci element
> "cmp eax , ebx "
> "jge .end_label "
> "mov edx , DWORD PTR [ ebp + 8 ] "
> "mov eax , DWORD PTR [ ebp + 12 ] "
> "add eax , edx "
> "mov DWORD PTR [ ebp + 16 ] , eax "
> "mov eax , DWORD PTR [ ebp + 12 ] "
> "mov DWORD PTR [ ebp + 8 ] , eax "
> "mov eax , DWORD PTR [ ebp + 16 ] "
> "mov DWORD PTR [ ebp + 12 ] , eax "
> "mov eax , DWORD PTR [ ebp + 4 ] "
> "mov ebx , 1 "
> "add eax , ebx "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "mov DWORD PTR [ ebp + 4 ] , eax "
> "jmp .loop_label "
> ":end_label "
> "mov eax , DWORD PTR [ ebp + 16 ] "
> "exit"_s);
> ```
>
> Version v1.0, from the github is simpler and works in a more naive way
> (e.g. every asm instruction is recursively expanded) but basically, it has
> same problems as the current version - there is a lot of types.
>
> I know that I can optimize the template code, but I'm sure that at some
> point I'll hit the barrier and the only way will be to modify llvm/clang,
> which I'm trying to understand and eventually do.
>
> Thanks,
> Stryku
>
>
>
> 2017-04-27 10:54 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>
>>
>>
>> On 27 April 2017 at 03:13, Brian Cain via cfe-dev <cfe-dev at lists.llvm.org
>> > wrote:
>>
>>> Do other compilers behave similarly in terms if memory consumption for
>>> your example code?
>>>
>>> On Apr 26, 2017 4:20 PM, "mateusz janek via cfe-dev" <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> Hi,
>>>>
>>>> Before I'll begin, I want to say that I don't want to implement below
>>>> 'feature' in a clang production code. I want to create my own custom build
>>>> for my own needs.
>>>>
>>>> Abstract:
>>>> I'm developing a hobby project which creates a lot of types in a
>>>> compile time, in one .cpp file. Really a lot. I tried to compile next
>>>> version, but it was compiling whole night and took ~30G of memory (16G RAM
>>>> + swap) and wasn't able to finish. I know that almost all of these types
>>>> are 'used' only once, they're kind of helpers.
>>>> Basically what I want to achieve is to decrease memory usage to not
>>>> need to use the swap, because it increases compilation time a lot.
>>>>
>>>> Question:
>>>> Is there a way to implement a removing no longer used types? E.g.
>>>>
>>>> template <typename FooParam>
>>>> struct Foo
>>>> {
>>>> using FooResult = typename Bar<FooParam>::BarResult;
>>>> };
>>>>
>>>> And I'd want to remove the 'Bar<FooParam>' type from a 'ASTContext'
>>>> because I know that I won't need it anymore. (then, if there will again
>>>> occur an initialization of very same 'Bar<FooParam>', I'm ok with that I'll
>>>> need to create it from the scratch).
>>>> So basically is there any way to detect if type is still referenced
>>>> somewhere, so I can remove it if it isn't?
>>>>
>>>
>> You will need ALL types (that are used at all) to be present at semantic
>> analysis, since types are used to determine for example implicit casts
>> (float to int, int to float, short to int, int to short - and even if your
>> type can't be converted, it still needs to be present during that phase).
>> Since Semantic analysis is done at the end of the compilation of a
>> translation unit. And of course, types are used during translation from AST
>> to IR, so need to be (at least somewhat) present during the IR generation
>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>> `size_t` or `int`.
>>
>> Also, are you sure that it's the number of types that is the real cause
>> of the problem?
>>
>> Can you provide a more complete example (preferrably one that compiles,
>> but does not take "over night", in a few minutes would be good, that
>> illustrates that it uses a lot more memory than a comparable piece of code
>> with less types introduced in it).
>>
>> --
>> Mats
>>
>>>
>>>> I hope I've described my problem well. I know that in fact I'm asking
>>>> for a explanation how types in clang are related to each other - that's a
>>>> lot, but I'd really like to learn and understand that (:
>>>>
>>>> Thanks in advance for help!
>>>>
>>>> Stryku
>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170427/f0bbeb74/attachment.html>
More information about the llvm-dev
mailing list