[llvm-dev] [cfe-dev] Remove unused types from ASTContext
mateusz janek via llvm-dev
llvm-dev at lists.llvm.org
Thu Apr 27 14:33:51 PDT 2017
My bad, I was sure that I'm "replying all", thanks for that.
Of course I don't want you to feel any kind of pressure from my side. I'm
just curious if I can somehow adjust llvm/clang to help me with such
problem.
I'll try to provide MCVE, but unfortunately probably tomorrow.
2017-04-27 21:56 GMT+02:00 mats petersson <mats at planetcatfish.com>:
> It would be good if you did "reply all" so that those other people that
> have read and is interested in this discussion can see the answers. I've
> added Brian and the mailing list back into this reply.
>
> That is an impressive amount of template-code. I've never really done much
> TMP, but I will see if I can at least somewhat analyse where the
> memory/time goes in this.
>
> It would probably have been better to have a SMALL example, that compiles
> in a few seconds to a few minutes, and a "plain" program that achieves
> roughly the same thing, but I'll see what I can do.
>
> When I get home (it's about 9pm, and I'm still at work), and I make no
> promises of ANY kind...
>
> --
> Mats
>
>
> On 27 April 2017 at 18:51, mateusz janek <stryku2393 at gmail.com> wrote:
>
>> Thanks for the replies (:
>>
>> @Brian Cain
>> >Do other compilers behave similarly in terms if memory consumption for
>> your example code?
>> Good point. I'll check that today with the g++.
>>
>>
>> @Mats Petersson
>> >You will need ALL types (that are used at all) to be present at semantic
>> analysis, since types are used to determine for example implicit casts
>> (float to int, int to float, short to int, int to short - and even if your
>> type can't be converted, it still needs to be present during that phase).
>> Since Semantic analysis is done at the end of the compilation of a
>> translation unit. And of course, types are used during translation from AST
>> to IR, so need to be (at least somewhat) present during the IR generation
>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>> `size_t` or `int`.
>> Thanks a lot for an explanation (:
>>
>> >Also, are you sure that it's the number of types that is the real cause
>> of the problem?
>> Yes, I'm pretty sure. Almost everything in this project is a template
>> type.
>>
>> >Can you provide a more complete example (preferrably one that compiles,
>> but does not take "over night", in a few minutes would be good, that
>> illustrates that it uses a lot more memory than a comparable piece of code
>> with less types introduced in it).
>> Of course. Check the repo: https://github.com/stryku/ctai (Not sure if
>> you've asked for this or for a complete new code which illustrates the
>> problem, if the new code - sorry and let me know I'll try to prepare
>> something)
>> There is still v1.0 version and it compiles just fine in about 1-2s. To
>> increase the time and memory usage you can just add a couple of dummy asm
>> instructions in main.cpp, e.g.:
>>
>> ```
>> using code = decltype(
>> "mov ebp , esp "
>> "push eax " // declare fourusing code = decltype(
>> "mov ebp , esp "
>> "push eax " // declare four variables
>> "push eax "
>> "push eax "
>> "push eax "
>> "mov DWORD PTR [ ebp + 8 ] , 0 "
>> "mov DWORD PTR [ ebp + 12 ] , 1 "
>> "mov DWORD PTR [ ebp + 16 ] , 1 "
>> "mov DWORD PTR [ ebp + 4 ] , 1 "
>> ":loop_label "
>> "mov eax , DWORD PTR [ ebp + 4 ] "
>> "mov ebx , 15 " //we want to get 15th fibonacci element
>> "cmp eax , ebx "
>> "jge .end_label "
>> "mov edx , DWORD PTR [ ebp + 8 ] "
>> "mov eax , DWORD PTR [ ebp + 12 ] "
>> "add eax , edx "
>> "mov DWORD PTR [ ebp + 16 ] , eax "
>> "mov eax , DWORD PTR [ ebp + 12 ] "
>> "mov DWORD PTR [ ebp + 8 ] , eax "
>> "mov eax , DWORD PTR [ ebp + 16 ] "
>> "mov DWORD PTR [ ebp + 12 ] , eax "
>> "mov eax , DWORD PTR [ ebp + 4 ] "
>> "mov ebx , 1 "
>> "add eax , ebx "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "mov DWORD PTR [ ebp + 4 ] , eax "
>> "jmp .loop_label "
>> ":end_label "
>> "mov eax , DWORD PTR [ ebp + 16 ] "
>> "exit"_s);
>> ```
>>
>> Version v1.0, from the github is simpler and works in a more naive way
>> (e.g. every asm instruction is recursively expanded) but basically, it has
>> same problems as the current version - there is a lot of types.
>>
>> I know that I can optimize the template code, but I'm sure that at some
>> point I'll hit the barrier and the only way will be to modify llvm/clang,
>> which I'm trying to understand and eventually do.
>>
>> Thanks,
>> Stryku
>>
>>
>>
>> 2017-04-27 10:54 GMT+02:00 mats petersson <mats at planetcatfish.com>:
>>
>>>
>>>
>>> On 27 April 2017 at 03:13, Brian Cain via cfe-dev <
>>> cfe-dev at lists.llvm.org> wrote:
>>>
>>>> Do other compilers behave similarly in terms if memory consumption for
>>>> your example code?
>>>>
>>>> On Apr 26, 2017 4:20 PM, "mateusz janek via cfe-dev" <
>>>> cfe-dev at lists.llvm.org> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> Before I'll begin, I want to say that I don't want to implement below
>>>>> 'feature' in a clang production code. I want to create my own custom build
>>>>> for my own needs.
>>>>>
>>>>> Abstract:
>>>>> I'm developing a hobby project which creates a lot of types in a
>>>>> compile time, in one .cpp file. Really a lot. I tried to compile next
>>>>> version, but it was compiling whole night and took ~30G of memory (16G RAM
>>>>> + swap) and wasn't able to finish. I know that almost all of these types
>>>>> are 'used' only once, they're kind of helpers.
>>>>> Basically what I want to achieve is to decrease memory usage to not
>>>>> need to use the swap, because it increases compilation time a lot.
>>>>>
>>>>> Question:
>>>>> Is there a way to implement a removing no longer used types? E.g.
>>>>>
>>>>> template <typename FooParam>
>>>>> struct Foo
>>>>> {
>>>>> using FooResult = typename Bar<FooParam>::BarResult;
>>>>> };
>>>>>
>>>>> And I'd want to remove the 'Bar<FooParam>' type from a 'ASTContext'
>>>>> because I know that I won't need it anymore. (then, if there will again
>>>>> occur an initialization of very same 'Bar<FooParam>', I'm ok with that I'll
>>>>> need to create it from the scratch).
>>>>> So basically is there any way to detect if type is still referenced
>>>>> somewhere, so I can remove it if it isn't?
>>>>>
>>>>
>>> You will need ALL types (that are used at all) to be present at semantic
>>> analysis, since types are used to determine for example implicit casts
>>> (float to int, int to float, short to int, int to short - and even if your
>>> type can't be converted, it still needs to be present during that phase).
>>> Since Semantic analysis is done at the end of the compilation of a
>>> translation unit. And of course, types are used during translation from AST
>>> to IR, so need to be (at least somewhat) present during the IR generation
>>> phase too. [And the compiler WILL need to know what `Bar<FooParam>` is to
>>> find `Bar<FooParam>::BarResult`, even if that is then translated to
>>> `size_t` or `int`.
>>>
>>> Also, are you sure that it's the number of types that is the real cause
>>> of the problem?
>>>
>>> Can you provide a more complete example (preferrably one that compiles,
>>> but does not take "over night", in a few minutes would be good, that
>>> illustrates that it uses a lot more memory than a comparable piece of code
>>> with less types introduced in it).
>>>
>>> --
>>> Mats
>>>
>>>>
>>>>> I hope I've described my problem well. I know that in fact I'm asking
>>>>> for a explanation how types in clang are related to each other - that's a
>>>>> lot, but I'd really like to learn and understand that (:
>>>>>
>>>>> Thanks in advance for help!
>>>>>
>>>>> Stryku
>>>>>
>>>>> _______________________________________________
>>>>> cfe-dev mailing list
>>>>> cfe-dev at lists.llvm.org
>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>>
>>>>>
>>>> _______________________________________________
>>>> cfe-dev mailing list
>>>> cfe-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170427/44a7279d/attachment-0001.html>
More information about the llvm-dev
mailing list