[cfe-dev] boost delay.c consumes huge amounts of memory

[Richard Smith via cfe-dev]

On Tue, Nov 10, 2015 at 3:16 PM, Jason Haslam <jason.haslam at gmail.com>
wrote:

> On Nov 10, 2015, at 2:53 PM, Richard Smith <richard at metafoo.co.uk> wrote:
>
> On Tue, Nov 10, 2015 at 10:11 AM, Jason Haslam via cfe-dev <
> cfe-dev at lists.llvm.org> wrote:
>
>> There’s a test file in boost at libs/preprocessor/doc/examples/delay.c
>> that causes clang to quickly consume large amounts of memory. Has anybody
>> seen this before? Is there some sort of workaround (like maybe a flag to
>> limit macro expansion depth)? I couldn’t find an issue in the bug tracker.
>>
>
> From that file:
>
> # /* The time complexity of DELAY(N) is O(2^N).
> #  *
> #  * Handy when recompiles are too fast to take a coffee break. :)
> #  *
> #  * Template metaprogramming can be used for implementing similar
> #  * delays. Unfortunately template instantiation consumes memory,
> #  * therefore compilers usually fail to fully compile long template
> #  * based delays, because they run out of memory.
> #  *
> #  * On many compilers (e.g. g++, MSVC++), this macro takes only a
> #  * small amount of memory to preprocess. On some compilers (e.g.
> #  * MWCW), however, this macro seems to consume huge amounts of
> #  * memory.
>
> The "problem" is likely to be that we're keeping track of all the macro
> expansions performed by this ridiculous code, so this will use 2^N memory
> as well as 2^N time. We don't have any way to turn that off. If you file a
> bug on this, it's likely to be resolved as invalid unless you can provide a
> good reason for why we should care about code that is deliberately trying
> to break the compiler.
>
>
> Okay, thanks. I don’t necessarily care about this code or expect anything
> reasonable to happen. The issue arises when users blindly try to slurp up
> and analyze everything in a directory. The outcome isn’t very friendly
> (especially on Windows).
>

OK, that makes sense. I wonder if the boost folks would consider deleting
this test, since it's going to cause (at least) slow processing for any
tool that tries to analyze the file (including performing full
preprocessing) while walking their test suite. I don't think a flag to
clang will really help all that much here (unless it's enabled by default),
since you would need to first hit and diagnose the problem before you'd
even know that a flag were necessary. Even a warning/error for the case
where we've hit a really complex macro expansion may not help, since
(depending on what style of report is requested) a user is unlikely to see
that until after the slow (and possibly non-terminating due to swap
thrashing) analyzer run finishes -- we'd need to actually abort translation.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20151110/fd801c1e/attachment.html>