[llvm] r312582 - Revert "[Decompression] Fail gracefully when out of memory"

[David Blaikie via llvm-commits]

On Fri, Sep 8, 2017 at 1:36 PM Kostya Serebryany <kcc at google.com> wrote:

> On Fri, Sep 8, 2017 at 1:32 PM, David Blaikie <dblaikie at gmail.com> wrote:
>
>>
>>
>> On Fri, Sep 8, 2017 at 1:24 PM Kostya Serebryany <kcc at google.com> wrote:
>>
>>>
>>>> I'm not sure I understand what you mean.
>>>>
>>>> It seems unfortunate to me to build fixed memory limits into programs -
>>>> it's nice if they scale up with bigger inputs and more memory on the
>>>> machine to do more of whatever it is they do - only limited by the memory
>>>> available on the machine, not some fixed limit in the program.
>>>>
>>>> If this means making programs robust to allocation failure (failing
>>>> gracefully rather than 'crashing') for them to be fuzzable - that seems, to
>>>> me at least, an OK tradeoff.
>>>>
>>>
>>> This is subtle, but let me try again.
>>> 1. IMHO any code that tries to recover from malloc (or new!!) returning
>>> nullptr is much more complicated than adding an artificial limit as a flag
>>> (by default the flag could be arbitrarily large).
>>>
>>
>> By 'recover' I'd be OK with "exits with an error code", so not too
>> complex, hopefully.
>>
>
> But we already have this, both from asan and libFuzzer.
> The problem is that 'exit' means a 'bug', which is where we started in
> this thread.
> A proper API should not exit on malformed input (nor leak, crash, OOM,
> timeout, etc) -- it should return an error code to the caller.
>

Then shouldn't we be plumbing through memory allocation failure as an error
to the caller, if that's the argument?


> I feel like plumbing through flags with bounds into every
>> program/allocation is a bit burdensome/awkward/uncommon. Maybe other folks
>> have other ideas/perspectives & I'm off in the weeds.
>>
>
> Not every allocation.
> Only those allocations that take the size from un-trusted user-provided
> inputs.
>

But it wouldn't just be those - the input might say "there are a million
things" and then for each thing you go and allocate a known size block. Or
other less entirely correlated - I mean most of the memory LLVM or any
program allocates is dictated by its inputs, even if it's not taking bytes
from the input and using it to determine the size of the allocation.

Or maybe the user input causes a just-barely-sane size, then the program
goes to allocate some normal data structure it always allocates and that
tips it over.

- Dave


>
>
>>
>>
>>> It's also much harder to test.
>>> 2. malloc is not guaranteed to return nullptr even in cases when there
>>> is actually not enough memory. It may instead return something backed by
>>> swap and this can easily kill your machine.
>>>
>>
>> Sure enough - though I'm not sure adding a flag to every program is going
>> to be a real path forward for protecting processes from user input. I'd
>> expect to use OS level protections for memory limits, etc, there.
>>
>> - Dave
>>
>>
>>>
>>>
>>>
>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170908/6e8b5511/attachment.html>