[llvm-dev] problems with objects larger than PTRDIFF_MAX

[Alexander Cherepanov via llvm-dev]

On 2016-05-29 21:16, David Majnemer wrote:
> On Sun, May 29, 2016 at 6:18 AM, Alexander Cherepanov <ch3root at openwall.com>
> wrote:
>
>> On 2016-05-20 19:58, David Majnemer via llvm-dev wrote:
>>
>>> I've come across this issue before and came to the following conclusion:
>>> - We are not obligated to support objects that large, C11 5.2.4.1/1 only
>>> requires that we support objects of size 65535!
>>>
>>
>> Right, the standard doesn't require it. But I guess you don't imply that
>> it's fine for clang to silently miscompile any program that works with
>> objects larger than 65535 bytes.
>
> I think that we should error when we can statically determine that an
> object is larger than PTRDIFF_MAX.

It seems we agree that the current situation is not good and something 
has to be fixed in clang/llvm. Ok.

Why don't you think that clang/llvm should just be fixed to support 
objects larger than PTRDIFF_MAX bytes?

>> - The expectation that PTRDIFF_MAX is more or less a proxy for the largest
>>> object size is not uncommon.   For example, C++'s std::count doesn't
>>> return
>>> a size_t but a iterator_traits<>::difference_type which is going to be a
>>> ptrdiff_t for things like std::vector.
>>
>> Bug in C++? Because max_size() of std::vector<char> has the type
>> `size_type` and returns SIZE_MAX in practice (even on x86-64).
>
> Some folks in the C++ community believe that using unsigned types in those
> places was the wrong decision.

Result of max_size() cannot be changed without changing its type?

>> Let's see other examples (on x86-32):
>>
>> - glibc's malloc is happy to allocate more than PTRDIFF_MAX bytes (and
>> this is important in practice so (at least some) glibc devs are reluctant
>> to "fix" it);
>>
>
> glibc is not the only game in town.

Right, for example musl doesn't create objects larger than PTRDIFF_MAX 
bytes. I think this is good that different libc implementations are free 
to choose their own policies.

The problem is that this is only possible when compilers support large 
objects. When compilers don't support large objects (like now), support 
in libc is not enough -- you get your large objects but your accesses to 
it will be miscompiled. Sometimes.

> Android prevents it: https://android-review.googlesource.com/#/c/170800/
> jemalloc prevents it:
> https://github.com/jemalloc/jemalloc/commit/0c516a00c4cb28cff55ce0995f756b5aae074c9e

A good share of the reasons for this is exactly the fact that clang/llvm 
don't support large objects. So this reasoning is circular:-)

> There are other implementations of malloc which also act similarly.

-- 
Alexander Cherepanov