[llvm-dev] problems with objects larger than PTRDIFF_MAX

[David Majnemer via llvm-dev]

On Sun, May 29, 2016 at 6:18 AM, Alexander Cherepanov <ch3root at openwall.com>
wrote:

> On 2016-05-20 19:58, David Majnemer via llvm-dev wrote:
>
>> I've come across this issue before and came to the following conclusion:
>> - We are not obligated to support objects that large, C11 5.2.4.1/1 only
>> requires that we support objects of size 65535!
>>
>
> Right, the standard doesn't require it. But I guess you don't imply that
> it's fine for clang to silently miscompile any program that works with
> objects larger than 65535 bytes.


I think that we should error when we can statically determine that an
object is larger than PTRDIFF_MAX.


>
>
> Their guidance for maximum
>> object size is stated to be half of SIZE_MAX in C11 K.3.4/4 which is
>> typically equivalent to PTRDIFF_MAX.
>>
>
> Whose guidance? Annex K is kinda alien to the rest of the standard and its
> future is not clear. See, e.g.,
> http://open-std.org/jtc1/sc22/wg14/www/docs/n1969.htm .


That document objects to much of Annex K but doesn't say anything about
K.3.4/4.


>
>
> - The expectation that PTRDIFF_MAX is more or less a proxy for the largest
>> object size is not uncommon.   For example, C++'s std::count doesn't
>> return
>> a size_t but a iterator_traits<>::difference_type which is going to be a
>> ptrdiff_t for things like std::vector.
>>
>
> Bug in C++? Because max_size() of std::vector<char> has the type
> `size_type` and returns SIZE_MAX in practice (even on x86-64).
>

Some folks in the C++ community believe that using unsigned types in those
places was the wrong decision.


>
> Let's see other examples (on x86-32):
>
> - glibc's malloc is happy to allocate more than PTRDIFF_MAX bytes (and
> this is important in practice so (at least some) glibc devs are reluctant
> to "fix" it);
>

glibc is not the only game in town.
Android prevents it: https://android-review.googlesource.com/#/c/170800/
jemalloc prevents it:
https://github.com/jemalloc/jemalloc/commit/0c516a00c4cb28cff55ce0995f756b5aae074c9e

There are other implementations of malloc which also act similarly.


> - clang has a compile-time limit for sizes of objects and types which is
> SIZE_MAX. For example, it compiles "char a[-1ul]; int main() {}" but
> complains about "char a[-1ul + 1ull]; int main() {}";
>
> - `new` for more than PTRDIFF_MAX works fine when compiled by clang++ (but
> throws std::bad_array_new_length when compiled by g++).
>
> On Fri, May 20, 2016 at 1:31 AM, John Regehr via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> It could be that 32-bit systems are disappearing so rapidly that nobody
>>> cares too much about this issue,
>>>
>>
> Please note that there is a whole new 32-bit architecture -- x32.
>
>
> but this blog post is still worth reading:
>>>
>>>    http://trust-in-soft.com/objects-larger-than-ptrdiff_max-bytes/
>>>
>>
> --
> Alexander Cherepanov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20160529/5659d353/attachment.html>