[llvm] r338913 - [Support] Don't initialize compressed buffer allocated by zlib::compress

[David Blaikie via llvm-commits]

Hmm, nope - my mistake. Just read the docs & you've got it right - don't
mind me. (:

On Fri, Aug 3, 2018 at 12:42 PM David Blaikie <dblaikie at gmail.com> wrote:

> I don't think this is what Eli was suggesting.
>
> set_size should be used instead of resize/reserve here - it acts like
> resize, but doesn't initialize the data. Using only 'reserve' violates the
> SmallVector contract - because data is then written beyond the 'size'.
>
> Instead, use 'set_size' before the compression, and resize down to the
> final size after the compressed bytes have been written.
>
>
> On Fri, Aug 3, 2018 at 12:38 PM Fangrui Song via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
>
>> Author: maskray
>> Date: Fri Aug  3 12:37:49 2018
>> New Revision: 338913
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=338913&view=rev
>> Log:
>> [Support] Don't initialize compressed buffer allocated by zlib::compress
>>
>> resize() (zeroing) makes every allocated page resident. The actual size
>> of the compressed buffer is usually much
>> smaller. Making every page resident is wasteful.
>>
>> When linking a test binary with ~1.9GiB uncompressed debug info with LLD,
>> this optimization decreases max RSS by ~1.5GiB.
>>
>> Differential Revision: https://reviews.llvm.org/50223
>>
>> Modified:
>>     llvm/trunk/lib/Support/Compression.cpp
>>
>> Modified: llvm/trunk/lib/Support/Compression.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/Compression.cpp?rev=338913&r1=338912&r2=338913&view=diff
>>
>> ==============================================================================
>> --- llvm/trunk/lib/Support/Compression.cpp (original)
>> +++ llvm/trunk/lib/Support/Compression.cpp Fri Aug  3 12:37:49 2018
>> @@ -61,7 +61,7 @@ Error zlib::compress(StringRef InputBuff
>>                       SmallVectorImpl<char> &CompressedBuffer,
>>                       CompressionLevel Level) {
>>    unsigned long CompressedSize = ::compressBound(InputBuffer.size());
>> -  CompressedBuffer.resize(CompressedSize);
>> +  CompressedBuffer.reserve(CompressedSize);
>>    int CLevel = encodeZlibCompressionLevel(Level);
>>    int Res = ::compress2((Bytef *)CompressedBuffer.data(),
>> &CompressedSize,
>>                          (const Bytef *)InputBuffer.data(),
>> InputBuffer.size(),
>> @@ -69,7 +69,7 @@ Error zlib::compress(StringRef InputBuff
>>    // Tell MemorySanitizer that zlib output buffer is fully initialized.
>>    // This avoids a false report when running LLVM with uninstrumented
>> ZLib.
>>    __msan_unpoison(CompressedBuffer.data(), CompressedSize);
>> -  CompressedBuffer.resize(CompressedSize);
>> +  CompressedBuffer.set_size(CompressedSize);
>>    return Res ? createError(convertZlibCodeToString(Res)) :
>> Error::success();
>>  }
>>
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180803/f6e8bd85/attachment.html>