[llvm] r338913 - [Support] Don't initialize compressed buffer allocated by zlib::compress

[David Blaikie via llvm-commits]

I don't think this is what Eli was suggesting.

set_size should be used instead of resize/reserve here - it acts like
resize, but doesn't initialize the data. Using only 'reserve' violates the
SmallVector contract - because data is then written beyond the 'size'.

Instead, use 'set_size' before the compression, and resize down to the
final size after the compressed bytes have been written.

On Fri, Aug 3, 2018 at 12:38 PM Fangrui Song via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> Author: maskray
> Date: Fri Aug  3 12:37:49 2018
> New Revision: 338913
>
> URL: http://llvm.org/viewvc/llvm-project?rev=338913&view=rev
> Log:
> [Support] Don't initialize compressed buffer allocated by zlib::compress
>
> resize() (zeroing) makes every allocated page resident. The actual size of
> the compressed buffer is usually much
> smaller. Making every page resident is wasteful.
>
> When linking a test binary with ~1.9GiB uncompressed debug info with LLD,
> this optimization decreases max RSS by ~1.5GiB.
>
> Differential Revision: https://reviews.llvm.org/50223
>
> Modified:
>     llvm/trunk/lib/Support/Compression.cpp
>
> Modified: llvm/trunk/lib/Support/Compression.cpp
> URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/Compression.cpp?rev=338913&r1=338912&r2=338913&view=diff
>
> ==============================================================================
> --- llvm/trunk/lib/Support/Compression.cpp (original)
> +++ llvm/trunk/lib/Support/Compression.cpp Fri Aug  3 12:37:49 2018
> @@ -61,7 +61,7 @@ Error zlib::compress(StringRef InputBuff
>                       SmallVectorImpl<char> &CompressedBuffer,
>                       CompressionLevel Level) {
>    unsigned long CompressedSize = ::compressBound(InputBuffer.size());
> -  CompressedBuffer.resize(CompressedSize);
> +  CompressedBuffer.reserve(CompressedSize);
>    int CLevel = encodeZlibCompressionLevel(Level);
>    int Res = ::compress2((Bytef *)CompressedBuffer.data(), &CompressedSize,
>                          (const Bytef *)InputBuffer.data(),
> InputBuffer.size(),
> @@ -69,7 +69,7 @@ Error zlib::compress(StringRef InputBuff
>    // Tell MemorySanitizer that zlib output buffer is fully initialized.
>    // This avoids a false report when running LLVM with uninstrumented
> ZLib.
>    __msan_unpoison(CompressedBuffer.data(), CompressedSize);
> -  CompressedBuffer.resize(CompressedSize);
> +  CompressedBuffer.set_size(CompressedSize);
>    return Res ? createError(convertZlibCodeToString(Res)) :
> Error::success();
>  }
>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20180803/7e28de99/attachment.html>