[llvm] r338913 - [Support] Don't initialize compressed buffer allocated by zlib::compress

Fāng-ruì Sòng via llvm-commits llvm-commits at lists.llvm.org
Mon Aug 6 21:12:21 PDT 2018


I think such local and temporary contract violation is allowed here.

This is how append() is implemente in SmallVector.h

  void append(size_type NumInputs, const T &Elt) {
    // Grow allocated space if needed.
    if (NumInputs > this->capacity() - this->size())
      this->grow(this->size()+NumInputs);

    // Copy the new elements over.
    std::uninitialized_fill_n(this->end(), NumInputs, Elt);
    this->set_size(this->size() + NumInputs);
  }

And many other functions (push_back, emplace_back, ...) set size after
the storage is changed.
On Fri, Aug 3, 2018 at 12:42 PM David Blaikie <dblaikie at gmail.com> wrote:
>
> I don't think this is what Eli was suggesting.
>
> set_size should be used instead of resize/reserve here - it acts like resize, but doesn't initialize the data. Using only 'reserve' violates the SmallVector contract - because data is then written beyond the 'size'.
>
> Instead, use 'set_size' before the compression, and resize down to the final size after the compressed bytes have been written.
>
> On Fri, Aug 3, 2018 at 12:38 PM Fangrui Song via llvm-commits <llvm-commits at lists.llvm.org> wrote:
>>
>> Author: maskray
>> Date: Fri Aug  3 12:37:49 2018
>> New Revision: 338913
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=338913&view=rev
>> Log:
>> [Support] Don't initialize compressed buffer allocated by zlib::compress
>>
>> resize() (zeroing) makes every allocated page resident. The actual size of the compressed buffer is usually much
>> smaller. Making every page resident is wasteful.
>>
>> When linking a test binary with ~1.9GiB uncompressed debug info with LLD, this optimization decreases max RSS by ~1.5GiB.
>>
>> Differential Revision: https://reviews.llvm.org/50223
>>
>> Modified:
>>     llvm/trunk/lib/Support/Compression.cpp
>>
>> Modified: llvm/trunk/lib/Support/Compression.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Support/Compression.cpp?rev=338913&r1=338912&r2=338913&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Support/Compression.cpp (original)
>> +++ llvm/trunk/lib/Support/Compression.cpp Fri Aug  3 12:37:49 2018
>> @@ -61,7 +61,7 @@ Error zlib::compress(StringRef InputBuff
>>                       SmallVectorImpl<char> &CompressedBuffer,
>>                       CompressionLevel Level) {
>>    unsigned long CompressedSize = ::compressBound(InputBuffer.size());
>> -  CompressedBuffer.resize(CompressedSize);
>> +  CompressedBuffer.reserve(CompressedSize);
>>    int CLevel = encodeZlibCompressionLevel(Level);
>>    int Res = ::compress2((Bytef *)CompressedBuffer.data(), &CompressedSize,
>>                          (const Bytef *)InputBuffer.data(), InputBuffer.size(),
>> @@ -69,7 +69,7 @@ Error zlib::compress(StringRef InputBuff
>>    // Tell MemorySanitizer that zlib output buffer is fully initialized.
>>    // This avoids a false report when running LLVM with uninstrumented ZLib.
>>    __msan_unpoison(CompressedBuffer.data(), CompressedSize);
>> -  CompressedBuffer.resize(CompressedSize);
>> +  CompressedBuffer.set_size(CompressedSize);
>>    return Res ? createError(convertZlibCodeToString(Res)) : Error::success();
>>  }
>>
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits



-- 
宋方睿


More information about the llvm-commits mailing list