<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jun 23, 2016 at 10:25 AM, Vedant Kumar via llvm-commits <span dir="ltr"><<a href="mailto:llvm-commits@lists.llvm.org" target="_blank">llvm-commits@lists.llvm.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I have zlib-compression support for the encoded data section implemented locally, but don't think it's worth pursuing. Here are the numbers:<br>
<br>
Size of bin/opt in Release mode: 76M<br>
Size of bin/opt in Release mode + covmappings: 196M<br>
Size of the covmappings in bin/opt: 120M<br>
Size of the compressed covmapping file: 70M<br></blockquote><div><br></div><div>Is this the result of compression on a per-file basis? What is the size of the covmap section if it is compressed after aggregation?</div><div><br></div><div>David </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
ISTM that there *is* a real opportunity to reduce the size of the covmappings. However, it appears that *most* of the data redundancies don't occur at the encoded data section level. Instead, they occur once all of the headers, function records, and encoded data sections are aggregated into one __llvm_covmap section. To some level that's to be expected. I didn't expect that turning on compression at the encoded data section level would produce insignificant results.<br>
<br>
This is just a cautionary note for anyone interested in experimenting with this in the future.<br>
<span class="HOEnZb"><font color="#888888"><br>
vedant<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
> On Jun 17, 2016, at 2:53 PM, Vedant Kumar via llvm-commits <<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a>> wrote:<br>
><br>
> Author: vedantk<br>
> Date: Fri Jun 17 16:53:31 2016<br>
> New Revision: 273055<br>
><br>
> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=273055&view=rev" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project?rev=273055&view=rev</a><br>
> Log:<br>
> [Coverage] Move logic to encode filenames and mappings into llvm (NFC)<br>
><br>
> Currently, frontends which emit source-based code coverage have to<br>
> duplicate logic to encode filenames and raw coverage mappings properly.<br>
> This violates an abstraction layer and forces frontends to copy tricky<br>
> code.<br>
><br>
> Introduce llvm::coverage::encodeFilenamesAndRawMappings() to take care<br>
> of this.<br>
><br>
> This will help us experiment with zlib-compressing coverage mapping<br>
> data.<br>
><br>
> Modified:<br>
> llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h<br>
> llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp<br>
><br>
> Modified: llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h?rev=273055&r1=273054&r2=273055&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h?rev=273055&r1=273054&r2=273055&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h (original)<br>
> +++ llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h Fri Jun 17 16:53:31 2016<br>
> @@ -23,19 +23,6 @@<br>
> namespace llvm {<br>
> namespace coverage {<br>
><br>
> -/// \brief Writer of the filenames section for the instrumentation<br>
> -/// based code coverage.<br>
> -class CoverageFilenamesSectionWriter {<br>
> - ArrayRef<StringRef> Filenames;<br>
> -<br>
> -public:<br>
> - CoverageFilenamesSectionWriter(ArrayRef<StringRef> Filenames)<br>
> - : Filenames(Filenames) {}<br>
> -<br>
> - /// \brief Write encoded filenames to the given output stream.<br>
> - void write(raw_ostream &OS);<br>
> -};<br>
> -<br>
> /// \brief Writer for instrumentation based coverage mapping data.<br>
> class CoverageMappingWriter {<br>
> ArrayRef<unsigned> VirtualFileMapping;<br>
> @@ -57,6 +44,17 @@ public:<br>
> void write(raw_ostream &OS);<br>
> };<br>
><br>
> +/// \brief Encode a list of filenames and raw coverage mapping data using the<br>
> +/// latest coverage data format.<br>
> +///<br>
> +/// Set \p FilenamesSize to the size of the filenames section.<br>
> +///<br>
> +/// Set \p CoverageMappingsSize to the size of the coverage mapping section<br>
> +/// (including any necessary padding bytes).<br>
> +Expected<std::string> encodeFilenamesAndRawMappings(<br>
> + ArrayRef<std::string> Filenames, ArrayRef<std::string> CoverageMappings,<br>
> + size_t &FilenamesSize, size_t &CoverageMappingsSize);<br>
> +<br>
> } // end namespace coverage<br>
> } // end namespace llvm<br>
><br>
><br>
> Modified: llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp<br>
> URL: <a href="http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp?rev=273055&r1=273054&r2=273055&view=diff" rel="noreferrer" target="_blank">http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp?rev=273055&r1=273054&r2=273055&view=diff</a><br>
> ==============================================================================<br>
> --- llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp (original)<br>
> +++ llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp Fri Jun 17 16:53:31 2016<br>
> @@ -18,14 +18,6 @@<br>
> using namespace llvm;<br>
> using namespace coverage;<br>
><br>
> -void CoverageFilenamesSectionWriter::write(raw_ostream &OS) {<br>
> - encodeULEB128(Filenames.size(), OS);<br>
> - for (const auto &Filename : Filenames) {<br>
> - encodeULEB128(Filename.size(), OS);<br>
> - OS << Filename;<br>
> - }<br>
> -}<br>
> -<br>
> namespace {<br>
> /// \brief Gather only the expressions that are used by the mapping<br>
> /// regions in this function.<br>
> @@ -181,3 +173,52 @@ void CoverageMappingWriter::write(raw_os<br>
> // Ensure that all file ids have at least one mapping region.<br>
> assert(CurrentFileID == (VirtualFileMapping.size() - 1));<br>
> }<br>
> +<br>
> +/// \brief Encode coverage data into \p OS.<br>
> +static void encodeCoverageData(ArrayRef<std::string> Filenames,<br>
> + ArrayRef<std::string> CoverageMappings,<br>
> + size_t &FilenamesSize,<br>
> + size_t &CoverageMappingsSize, raw_ostream &OS) {<br>
> + size_t OSOffset = OS.GetNumBytesInBuffer();<br>
> +<br>
> + // Encode the filenames.<br>
> + encodeULEB128(Filenames.size(), OS);<br>
> + for (const auto &Filename : Filenames) {<br>
> + encodeULEB128(Filename.size(), OS);<br>
> + OS << Filename;<br>
> + }<br>
> +<br>
> + FilenamesSize = OS.GetNumBytesInBuffer() - OSOffset;<br>
> +<br>
> + // Encode the coverage mappings.<br>
> + for (const auto &RawMapping : CoverageMappings)<br>
> + OS << RawMapping;<br>
> +<br>
> + // Pad the output stream to an 8-byte boundary. Account for the padding bytes<br>
> + // in \p CoverageMappingsSize.<br>
> + if (size_t Rem = OS.GetNumBytesInBuffer() % 8) {<br>
> + CoverageMappingsSize += 8 - Rem;<br>
> + for (size_t I = 0, S = 8 - Rem; I < S; ++I)<br>
> + OS << '\0';<br>
> + }<br>
> +<br>
> + CoverageMappingsSize = OS.GetNumBytesInBuffer() - FilenamesSize - OSOffset;<br>
> +}<br>
> +<br>
> +namespace llvm {<br>
> +namespace coverage {<br>
> +<br>
> +Expected<std::string> encodeFilenamesAndRawMappings(<br>
> + ArrayRef<std::string> Filenames, ArrayRef<std::string> CoverageMappings,<br>
> + size_t &FilenamesSize, size_t &CoverageMappingsSize) {<br>
> + std::string CoverageData;<br>
> + {<br>
> + raw_string_ostream OS{CoverageData};<br>
> + encodeCoverageData(Filenames, CoverageMappings, FilenamesSize,<br>
> + CoverageMappingsSize, OS);<br>
> + }<br>
> + return std::move(CoverageData);<br>
> +}<br>
> +<br>
> +} // end namespace coverage<br>
> +} // end namespace llvm<br>
><br>
><br>
> _______________________________________________<br>
> llvm-commits mailing list<br>
> <a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
<br>
_______________________________________________<br>
llvm-commits mailing list<br>
<a href="mailto:llvm-commits@lists.llvm.org">llvm-commits@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits</a><br>
</div></div></blockquote></div><br></div></div>