[llvm] r273055 - [Coverage] Move logic to encode filenames and mappings into llvm (NFC)

Thu Jun 23 10:38:06 PDT 2016

On Thu, Jun 23, 2016 at 10:25 AM, Vedant Kumar via llvm-commits <
llvm-commits at lists.llvm.org> wrote:

> I have zlib-compression support for the encoded data section implemented
> locally, but don't think it's worth pursuing. Here are the numbers:
>
>   Size of bin/opt in Release mode: 76M
>   Size of bin/opt in Release mode + covmappings: 196M
>   Size of the covmappings in bin/opt: 120M
>   Size of the compressed covmapping file: 70M
>

Is this the result of compression on a per-file basis?  What is the size of
the covmap section if it is compressed after aggregation?

David

>
> ISTM that there *is* a real opportunity to reduce the size of the
> covmappings. However, it appears that *most* of the data redundancies don't
> occur at the encoded data section level. Instead, they occur once all of
> the headers, function records, and encoded data sections are aggregated
> into one __llvm_covmap section. To some level that's to be expected. I
> didn't expect that turning on compression at the encoded data section level
> would produce insignificant results.
>
> This is just a cautionary note for anyone interested in experimenting with
> this in the future.
>
> vedant
>
> > On Jun 17, 2016, at 2:53 PM, Vedant Kumar via llvm-commits <
> llvm-commits at lists.llvm.org> wrote:
> >
> > Author: vedantk
> > Date: Fri Jun 17 16:53:31 2016
> > New Revision: 273055
> >
> > URL: http://llvm.org/viewvc/llvm-project?rev=273055&view=rev
> > Log:
> > [Coverage] Move logic to encode filenames and mappings into llvm (NFC)
> >
> > Currently, frontends which emit source-based code coverage have to
> > duplicate logic to encode filenames and raw coverage mappings properly.
> > This violates an abstraction layer and forces frontends to copy tricky
> > code.
> >
> > Introduce llvm::coverage::encodeFilenamesAndRawMappings() to take care
> > of this.
> >
> > This will help us experiment with zlib-compressing coverage mapping
> > data.
> >
> > Modified:
> >    llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h
> >    llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp
> >
> > Modified:
> llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h
> > URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h?rev=273055&r1=273054&r2=273055&view=diff
> >
> ==============================================================================
> > --- llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h
> (original)
> > +++ llvm/trunk/include/llvm/ProfileData/Coverage/CoverageMappingWriter.h
> Fri Jun 17 16:53:31 2016
> > @@ -23,19 +23,6 @@
> > namespace llvm {
> > namespace coverage {
> >
> > -/// \brief Writer of the filenames section for the instrumentation
> > -/// based code coverage.
> > -class CoverageFilenamesSectionWriter {
> > -  ArrayRef<StringRef> Filenames;
> > -
> > -public:
> > -  CoverageFilenamesSectionWriter(ArrayRef<StringRef> Filenames)
> > -      : Filenames(Filenames) {}
> > -
> > -  /// \brief Write encoded filenames to the given output stream.
> > -  void write(raw_ostream &OS);
> > -};
> > -
> > /// \brief Writer for instrumentation based coverage mapping data.
> > class CoverageMappingWriter {
> >   ArrayRef<unsigned> VirtualFileMapping;
> > @@ -57,6 +44,17 @@ public:
> >   void write(raw_ostream &OS);
> > };
> >
> > +/// \brief Encode a list of filenames and raw coverage mapping data
> using the
> > +/// latest coverage data format.
> > +///
> > +/// Set \p FilenamesSize to the size of the filenames section.
> > +///
> > +/// Set \p CoverageMappingsSize to the size of the coverage mapping
> section
> > +/// (including any necessary padding bytes).
> > +Expected<std::string> encodeFilenamesAndRawMappings(
> > +    ArrayRef<std::string> Filenames, ArrayRef<std::string>
> CoverageMappings,
> > +    size_t &FilenamesSize, size_t &CoverageMappingsSize);
> > +
> > } // end namespace coverage
> > } // end namespace llvm
> >
> >
> > Modified: llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp
> > URL:
> http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp?rev=273055&r1=273054&r2=273055&view=diff
> >
> ==============================================================================
> > --- llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp
> (original)
> > +++ llvm/trunk/lib/ProfileData/Coverage/CoverageMappingWriter.cpp Fri
> Jun 17 16:53:31 2016
> > @@ -18,14 +18,6 @@
> > using namespace llvm;
> > using namespace coverage;
> >
> > -void CoverageFilenamesSectionWriter::write(raw_ostream &OS) {
> > -  encodeULEB128(Filenames.size(), OS);
> > -  for (const auto &Filename : Filenames) {
> > -    encodeULEB128(Filename.size(), OS);
> > -    OS << Filename;
> > -  }
> > -}
> > -
> > namespace {
> > /// \brief Gather only the expressions that are used by the mapping
> > /// regions in this function.
> > @@ -181,3 +173,52 @@ void CoverageMappingWriter::write(raw_os
> >   // Ensure that all file ids have at least one mapping region.
> >   assert(CurrentFileID == (VirtualFileMapping.size() - 1));
> > }
> > +
> > +/// \brief Encode coverage data into \p OS.
> > +static void encodeCoverageData(ArrayRef<std::string> Filenames,
> > +                               ArrayRef<std::string> CoverageMappings,
> > +                               size_t &FilenamesSize,
> > +                               size_t &CoverageMappingsSize,
> raw_ostream &OS) {
> > +  size_t OSOffset = OS.GetNumBytesInBuffer();
> > +
> > +  // Encode the filenames.
> > +  encodeULEB128(Filenames.size(), OS);
> > +  for (const auto &Filename : Filenames) {
> > +    encodeULEB128(Filename.size(), OS);
> > +    OS << Filename;
> > +  }
> > +
> > +  FilenamesSize = OS.GetNumBytesInBuffer() - OSOffset;
> > +
> > +  // Encode the coverage mappings.
> > +  for (const auto &RawMapping : CoverageMappings)
> > +    OS << RawMapping;
> > +
> > +  // Pad the output stream to an 8-byte boundary. Account for the
> padding bytes
> > +  // in \p CoverageMappingsSize.
> > +  if (size_t Rem = OS.GetNumBytesInBuffer() % 8) {
> > +    CoverageMappingsSize += 8 - Rem;
> > +    for (size_t I = 0, S = 8 - Rem; I < S; ++I)
> > +      OS << '\0';
> > +  }
> > +
> > +  CoverageMappingsSize = OS.GetNumBytesInBuffer() - FilenamesSize -
> OSOffset;
> > +}
> > +
> > +namespace llvm {
> > +namespace coverage {
> > +
> > +Expected<std::string> encodeFilenamesAndRawMappings(
> > +    ArrayRef<std::string> Filenames, ArrayRef<std::string>
> CoverageMappings,
> > +    size_t &FilenamesSize, size_t &CoverageMappingsSize) {
> > +  std::string CoverageData;
> > +  {
> > +    raw_string_ostream OS{CoverageData};
> > +    encodeCoverageData(Filenames, CoverageMappings, FilenamesSize,
> > +                       CoverageMappingsSize, OS);
> > +  }
> > +  return std::move(CoverageData);
> > +}
> > +
> > +} // end namespace coverage
> > +} // end namespace llvm
> >
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at lists.llvm.org
> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160623/921e6b78/attachment.html>