[llvm-dev] InstrProf backward compatibility

[Johan Engelen via llvm-dev]

Thanks for the clear answers.

At the moment, my only request would be to also make sure that profile lib
works with MSVC (3.7 needed small patching, trunk is fine now).

Thanks!
  Johan


On Fri, Dec 18, 2015 at 7:49 PM, Xinliang David Li <davidxl at google.com>
wrote:

> On Fri, Dec 18, 2015 at 5:37 AM, Johan Engelen <jbc.engelen at gmail.com>
> wrote:
> > Hi all,
> >   I am working on adding PGO to LDC (LLVM D Compiler). The current
> > implementation
> > 1) uses LLVM's InstrProf pass to generate an instrumented executable
> > 2) links to compiler-rt/lib/profile for the runtime functionality to
> write a
> > raw profile data file
> > 3) uses llvm-profdata to merge profile data and convert from profraw to
> > profdata format
> > 4) uses llvm::IndexedInstrProfReader to read-in the profdata file for the
> > 2nd PGO compile pass
> >
> > This all works well with LLVM 3.7. While testing this with LLVM trunk, I
> run
> > into a version compatibility problem between different versions of LLVM:
> I
> > made a local copy of the compiler-rt/lib/profile release_37 version
> (with a
> > few fixes for MSVC) and so steps 1,2, and 4 use LLVM trunk, but step 2
> uses
> > an older version of compiler-rt/lib/profile. Apparently, trunk's
>
> Runtime and InstrProf pass have to be in sync. You can not use 3.7
> version of runtime with trunk version of 1).
>
> Perhaps you want to explain more about your use cases? If you want to
> use 3.7 compiler/runtime to produce profile to be used by trunk
> compiler -- that is doable.
>
> 1) instrument the code with 3.7 compiler and link with 3.7 runtime
> 2) run and dump the profile
> 3) use 3.7 llvm-profdata tool to do a merge and generate indexed format
> profile
> 4) use that profile with the trunk compiler
>
> However, the other direction does not work -- you can not expect older
> version of compiler to consume indexed format profile of the new
> compiler.
>
>
> > llvm-profdata cannot read raw profile data files generated by compiler-rt
> > 3.7. Is this intended?
>
> yes. New version of raw profile reader does not understand old version
> of raw profile data. You need to use older version of llvm-profdata to
> produce old version of indexed format, and then use new version of
> llvm-profdata to convert the indexed format into new version. Here is
> flow:
>
> 1) llvm-profdata_3.7 merge -o my_profdata_old.profdata
> my_profdata_old.profraw
>
> At this point, my_profdata_old.profdata *can* be consumed by trunk
> version of compiler already, but if you want to convert it to new
> version of indexed format, do this:
>
> 2) llvm-profdata_trunk merge -o my_profdata_new.profdata
> my_profdata_old.profdata
>
> Now my_profdata_new.profdata is in shiny new version (indexed).
>
>
>  Is llvm-profdata's output also incompatible between
> > different versions of LLVM (InstrProfWriter/Reader are not meant to be
> > compatible across LLVM versions) ?
>
> The merge output format (indexed format) is backward compatible as
> explained by Vedant.
>
> >
> > LDC is currently compatible with LLVM 3.5 up to trunk. To support PGO for
> > multiple LLVM versions (>= 3.7), I think that we will need to have a
> copy of
> > compiler-rt/lib/profile for each version of LLVM >= 3.7. We also need to
> > have a copy of llvm-profdata for each version of LLVM >= 3.7. We already
> do
> > this with the FileCheck utility and will not be a big problem.
> >
>
> yes -- see above. Remember compiler instrumentation and supporting
> runtime must be in sync.
>
> thanks,
>
> David
>
>
> > Does that make sense? What do you advise?
> >
> > Thanks very much for your comments,
> >   Johan
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151219/53d1dc72/attachment.html>