[llvm-dev] InstrProf backward compatibility

[Vedant Kumar via llvm-dev]

Hi Johan,

> On Dec 18, 2015, at 5:37 AM, Johan Engelen via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> Hi all,
>   I am working on adding PGO to LDC (LLVM D Compiler). The current implementation
> 1) uses LLVM's InstrProf pass to generate an instrumented executable
> 2) links to compiler-rt/lib/profile for the runtime functionality to write a raw profile data file
> 3) uses llvm-profdata to merge profile data and convert from profraw to profdata format
> 4) uses llvm::IndexedInstrProfReader to read-in the profdata file for the 2nd PGO compile pass
> 
> This all works well with LLVM 3.7. While testing this with LLVM trunk, I run into a version compatibility problem between different versions of LLVM: I made a local copy of the compiler-rt/lib/profile release_37 version (with a few fixes for MSVC) and so steps 1,2, and 4 use LLVM trunk, but step 2 uses an older version of compiler-rt/lib/profile. Apparently, trunk's llvm-profdata cannot read raw profile data files generated by compiler-rt 3.7. Is this intended?

Yes, as far as I'm aware, the raw profile format is meant to buy llvm flexibility. There are backwards compatibility guarantees for the indexed (*.profdata) format. I'm not sure if these guarantees have been written down yet -- if they haven't I'll put a document together.

The profraw/profdata split is analogous to the textual IR / bitcode split.


> Is llvm-profdata's output also incompatible between different versions of LLVM (InstrProfWriter/Reader are not meant to be compatible across LLVM versions) ?

The indexed profdata reader for version X+1 of the format should be able to read versions <= X. There are no forward-compatibility promises: the reader for version X may not be able to read version X+k files.


> LDC is currently compatible with LLVM 3.5 up to trunk. To support PGO for multiple LLVM versions (>= 3.7), I think that we will need to have a copy of compiler-rt/lib/profile for each version of LLVM >= 3.7. We also need to have a copy of llvm-profdata for each version of LLVM >= 3.7. We already do this with the FileCheck utility and will not be a big problem.

You should only ever need the latest compiler-rt you can use (i.e, one compatible with your version of llvm). That should work if you only rely on backwards compatibility for *.profdata files.

Does this work for your purposes? Is there anything we can do to make life easier for downstream projects?


best,
vedant