[llvm-dev] [llvm-pdbutil] : merge not working properly

[Zachary Turner via llvm-dev]

On Tue, Jan 15, 2019 at 2:50 AM Vivien Millet <vivien.millet at gmail.com>
wrote:

> Hello Zachary !
> Thanks for your time !
> So you are one of the happy guys who suffered from the lack of PDB format
> information :)
>
Yes, that would be me :)


> To be honest I'm really a beginner in the PDB stuff, I just read some llvm
> documentation to understand what went wrong when merging my PDBs.
> In my case, what I do with my team and try to achieve is this :
> - Run our application under a visual studio debugger
> - Generate JIT code ( using llvm MCJIT  )
> - Then, either :
>    - export as COFF obj file with dwarf information and then convert it
> with cv2pdb to obtain a pdb of my JIT symbols (what I do now)
>    - export directly to PDB my JIT debug info (what i would like to do, if
> you have an idea how..)
> - Detach the visual studio debugger
> - Merge my JIT pdb into a copy of the executable pdb (where things start
> to go bad..)
> - Replace original executable by the copy (creating a backup of original)
> - Reattach  the visual studio debugger to my executable (loading the new
> pdb version)
> - Debug JIT code with visual studio.
> - On each JIT rebuild, restart these steps from the original native
> executable PDB to avoid merge conflict between the multiple JIT iterations
>
Yea, it's an interesting use case.  It makes me think it would be nice if
the PDB format supported some way of having a symbol which simply refers to
another PDB file, that way you could re-write that PDB file at runtime once
all your code is jitted, and when the debugger tries to look up that
symbol, it finds a record that tells it to go check the other PDB file.

So, here are the things I think you would need to do:

1) Create a JIT module in the module list with a unique name.  All symbols
will go here.  llvm-pdbutil dump -modules shows you the list.  Be careful
about putting it at the end though, because there's already one at the end
called * LINKER * that is kind of special.  On the other hand, you don't
want to put it first because it means you will have to do lots of fixups on
the EXE PDB.  It's probably best to add it right before the linker module,
this has the least chance of breaking anything.

2) In the debug stream for this module, add all symbols.  You will need to
fix up their type indices.  As you noticed, llvm-pdbutil already merges
type information from the JIT PDB, so after merging the type indices in the
EXE PDB will be different than they were in the JIT PDB, but the symbol
records will refer to the JIT PDB type indices.  So these need to be fixed
up.  LLD already has code to do this, you can probably borrow a similar
algorithm with some slight modifications (lldb/COFF/PDB.cpp, search for
mergeSymbolRecords)

3) Merge in the new section contributions and section map.  See LLD again
for how to modify these.  Hopefully the object file you exported contains
relocated symbol addresses so you don't have to do any fixups here.

4) Merge in the publics and globals.  This shouldn't be too hard, I think
you can just iterate over them in the JIT PDB and add them to the new EXE
PDB.

You're kind of in uncharted territory here, so this is just a rough idea of
what needs to be done.  There may be other issues that you don't encounter
until you actually try it out.

Unfortunately I don't personally have the time to work on this, but it
sounds neat, and I'm happy to help if you run into questions or problems
along the way.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20190115/cfd5270a/attachment.html>