[LLVMdev] bit code file incompatibility due to debug info changes

[Eric Christopher]

On Thu, Nov 21, 2013 at 11:06 AM, Manman Ren <manman.ren at gmail.com> wrote:
>
>
>
> On Thu, Nov 21, 2013 at 10:57 AM, David Blaikie <dblaikie at gmail.com> wrote:
>>
>>
>>
>>
>> On Thu, Nov 21, 2013 at 10:38 AM, Manman Ren <manman.ren at gmail.com> wrote:
>>>
>>>
>>>
>>>
>>> On Tue, Nov 19, 2013 at 8:47 PM, Manman Ren <manman.ren at gmail.com> wrote:
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Nov 19, 2013 at 5:26 PM, Manman Ren <manman.ren at gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Nov 18, 2013 at 11:00 AM, Chandler Carruth
>>>>> <chandlerc at google.com> wrote:
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 18, 2013 at 10:55 AM, David Blaikie <dblaikie at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> It depends a bit, also, on what kind of guarantees we need to offer.
>>>>>>> If the guarantee when reading IR from disk is "will not crash" then there's
>>>>>>> nothing for it but to run full debug info verification.
>>>>>>>
>>>>>>> On the other hand, if we can assume that some specific metadata
>>>>>>> implies the correctness of some other metadata, then all we need to do is
>>>>>>> check a known debug info version number. If it's the current number, do
>>>>>>> nothing (assume the rest of the debug info is up to date and well formed),
>>>>>>> otherwise do all the work to find the debug info and dump it (no need to
>>>>>>> verify it - we just have to do the work to find it, so we don't go following
>>>>>>> bad links later on - that should be as easy as dropping the llvm.dbg.cu
>>>>>>> named node and removing all debug intrinsics and the instruction metadata
>>>>>>> line references). But this latter scheme isn't robust against arbitrary
>>>>>>> metadata (that could, coincidentally, have the right version number and
>>>>>>> arbitrary metadata that breaks all our debug info metadata assumptions)
>>>>>>>
>>>>>>> If the latter is sufficient for everyone's needs/principles, great.
>>>>>>
>>>>>>
>>>>>> This makes sense to me, but I see Eric's fundamental concern with
>>>>>> upgrading test cases.
>>>>>>
>>>>>>
>>>>>> One other possible idea: we could artificially force coarseness on
>>>>>> metadata while things are still iterating rapidly by just having a version
>>>>>> number that rotates every "release". (Even non-open-source-releases could be
>>>>>> handled by letting anyone, any time they need to rev it, update what the
>>>>>> current version is and update every test case to reference that version.) If
>>>>>> the version is nicely factored, it should at least be an obvious diff if
>>>>>> still huge.
>>>>>
>>>>>
>>>>> Adding a debug info version number and ignoring the debug info MDNodes
>>>>> when version does not match makes sense. And changing the version number
>>>>> with every release sounds reasonable.
>>>>>
>>>>> One implementation is
>>>>> 1> completely get rid of version number in the tag
>>>>>   LLVMDebugVersion is only used in two places and we should be able to
>>>>> remove it.
>>>>>     include/llvm/DebugInfo.h:    return getUnsignedField(0) &
>>>>> ~LLVMDebugVersionMask;
>>>>>     lib/IR/DIBuilder.cpp:  assert((Tag & LLVMDebugVersionMask) == 0 &&
>>>>>     lib/IR/DIBuilder.cpp:  return
>>>>> ConstantInt::get(Type::getInt32Ty(VMContext), Tag | LLVMDebugVersion);
>>>>>   I remember there was something that blocks David from completely
>>>>> getting rid of version number, but it seems the blocker is gone now.
>>>>>   This can be done separately as well.
>>>>>
>>>>> 2> introduce a debug info version in module flags similar to "dwarf
>>>>> version" module flag
>>>>>      LTO can correctly handle the module flag during linking and we
>>>>> only need to change one line in the debug info testing cases when we update
>>>>> debug info version number.
>>>>>      One issue is that we don't have a mode that emits a warning and
>>>>> outputs the largest value (we have a mode that emits a warning and outputs
>>>>> the value from the first module, but we can definitely add another mode if
>>>>> necessary)
>>>>>
>>>>> 3> when to drop the debug info MDNodes?
>>>>>      We could do this in the bit code loader but it seems a violation
>>>>> of layers. One possibility is to run a module pass right after loading the
>>>>> bit code, to remove debug intrinsics and debug tags on instructions.
>>>>
>>>>
>>>> We have an existing pass StripSymbols that can strip debug info in the
>>>> module.
>>>> So it is just a matter of adding that pass after bit code loader if the
>>>> debug info version number does not match.
>>>
>>>
>>> Can I assume no objection to the approach? :)
>>> Bill, are you okay with adding another mode to module flags? i.e. emits a
>>> warning and outputs the largest value.
>>
>>
>> How will this work as a module flag, though? If the flag gets merged and
>> maximized, how will we know which compile units have out of date debug info
>> metadata and drop them?
>
>
> The dropping part comes from item 3 above: adding StripSymbols pass right
> after bit code loader if the debug info version number does not match.
> Item 3 is shared between non-LTO and LTO. For LTO, we want the debug info
> version number gets merged and maximized.
>

You'll want to do it at loading time and not worry about merging at
all. I think this will need to go just after reading the module and
check the version against the hard coded version in the bitcode
reader/somewhere. I think we should do it there rather than as a later
pass as we really don't want to do anything to the metadata other than
strip it.

-eric

> Thanks
> Manman
>
>>
>>
>> - David
>>
>>>
>>>
>>> Thanks,
>>> Manman
>>>
>>>>
>>>> 4> how to report warning?
>>>>     Is errs() << "WARNING: " good enough? BTW this is how we emit
>>>> warning when linking module flags.
>>>>
>>>> Any objection to the above? Any other suggestion?
>>>>
>>>> Thanks,
>>>> Manman
>>>>
>>>>>
>>>>> _______________________________________________
>>>>> LLVM Developers mailing list
>>>>> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>>>>>
>>>>
>>>
>>>
>>
>