[llvm-dev] Opaque Pointers Help Wanted

Thu Jul 1 11:06:26 PDT 2021

On Thu, Jul 1, 2021 at 7:43 PM David Blaikie <dblaikie at gmail.com> wrote:

>
> On Thu, Jul 1, 2021 at 8:00 AM Nikita Popov <nikita.ppv at gmail.com> wrote:
>
>> On Thu, Jul 1, 2021 at 4:39 PM <paul.robinson at sony.com> wrote:
>>
>>> | My expectation is that after the opaque pointer migration is complete,
>>> the time the non-opaque pointer mode remains available will be measured in
>>> days, not years.
>>>
>>>
>>>
>>> That can be true for textual IR, but we need to retain backward
>>> compatibility (auto-upgrade) for bitcode for much longer.  I don’t have a
>>> clear idea of what’s involved (haven’t really been tracking this project)
>>> but many users/scenarios require bitcode compatibility for long periods.  I
>>> don’t think we’ve had a true format break since about 4.0?
>>>
>>> --paulr
>>>
>>
>> Sure, we definitely need to retain bitcode compatibility (i.e.
>> auto-upgrade to opaque pointers).
>>
>> My comment was targeted at maintainers of out-of-free frontends and
>> targets, who shouldn't assume there will be a long-term period of dual
>> compatibility, like there was/is with the new pass manager. If you don't
>> want to be stuck on an old toolchain, you need to be ready by the time the
>> switch happens.
>>
>
> Correct me if I'm wrong, but to clarify, I think: LLVM IR API likely won't
> have a long period of overlap supporting both opaque and non-opaque
> pointers and especially the ability to mix/match in IR may never exist
> (hard cutover) - as discussed with things like intrinsics recently, we'll
> likely have to have a hard cutover to opaque pointers. Textual IR may be
> able to be auto-upgraded for a time, and Bitcode certainly will be - so, in
> the worst case, if a downstream project needed to pickup a newer version of
> LLVM but didn't have time to invest it migrating their IR generation -
> maybe they could get away with linking an older LLVM into their frontend,
> then serializing roundtrip via bitcode back into a process with the newer
> LLVM that would convert to opaque pointers on loading.
>
> Is that about right?
>

Yeah, I believe that should work.

Regards,
Nikita

>   *From:* llvm-dev <llvm-dev-bounces at lists.llvm.org> *On Behalf Of *Nikita
> Popov via llvm-dev
>
>> *Sent:* Wednesday, June 30, 2021 2:02 PM
>> *To:* David Blaikie <dblaikie at gmail.com>
>> *Cc:* llvm-dev at lists.llvm.org
>> *Subject:* Re: [llvm-dev] Opaque Pointers Help Wanted
>>
>>
>>
>> On Tue, Jun 22, 2021 at 7:12 PM David Blaikie via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>
>>
>>
>>
>> On Tue, Jun 22, 2021 at 9:39 AM Kaylor, Andrew via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>> > I don't think metadata can reference an LLVM type. My previous hacky
>> suggestion was to add a new overloaded parameter with the type you want and
>> pass undef/poison to it. In any case, we'll have to find a way to fix these
>> sorts of intrinsics, we shouldn't block the opaque pointers project on some
>> intrinsics.
>>
>>
>>
>> It looks like I just missed your response before getting my response to
>> David in flight. Yes, I see the metadata problem. I don’t like the
>> undef/poison approach, but I agree that it would technically work.
>>
>>
>>
>> I completely agree that we should solve this problem rather than block
>> the opaque pointer project because of it. Nevertheless, we’ll need to solve
>> the problem.
>>
>>
>>
>>
>>
>> > LLVM already (mostly) treats memory as untyped, what is your intrinsic
>> attempting to do?
>>
>>
>>
>> It’s kind of like a GEP but more specialized. It’s doing a pointer
>> calculation based on multidimensional composite types with a shape that is
>> known at runtime but unknown at compile time, and we have a front end
>> that’s able to supply enough info to make this useful.
>>
>>
>>
>>
>>
>> > As for the timeline, we'll have to support mixed opaque pointers and
>> legacy pointers for a while, we don't want out of tree users to immediately
>> break as soon as the opaque pointers work is finished up in-tree.
>>
>>
>>
>> Is there any consensus on the scope of “for a while”? Like, how many
>> major releases after the opaque pointer work is complete? My manager would
>> very much like to know the answer to this question. :-) I’ve been trying to
>> prepare for the buffer being as little as one major release after the
>> in-tree work is done but hoping for more. I expect there are others with a
>> similar interest.
>>
>>
>>
>> By default I'd vote for one major release - but likely the ongoing burden
>> of carrying it for a bit longer if some significant contributors would
>> benefit from extra time it's probably going to be pretty harmless to keep
>> it around a bit longer, I think?
>>
>>
>>
>> Opaque pointers are not like the pass manager switch, where we can retain
>> support for the legacy pass manager at close to zero cost. Nearly all tests
>> work the same on both pass managers, so there is little ongoing maintenance
>> cost.
>>
>>
>>
>> This is not going to be the case with opaque pointers. Doing the switch
>> will require changes to nearly the whole test suite. This means that either
>> typed pointers will end up being entirely untested (or very weakly tested),
>> or we need to duplicate a very large fraction of our tests.
>>
>>
>>
>> My expectation is that after the opaque pointer migration is complete,
>> the time the non-opaque pointer mode remains available will be measured in
>> days, not years.
>>
>>
>>
>> Regards,
>>
>> Nikita
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *From:* Arthur Eubanks <aeubanks at google.com>
>> *Sent:* Tuesday, June 22, 2021 12:15 PM
>> *To:* Kaylor, Andrew <andrew.kaylor at intel.com>
>> *Cc:* llvm-dev at lists.llvm.org
>> *Subject:* Re: [llvm-dev] Opaque Pointers Help Wanted
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Jun 21, 2021 at 3:27 PM Kaylor, Andrew <andrew.kaylor at intel.com>
>> wrote:
>>
>> This has probably been discussed somewhere, but I missed it. Can you
>> elaborate a bit on this?
>>
>>
>>
>>    - Allow bitcode auto-upgrade of legacy pointer type to the new opaque
>>    pointer type (not to be turned on until ready)
>>
>>
>>    - To support legacy bitcode, such as legacy stores/loads, we need to
>>       track pointee types for all values since legacy instructions may infer the
>>       types from a pointer operand's pointee type
>>
>> I‘m specifically trying to understand what will happen when typed pointer
>> support is removed. How will IR with typed pointers be auto-upgraded to
>> pure opaque pointer IR? Will the bitcode reader keep some level of typed
>> pointer support indefinitely?
>>
>> Yes, the plan is something along the lines of associating each Value with
>> a possible pointee type inside the bitcode reader.
>>
>>
>>
>>
>>
>> Also, do you have a plan for replacing intrinsics that currently rely on
>> pointee types? For example, the load instruction was updated to take an
>> explicit type operand but I don’t think we can do the same thing for an
>> intrinsic like llvm.masked.load since there is Value for Type. This is an
>> easy problem to work around for something like masked.load, but more
>> complicated if anyone has a downstream GEP-like intrinsic that needs more
>> than the size of an element (spoiler alert: I do have such an intrinsic).
>> Would you use a metadata argument?
>>
>> I don't think metadata can reference an LLVM type. My previous hacky
>> suggestion was to add a new overloaded parameter with the type you want and
>> pass undef/poison to it. In any case, we'll have to find a way to fix these
>> sorts of intrinsics, we shouldn't block the opaque pointers project on some
>> intrinsics.
>>
>>
>>
>> LLVM already (mostly) treats memory as untyped, what is your intrinsic
>> attempting to do?
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> <https://urldefense.com/v3/__https:/lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!qYr6p_UsPommoeDmCaNhSX4hvxR3Uy9ym1IhWIFjgm_GXRDjrn7UxxzTONUejGpYGA$>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>> <https://urldefense.com/v3/__https:/lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!qYr6p_UsPommoeDmCaNhSX4hvxR3Uy9ym1IhWIFjgm_GXRDjrn7UxxzTONUejGpYGA$>
>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210701/20b07cd5/attachment.html>