<div dir="ltr"><div>Hi,</div><div><br></div><div>Which version of LLVM and CLANG do I need to test opaque pointers?</div><div>What command line options do I need to test opaque pointers?</div><div>When I say "test", I mean more like "experiment with".</div><div>I would like to use the LLVM IR Builder to create IR code that uses opaque pointers.<br></div><div>I would also like to use CLANG to create LLVM IR code that uses opaque pointers.</div><div></div><div><br></div><div>Kind Regards</div><div><br></div><div>James</div><div><br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 1 Jul 2021 at 22:49, Kaylor, Andrew via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="overflow-wrap: break-word;" lang="EN-US">
<div class="gmail-m_8848331123081085618WordSection1">
<p class="MsoNormal">> My comment was targeted at maintainers of out-of-free frontends and targets, who shouldn't assume there will be a long-term period of dual compatibility, like there was/is with the new pass manager. If you don't want to be stuck on an
old toolchain, you need to be ready by the time the switch happens.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I have serious concerns about this. I understand why you want to limit the time non-opaque pointers are supported, but having a sharp cutoff feels somewhat hostile to those who have a significant amount of out-of-tree code.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">I know the opaque pointer work has been well-publicized and ongoing for quite some time, but that’s part of the problem. A lot of work has gone into making opaque pointers work, and a corresponding amount if work may be required for out-of-tree
projects. My company, for example, has been following the work in LLVM and working on our own updates, but we’ve been allocating resources based on the pace of the open source work, which until recently has been fairly slow. We are adjusting our plans in accordance
with the new push, but it is going to take us some time to catch up.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">At a minimum, I think we need to agree upon a schedule for the transition so people can plan their work. Saying that we’re going to cut off support for a fundamental feature like this “when the migration is done” makes planning impossible.
Beyond that, I would vote for keeping the duplicate tests around for some period of time.<u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal">-Andy <u></u><u></u></p>
<p class="MsoNormal"><u></u> <u></u></p>
<div style="border-color:rgb(225,225,225) currentcolor currentcolor;border-style:solid none none;border-width:1pt medium medium;padding:3pt 0in 0in">
<p class="MsoNormal"><b>From:</b> llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>> <b>On Behalf Of
</b>Nikita Popov via llvm-dev<br>
<b>Sent:</b> Thursday, July 01, 2021 11:00 AM<br>
<b>To:</b> <a href="mailto:paul.robinson@sony.com" target="_blank">paul.robinson@sony.com</a><br>
<b>Cc:</b> llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] Opaque Pointers Help Wanted<u></u><u></u></p>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Thu, Jul 1, 2021 at 4:39 PM <<a href="mailto:paul.robinson@sony.com" target="_blank">paul.robinson@sony.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal">| My expectation is that after the opaque pointer migration is complete, the time the non-opaque pointer mode remains available will be measured in days, not years.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">That can be true for textual IR, but we need to retain backward compatibility (auto-upgrade) for bitcode for much longer. I don’t have a clear idea of what’s involved (haven’t
really been tracking this project) but many users/scenarios require bitcode compatibility for long periods. I don’t think we’ve had a true format break since about 4.0?<u></u><u></u></p>
<p class="MsoNormal">--paulr<u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Sure, we definitely need to retain bitcode compatibility (i.e. auto-upgrade to opaque pointers).<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">My comment was targeted at maintainers of out-of-free frontends and targets, who shouldn't assume there will be a long-term period of dual compatibility, like there was/is with the new pass manager. If you don't want to be stuck on an old
toolchain, you need to be ready by the time the switch happens.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Nikita<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-color:currentcolor currentcolor currentcolor rgb(204,204,204);border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal"> <b>From:</b> llvm-dev <<a href="mailto:llvm-dev-bounces@lists.llvm.org" target="_blank">llvm-dev-bounces@lists.llvm.org</a>>
<b>On Behalf Of </b>Nikita Popov via llvm-dev<br>
<b>Sent:</b> Wednesday, June 30, 2021 2:02 PM<br>
<b>To:</b> David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>><br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] Opaque Pointers Help Wanted<u></u><u></u></p>
<div style="border-style:none none none solid;border-width:medium medium medium 1.5pt;padding:0in 0in 0in 4pt;border-color:currentcolor currentcolor currentcolor blue">
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<div>
<p class="MsoNormal">On Tue, Jun 22, 2021 at 7:12 PM David Blaikie via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">On Tue, Jun 22, 2021 at 9:39 AM Kaylor, Andrew via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal">> I don't think metadata can reference an LLVM type. My previous hacky suggestion was to add a new overloaded parameter with the type you want and pass undef/poison to it. In any
case, we'll have to find a way to fix these sorts of intrinsics, we shouldn't block the opaque pointers project on some intrinsics.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">It looks like I just missed your response before getting my response to David in flight. Yes, I see the metadata problem. I don’t like the undef/poison approach, but I agree that
it would technically work.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">I completely agree that we should solve this problem rather than block the opaque pointer project because of it. Nevertheless, we’ll need to solve the problem.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">> LLVM already (mostly) treats memory as untyped, what is your intrinsic attempting to do?<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">It’s kind of like a GEP but more specialized. It’s doing a pointer calculation based on multidimensional composite types with a shape that is known at runtime but unknown at compile
time, and we have a front end that’s able to supply enough info to make this useful.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">> As for the timeline, we'll have to support mixed opaque pointers and legacy pointers for a while, we don't want out of tree users to immediately break as soon as the opaque pointers
work is finished up in-tree.<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Is there any consensus on the scope of “for a while”? Like, how many major releases after the opaque pointer work is complete? My manager would very much like to know the answer
to this question. :-) I’ve been trying to prepare for the buffer being as little as one major release after the in-tree work is done but hoping for more. I expect there are others with a similar interest.<u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">By default I'd vote for one major release - but likely the ongoing burden of carrying it for a bit longer if some significant contributors would benefit from extra time it's probably
going to be pretty harmless to keep it around a bit longer, I think?<u></u><u></u></p>
</div>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Opaque pointers are not like the pass manager switch, where we can retain support for the legacy pass manager at close to zero cost. Nearly all tests work the same on both pass
managers, so there is little ongoing maintenance cost.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">This is not going to be the case with opaque pointers. Doing the switch will require changes to nearly the whole test suite. This means that either typed pointers will end up being
entirely untested (or very weakly tested), or we need to duplicate a very large fraction of our tests.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">My expectation is that after the opaque pointer migration is complete, the time the non-opaque pointer mode remains available will be measured in days, not years.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Regards,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Nikita<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<div>
<p class="MsoNormal">
<u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<div style="border-style:solid none none;border-width:1pt medium medium;padding:3pt 0in 0in;border-color:currentcolor">
<p class="MsoNormal"><b>From:</b> Arthur Eubanks <<a href="mailto:aeubanks@google.com" target="_blank">aeubanks@google.com</a>>
<br>
<b>Sent:</b> Tuesday, June 22, 2021 12:15 PM<br>
<b>To:</b> Kaylor, Andrew <<a href="mailto:andrew.kaylor@intel.com" target="_blank">andrew.kaylor@intel.com</a>><br>
<b>Cc:</b> <a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<b>Subject:</b> Re: [llvm-dev] Opaque Pointers Help Wanted<u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<p class="MsoNormal">On Mon, Jun 21, 2021 at 3:27 PM Kaylor, Andrew <<a href="mailto:andrew.kaylor@intel.com" target="_blank">andrew.kaylor@intel.com</a>> wrote:<u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal">This has probably been discussed somewhere, but I missed it. Can you elaborate a bit on this?<u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoNormal" style="color:rgb(36,41,46);margin-top:3pt;background:white none repeat scroll 0% 0%">
<span style="font-size:12pt;font-family:"-apple-system",serif">Allow bitcode auto-upgrade of legacy pointer type to the new opaque pointer type (not to be turned on until ready)</span><u></u><u></u></li></ul>
<ul type="disc">
<ul type="circle">
<li class="MsoNormal" style="color:rgb(36,41,46);background:white none repeat scroll 0% 0%">
<span style="font-size:12pt;font-family:"-apple-system",serif">To support legacy bitcode, such as legacy stores/loads, we need to track pointee types for all values since legacy instructions may infer the types from a pointer operand's pointee type</span><u></u><u></u></li></ul>
</ul>
<p class="MsoNormal">I‘m specifically trying to understand what will happen when typed pointer support is removed. How will IR with typed pointers be auto-upgraded to pure opaque pointer IR? Will the
bitcode reader keep some level of typed pointer support indefinitely?<u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal">Yes, the plan is something along the lines of associating each Value with a possible pointee type inside the bitcode reader.<u></u><u></u></p>
</div>
<blockquote style="border-style:none none none solid;border-width:medium medium medium 1pt;padding:0in 0in 0in 6pt;margin:5pt 0in 5pt 4.8pt;border-color:currentcolor currentcolor currentcolor rgb(204,204,204)">
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal"> <u></u><u></u></p>
<p class="MsoNormal">Also, do you have a plan for replacing intrinsics that currently rely on pointee types? For example, the load instruction was updated to take an explicit type operand but I don’t
think we can do the same thing for an intrinsic like llvm.masked.load since there is Value for Type. This is an easy problem to work around for something like masked.load, but more complicated if anyone has a downstream GEP-like intrinsic that needs more than
the size of an element (spoiler alert: I do have such an intrinsic). Would you use a metadata argument?<u></u><u></u></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal">I don't think metadata can reference an LLVM type. My previous hacky suggestion was to add a new overloaded parameter with the type you want and pass undef/poison to it. In any
case, we'll have to find a way to fix these sorts of intrinsics, we shouldn't block the opaque pointers project on some intrinsics.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">LLVM already (mostly) treats memory as untyped, what is your intrinsic attempting to do?<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://urldefense.com/v3/__https:/lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!qYr6p_UsPommoeDmCaNhSX4hvxR3Uy9ym1IhWIFjgm_GXRDjrn7UxxzTONUejGpYGA$" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
</div>
<p class="MsoNormal">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://urldefense.com/v3/__https:/lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev__;!!JmoZiZGBv3RvKRSx!qYr6p_UsPommoeDmCaNhSX4hvxR3Uy9ym1IhWIFjgm_GXRDjrn7UxxzTONUejGpYGA$" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><u></u><u></u></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" rel="noreferrer" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><br>
</blockquote></div>