<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:DengXian;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@DengXian";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:294796631;
mso-list-template-ids:-1010663202;}
@list l0:level1
{mso-level-start-at:2;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1
{mso-list-id:490099666;
mso-list-template-ids:-990379300;}
@list l1:level1
{mso-level-start-at:3;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2
{mso-list-id:725952578;
mso-list-template-ids:1333276444;}
@list l2:level1
{mso-level-start-at:2;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3
{mso-list-id:1572345384;
mso-list-template-ids:1299887118;}
@list l3:level1
{mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l3:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">I write a patch (<a href="https://reviews.llvm.org/D93788">https://reviews.llvm.org/D93788</a>) to transform the load/store x86_amx* to amx intrinsics. The effort is much more than disable the bitcast from load/store <256 x i32>* to load/store
x86_amx*.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Luo, Yuanke <br>
<b>Sent:</b> Friday, March 19, 2021 9:58 AM<br>
<b>To:</b> James Y Knight <jyknight@google.com><br>
<b>Cc:</b> Florian Hahn <florian_hahn@apple.com>; Wang, Pengfei <Pengfei.Wang@intel.com>; llvm-dev <llvm-dev@lists.llvm.org><br>
<b>Subject:</b> RE: [llvm-dev] Does middle-end pass need to consider some special type when doing optimization? Or letting back-end to revert the optimization accordingly?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi James,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thank you for taking the time to deep dive the issue. It is very constructive. I agree we can transform “load x86_amx*” to amx load intrinsic. But it seems we need more effort to do the transform than preventing generate “load x86_amx*”.
I can support transforming “load x86_amx*” to amx load intrinsic if people like this approach.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I also think Florian raise a good question. What the semantics about “load x86_amx*”. Is it different semantics than regular LLVM pointer types? What’s your opinions on it?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks<o:p></o:p></p>
<p class="MsoNormal">Yuanke<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> James Y Knight <<a href="mailto:jyknight@google.com">jyknight@google.com</a>>
<br>
<b>Sent:</b> Friday, March 19, 2021 9:28 AM<br>
<b>To:</b> Luo, Yuanke <<a href="mailto:yuanke.luo@intel.com">yuanke.luo@intel.com</a>><br>
<b>Cc:</b> Florian Hahn <<a href="mailto:florian_hahn@apple.com">florian_hahn@apple.com</a>>; Wang, Pengfei <<a href="mailto:pengfei.wang@intel.com">pengfei.wang@intel.com</a>>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] Does middle-end pass need to consider some special type when doing optimization? Or letting back-end to revert the optimization accordingly?<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Why is that harder than lowering a load <256 x i32> and then bitcast to x86_amx?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">E.g., I see there is in llvm/lib/Target/X86/X86LowerAMXType.cpp a transform:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%src = load <256 x i32>, <256 x i32>* %addr, align 64<br>
%2 = bitcast <256 x i32> %src to x86_amx<br>
--><br>
%2 = call x86_amx @llvm.x86.tileloadd64.internal(i16 %row, i16 %col, i8* %addr, i64 %stride64)<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Isn't it equivalent, then, to do:<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%2 = load x86_amx, x86_amx* %addr, align 64<br>
--><br>
%2 = call x86_amx @llvm.x86.tileloadd64.internal(i16 %row, i16 %col, i8* %addr, i64 %stride64)<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Thu, Mar 18, 2021 at 9:29 AM Luo, Yuanke <<a href="mailto:yuanke.luo@intel.com" target="_blank">yuanke.luo@intel.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">But x86_amx represent a tile. The semantics of hardware instruction tileloadd is something like ‘llvm.matrix.row.major.load’. How do we lower `%v = load x86_amx, x86_amx* %ptr`
to tileloadd?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> James Y Knight <<a href="mailto:jyknight@google.com" target="_blank">jyknight@google.com</a>>
<br>
<b>Sent:</b> Thursday, March 18, 2021 9:09 PM<br>
<b>To:</b> Luo, Yuanke <<a href="mailto:yuanke.luo@intel.com" target="_blank">yuanke.luo@intel.com</a>><br>
<b>Cc:</b> Florian Hahn <<a href="mailto:florian_hahn@apple.com" target="_blank">florian_hahn@apple.com</a>>; Wang, Pengfei <<a href="mailto:pengfei.wang@intel.com" target="_blank">pengfei.wang@intel.com</a>>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] Does middle-end pass need to consider some special type when doing optimization? Or letting back-end to revert the optimization accordingly?<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Since the x86_amx type has a fixed size of 1024, I would expect `%v = load x86_amx, x86_amx* %ptr` to load 1024 bytes of contiguous memory starting at %ptr -- I don't see why this
should be invalid?<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Thu, Mar 18, 2021 at 8:53 AM Luo, Yuanke <<a href="mailto:yuanke.luo@intel.com" target="_blank">yuanke.luo@intel.com</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I mean transforming from “load <256 x i32>*” to “load x86_amx*” is not invalid because x86_amx represent a tile and “load x86_amx*” doesn’t express its semantics without a stride.
Now it looks to me “load x86_amx*” is invalid.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> James Y Knight <<a href="mailto:jyknight@google.com" target="_blank">jyknight@google.com</a>>
<br>
<b>Sent:</b> Thursday, March 18, 2021 8:41 PM<br>
<b>To:</b> Luo, Yuanke <<a href="mailto:yuanke.luo@intel.com" target="_blank">yuanke.luo@intel.com</a>><br>
<b>Cc:</b> Florian Hahn <<a href="mailto:florian_hahn@apple.com" target="_blank">florian_hahn@apple.com</a>>; Wang, Pengfei <<a href="mailto:pengfei.wang@intel.com" target="_blank">pengfei.wang@intel.com</a>>; llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>><br>
<b>Subject:</b> Re: [llvm-dev] Does middle-end pass need to consider some special type when doing optimization? Or letting back-end to revert the optimization accordingly?<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Err...are you saying this is the expected semantics of a "load x86_amx" operation today? That doesn't make much sense...Surely a strided-load operation should be spelled `llvm.matrix.column.major.load`
in the IR, not `load`?<o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Thu, Mar 18, 2021 at 8:17 AM Luo, Yuanke via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thank Florian. I agree with you that pointers to `x86_amx` have different semantics than regular LLVM pointer types. First the x86_amx pointer point to a 2D array of a big matrix.
The data of each row is contiguous, but the data on contiguous row is not contiguous in memory. Below picture shows the x86_amx load semantics. We need another operand stride to describe the stride of each rows. So the semantics for “load <256xi32>*” and
“load x86_amx” is different. Because “load <256 x i32>* assume the memory is contiguous and load a flat vector.<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">You also mention that there is no documentation of x86_amx in the langref. I’d like to add x86_amx to the document. Is there any process to document for a type?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><sup> </sup><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><img border="0" width="643" height="245" style="width:6.7013in;height:2.5555in" id="gmail-m_2182852942543212951gmail-m_-8126207066208387751gmail-m_891449304947194046gmail-m_4077126272188660151Picture_x0020_1" src="cid:image001.jpg@01D71DCA.BC549CA0"><o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Yuanke<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b>From:</b> Florian Hahn <<a href="mailto:florian_hahn@apple.com" target="_blank">florian_hahn@apple.com</a>>
<br>
<b>Sent:</b> Thursday, March 18, 2021 6:03 PM<br>
<b>To:</b> Wang, Pengfei <<a href="mailto:pengfei.wang@intel.com" target="_blank">pengfei.wang@intel.com</a>><br>
<b>Cc:</b> llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>>; Luo, Yuanke <<a href="mailto:yuanke.luo@intel.com" target="_blank">yuanke.luo@intel.com</a>><br>
<b>Subject:</b> Re: [llvm-dev] Does middle-end pass need to consider some special type when doing optimization? Or letting back-end to revert the optimization accordingly?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;margin-bottom:12.0pt"> <o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">On Mar 17, 2021, at 10:11, Wang, Pengfei via llvm-dev <<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We are developing prototypes for Intel Advanced Matrix Extensions (AMX) [1] programing model in Clang and LLVM [2].<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">We met several cases when the certain type we added are optimized unexpectedly in the middle-end. E.g. optimizing phi + biscast + load:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">From<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%a = load <256 x i32>, <256 x i32>* %mem, align 64<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">… …<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%b = phi <256 x i32> [ %a, %label1 ], [%someother, %label2]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%c = bitcast <256 x i32> %b to x86_amx<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">To<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%a = bitcast <256 x i32>* %mem to x86_amx*<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%b = load x86_amx, x86_amx*, align 64<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">… …<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">%c = phi x86_amx [ %b, %label1 ], [%someother, %label2]<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">To prevent such unexpected transforms, we concretely added the type check in each point of the optimizations.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Roman pointed out the changes are not the right direction [3], and thought it’s bug for backend. While we agreed backend might be able to handle it for the functionality, we think
it is better to handle it in the midden-end since they are negative optimizations for AMX.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">First, let me put some background here:<o:p></o:p></p>
</div>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoNormal" style="mso-list:l3 level1 lfo1">x86_amx* is different from trivial pointers.<o:p></o:p></li></ol>
<div style="margin-left:.5in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The AMX load instruction is much different from other load instructions. It is not only need the memory address but also the shape / stride of the tile register. We did some extra
work in the backend to deduce the shape information from the context. We don’t want the pass to add new x86_amx related usage because this will result in the difficulty in deduction. That said bitcasting other pointer types to x86_amx* is not trivial as assumed
here.<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">The problem appears to be that this difference is not modeled or specified in LLVM IR AFAICT. The current LangRef does not appear to specific `x86_amx` to start with. If pointers
to `x86_amx` have different semantics than regular LLVM pointer types, using regular LLVM pointer types for pointers to `x86_amx` may not be appropriate. I’ve not followed the previous AMX discussions closely, but it sounds like it may be good to reconsider
how x86_amx pointers are modeled in LVM IR.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Also note that `bitcast` is specified as `no-op` (<a href="https://llvm.org/docs/LangRef.html#id293" target="_blank">https://llvm.org/docs/LangRef.html#id293</a>) (expect for pointers
with different address spaces), but from what you mentioned above this does not match the semantics for `x86_amx*`. It sounds like this is the underlying problem that should be addressed, because trying to update various middle end optimization tot ry to enforce
the special semantics does not seem to be a scalable solution. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">As Nuno mentioned, you could try and use a separate address space for `x86_amx` pointers to avoid pointer optimizations.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<ol style="margin-top:0in" start="2" type="1">
<li class="MsoNormal" style="mso-list:l0 level1 lfo2">The physical tile registers have more limitations. <o:p></o:p></li></ol>
<ol style="margin-top:0in" start="2" type="1">
<ol style="margin-top:0in" start="1" type="a">
<li class="MsoNormal" style="mso-list:l2 level2 lfo3">No copy instruction between tile registers.<o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level2 lfo3">Spilling / reload a tile register is expensive in light of its size is 1024 bytes.<o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level2 lfo3">The shapes of tile registers need to be pre-configured before use and all data in tile registers will turn into invalid once re-configured. That said we need to dominate as more tile registers as possible
to configure their shapes with one configure instruction, otherwise we need to spill and reload the live registers once we need to re-configure.<o:p></o:p></li><li class="MsoNormal" style="mso-list:l2 level2 lfo3">The number of tile registers is rather small (only 8) and different shapes cannot be reused.<o:p></o:p></li></ol>
</ol>
<div style="margin-left:.5in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Based on the limitations, we need to reduce the use / live range of tile registers. But optimizations may increase the opportunity of the use. So even we can handle some combined
operation for AMX type, we still prefer to prevent it from the beginning. Unless we can totally roll back the optimization. Which is also not a good solution in my opinion.<o:p></o:p></p>
</div>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<ol style="margin-top:0in" start="3" type="1">
<li class="MsoNormal" style="mso-list:l1 level1 lfo4">For more information, please refer to discussion in [3].<o:p></o:p></li></ol>
</div>
</blockquote>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">For other optimization points, please refer [4][5].<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">I think the main controversy from Roman is if middle-end pass should consider some special type when doing optimization. I tend to let middle-end do the type check on account of
the peculiarity of AMX type. But I’m not sure if we have precedent to handle the similar issue in other targets. I’m open and glad to do it either way so long as we have an elegant solution.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Any suggestions are welcome.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
</div>
</blockquote>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">IIUC the main problem is not that middle-end passes perform or not perform optimizations based on certain types. To me it sounds like the actual problem is that pointers to `x86_amx`
do not behave like regular LLVM IR pointers and you are trying to enforce extra restrictions for bit casts.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Cheers,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Florian<o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">_______________________________________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank">llvm-dev@lists.llvm.org</a><br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</body>
</html>