<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:SimSun;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@SimSun";
panose-1:2 1 6 0 3 1 1 1 1 1;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:452409354;
mso-list-type:hybrid;
mso-list-template-ids:1571083742 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1
{mso-list-id:542442239;
mso-list-template-ids:1132763184;}
@list l1:level1
{mso-level-start-at:3;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l2
{mso-list-id:617101377;
mso-list-type:hybrid;
mso-list-template-ids:269141050 733665766 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l2:level1
{mso-level-start-at:372;
mso-level-number-format:bullet;
mso-level-text:\F06E;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;
mso-fareast-font-family:SimSun;
mso-bidi-font-family:"Times New Roman";}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l3
{mso-list-id:1050038813;
mso-list-type:hybrid;
mso-list-template-ids:736769252 -730672338 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l3:level1
{mso-level-start-at:372;
mso-level-number-format:bullet;
mso-level-text:\F06E;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;
mso-fareast-font-family:SimSun;
mso-bidi-font-family:"Times New Roman";}
@list l3:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l3:level4
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l3:level7
{mso-level-number-format:bullet;
mso-level-text:\F0B7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l3:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l3:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l4
{mso-list-id:1431118370;
mso-list-template-ids:845070022;}
@list l5
{mso-list-id:1579827173;
mso-list-template-ids:-80676614;}
@list l5:level1
{mso-level-start-at:2;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l5:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><a id="OWAAM44DB084C66B74C0F8E6CFD1C71D8747E" href="mailto:lebedev.ri@gmail.com"><span style="font-family:"Calibri",sans-serif;text-decoration:none">@lebedev.ri@gmail.com</span></a><o:p></o:p></p>
<p class="MsoNormal">Current we see “ if (Ty.isVectorTy()) {…}” is make sense in Mid-End.
<o:p></o:p></p>
<p class="MsoNormal">Why we can’t see “if (Ty.isX86_AMXTy()){…}” is make sense ? <o:p>
</o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Just because more targets support the VectorTy, less target (only x86) support the AMXTy ?<o:p></o:p></p>
<p class="MsoNormal">The logic is not make sense.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">-xiang<o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Wang, Pengfei <pengfei.wang@intel.com> <br>
<b>Sent:</b> Wednesday, March 17, 2021 6:11 PM<br>
<b>To:</b> llvm-dev <llvm-dev@lists.llvm.org><br>
<b>Cc:</b> lebedev.ri@gmail.com; Luo, Yuanke <yuanke.luo@intel.com>; Zhang, Xiang1 <xiang1.zhang@intel.com><br>
<b>Subject:</b> Does middle-end pass need to consider some special type when doing optimization? Or letting back-end to revert the optimization accordingly?<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We are developing prototypes for Intel Advanced Matrix Extensions (AMX) [1] programing model in Clang and LLVM [2].<o:p></o:p></p>
<p class="MsoNormal">We met several cases when the certain type we added are optimized unexpectedly in the middle-end. E.g. optimizing phi + biscast + load:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">From<o:p></o:p></p>
<p class="MsoNormal">%a = load <256 x i32>, <256 x i32>* %mem, align 64<o:p></o:p></p>
<p class="MsoNormal">… …<o:p></o:p></p>
<p class="MsoNormal">%b = phi <256 x i32> [ %a, %label1 ], [%someother, %label2]<o:p></o:p></p>
<p class="MsoNormal">%c = bitcast <256 x i32> %b to x86_amx<o:p></o:p></p>
<p class="MsoNormal">To<o:p></o:p></p>
<p class="MsoNormal">%a = bitcast <256 x i32>* %mem to x86_amx*<o:p></o:p></p>
<p class="MsoNormal">%b = load x86_amx, x86_amx*, align 64<o:p></o:p></p>
<p class="MsoNormal">… …<o:p></o:p></p>
<p class="MsoNormal">%c = phi x86_amx [ %b, %label1 ], [%someother, %label2]<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">To prevent such unexpected transforms, we concretely added the type check in each point of the optimizations.
<o:p></o:p></p>
<p class="MsoNormal">Roman pointed out the changes are not the right direction [3], and thought it’s bug for backend. While we agreed backend might be able to handle it for the functionality, we think it is better to handle it in the midden-end since they are
negative optimizations for AMX.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">First, let me put some background here:<o:p></o:p></p>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo3">x86_amx* is different from trivial pointers.<o:p></o:p></li></ol>
<p class="MsoListParagraph">The AMX load instruction is much different from other load instructions. It is not only need the memory address but also the shape / stride of the tile register. We did some extra work in the backend to deduce the shape information
from the context. We don’t want the pass to add new x86_amx related usage because this will result in the difficulty in deduction. That said bitcasting other pointer types to x86_amx* is not trivial as assumed here.<o:p></o:p></p>
<ol style="margin-top:0in" start="2" type="1">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo3">The physical tile registers have more limitations.
<o:p></o:p></li><ol style="margin-top:0in" start="1" type="a">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level2 lfo3">No copy instruction between tile registers.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level2 lfo3">Spilling / reload a tile register is expensive in light of its size is 1024 bytes.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level2 lfo3">The shapes of tile registers need to be pre-configured before use and all data in tile registers will turn into invalid once re-configured. That said we need to dominate as more tile
registers as possible to configure their shapes with one configure instruction, otherwise we need to spill and reload the live registers once we need to re-configure.<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level2 lfo3">The number of tile registers is rather small (only 8) and different shapes cannot be reused.<o:p></o:p></li></ol>
</ol>
<p class="MsoNormal" style="margin-left:.5in">Based on the limitations, we need to reduce the use / live range of tile registers. But optimizations may increase the opportunity of the use. So even we can handle some combined operation for AMX type, we still
prefer to prevent it from the beginning. Unless we can totally roll back the optimization. Which is also not a good solution in my opinion.<o:p></o:p></p>
<ol style="margin-top:0in" start="3" type="1">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo3">For more information, please refer to discussion in [3].<o:p></o:p></li></ol>
<p class="MsoNormal">For other optimization points, please refer [4][5].<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I think the main controversy from Roman is if middle-end pass should consider some special type when doing optimization. I tend to let middle-end do the type check on account of the peculiarity of AMX type. But I’m not sure if we have precedent
to handle the similar issue in other targets. I’m open and glad to do it either way so long as we have an elegant solution.<o:p></o:p></p>
<p class="MsoNormal">Any suggestions are welcome.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">[1] <a href="https://software.intel.com/content/www/us/en/develop/articles/intel-sdm.html#architecture">
https://software.intel.com/content/www/us/en/develop/articles/intel-sdm.html#architecture</a><o:p></o:p></p>
<p class="MsoNormal">[2] <a href="https://lists.llvm.org/pipermail/llvm-dev/2020-November/146770.html">
https://lists.llvm.org/pipermail/llvm-dev/2020-November/146770.html</a><o:p></o:p></p>
<p class="MsoNormal">[3] <a href="https://reviews.llvm.org/D98247">https://reviews.llvm.org/D98247</a><o:p></o:p></p>
<p class="MsoNormal">[4] <a href="https://reviews.llvm.org/D98595">https://reviews.llvm.org/D98595</a><o:p></o:p></p>
<p class="MsoNormal">[5] <a href="https://reviews.llvm.org/D98757">https://reviews.llvm.org/D98757</a><o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks<o:p></o:p></p>
<p class="MsoNormal">Pengfei<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</body>
</html>