<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<div class="moz-cite-prefix">On 07/09/2015 04:36 PM, Andrew Trick
wrote:<br>
</div>
<blockquote
cite="mid:7AB975A6-6437-42CE-83AC-010EA76312D9@apple.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<br class="">
<div>
<blockquote type="cite" class="">
<div class="">On Jul 9, 2015, at 3:33 PM, Swaroop Sridhar <<a moz-do-not-send="true" href="mailto:Swaroop.Sridhar@microsoft.com" class="">Swaroop.Sridhar@microsoft.com</a>>
wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<meta http-equiv="Content-Type" content="text/html;
charset=utf-8" class="">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)" class="">
<style class=""><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Courier New";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:"Courier New";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1605916574;
mso-list-type:hybrid;
mso-list-template-ids:-524383778 67698705 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-text:"%1\)";
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div link="#0563C1" vlink="#954F72" class="" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">Regarding Call-site size specification:<o:p
class=""></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><o:p class=""> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">CoreCLR (<a moz-do-not-send="true" href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_dotnet_coreclr&d=AwMDaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=Mfk2qtn1LTDThVkh6-oGglNfMADXfJdty4_bhmuhMHA&m=-xaSAsXcu4p_Jo4Wr_5OjzWnxqS_X774UX8_-QRFhjk&s=rT0rLwHfbyLgYtDyln3PU3AXyRWJP9594ZzmLIF2Pzo&e=" class="">https://github.com/dotnet/coreclr</a>)
requires the size of the Call-instruction to be
reported in the GCInfo encoding.<o:p class=""></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><o:p class=""> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">The runtime performs querries for StackMap
records using instruction offsets as follows:<o:p
class=""></o:p></span></p>
<p class="MsoListParagraph"
style="text-indent:-.25in;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><span style="mso-list:Ignore" class="">1)<span
style="font:7.0pt "Times New Roman""
class="">
</span></span></span><!--[endif]--><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">Offset at the end of the call instruction
(offset of next instruction-1) if the call
instruction occurs in code where GC can only take
control at safe-points.</span></p>
</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
As part of this change it would be great if LLVM could now
guarantee that the call will be emitted at the end of the
patchable space. It currently happens to emit at the beginning,
but makes no guarantee. Emitting at the end works better for
tracking the return address.</div>
</blockquote>
This sounds reasonable. I can't see an immediate reason why this
would restrict future implementation flexibility. <br>
<blockquote
cite="mid:7AB975A6-6437-42CE-83AC-010EA76312D9@apple.com"
type="cite">
<div><br class="">
<blockquote type="cite" class="">
<div class="">
<div link="#0563C1" vlink="#954F72" class="" lang="EN-US">
<div class="WordSection1">
<p class="MsoListParagraph"
style="text-indent:-.25in;mso-list:l0 level1 lfo1"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><o:p class=""></o:p></span></p>
<p class="MsoListParagraph"
style="text-indent:-.25in;mso-list:l0 level1 lfo1"><!--[if !supportLists]--><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><span style="mso-list:Ignore" class="">2)<span
style="font:7.0pt "Times New Roman""
class="">
</span></span></span><!--[endif]--><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">Offset of the start of the call instruction
if the call instruction occurs within a code-range
that allows full interruption (that is, all
instructions in a range are considered safe-points)</span></p>
</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>If the JIT requires knowledge of call's encoding, it should
probably be emitting the call instruction itself within the
patchable space reserved by LLVM. </div>
<div><br class="">
</div>
<div>Note that the patchpoint may include a mov in addition to
call, so the patchpoint address is not the same as the call
address.</div>
</div>
</blockquote>
Just to be clear, the problem with (2) has nothing to do with
StackMaps. It has to do with the fact our entire backend confuses
pointers and non-pointers. To have option (2), I believe we'd need
to effectively have a stackmap for every instruction. That's a lot
more work than anyone has serious proposed to date. :)<br>
<blockquote
cite="mid:7AB975A6-6437-42CE-83AC-010EA76312D9@apple.com"
type="cite">
<div>
<div><br class="">
</div>
<blockquote type="cite" class="">
<div class="">
<div link="#0563C1" vlink="#954F72" class="" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><o:p class=""> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">LLVM/statepoint GC does not support option
(2), but the CoreCLR’s GC-table Encoding has an
interface designed to suite both modes.<o:p class=""></o:p></span></p>
<p class="MsoNormal" style="background:white"><span
style="font-size:10.0pt;font-family:Consolas;color:blue"
class="">void</span><span style="font-size: 10pt;
font-family: Consolas;" class=""> DefineCallSites(</span><span
style="font-size:10.0pt;font-family:Consolas;color:#2B91AF" class="">UINT32</span><span
style="font-size: 10pt; font-family: Consolas;"
class="">* </span><span
style="font-size:10.0pt;font-family:Consolas;color:gray"
class="">pCallSites</span><span style="font-size:
10pt; font-family: Consolas;" class="">, </span><span
style="font-size:10.0pt;font-family:Consolas;color:#2B91AF" class="">BYTE</span><span
style="font-size: 10pt; font-family: Consolas;"
class="">* </span><span
style="font-size:10.0pt;font-family:Consolas;color:gray"
class="">pCallSiteSizes</span><span
style="font-size: 10pt; font-family: Consolas;"
class="">, </span><span
style="font-size:10.0pt;font-family:Consolas;color:#2B91AF"
class="">UINT32</span><span style="font-size: 10pt;
font-family: Consolas;" class=""> </span><span
style="font-size:10.0pt;font-family:Consolas;color:gray"
class="">numCallSites</span><span style="font-size:
10pt; font-family: Consolas;" class="">)<o:p
class=""></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><o:p class=""> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">Therefore, it is helpful to have Call-Site
size specified in
</span><span style="font-family:"Courier
New"" class="">StackMapRecord</span><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">.<o:p class=""></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">I agree with Andy, that the call-site size
should include all bytes between the start of the
call instruction and the start of the next
instruction.</span></p>
</div>
</div>
</div>
</blockquote>
<div><br class="">
</div>
<div>I suggested this because we want to support a dynamic
callback that determines the patchpoint size (given the set of
register arguments), and it would be nice to record that
decision within the stack map. This is important information
for any code that is responsible for patching because it must
patch over all reserved bytes.</div>
<div><br class="">
</div>
<div>The alternative would be for LLVM to emit the call at the
end and record just the size of the call instruction encoding.
That seems like a silly, x86-specific waste of stackmap space
though. The JIT can either do the encoding and keep track of
the info, or a small nop+move_immediate decoder can figure it
out for all reasonable cases.</div>
</div>
</blockquote>
I would argue in favor of recording the number of bytes recorded for
patching. I think that gives us everything we need in practice.
Any non-destructive or concurrent patch mechanism is going to need
extremely fine grained control anyways. Trying to do that partly
within LLVM and partly without just seems like a mistake. <br>
<blockquote
cite="mid:7AB975A6-6437-42CE-83AC-010EA76312D9@apple.com"
type="cite">
<div>
<div><br class="">
</div>
<div>Andy</div>
<div><br class="">
</div>
<blockquote type="cite" class="">
<div class="">
<div link="#0563C1" vlink="#954F72" class="" lang="EN-US">
<div class="WordSection1">
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><o:p class=""> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">Thanks,<o:p class=""></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class="">Swaroop.<o:p class=""></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"
class=""><o:p class=""> </o:p></span></p>
<div class="">
<div style="border:none;border-top:solid #E1E1E1
1.0pt;padding:3.0pt 0in 0in 0in" class="">
<p class="MsoNormal"><b class=""><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif"
class="">From:</span></b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif"
class=""> Juergen Ributzka [<a moz-do-not-send="true" href="mailto:juergen@apple.com" class="">mailto:juergen@apple.com</a>]
<br class="">
<b class="">Sent:</b> Thursday, July 9, 2015
2:04 PM<br class="">
<b class="">To:</b> LLVM Dev<br class="">
<b class="">Cc:</b> Lang Hames; Andrew Trick;
Phil Pizlo; Philip Reames; Sanjoy Das; Swaroop
Sridhar; Russell Hadley<br class="">
<b class="">Subject:</b> [RFC] New StackMap
format proposal (StackMap v2)<o:p class=""></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p class=""> </o:p></p>
<p class="MsoNormal">Hi @ll,<o:p class=""></o:p></p>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">over the past year we gained more
experience with the patchpoint/stackmap/statepoint
intrinsics and it exposed limitations in the
stackmap format.<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">The following proposal includes
feedback and request from several interested parties
and I would like to hear your feedback.<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">Missing correlation between
functions and stackmap records:<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">Originally the client had to keep
track of the ID to know which stackmap record
belongs to which function, but this would stop
working once functions are inlined.<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">The new format fixes that by
adding a direct reference from the function to the
stackmap records.<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">Call Size and Function Size:<o:p
class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">These are additional information
that are of interest and have been added to the
format.<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">@Swaroop: Could you please
provide a little more detailed explanation on the
"Call Size" field and what exactly is there
recorded. Is it just the call instruction<o:p
class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">or also the materialization code
for the address? For what is this used for?<o:p
class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">Flat format:<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">We think moving to a flat form
will make parsing easier, because every record has a
fixed size and offsets can be calculated easily. The
plan is to also<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">provide parsers for both stackmap
versions (there is already one for the first format
in tree) and a corresponding C-API to make it easier
for clients to<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">adopt the new format. There is no
plan to drop the original format and we will
continue to support both formats. I will ask for
feedback on the C API in a<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">separate RFC. <o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">Another benefit we hope to
achieve from this format is to optimize for size by
uniquing entries - but that is independent
optimization and not required.<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">More detailed frame record:<o:p
class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">Clients require more information
about the function frame, such as spilled registers,
etc. The frame base register i.e. might change when
dynamic stack<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">realignment is performed on X86.<o:p
class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">If there is anything missing
please let me know.<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">Thanks<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal">Cheers,<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal">Juergen<o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<div class="">
<p class="MsoNormal"><span
style="font-family:"Courier New""
class="">Header v2 {<br class="">
uint8 : Stack Map Version (2)<br class="">
uint8 : Reserved [3] (0)<br class="">
uint32 : Constants Offset (bytes)<br class="">
uint32 : Frame Records Offset (bytes)<br
class="">
uint32 : Frame Registers Offset (bytes)<br
class="">
uint32 : StackMap Records Offset (bytes)<br
class="">
uint32 : Locations Offset (bytes)<br class="">
uint32 : LiveOuts Offset (bytes)</span><o:p
class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><span
style="font-family:"Courier New""
class="">}<br class="">
<br class="">
align to 8 bytes<br class="">
Constants[] {<br class="">
uint64 : LargeConstant<br class="">
}<br class="">
<br class="">
align to 8 bytes<br class="">
FrameRecord[] {<br class="">
uint64 : Function Address<br class="">
uint32 : Function Size<br class="">
uint32 : Stack Size<br class="">
uint16 : Flags {<br class="">
bool : HasFrame<br class="">
bool : HasVariableSizeAlloca<br class="">
bool : HasStackRealignment<br class="">
bool : HasLiveOutInfo<br class="">
bool : Reserved [12]<br class="">
}<br class="">
uint16 : Frame Base Register Dwarf RegNum<br
class="">
uint16 : Num Frame Registers<br class="">
uint16 : Frame Register Index<br class="">
uint16 : Num StackMap Records<br class="">
uint16 : StackMap Record Index</span><o:p
class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><span
style="font-family:"Courier New""
class="">}<br class="">
<br class="">
align to 4 bytes<br class="">
FrameRegister[] {<br class="">
uint16 : Dwarf RegNum<br class="">
int16 : Offset<br class="">
uint8 : Size in Bytes<br class="">
uint8 : Flags {<br class="">
bool : IsSpilled<br class="">
bool : Reserved [7]<br class="">
}<br class="">
}<br class="">
<br class="">
align to 8 bytes<br class="">
StackMapRecord[] {<br class="">
uint64 : PatchPoint ID<br class="">
uint32 : Instruction Offset<br class="">
uint8 : Call size (bytes)<br class="">
uint8 : Flags {<br class="">
bool : HasLiveOutInfo<br class="">
bool : Reserved [7]<br class="">
}<br class="">
uint16 : Num Locations<br class="">
uint16 : Location Index<br class="">
uint16 : Num LiveOuts<br class="">
uint16 : LiveOut Index<br class="">
}</span><o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
<div class="">
<p class="MsoNormal"><span
style="font-family:"Courier New""
class="">align to 4 bytes</span><o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><span
style="font-family:"Courier New""
class="">Location[] {<br class="">
uint8 : Register | Direct | Indirect |
Constant | ConstantIndex<br class="">
uint8 : Reserved (location flags)<br class="">
uint16 : Dwarf RegNum<br class="">
int32 : Offset or SmallConstant<br class="">
}<br class="">
<br class="">
align to 2 bytes<br class="">
LiveOuts[] {<br class="">
uint16 : Dwarf RegNum<br class="">
uint8 : Reserved<br class="">
uint8 : Size in Bytes<br class="">
}</span><o:p class=""></o:p></p>
</div>
<div class="">
<p class="MsoNormal"><o:p class=""> </o:p></p>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</blockquote>
<br>
</body>
</html>