<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:8.0pt;
margin-left:.5in;
mso-add-space:auto;
line-height:106%;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
mso-add-space:auto;
line-height:106%;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
mso-add-space:auto;
line-height:106%;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-type:export-only;
margin-top:0in;
margin-right:0in;
margin-bottom:8.0pt;
margin-left:.5in;
mso-add-space:auto;
line-height:106%;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1634213119;
mso-list-type:hybrid;
mso-list-template-ids:-2076947862 67698703 67698713 67698715 67698703 67698713 67698715 67698703 67698713 67698715;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style="text-align:justify">Following approach #2 from the previous RFC, we are preparing some updates to the address sanitizer<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify">runtime and we would like to get some feedback on how to approach these changes.<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify"><o:p> </o:p></p>
<p class="MsoNormal" style="text-align:justify">In order to isolate the impact of our changes on others, we are thinking to guard our changes inside a<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify">distinguishable macro, for example, SANITIZER_AMDGPU, and to localize our changes as much as possible<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify">within new files whose name includes AMDGPU. We would your feedback on this, and suggestions<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify">about how to trigger the definition of the macro. The host runtime will always be targeted to the host<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify">which implies that we will not be able to check the target architecture and set the macro, like other<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify">architectures. We propose two ways to achieve this:
<o:p></o:p></p>
<p class="MsoNormal" style="text-align:justify"><o:p> </o:p></p>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoListParagraphCxSpFirst" style="margin-left:0in;mso-add-space:auto;mso-list:l0 level1 lfo1">
Based on LLVM_TARGETS_TO_BUILD value <o:p></o:p></li></ol>
<p class="MsoListParagraphCxSpMiddle">We can look for AMDGPU in the LLVM_TARGETS_TO_BUILD string and trigger the AMDGPU<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle">specific macro. We will make sure it does not trigger with the default option which is the list of<o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle">all the supported architectures. <o:p></o:p></p>
<p class="MsoListParagraphCxSpMiddle"><o:p> </o:p></p>
<ol style="margin-top:0in" start="2" type="1">
<li class="MsoListParagraphCxSpMiddle" style="margin-left:0in;mso-add-space:auto;mso-list:l0 level1 lfo1">
Adding a new cmake flag<o:p></o:p></li></ol>
<p class="MsoListParagraphCxSpMiddle">We could require a specific cmake variable be set which when used will trigger the AMDGPU<o:p></o:p></p>
<p class="MsoListParagraphCxSpLast">specific macro. <o:p></o:p></p>
<p class="MsoNormal">We are definitely open to other alternatives.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Many thanks,<o:p></o:p></p>
<p class="MsoNormal">Reshabh<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Sharma, Reshabh Kumar <br>
<b>Sent:</b> Thursday, October 29, 2020 9:34 PM<br>
<b>To:</b> 'llvm-dev@lists.llvm.org' <llvm-dev@lists.llvm.org><br>
<b>Cc:</b> Sumner, Brian <Brian.Sumner@amd.com><br>
<b>Subject:</b> [RFC] Implementing the sanitizer runtimes for heterogeneous devices<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Hi all,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We are thinking of extending the LLVM sanitizers, starting with asan, to heterogeneous situations<o:p></o:p></p>
<p class="MsoNormal">such as those found in OpenCL and HIP. We have already started thinking about the way<o:p></o:p></p>
<p class="MsoNormal">instrumentation will look like for different address spaces and posted a small RFC<o:p></o:p></p>
<p class="MsoNormal">(<a href="http://lists.llvm.org/pipermail/llvm-dev/2020-July/143310.html">http://lists.llvm.org/pipermail/llvm-dev/2020-July/143310.html</a>) about removing the generated<o:p></o:p></p>
<p class="MsoNormal">inttoptr and ptrtoint pair from instrumented IR. We have started to look into the runtime<o:p></o:p></p>
<p class="MsoNormal">implementation, and we will be happy to get feedback and suggestion from the community so that we<o:p></o:p></p>
<p class="MsoNormal">can incorporate them in the design itself. We are looking at implementing a sanitizer runtime that<o:p></o:p></p>
<p class="MsoNormal">would support heterogeneous devices such as AMD GPUs and we present some directions that we can take<o:p></o:p></p>
<p class="MsoNormal">to make that happen:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> 1. Add support for compilation of the compiler-rt sanitizer runtime by a device compiler<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> We propose to update the sanitizer parts of compiler-rt so they can additionally be compiled<o:p></o:p></p>
<p class="MsoNormal"> with the HIP compiler and generate device code. This would be our first choice. It provides<o:p></o:p></p>
<p class="MsoNormal"> obvious advantages over keeping separate sanitizer runtimes for heterogeneous devices and<o:p></o:p></p>
<p class="MsoNormal"> easier maintenance while ensuring feature parity. This can be broadly summarized in the<o:p></o:p></p>
<p class="MsoNormal"> following activities:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> a. Supporting device code generation<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> We propose to add suitably protected changes which allow the code to be compiled for the<o:p></o:p></p>
<p class="MsoNormal"> HIP language and generate device code.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> b. Supporting device and host communication<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> Sanitizer features like violation reporting requires communication from device to host.<o:p></o:p></p>
<p class="MsoNormal"> We propose to add support for the device code to communicate with the host for<o:p></o:p></p>
<p class="MsoNormal"> reporting. We see this going into a target specific sub-folder.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> c. Adding new interceptors<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> We foresee a need for new interceptors, for example, device specific memory allocators.<o:p></o:p></p>
<p class="MsoNormal"> We propose to add these interceptors.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> d. Updating the build system<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> We propose to update the build system to build device code when requested, as well as<o:p></o:p></p>
<p class="MsoNormal"> host code.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> 2. Implement the device side sanitizer runtime separately<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> In this scenario, the device side of the sanitizer runtime would be implemented in a<o:p></o:p></p>
<p class="MsoNormal"> separate folder, possibly in a separate repository. For example, this could be added to our<o:p></o:p></p>
<p class="MsoNormal"> existing device side runtime. The changes mentioned in 1b and 1c to the existing host<o:p></o:p></p>
<p class="MsoNormal"> runtime would still be required.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Combinations of different approaches listed above are also possible. We will be happy to hear about<o:p></o:p></p>
<p class="MsoNormal">other approaches and ideas to move forward with them as well. Adding direct support for<o:p></o:p></p>
<p class="MsoNormal">heterogeneous device sanitizer runtime in compiler-rt will also be helpful for other heterogeneous<o:p></o:p></p>
<p class="MsoNormal">devices in future.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Many thanks,<o:p></o:p></p>
<p class="MsoNormal">Reshabh and Brian<o:p></o:p></p>
</div>
</body>
</html>