<html>
<head>
<meta name="generator" content="Windows Mail 17.5.9600.20413">
<style data-externalstyle="true"><!--
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph {
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
}
p.MsoNormal, li.MsoNormal, div.MsoNormal {
margin:0in;
margin-bottom:.0001pt;
}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst,
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle,
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast {
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
line-height:115%;
}
--></style></head>
<body dir="ltr">
<div data-externalstyle="false" dir="ltr" style="font-family: 'Calibri', 'Microsoft YaHei UI', 'Segoe UI', 'Meiryo', 'Microsoft JhengHei UI', 'Malgun Gothic', 'sans-serif';font-size:12pt;"><div>Eli, </div><div><br></div><div>You are right that the NVPTX backend is tightly coupled with the driver. This is so because of what we call compiler-driver interface. Printf is not part of this interface. It is a library call. </div><div><br></div><div>This printf patch will cause trouble if the implementation of a user defined printf is in another LLVM IR module which is compiled to PTX, or when the implementation is in provided in a SASS or PTX library in the future.<br></div><div data-signatureblock="true"><br></div><div data-signatureblock="true">For all practical reasons, I do understand the desire to support printf and we don’t have a perfect solution here. I'd suggest you move the printf to vprintf lowering to the NVPTX specific part of Clang. This is cleaner, because printf is a C library feature, the vprintf is a CUDA driver implementation, and LLVM IR is language neutral. You are considering CUDA printf, what about OpenCL printf?</div><div data-signatureblock="true"><br></div><div data-signatureblock="true">For the same reason, we don’t have printf to vprintf lowering in libNVVM. In fact, libNVVM does not even understand either printf or vprintf. We expect libNVVM users to generate calls to the vprintf syscall directly, though everyone should be aware that vprintf is currently a private and evolving interface. </div><div data-signatureblock="true"> </div><div data-signatureblock="true">A less desirable approach is to keep this pass in the NVPTX backend module. A compiler builder can cherry pick it in his/her custom compiler. But this pass needs be turned off by default.</div><div data-signatureblock="true"><br></div><div data-signatureblock="true">Yuan</div><div data-signatureblock="true"><br></div><div data-signatureblock="true"><br></div><div style="padding-top: 5px; border-top-color: rgb(229, 229, 229); border-top-width: 1px; border-top-style: solid;"><div><font face=" 'Calibri', 'Microsoft YaHei UI', 'Segoe UI', 'Meiryo', 'Microsoft JhengHei UI', 'Malgun Gothic', 'sans-serif'" style='line-height: 15pt; letter-spacing: 0.02em; font-family: "Calibri", "Microsoft YaHei UI", "Segoe UI", "Meiryo", "Microsoft JhengHei UI", "Malgun Gothic", "sans-serif"; font-size: 12pt;'><b>From:</b> <a href="mailto:eliben@google.com" target="_parent">Eli Bendersky</a><br><b>Sent:</b> Friday, March 28, 2014 4:25 PM<br><b>To:</b> <a href="mailto:yulin@nvidia.com" target="_parent">Yuan Lin</a><br><b>Cc:</b> <a href="mailto:llvm-commits@cs.uiuc.edu" target="_parent">llvm-commits@cs.uiuc.edu</a>, <a href="mailto:justin.holewinski@gmail.com" target="_parent">Justin Holewinski</a>, <a href="mailto:jholewinski@nvidia.com" target="_parent">Justin Holewinski</a>, <a href="mailto:jingyue@google.com" target="_parent">Jingyue Wu</a></font></div></div><div><br></div><div dir="">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">On Fri, Mar 28, 2014 at 3:56 PM, Yuan Lin <span dir="ltr">
<<a href="mailto:yulin@nvidia.com" target="_parent">yulin@nvidia.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;">
<div lang="EN-US">
<div>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;">Eli,
<u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;">I am sure you are aware this is a very specific pass for translating calls to printf functions for the NVPTX backend with current existing CUDA driver implementation.</span></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div>Hi Yuan, thanks for your comments.</div>
<div><br>
</div>
<div>I'm note sure I fully understand. A lot in the NVPTX backend is tightly coupled with the driver, which further lowers the PTX into something. Are you suggesting it would be better to create this coupling between the front-end and the driver?</div>
<div> <br>
</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;">
<div lang="EN-US">
<div>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;">With your patch, the NVPTX backend won’t be able to handle the input IR that contains a user defined ‘printf’ function.</span></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>The pass checks whether printf is *defined* in the module (note that this is the final, linked IR module before code generation). If yes, it doesn't do its thing, so IR containing a user-defined 'printf' will work. I can add a tests case for that upstream
just to be on the safe side.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;">
<div lang="EN-US">
<div>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;">It will also cause trouble for implementation that implements ‘printf’ as an external llibrary call.</span></p>
</div>
</div>
</blockquote>
<div><br>
</div>
<div>Can you clarify -- is there PTX-level linking here? NVPTX gets the fully linked LLVM IR and code-gens it. At this stage I believe there are no external library calls, only PTX syscalls.</div>
<div> </div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;">
<div lang="EN-US">
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;"><u></u><u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;">Instead of adding this pass to the NVPTX backend, it would be better to do the translation in the compiler module (e.g. clang or other front-ends) that generates
the LLVM IR for the NVPTX backend.</span></p>
</div>
</blockquote>
<div><br>
</div>
<div>So you're suggesting to expose vprintf as a formal part of the NVVM IR specification (<a href="http://docs.nvidia.com/cuda/nvvm-ir-spec/index.html" target="_parent">http://docs.nvidia.com/cuda/nvvm-ir-spec/index.html</a>)? Because it's not currently there.</div>
<div><br>
</div>
<div>Neither is printf, to be fair. But printf is at least documented & known at the CUDA level.</div>
<div><br>
</div>
<div>In other words, when you expect someone to target libNVVM for some alternative front-end language (let's say compiling Matlab or whatever other language), how can he print anything? Are you going to expose vprintf as a libNVVM API?</div>
<div><br>
</div>
<div>As long as it's not exposed in this way, it makes more sense to me to make it a NVPTX explicit pass, because conceptually only NVPTX "knows" what vprintf is (in the same way it "knows" how PTX instructions should look). And this is then a viable way for
someone to develop an open-source toolchain for CUDA based on Clang and LLVM with the NVPTX backend.</div>
<div><br>
</div>
<div>Eli</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div> </div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; padding-left: 1ex; border-left-color: rgb(204, 204, 204); border-left-width: 1px; border-left-style: solid;">
<div lang="EN-US">
<div>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;"> <u></u></span></p>
<p class="MsoNormal"><span style="color: rgb(31, 73, 125); font-family: Calibri,sans-serif; font-size: 11pt;"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-family: Calibri,sans-serif; font-size: 11pt;">From:</span></b><span style="font-family: Calibri,sans-serif; font-size: 11pt;"> Eli Bendersky [mailto:<a href="mailto:eliben@google.com" target="_parent">eliben@google.com</a>]
<br>
<b>Sent:</b> Wednesday, March 26, 2014 9:51 AM<br>
<b>To:</b> <a href="mailto:llvm-commits@cs.uiuc.edu" target="_parent">llvm-commits@cs.uiuc.edu</a>; Justin Holewinski; Justin Holewinski<br>
<b>Cc:</b> Yuan Lin; Jingyue Wu<br>
<b>Subject:</b> PATCH: implement printf call lowering in the NVPTX backend<u></u><u></u></span></p>
<div>
<div class="h5">
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">Hello,<u></u><u></u></p>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">This patch implements lowering of the printf call to the vprintf syscall in the NVPTX backend. It is implemented as a target-specific IR pass that runs before codegen. The pass builds the required data structure from the varargs to pass
to the vprintf calls.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">Tests (IR to IR, and IR to PTX) included.<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="MsoNormal">PTAL,<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal">Eli<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</div>
</div>
<div>
<hr>
</div>
<div>This email message is for the sole use of the intended recipient(s) and may contain confidential information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message. </div>
<div>
<hr>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div></div>
</body>
</html>