<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Exchange Server">
<!-- converted from text --><style><!-- .EmailQuote { margin-left: 1pt; padding-left: 4pt; border-left: #800000 2px solid; } --></style>
</head>
<body>
<div>
<div dir="auto" style="direction:ltr; margin:0; padding:0; font-family:sans-serif; font-size:11pt; color:black">
Good point. <br>
<br>
</div>
<div dir="auto" style="direction:ltr; margin:0; padding:0; font-family:sans-serif; font-size:11pt; color:black">
I would say we change it where it makes sense, e.g., we allow to override some parts with a different logic. Though, at the end of the day, it is the interface that needs to be implemented, how is up to the deviceRTLs. I want is to avoid duplication as much
as possible, so not to force some logic, e.g., the current gpu-focused one, onto an implementation but to offer it. Maybe we will end up with multiple "common logics", but any reuse we can get out of it is worth separating the device specific parts, especially
since there is no runtime cost to it.<br>
<br>
</div>
<div dir="auto" style="direction:ltr; margin:0; padding:0; font-family:sans-serif; font-size:11pt; color:black">
Does that make sense and answer your question?<br>
<br>
</div>
<div dir="auto" style="direction:ltr; margin:0; padding:0; font-family:sans-serif; font-size:11pt; color:black">
<span id="x_OutlookSignature">
<div dir="auto" style="direction:ltr; margin:0; padding:0; font-family:sans-serif; font-size:11pt; color:black">
Get <a href="https://aka.ms/ghei36">Outlook for Android</a></div>
</span><br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Narayanaswamy, Ravi <ravi.narayanaswamy@intel.com><br>
<b>Sent:</b> Thursday, July 11, 2019 11:24:37 AM<br>
<b>To:</b> Doerfert, Johannes; Bae, Hansang; Tian, Xinmin<br>
<b>Cc:</b> Finkel, Hal J.; openmp-dev@lists.llvm.org<br>
<b>Subject:</b> RE: [RFC] Device runtime library (re)design</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:11pt;">
<div class="PlainText">Johannes,<br>
Suppose we define today some common logic based on existing deviceRTLs. Are you requiring that all future deviceRTLs support this. What if they cannot, are you going to change the common logic or claim that the new deviceRTLs is not compliant.<br>
Thanks<br>
Ravi<br>
<br>
-----Original Message-----<br>
From: Doerfert, Johannes [<a href="mailto:jdoerfert@anl.gov">mailto:jdoerfert@anl.gov</a>]
<br>
Sent: Wednesday, July 10, 2019 4:32 PM<br>
To: Bae, Hansang <hansang.bae@intel.com>; Narayanaswamy, Ravi <ravi.narayanaswamy@intel.com>; Tian, Xinmin <xinmin.tian@intel.com><br>
Cc: Finkel, Hal J. <hfinkel@anl.gov>; openmp-dev@lists.llvm.org<br>
Subject: Re: [RFC] Device runtime library (re)design<br>
<br>
Hi Hansang,<br>
<br>
I have the feeling the file ending and modeline (file comment) changes I made overshadow the important parts, namely the separation of device agnostic runtime logic and device specific implementation.<br>
<br>
To exaggerate a bit:<br>
If people really want the modeline to say "CUDA" and the file ending to be ".cu", that is at the end of the day OK with me.
<br>
After this rewrite, with the separation of common logic and device impl., I can simply interpret the common logic "CUDA" files if I need to, e.g., as HIP for AMDGPU.<br>
<br>
I mentioned already in the other discussion [0], we can easily have a design in which none of the files has a ".cpp" ending.<br>
Even if they have, that does not mean we actually interpret/compile them as C++ files ("clang -x cuda a.cpp" works just fine).<br>
Calling them CUDA is simply misleading if they are not CUDA specific. <br>
Calling them C++ might be misleading as well, I agree, but for now the logic part is basically C++ code.<br>
During the rewrite we can make the code C, though I would prefer not to.<br>
<br>
Thanks,<br>
Johannes<br>
<br>
[0] <a href="http://lists.llvm.org/pipermail/openmp-dev/2019-July/002589.html">http://lists.llvm.org/pipermail/openmp-dev/2019-July/002589.html</a><br>
<br>
________________________________________<br>
From: Bae, Hansang <hansang.bae@intel.com><br>
Sent: Wednesday, July 10, 2019 18:01<br>
To: Narayanaswamy, Ravi; Doerfert, Johannes; Tian, Xinmin<br>
Cc: Finkel, Hal J.; openmp-dev@lists.llvm.org<br>
Subject: RE: [RFC] Device runtime library (re)design<br>
<br>
I think there are already some discussions on the base language for this effort, and it is also one of my concern.<br>
<br>
There can be a situation where C++ is not the best language to use for a certain device RTL development, so it may not be a good idea to write the "common" code in C++.<br>
<br>
Thanks,<br>
Hansang<br>
<br>
<br>
-----Original Message-----<br>
From: Narayanaswamy, Ravi<br>
Sent: Wednesday, July 10, 2019 5:21 PM<br>
To: Doerfert, Johannes <jdoerfert@anl.gov>; Tian, Xinmin <xinmin.tian@intel.com><br>
Cc: Finkel, Hal J. <hfinkel@anl.gov>; Bae, Hansang <hansang.bae@intel.com><br>
Subject: RE: [RFC] Device runtime library (re)design<br>
<br>
I am responding to this mail to clarify that I mixed up my deviceRTL and plugin.<br>
Hansang who implemented our library has some comments and will respond to the general mailing list.<br>
<br>
-----Original Message-----<br>
From: Doerfert, Johannes [<a href="mailto:jdoerfert@anl.gov">mailto:jdoerfert@anl.gov</a>]<br>
Sent: Wednesday, July 10, 2019 3:11 PM<br>
To: Narayanaswamy, Ravi <ravi.narayanaswamy@intel.com>; Tian, Xinmin <xinmin.tian@intel.com><br>
Cc: Finkel, Hal J. <hfinkel@anl.gov><br>
Subject: Re: [RFC] Device runtime library (re)design<br>
<br>
We should have this conversation on the list but anyway.<br>
<br>
deviceRTL contains code that is executed on the device. It doesn't need to be device specific, most of it at least. Once we refactor we can reuse the logic for all devices and provide small device specific parts (on a function level basis) where necessary.<br>
libomptarget, depending on what you mean by it, contains code that is executed on the host to prepare the device and device execution. That code is also partially device specific and in parts generic.<br>
<br>
________________________________________<br>
From: Narayanaswamy, Ravi <ravi.narayanaswamy@intel.com><br>
Sent: Wednesday, July 10, 2019 16:37<br>
To: Doerfert, Johannes; Tian, Xinmin<br>
Cc: Finkel, Hal J.<br>
Subject: RE: [RFC] Device runtime library (re)design<br>
<br>
Johannes,<br>
My initial question is if it can be made common to all devices then why doesn't it belong in libomptarget. The intent of deviceRTL is that the code there are device specific.<br>
Thanks<br>
Ravi<br>
<br>
-----Original Message-----<br>
From: Doerfert, Johannes [<a href="mailto:jdoerfert@anl.gov">mailto:jdoerfert@anl.gov</a>]<br>
Sent: Wednesday, July 10, 2019 1:28 PM<br>
To: Tian, Xinmin <xinmin.tian@intel.com><br>
Cc: Narayanaswamy, Ravi <ravi.narayanaswamy@intel.com>; Finkel, Hal J. <hfinkel@anl.gov><br>
Subject: Fw: [RFC] Device runtime library (re)design<br>
<br>
Hi Xinmin, Ravi,<br>
<br>
In case you haven't seen the RFC for the redesign of the OpenMP runtime (attached), please take a look and/or forward it to relevant people.<br>
I think the proposed changes should it make easier for you guys to upstream as well. In any case, feedback through the list would be appreciated.<br>
<br>
Thanks,<br>
Johannes<br>
<br>
<br>
---------------------------------------<br>
Johannes Doerfert<br>
Researcher<br>
<br>
Argonne National Laboratory<br>
Lemont, IL 60439, USA<br>
<br>
jdoerfert@anl.gov<br>
<br>
________________________________________<br>
From: Openmp-dev <openmp-dev-bounces@lists.llvm.org> on behalf of Doerfert, Johannes via Openmp-dev <openmp-dev@lists.llvm.org><br>
Sent: Friday, July 5, 2019 12:16<br>
To: openmp-dev@lists.llvm.org<br>
Cc: Lingda Li; Hernandez, Oscar R.; Alexey Bataev; Gregory.Rodgers@amd.com; Xinmin Tian; Denny, Joel<br>
Subject: [Openmp-dev] [RFC] Device runtime library (re)design<br>
<br>
Tl;dr<br>
We should extract device specific code out of the OpenMP deviceRTL such that we can reuse the common logic (>90%) for all devices.<br>
We also need to improve the documentation and we should think about bringing the code into the LLVM coding style.<br>
<br>
<br>
Requested changes:<br>
I would like is to change the OpenMP device runtime library design (openmp/libomptarget/deviceRTLs) towards the following goals:<br>
1) Allow reuse of common logic between different devices in a clean and extensible way.<br>
2) Improve the documentation, e.g., doxygen comments and code comments, for the code.<br>
3) Follow the same coding style as LLVM core.<br>
<br>
Disclaimer:<br>
First, I do not want to say it currently is impossible the reuse the code for other devices or the code is not documented at all. What I think is that we can improve both substantially if we choose to do so. Also, a change in coding style is easier now than
later, so if we decide to do refactoring, that can be included without adding to much churn.<br>
<br>
Motivation:<br>
Now we can discuss if we should do any of the proposed changes but I guess most of them have clear benefits. I am also not the first to suggest them. Point 1 was mentioned with the initial drop of the device runtime [0], but it was rejected for time reasons.
Point 2 was recently discussed as a pressing issue in multiple reviews. Point 3 is a general observation as writing and reviewing code for the openmp sub project is unnecessarily hard for LLVM developers due to the different coding style.<br>
<br>
Proposed structure:<br>
In order to ease the reuse by new devices we should have a common core with device independent logic, e.g., in<br>
openmp/ibomptarget/deviceRTLs/common<br>
including an interface that declares all device specific methods needed by the core logic. The interface is then the only thing implemented in the device subfolders, e.g.,<br>
openmp/ibomptarget/deviceRTLs/nvptx, openmp/ibomptarget/deviceRTLs/amdgpu, ...<br>
To get to this goal, all device specific code has to be extracted from the core logic. The prototypes below show that this is fairly easy to do.<br>
<br>
<br>
Feasibility and prototypes:<br>
To showcase the direction I would like is to move to I "redesigned" three files (out of ~20) with the above goals in mind. The patches can be found here:<br>
<a href="https://reviews.llvm.org/D64217">https://reviews.llvm.org/D64217</a><br>
<a href="https://reviews.llvm.org/D64218">https://reviews.llvm.org/D64218</a><br>
<a href="https://reviews.llvm.org/D64219">https://reviews.llvm.org/D64219</a><br>
Note that there is a vast design space even if we agree to the above three goals. As a consequence, I'd like us to use the patches to discuss general design decisions not specific ones until we agreed on a path forward.<br>
<br>
<br>
Please let me know what you think,<br>
Johannes<br>
<br>
<br>
<br>
[0] <a href="https://reviews.llvm.org/D14254#949985">https://reviews.llvm.org/D14254#949985</a><br>
<br>
<br>
---------------------------------------<br>
Johannes Doerfert<br>
Researcher<br>
<br>
Argonne National Laboratory<br>
Lemont, IL 60439, USA<br>
<br>
jdoerfert@anl.gov<br>
_______________________________________________<br>
Openmp-dev mailing list<br>
Openmp-dev@lists.llvm.org<br>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev">https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a><br>
</div>
</span></font>
</body>
</html>