<div dir="auto"><div><div class="gmail_extra"><div class="gmail_quote">On Dec 9, 2017 6:39 AM, "Hal Finkel" <<a href="mailto:hfinkel@anl.gov">hfinkel@anl.gov</a>> wrote:<br type="attribution"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><div class="quoted-text">
<p><br>
</p>
<div class="m_2990692331795416324moz-cite-prefix">On 12/08/2017 09:58 AM, Joel E. Denny
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hi Hal,<br>
<br>
</div>
Thanks for your feedback. It sounds like we're basically in
agreement, but I've added a few thoughts inline below.
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Dec 6, 2017 at 4:02 AM, Hal
Finkel <span dir="ltr"><<a href="mailto:hfinkel@anl.gov" target="_blank">hfinkel@anl.gov</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span class="m_2990692331795416324gmail-">
<p><br>
</p>
<div class="m_2990692331795416324gmail-m_-6492443356502168075moz-cite-prefix">On
12/05/2017 01:06 PM, Joel E. Denny wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span>Hi,<br>
<br>
We are working on a new project, clacc, that
extends clang with OpenACC support. Clacc's
approach is to translate OpenACC (a
descriptive language) to OpenMP (a
prescriptive language) and thus to build on
clang's existing OpenMP support. While we
plan to develop clacc to support our own
research, an important goal is to contribute
clacc as a production-quality component of
upstream clang.<br>
</span></span></div>
</div>
</blockquote>
<br>
</span> Great.<span class="m_2990692331795416324gmail-"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><br>
We have begun implementing an early
prototype of clacc. Before we get too far
into the implementation, we would like to
get feedback from the LLVM community to help
ensure our design would ultimately be
acceptable for contribution. For that
purpose, below is an analysis of several
high-level design alternatives we have
considered and their various features. We
welcome any feedback.<br>
<br>
Thanks.<br>
<br>
Joel E. Denny</span></span></div>
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span>Future
Technologies Group<br>
Oak Ridge National Laboratory</span></span></div>
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><br>
</span></span></div>
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><span style="font-family:monospace,monospace"><br>
</span></span></span></div>
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><span style="font-family:monospace,monospace">Design
Alternatives<br>
-------------------<br>
<br>
We have considered three design
alternatives for the clacc compiler:<br>
<br>
1. acc src
--parser--> <wbr>
omp AST --codegen--> LLVM IR + omp rt
calls<br>
</span></span></span></div>
</div>
</blockquote>
<br>
</span> I don't think that we want this option because,
if nothing else, it will preclude builting source-level
tooling for OpenACC.<span class="m_2990692331795416324gmail-"><br>
</span></div>
</blockquote>
<div><br>
</div>
<div>Agreed.</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span class="m_2990692331795416324gmail-"> <br>
<blockquote type="cite">
<div dir="ltr">
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><span style="font-family:monospace,monospace">2.
acc src --parser--> acc
AST --codegen-->
LLVM IR + omp rt calls<br>
3. acc src --parser--> acc AST
--ttx--> omp AST --codegen-->
LLVM IR + omp rt calls<br>
</span></span></span></div>
</div>
</blockquote>
<br>
</span> My recommendation: We should think about the
very best way we could refactor the code to implement
(2), and if that is too ugly (or otherwise significantly
degrades maintainability of the OpenMP code), then we
should choose (3).</div>
</blockquote>
<div><br>
</div>
<div>I started out with design 2 in the early prototype I'm
experimenting with. Eventually I figured out some
possibilities for how to implement the ttx component above
(I'd be happy to discuss that)</div>
</div>
</div>
</div>
</blockquote>
<br></div>
That's probably a good idea. Please share some details on this
front.</div></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto"><br></div><div dir="auto">Will do. However, due to the holidays, it might take me a little time to put that in an email. </div><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div class="quoted-text"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div>, and I switched to design 3. So far, I'm finding
design 3 to be easier to implement. Moreover, I can use
-ast-print combined with a custom option to print either
OpenACC source, OpenMP source, or both with one commented
out. I like that capability. However, I think it's clear
that design 3 has greater potential for running into
difficulties as I move forward to more complex OpenACC
constructs.</div>
</div>
</div>
</div>
</blockquote>
<br></div>
It is this last part that is potentially concerning. If you try it,
however, and it sounds like you are, then we'll know for sure soon
enough.<br>
<br>
Obviously the most efficient way to write some piece of code, and
the way to write it to maximize maintainability and ease of
extension, may be different. To the extent that they're the same, in
terms of upstream functionality, we'll learn something.</div></blockquote></div></div></div><div dir="auto"><br></div><div dir="auto">Agreed. My hope is that we can get the best of both if design 3 and design 2 prove easily composable.</div><div dir="auto"><br></div><div dir="auto">Thanks.</div><div dir="auto"><br></div><div dir="auto">Joel</div><div dir="auto"><br></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000"><div class="elided-text"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div>
<div class="m_2990692331795416324gmail-h5"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><span style="font-family:monospace,monospace"><br>
In the above diagram:<br>
<br>
* acc src = C source code containing acc
constructs.<br>
* acc AST = a clang AST in which acc
constructs are represented by<br>
nodes with acc node types. Of course,
such node types do not<br>
already exist in clang's
implementation.<br>
* omp AST = a clang AST in which acc
constructs have been lowered<br>
to omp constructs represented by nodes
with omp node types. Of<br>
course, such node types do already
exist in clang's<br>
implementation.<br>
* parser = the existing clang parser and
semantic analyzer,<br>
extended to handle acc constructs.<br>
* codegen = the existing clang backend
that translates a clang AST<br>
to LLVM IR, extended if necessary
(depending on which design is<br>
chosen) to perform codegen from acc
nodes.<br>
* ttx (tree transformer) = a new clang
component that transforms<br>
acc to omp in clang ASTs.<br>
<br>
Design Features<br>
---------------<br>
<br>
There are several features to consider
when choosing among the designs<br>
in the previous section:<br>
<br>
1. acc AST as an artifact -- Because
they create acc AST nodes,<br>
designs 2 and 3 best facilitate the
creation of additional acc<br>
source-level tools (such as pretty
printers, analyzers, lint-like<br>
tools, and editor extensions). Some
of these tools, such as pretty<br>
printing, would be available
immediately or as minor extensions of<br>
tools that already exist in clang's
ecosystem.<br>
<br>
2. omp AST/source as an artifact --
Because they create omp AST<br>
nodes, designs 1 and 3 best
facilitate the use of source-level<br>
tools to help an application
developer discover how clacc has<br>
mapped his acc to omp, possibly in
order to debug a mapping<br>
specification he has supplied. With
design 2 instead, an<br>
application developer has to examine
low-level LLVM IR + omp rt<br>
calls. Moreover, with designs 1 and
3, permanently migrating an<br>
application's acc source to omp
source can be automated.<br>
<br>
3. omp AST for mapping implementation --
Designs 1 and 3 might<br>
also make it easier for the compiler
developer to reason about and<br>
implement mappings from acc to omp.
That is, because acc and omp<br>
syntax is so similar, implementing
the translation at the level of<br>
a syntactic representation is
probably easier than translating to<br>
LLVM IR.<br>
<br>
4. omp AST for codegen -- Designs 1 and
3 simplify the<br>
compiler implementation by enabling
reuse of clang's existing omp<br>
support for codegen. In contrast,
design 2 requires at least some<br>
extensions to clang codegen to
support acc nodes.<br>
<br>
5. Full acc AST for mapping -- Designs 2
and 3 potentially<br>
enable the compiler to analyze the
entire source (as opposed to<br>
just the acc construct currently
being parsed) while choosing the<br>
mapping to omp. It is not clear if
this feature will prove useful,<br>
but it might enable more
optimizations and compiler research<br>
opportunities.<br>
</span></span></span></div>
</div>
</blockquote>
<br>
</div>
</div>
We'll end up doing this, but most of this falls within
the scope of the "parallel IR" designs that many of us
are working on. Doing this kind of analysis in the
frontend is hard (because it essentially requires it to
do inlining, simplification, and analysis akin to what
the optimizer itself does).<span class="m_2990692331795416324gmail-"><br>
</span></div>
</blockquote>
<div><br>
</div>
<div>I agree. However, before the parallel IR efforts
mature, I need to make progress. Also, I want to keep my
options open, especially at this early stage, so I can
experiment with different possibilities.</div>
</div>
</div>
</div>
</blockquote>
<br></div>
You're free to prototype things however you'd like :-)</div></blockquote></div></div></div><div dir="auto"></div><div dir="auto"><div class="gmail_extra"><div class="gmail_quote"><blockquote class="quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div bgcolor="#FFFFFF" text="#000000">
<br>
Thanks again,<br>
Hal<div class="elided-text"><br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_extra">
<div class="gmail_quote">
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span class="m_2990692331795416324gmail-"> <br>
<blockquote type="cite">
<div dir="ltr">
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><span style="font-family:monospace,monospace"><br>
6. No acc node classes -- Design 1
simplifies the compiler<br>
implementation by eliminating the need
to implement many acc node<br>
classes. While we have so far found
that implementing these<br>
classes is mostly mechanical, it does
take a non-trivial amount of<br>
time.<br>
</span></span></span></div>
<span style="font-family:monospace,monospace"><br>
7. No omp mapping -- Design 2 does not require
acc to be mapped to<br>
omp. That is, it is conceivable that, for
some acc constructs,<br>
there will prove to be no omp syntax to
capture the semantics we<br>
wish to implement. <br>
</span></div>
</blockquote>
<br>
</span> I'm fairly certain that not everything maps
exactly. They'll be some things we need to deal with
explicitly in CodeGen.<span class="m_2990692331795416324gmail-"><br>
<br>
<blockquote type="cite">
<div dir="ltr"><span style="font-family:monospace,monospace"> It is
also conceivable that we might one day<br>
want to represent some acc constructs
directly as extensions to<br>
LLVM IR, where some acc analyses or
optimizations might be more<br>
feasible to implement. This possibility
dovetails with recent<br>
discussions in the LLVM community about
developing LLVM IR<br>
extensions for various parallel programming
models.</span><span style="font-family:monospace,monospace"><br>
</span></div>
</blockquote>
<br>
</span> +1<span class="m_2990692331795416324gmail-"><br>
<br>
<blockquote type="cite">
<div dir="ltr"><span style="font-family:monospace,monospace"><br>
<span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span></span></span></span>
<div>
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><span style="font-family:monospace,monospace">Because
of features 4 and 6, design 1 is likely
the fastest design to<br>
implement, at least at first while we
focus on simple acc features and<br>
simple mappings to omp. However, we
have so far found no advantage<br>
that design 1 has but that design 3 does
not have except for feature<br>
6, which we see as the least important
of the above features in the<br>
long term.<br>
<br>
The only advantage we have found that
design 2 has but that design 3<br>
does not have is feature 7. It should
be possible to choose design 3<br>
as the default but, for certain acc
constructs or scenarios where<br>
feature 7 proves important (if any),
incorporate design 2. In other<br>
words, if we decide not to map a
particular acc construct to any omp<br>
construct, ttx would leave it alone, and
we would extend codegen to<br>
handle it directly.<br>
</span></span></span></div>
</div>
</div>
</blockquote>
<br>
</span> This makes sense to me, and I think is most
likely to leave the CodeGen code easiest to maintain
(and has good separation of concerns). Nevertheless, I
think we should go through the mental refactoring
exercise for (2) to decide on the value of (3).<br>
</div>
</blockquote>
<div><br>
</div>
<div>At this moment, I'm finding that the easiest way to
explore is to just push forward with design 3. Even so,
if developers who have a deeper understanding than I do of
clang's OpenMP implementation would like to have an email
discussion on the refactoring exercise for design 2, I
agree that would be helpful.</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> <br>
Thanks again,<br>
Hal<span class="m_2990692331795416324gmail-"><br>
</span></div>
</blockquote>
<div><br>
</div>
<div>Thanks.</div>
<div><br>
</div>
<div>Joel</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"><span class="m_2990692331795416324gmail-"> <br>
<blockquote type="cite">
<div dir="ltr">
<div>
<div><span class="m_2990692331795416324gmail-m_-6492443356502168075gmail-m_1339134059082687238gmail-m_-1727004882907755355gmail-gI"><span><span style="font-family:monospace,monospace"><br>
Conclusions<br>
-----------<br>
<br>
For the above reasons, and because
design 3 offers the cleanest<br>
separation of concerns, we have chosen
design 3 with the possibility<br>
of incorporating design 2 where it
proves useful.<br>
<br>
Because of the immutability of clang's
AST, the design of our proposed<br>
ttx component requires careful
consideration. To shorten this initial<br>
email, we have omitted those details for
now, but we will be happy to<br>
include them as the discussion
progresses.</span><br>
</span></span></div>
</div>
</div>
</blockquote>
<br>
</span><span class="m_2990692331795416324gmail-">
<pre class="m_2990692331795416324gmail-m_-6492443356502168075moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</span></div>
</blockquote>
</div>
<br>
</div>
</div>
</blockquote>
<br>
<pre class="m_2990692331795416324moz-signature" cols="72">--
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
</div></div>
</blockquote></div><br></div></div></div>