<html><head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body>
    <p><br>
    </p>
    <div class="moz-cite-prefix">On 7/10/20 4:00 PM, JF Bastien wrote:<br>
    </div>
    <blockquote type="cite" cite="mid:FAF919EA-DD62-4BE3-AB8D-4B5CF321BBA2@apple.com">
      
      <br class="">
      <div><br class="">
        <blockquote type="cite" class="">
          <div class="">On Jul 10, 2020, at 1:55 PM, Hal Finkel <<a href="mailto:hfinkel@anl.gov" class="" moz-do-not-send="true">hfinkel@anl.gov</a>> wrote:</div>
          <br class="Apple-interchange-newline">
          <div class=""><br style="caret-color: rgb(0, 0, 0);
              font-family: Helvetica; font-size: 12px; font-style:
              normal; font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none;" class="">
            <span style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class="">On 7/10/20 1:57 PM, Vassil Vassilev
              wrote:</span><br style="caret-color: rgb(0, 0, 0);
              font-family: Helvetica; font-size: 12px; font-style:
              normal; font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none;" class="">
            <blockquote type="cite" style="font-family: Helvetica;
              font-size: 12px; font-style: normal; font-variant-caps:
              normal; font-weight: normal; letter-spacing: normal;
              orphans: auto; text-align: start; text-indent: 0px;
              text-transform: none; white-space: normal; widows: auto;
              word-spacing: 0px; -webkit-text-size-adjust: auto;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class="">On 7/10/20 6:43 AM, JF Bastien wrote:<br class="">
              <blockquote type="cite" class="">I like cling, and having
                it integrated with the rest of the project would be
                neat. I agree with Hal’s suggestion to explain the
                design of what remains. It sounds like a pretty small
                amount of code.<br class="">
              </blockquote>
              <br class="">
              <br class="">
               <span class="Apple-converted-space"> </span>JF, Hal, did
              you mean you want a design document of how cling in
              general or a design RFC for the patches we have? A design
              document for cling would be quite large and will take us
              some time to write up. OTOH, we could relatively easily
              give a rationale for each patch.<br class="">
            </blockquote>
            <br style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none;" class="">
            <br style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none;" class="">
            <span style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class="">I had in mind something that's
              probably in between. Something that explains the patches
              and enough about how they fit into a larger system that we
              can reason about the context.</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica;
              font-size: 12px; font-style: normal; font-variant-caps:
              normal; font-weight: normal; letter-spacing: normal;
              text-align: start; text-indent: 0px; text-transform: none;
              white-space: normal; word-spacing: 0px;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
          </div>
        </blockquote>
        <div><br class="">
        </div>
        <div>Maybe a purpose would be more useful to understand your
          request? I assume you meant “I’d like us to understand what
          we’re signing up to maintain, and why it’s useful to do things
          this way”. In particular, if there’s undue burden in a
          particular component, and the code could be changed to work
          differently with less support overhead, then we’d want to
          identify this fact ahead of time.</div>
        <div><br class="">
        </div>
        <div>I’m guessing at what Hal is asking, LMK if that’s not what
          you had in mind!</div>
      </div>
    </blockquote>
    <p><br>
    </p>
    <p>Yes. To understand how all of the pieces fit together to enable
      support for incremental compilation of C++ code. Once everything
      is in place, if I wanted to use the infrastructure to do some kind
      of incremental compilation of C++, what would I do? And what do
      the set of patches aim to do to get us there?<br>
    </p>
    <p> -Hal<br>
    </p>
    <p><br>
    </p>
    <blockquote type="cite" cite="mid:FAF919EA-DD62-4BE3-AB8D-4B5CF321BBA2@apple.com">
      <div>
        <div><br class="">
        </div>
        <br class="">
        <blockquote type="cite" class="">
          <div class=""><span style="caret-color: rgb(0, 0, 0);
              font-family: Helvetica; font-size: 12px; font-style:
              normal; font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class=""> -Hal</span><br style="caret-color:
              rgb(0, 0, 0); font-family: Helvetica; font-size: 12px;
              font-style: normal; font-variant-caps: normal;
              font-weight: normal; letter-spacing: normal; text-align:
              start; text-indent: 0px; text-transform: none;
              white-space: normal; word-spacing: 0px;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
            <br style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none;" class="">
            <br style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none;" class="">
            <blockquote type="cite" style="font-family: Helvetica;
              font-size: 12px; font-style: normal; font-variant-caps:
              normal; font-weight: normal; letter-spacing: normal;
              orphans: auto; text-align: start; text-indent: 0px;
              text-transform: none; white-space: normal; widows: auto;
              word-spacing: 0px; -webkit-text-size-adjust: auto;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><br class="">
              <br class="">
              <blockquote type="cite" class=""><br class="">
                <br class="">
                <blockquote type="cite" class="">On Jul 9, 2020, at 7:25
                  PM, Hal Finkel via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" class="" moz-do-not-send="true">cfe-dev@lists.llvm.org</a>>
                  wrote:<br class="">
                  <br class="">
                  I think that it would be great to have infrastructure
                  for incremental C++ compilation, supporting
                  interactive use, just-in-time compilation, and so on.
                  I think that the best way to deal with the patches,
                  etc., as well as IncrementalAction, is to first send
                  an RFC explaining the overall design.<br class="">
                  <br class="">
                   <span class="Apple-converted-space"> </span>-Hal<br class="">
                  <br class="">
                  On 7/9/20 3:46 PM, Vassil Vassilev via cfe-dev wrote:<br class="">
                  <blockquote type="cite" class="">Motivation<br class="">
                    ===<br class="">
                    <br class="">
                    Over the last decade we have developed an
                    interactive, interpretative C++ (aka REPL) as part
                    of the high-energy physics (HEP) data analysis
                    project -- ROOT [1-2]. We invested a significant 
                    effort to replace the CINT C++ interpreter with a
                    newly implemented REPL based on llvm -- cling [3].
                    The cling infrastructure is a core component of the
                    data analysis framework of ROOT and runs in
                    production for approximately 5 years.<br class="">
                    <br class="">
                    Cling is also  a standalone tool, which has a
                    growing community outside of our field. Cling’s user
                    community includes users in finance, biology and in
                    a few companies with proprietary software. For
                    example, there is a xeus-cling jupyter kernel [4].
                    One of the major challenges we face to foster that
                    community is  our cling-related patches in llvm and
                    clang forks. The benefits of using the LLVM
                    community standards for code reviews, release cycles
                    and integration has been mentioned a number of times
                    by our "external" users.<br class="">
                    <br class="">
                    Last year we were awarded an NSF grant to improve
                    cling's sustainability and make it a standalone
                    tool. We thank the LLVM Foundation Board for
                    supporting us with a non-binding letter of
                    collaboration which was essential for getting this
                    grant.<br class="">
                    <br class="">
                    <br class="">
                    Background<br class="">
                    ===<br class="">
                    <br class="">
                    Cling is a C++ interpreter built on top of clang and
                    llvm. In a nutshell, it uses clang's incremental
                    compilation facilities to process code
                    chunk-by-chunk by assuming an ever-growing
                    translation unit [5]. Then code is lowered into llvm
                    IR and run by the llvm jit. Cling has implemented
                    some language "extensions" such as execution
                    statements on the global scope and error recovery.
                    Cling is in the core of HEP -- it is heavily used
                    during data analysis of exabytes of particle physics
                    data coming from the Large Hadron Collider (LHC) and
                    other particle physics experiments.<br class="">
                    <br class="">
                    <br class="">
                    Plans<br class="">
                    ===<br class="">
                    <br class="">
                    The project foresees three main directions -- move
                    parts of cling upstream along with the clang and
                    llvm features that enable them; extend and
                    generalize the language interoperability layer
                    around cling; and extend and generalize the
                    OpenCL/CUDA support in cling. We are at the early
                    stages of the project and this email intends to be
                    an RFC for the first part -- upstreaming parts of
                    cling. Please do share your thoughts on the rest,
                    too.<br class="">
                    <br class="">
                    <br class="">
                    Moving Parts of Cling Upstream<br class="">
                    ---<br class="">
                    <br class="">
                    Over the years we have slowly moved some patches
                    upstream. However we still have around 100 patches
                    in the clang fork. Most of them are in the context
                    of extending the incremental compilation support for
                    clang. The incremental compilation poses some
                    challenges in the clang infrastructure. For example,
                    we need to tune CodeGen to work with multiple
                    llvm::Module instances, and finalize per each
                    end-of-translation unit (we have multiple of them).
                    Other changes include small adjustments in the
                    FileManager's caching mechanism, and bug fixes in
                    the SourceManager (code which can be reached mostly
                    from within our setup). One conclusion we can draw
                    from our research is that the clang infrastructure
                    fits amazingly well to something which was not its
                    main use case. The grand total of our diffs against
                    clang-9 is: `62 files changed, 1294 insertions(+),
                    231 deletions(-)`. Cling is currently being upgraded
                    from llvm-5 to llvm-9.<br class="">
                    <br class="">
                    A major weakness of cling's infrastructure is that
                    it does not work with the clang Action
                    infrastructure due to the lack of an
                    IncrementalAction.  A possible way forward would be
                    to implement a clang::IncrementalAction as a
                    starting point. This way we should be able to reduce
                    the amount of setup necessary to use the incremental
                    infrastructure in clang. However, this will be a bit
                    of a testing challenge -- cling lives downstream and
                    some of the new code may be impossible to pick
                    straight away and use. Building a mainline example
                    tool such as clang-repl which gives us a way to test
                    that incremental case or repurpose the already
                    existing clang-interpreter may  be able to address
                    the issue. The major risk of the task is avoiding
                    code in the clang mainline which is untested by its
                    HEP production environment.<br class="">
                    There are several other types of patches to the ROOT
                    fork of Clang, including ones  in the context of
                    performance,towards  C++ modules support (D41416),
                    and storage (does not have a patch yet but has an
                    open projects entry and somebody working on it).
                    These patches can be considered in parallel
                    independently on the rest.<br class="">
                    <br class="">
                    Extend and Generalize the Language Interoperability
                    Layer Around Cling<br class="">
                    ---<br class="">
                    <br class="">
                    HEP has extensive experience with on-demand python
                    interoperability using cppyy[6], which is built
                    around the type information provided by cling.
                    Unlike tools with custom parsers such as swig and
                    sip and tools built on top of C-APIs such as
                    boost.python and pybind11, cling can provide
                    information about memory management patterns (eg
                    refcounting) and instantiate templates on the fly.We
                    feel that functionality may not be of general
                    interest to the llvm community but we will prepare
                    another RFC and send it here later on to gather
                    feedback.<br class="">
                    <br class="">
                    <br class="">
                    Extend and Generalize the OpenCL/CUDA Support in
                    Cling<br class="">
                    ---<br class="">
                    <br class="">
                    Cling can incrementally compile CUDA code [7-8]
                    allowing easier set up and enabling some interesting
                    use cases. There are a number of planned
                    improvements including talking to HIP [9] and SYCL
                    to support more hardware architectures.<br class="">
                    <br class="">
                    <br class="">
                    <br class="">
                    The primary focus of our work is to upstreaming
                    functionality required to build an incremental
                    compiler and rework cling build against vanilla
                    clang and llvm. The last two points are to give the
                    scope of the work which we will be doing the next
                    2-3 years. We will send here RFCs for both of them
                    to trigger technical discussion if there is interest
                    in pursuing this direction.<br class="">
                    <br class="">
                    <br class="">
                    Collaboration<br class="">
                    ===<br class="">
                    <br class="">
                    Open source development nowadays relies on
                    reviewers. LLVM is no different and we will probably
                    disturb a good number of people in the community
                    ;)We would like to invite anybody interested in
                    joining our incremental C++ activities to our open
                    every second week calls. Announcements will be done
                    via google group: compiler-research-announce (<a href="https://groups.google.com/g/compiler-research-announce" class="" moz-do-not-send="true">https://groups.google.com/g/compiler-research-announce</a>).<br class="">
                    <br class="">
                    <br class="">
                    <br class="">
                    Many thanks!<br class="">
                    <br class="">
                    <br class="">
                    David & Vassil<br class="">
                    <br class="">
                    References<br class="">
                    ===<br class="">
                    [1] ROOT GitHub <a href="https://github.com/root-project/root" class="" moz-do-not-send="true">https://github.com/root-project/root</a><br class="">
                    [2] ROOT <a href="https://root.cern" class="" moz-do-not-send="true">https://root.cern</a><br class="">
                    [3] Cling <a href="https://github.com/root-project/cling" class="" moz-do-not-send="true">https://github.com/root-project/cling</a><br class="">
                    [4] Xeus-Cling <a href="https://blog.jupyter.org/xeus-is-now-a-jupyter-subproject-c4ec5a1bf30b" class="" moz-do-not-send="true">https://blog.jupyter.org/xeus-is-now-a-jupyter-subproject-c4ec5a1bf30b</a><br class="">
                    [5] Cling – The New Interactive Interpreter for ROOT
                    6, <a href="https://iopscience.iop.org/article/10.1088/1742-6596/396/5/052071" class="" moz-do-not-send="true">https://iopscience.iop.org/article/10.1088/1742-6596/396/5/052071</a><br class="">
                    [6] High-performance Python-C++ bindings with PyPy
                    and Cling, <a href="https://dl.acm.org/doi/10.5555/3019083.3019087" class="" moz-do-not-send="true">https://dl.acm.org/doi/10.5555/3019083.3019087</a><br class="">
                    [7] <a href="https://indico.cern.ch/event/697389/contributions/3085538/attachments/1712698/2761717/2018_09_10_cling_CUDA.pdf" class="" moz-do-not-send="true">https://indico.cern.ch/event/697389/contributions/3085538/attachments/1712698/2761717/2018_09_10_cling_CUDA.pdf</a><br class="">
                    [8] CUDA C++ in Jupyter: Adding CUDA Runtime Support
                    to Cling', <a href="https://zenodo.org/record/3713753#.Xu8jqvJRXxU" class="" moz-do-not-send="true">https://zenodo.org/record/3713753#.Xu8jqvJRXxU</a><br class="">
                    [9] HIP Programming Guide <a href="https://rocmdocs.amd.com/en/latest/Programming_Guides/HIP-GUIDE.html" class="" moz-do-not-send="true">https://rocmdocs.amd.com/en/latest/Programming_Guides/HIP-GUIDE.html</a><br class="">
                    <br class="">
                    _______________________________________________<br class="">
                    cfe-dev mailing list<br class="">
                    <a href="mailto:cfe-dev@lists.llvm.org" class="" moz-do-not-send="true">cfe-dev@lists.llvm.org</a><br class="">
<a class="moz-txt-link-freetext" href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br class="">
                  </blockquote>
                  --<span class="Apple-converted-space"> </span><br class="">
                  Hal Finkel<br class="">
                  Lead, Compiler Technology and Programming Languages<br class="">
                  Leadership Computing Facility<br class="">
                  Argonne National Laboratory<br class="">
                  <br class="">
                  _______________________________________________<br class="">
                  cfe-dev mailing list<br class="">
                  <a href="mailto:cfe-dev@lists.llvm.org" class="" moz-do-not-send="true">cfe-dev@lists.llvm.org</a><br class="">
<a class="moz-txt-link-freetext" href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br class="">
                </blockquote>
              </blockquote>
              <br class="">
              <br class="">
            </blockquote>
            <span style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class="">--<span class="Apple-converted-space"> </span></span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica;
              font-size: 12px; font-style: normal; font-variant-caps:
              normal; font-weight: normal; letter-spacing: normal;
              text-align: start; text-indent: 0px; text-transform: none;
              white-space: normal; word-spacing: 0px;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
            <span style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class="">Hal Finkel</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica;
              font-size: 12px; font-style: normal; font-variant-caps:
              normal; font-weight: normal; letter-spacing: normal;
              text-align: start; text-indent: 0px; text-transform: none;
              white-space: normal; word-spacing: 0px;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
            <span style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class="">Lead, Compiler Technology and
              Programming Languages</span><br style="caret-color: rgb(0,
              0, 0); font-family: Helvetica; font-size: 12px;
              font-style: normal; font-variant-caps: normal;
              font-weight: normal; letter-spacing: normal; text-align:
              start; text-indent: 0px; text-transform: none;
              white-space: normal; word-spacing: 0px;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
            <span style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class="">Leadership Computing Facility</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica;
              font-size: 12px; font-style: normal; font-variant-caps:
              normal; font-weight: normal; letter-spacing: normal;
              text-align: start; text-indent: 0px; text-transform: none;
              white-space: normal; word-spacing: 0px;
              -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
            <span style="caret-color: rgb(0, 0, 0); font-family:
              Helvetica; font-size: 12px; font-style: normal;
              font-variant-caps: normal; font-weight: normal;
              letter-spacing: normal; text-align: start; text-indent:
              0px; text-transform: none; white-space: normal;
              word-spacing: 0px; -webkit-text-stroke-width: 0px;
              text-decoration: none; float: none; display: inline
              !important;" class="">Argonne National Laboratory</span></div>
        </blockquote>
      </div>
      <br class="">
    </blockquote>
    <pre class="moz-signature" cols="72">-- 
Hal Finkel
Lead, Compiler Technology and Programming Languages
Leadership Computing Facility
Argonne National Laboratory</pre>
  </body>
</html>