[cfe-dev] moving the clang-omp merge along

[Hal Finkel]

----- Original Message -----
> From: "Jack Howarth" <howarth.mailing.lists at gmail.com>
> To: "Hal Finkel" <hfinkel at anl.gov>
> Cc: "cfe-dev" <cfe-dev at cs.uiuc.edu>, "Andrey Bokhanko" <andreybokhanko at gmail.com>
> Sent: Thursday, May 29, 2014 12:12:00 PM
> Subject: Re: [cfe-dev] moving the clang-omp merge along
> 
> 
> Hal,
> I didn't mean to imply anyone was hiding fixes but whether there
> multiple repositories for openmp which just weren't well advertised.
> In particular, the existence of the https://www.openmprtl.org site
> and how that site figures in the general transmission of bug fixes
> between it and the various compiler projects that might use openmp.

The releases on the openmprtl are managed by Intel, and the site existed before the library was contributed to the LLVM project, but certainly lag the code in the LLVM openmp repository at this point.

 -Hal

> Jack
> 
> 
> 
> On Thu, May 29, 2014 at 1:03 PM, Hal Finkel < hfinkel at anl.gov >
> wrote:
> 
> 
> 
> ----- Original Message -----
> > From: "Jack Howarth" < howarth.mailing.lists at gmail.com >
> > To: "Andrey Bokhanko" < andreybokhanko at gmail.com >
> > Cc: "cfe-dev" < cfe-dev at cs.uiuc.edu >
> > Sent: Thursday, May 29, 2014 10:36:47 AM
> > Subject: Re: [cfe-dev] moving the clang-omp merge along
> > 
> > Andrey,
> > Thanks for the clarifications. Everyone I asked had been telling me
> > that llvm.org openmp svn was *the*
> > development repository for openmp but I thought that was unlikely.
> > Where is that active tree at so we can
> > monitor for changes that need to be migrated to the llvm.org copy
> > of
> > openmp? In particular, I am interested
> > in finding the fix failure of…
> 
> Jack,
> 
> Let me try to clarify:
> 
> - OpenMP language support
> 
> - Intel has released its clang-omp branch to the public. On the
> clang-omp github page, there are also trunk-rebased repositories
> that, with some help from Intel, I personally maintain.
> Nevertheless, this code is not part of the Clang project. We have a
> process in place, involving extensive code reviews, to ensure
> implementation quality and maintainability, to help ensure multiple
> developers are familiar with the different areas of the codebase,
> etc. that the OpenMP implementation much undergo. This process,
> considering the limited number of qualified reviewers and the
> overall quantity of code, is operating at a reasonable pace.
> Personally, I would love for it to go much faster (recall, *I*
> maintain the trunk-rebased branch), but that is not necessarily in
> the best interest of the community. Whether this makes the 3.5
> release in total, or takes until 3.6, in the long run, seems to
> matter very little. There has been steady (and now accelerating)
> progress toward contributing OpenMP support to Clang itself. Anyone
> who *needs* OpenMP support now can get it (albeit by applying some
> extra patches), and the pieces that are in the upstream Clang
> repository are of notably higher quality than the original code from
> Intel's clang-omp branch. In the end, everyone will benefit from the
> improved quality of the implementation.
> 
> - OpenMP runtime library
> 
> - This is the piece that is currently on llvm.org as a separate
> project. The openmp repository on llvm.org *is* *the* repository for
> this, at least as it exists as an open-source project. I know of no
> other. Obviously, other entities can have internal branches of any
> project, as Intel may have in this case, and these internal branches
> may contain bug fixes not yet contributed to the open source
> project. When this happens it is unfortunate, and it seems perfectly
> reasonable to push anyone with such internal fixes to contribute
> them sooner rather than later. Nevertheless, no one has been
> withholding otherwise-public code from you -- or, if they have,
> they've been withholding it from me too ;) If you'd like to push for
> such a code release, the openmp-dev list is the appropriate place
> for this, not here.
> 
> I hope this helps.
> 
> -Hal
> 
> 
> > 
> > 
> > 
> > Testing for "omp_taskyield":
> > Generating sources .............. success
> > Compiling soures ................ success
> > Running test with 8 threads ..... success ...sh: line 1: 87638
> > Segmentation fault: 11 ./bin/c/ctest_omp_taskyield >
> > bin/c/ctest_omp_taskyield.out
> > ... and verified with 139% certainty
> > 
> > 
> > in the OpenMP3.1_Validation test suite on darwin. My understanding
> > is
> > that this is to be fixed in
> > a new version of openmp but is unclear where to find that copy.
> > Since
> > the upstream developers of
> > OpenMP3.1_Validation asked for it to be added to the openmp trunk
> > in
> > llvm.org , it would be nice
> 
> 
> > 
> > to fix that for the copy currently checked into llvm.org 's openmp
> > trunk.
> > I assume you intend to eventually move the clang-omp development
> > completely into clang trunk, no?
> > That would be a worthwhile goal for clang 3.5 so that you don't
> > have
> > to constantly rebase the clang-omp
> > branch to the latest release of clang.
> > Jack
> > 
> > 
> > 
> > 
> > On Thu, May 29, 2014 at 9:46 AM, Andrey Bokhanko <
> > andreybokhanko at gmail.com > wrote:
> > 
> > 
> > 
> > All,
> > 
> > To clarify, what I meant is getting OMP runtime library (not
> > clang-omp branch!) as a part of 3.5 release. I specifically
> > referred
> > to this thread:
> > http://lists.cs.uiuc.edu/pipermail/llvmdev/2014-May/thread.html#73025
> > started by Jack. I thought he means libiomp when saying "openmp
> > support" -- apparently, I was mistaken. Hence the confusion.
> > 
> > After so many talks with Chandler, I know very well that
> > upstreaming
> > full OpenMP support is a long way to go. :-)
> > 
> > Also, the whole desire to put the library into 3.5 release is a
> > responce to the criticism expressed by Chandler in this message:
> > http://lists.cs.uiuc.edu/pipermail/cfe-commits/Week-of-Mon-20140217/099477.html
> > .
> > 
> > While we are on the topic, let me update on some other things
> > happening in responce to other issues highlighted by Chandler:
> > 
> > - This library is not being developed as an active part of the LLVM
> > community, even if it is checked into SVN as part of the LLVM
> > project
> > and
> > under its license. See r197914 where there is a code drop of many
> > months
> > worth of development with *no* change log, attribution,
> > information,
> > or
> > other participation in any part of the community.
> > 
> > This is changing, and many developers joining the whole OpenMP in
> > clang support effort. I can say that Michael Wong and many of his
> > colleagues from IBM are involved; Eric Stotzer and his colleagues
> > from TI are involved; Barbara Chapman and her group from UofHouston
> > is involved; Guansong Zhang from AMD is involved. Obviously, Hal
> > Finkel, Chris Bergstrom, Steve Noonan and many others continue to
> > be
> > actively involved as well.
> > 
> > Take a look at the recent activity in openmp-dev mailing list. More
> > to come.
> > 
> > 
> > - There has been essentially *zero* discussion with the rest of the
> > clang
> > or llvm community about this library. There are separate mailing
> > lists
> > which have nearly no traffic since the code drop.
> > 
> > Take a look at this month's messages:
> > http://lists.cs.uiuc.edu/pipermail/openmp-dev/2014-May/thread.html
> > .
> > In general, as more people get interested, more traffic became
> > generated. It's a chicken and egg problem...
> > 
> > - There has been no effort to make this library even work properly
> > with
> > Clang as a host compiler. See the copious notes that only Clang 3.3
> > is
> > supported, and that not full featured.
> > 
> > It is buildable with clang now. Moreover, regular buildbots, with
> > both gcc and clang, are running:
> > 
> > http://lab.llvm.org:8011/builders/libiomp5-gcc-x86_64-linux-debian
> > http://lab.llvm.org:8011/builders/libiomp5-clang-x86_64-linux-debian
> > 
> > - The build system is totally disjoint from LLVM's, in fact it is
> > an
> > entirely custom Perl build system that is unlike anything in use by
> > the
> > LLVM project.
> > 
> > We started to work on improving build system. Stay tuned.
> > 
> > - There are *zero tests* in the open source repository!!!! This is
> > even
> > called out in the original submission and on the primary website.
> > We
> > simply
> > *cannot* ship and link against a runtime library which has no
> > tests!
> > 
> > University of Houston contributed OpenUH test suite:
> > http://lists.cs.uiuc.edu/pipermail/openmp-commits/2014-May/000019.html
> > . Sunita Chandrasekaran from the University works on integrating
> > this suite into LLVM test system.
> > 
> > BTW, any advice with how to approach this would be *much*
> > appreciated!
> > 
> > - No part of this library has gone through an LLVM release process
> > either,
> > not even as a "new" or "beta" project.
> > 
> > Aha! So, you support inclusion of openmp (library, not compiler) in
> > 3.5 as well? :-)
> > 
> > Andrey
> > 
> > 
> > 
> > 
> > 
> > 
> > On Thu, May 29, 2014 at 2:19 AM, Jack Howarth <
> > howarth.mailing.lists at gmail.com > wrote:
> > 
> > 
> > 
> > 
> > Andrey Bokhanko expressed interest in getting the clang-omp merge
> > done in time for the 3.5 release but wants guidance on the process.
> > I suggested starting with the creation a new clang-omp branch
> > upstream rebased on clang trunk for generation of merge patch.
> > Unfortunately merging the current changes from the clang-omp (based
> > on clang 3.4) to a clang-omp (based on clang trunk) looks very
> > difficult as selective patches have been committed into clang trunk
> > from clang-omp and don't appear to have been kept synchronized with
> > the current changes from upstream. Does anyone know if these new
> > files from previous commits out of clang-omp contain any local
> > changes which haven't been back ported to clang-omp? It would seem
> > that postponing this merge will just make the process harder as
> > time
> > goes on if selective merges from clang-omp into clang trunk
> > continue
> > in the interim. Hopefully the folks who did the original selective
> > commits would help detangle this mess.
> > Jack
> > 
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> > 
> > 
> > 
> > 
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
> > 
> 
> --
> Hal Finkel
> Assistant Computational Scientist
> Leadership Computing Facility
> Argonne National Laboratory
> 
>