<html>
  <head>
    <meta content="text/html; charset=utf-8" http-equiv="Content-Type">
  </head>
  <body bgcolor="#FFFFFF" text="#000000">
    César,<br>
    Currently clang does not support array sections. Initial
    implementation of array sections is under review.<br>
    <pre class="moz-signature" cols="72">Best regards,
Alexey Bataev
=============
Software Engineer
Intel Compiler Team</pre>
    <div class="moz-cite-prefix">19.08.2015 20:26, César via Openmp-dev
      пишет:<br>
    </div>
    <blockquote
cite="mid:CAHN8PeeDwvVGHbFeC0mSnkBpO5EdBFRLuCM2LWqgrEg1J6201w@mail.gmail.com"
      type="cite">
      <div dir="ltr">Thanks Jonathan, using -fopenmp=libomp I was able
        to link against libomp. However, for some benchmarks I get
        syntax error:
        <div><br>
        </div>
        <div>
          <div>src/sparselu-task-dep.c:35:89: error: expected ']'</div>
          <div>                        #pragma omp task firstprivate(kk)
            shared(BENCH) depend(inout:
            BENCH[kk*matrix_size+kk:submatrix_size*submatrix_size])</div>
        </div>
        <div>
          <div>                                                         
                                                                       
                                                                 ^</div>
        </div>
        <div><br>
        </div>
        <div>the same benchmark compile with the [older] Github version
          of Clang, with clang-3.8 -fopenmp and also with GCC.</div>
        <div><br>
        </div>
        <div>Doesn't Clang support specifying memory ranges for the
          dependencies or does it just uses another syntax?</div>
        <div><br>
        </div>
        <div><br>
        </div>
        <div><br>
        </div>
      </div>
      <div class="gmail_extra"><br clear="all">
        <div>
          <div class="gmail_signature"><br>
            <br>
            César.</div>
        </div>
        <br>
        <div class="gmail_quote">On Wed, Aug 19, 2015 at 1:37 PM,
          Peyton, Jonathan L <span dir="ltr"><<a
              moz-do-not-send="true"
              href="mailto:jonathan.l.peyton@intel.com" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:jonathan.l.peyton@intel.com">jonathan.l.peyton@intel.com</a></a>></span>
          wrote:<br>
          <blockquote class="gmail_quote" style="margin:0 0 0
            .8ex;border-left:1px #ccc solid;padding-left:1ex">
            <div link="blue" vlink="purple" lang="EN-US">
              <div>
                <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">Currently
                    to use libomp, the flag is: -fopenmp=libomp</span></p>
                <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></p>
                <p class="MsoNormal"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d">--
                    Johnny</span></p>
                <p class="MsoNormal"><a moz-do-not-send="true"
                    name="14f46d30b9324386__MailEndCompose"><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1f497d"> </span></a></p>
                <p class="MsoNormal"><b><span
                      style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span
style="font-size:11.0pt;font-family:"Calibri",sans-serif">
                    Openmp-dev [mailto:<a moz-do-not-send="true"
                      href="mailto:openmp-dev-bounces@lists.llvm.org"
                      target="_blank">openmp-dev-bounces@lists.llvm.org</a>]
                    <b>On Behalf Of </b>César via Openmp-dev<br>
                    <b>Sent:</b> Wednesday, August 19, 2015 11:30 AM<br>
                    <b>To:</b> andreybokhanko<span class=""><br>
                      <b>Cc:</b> <a moz-do-not-send="true"
                        href="mailto:openmp-dev@lists.llvm.org"
                        target="_blank">openmp-dev@lists.llvm.org</a><br>
                      <b>Subject:</b> Re: [Openmp-dev] Performance
                      slowdown</span></span></p>
                <p class="MsoNormal"> </p>
                <div>
                  <div>
                    <div>
                      <p class="MsoNormal">On Wed, Aug 19, 2015 at 12:59
                        PM, andreybokhanko <<a moz-do-not-send="true"
                          href="mailto:andreybokhanko@gmail.com"
                          target="_blank">andreybokhanko@gmail.com</a>>
                        wrote:</p>
                      <div>
                        <div class="h5">
                          <blockquote
                            style="border:none;border-left:solid #cccccc
                            1.0pt;padding:0in 0in 0in
                            6.0pt;margin-left:4.8pt;margin-right:0in">
                            <div>
                              <div>
                                <p class="MsoNormal">Indeed, I meant
                                  official released 3.5. Did you get
                                  your compiler from clang-omp.github?
                                  It's probably outdated and can't be
                                  used for reliable performance
                                  measurements.</p>
                              </div>
                              <div>
                                <p class="MsoNormal"> </p>
                              </div>
                              <div>
                                <p class="MsoNormal"
                                  style="margin-bottom:12.0pt">I will
                                  update clang-omp.github home page to
                                  avoid further confusion.</p>
                              </div>
                            </div>
                          </blockquote>
                          <div>
                            <p class="MsoNormal"> </p>
                          </div>
                          <div>
                            <p class="MsoNormal">Yes, that was exactly
                              what happend! Thanks Alexey/Andrey. </p>
                          </div>
                          <div>
                            <p class="MsoNormal"> </p>
                          </div>
                          <div>
                            <p class="MsoNormal">I have just downloaded
                              clang-3.8 (trunk) and started some
                              experiments, however I see that clang is
                              trying to link with "lib gomp" (GNU) and
                              not "libomp" (Intel). Is this really the
                              intended default behavior? How can I tell
                              clang to use the Intel OMP?</p>
                          </div>
                          <div>
                            <p class="MsoNormal"> </p>
                          </div>
                          <div>
                            <p class="MsoNormal"> </p>
                          </div>
                          <div>
                            <p class="MsoNormal"> </p>
                          </div>
                          <div>
                            <p class="MsoNormal"> </p>
                          </div>
                          <blockquote
                            style="border:none;border-left:solid #cccccc
                            1.0pt;padding:0in 0in 0in
                            6.0pt;margin-left:4.8pt;margin-right:0in">
                            <div>
                              <div>
                                <p class="MsoNormal">Yours,</p>
                                <div>
                                  <p class="MsoNormal">Andrey</p>
                                </div>
                              </div>
                              <div>
                                <p class="MsoNormal"
                                  style="margin-bottom:12.0pt"><br>
                                  19 авг. 2015 г., в 8:17, Bataev,
                                  Alexey <<a moz-do-not-send="true"
                                    href="mailto:a.bataev@hotmail.com"
                                    target="_blank">a.bataev@hotmail.com</a>>
                                  написал(а):</p>
                              </div>
                              <div>
                                <div>
                                  <blockquote
                                    style="margin-top:5.0pt;margin-bottom:5.0pt">
                                    <div>
                                      <p class="MsoNormal">If you're
                                        using 3.5, then this is
                                        unofficial version with OpenMP.
                                        Use 3.7 instead just like Andrey
                                        said<br>
                                        <br>
                                      </p>
                                      <pre>Best regards,</pre>
                                      <pre>Alexey Bataev</pre>
                                      <pre>=============</pre>
                                      <pre>Software Engineer</pre>
                                      <pre>Intel Compiler Team</pre>
                                      <div>
                                        <p class="MsoNormal">19.08.2015
                                          1:25, César via Openmp-dev
                                          пишет:</p>
                                      </div>
                                      <blockquote
                                        style="margin-top:5.0pt;margin-bottom:5.0pt">
                                        <div>
                                          <p class="MsoNormal">Hi
                                            Andrey, </p>
                                          <div>
                                            <p class="MsoNormal"> </p>
                                          </div>
                                          <div>
                                            <p class="MsoNormal">this is
                                              strange because when I
                                              compile with "clang-3.5
                                              -fopenmp" the executable
                                              that is produced is
                                              parallel. I am sure of
                                              this because I'm able to
                                              see the threads and also
                                              because I can see the
                                              symbols used by the IOMP
                                              runtime in the binary.</p>
                                          </div>
                                          <div>
                                            <p class="MsoNormal"> </p>
                                          </div>
                                          <div>
                                            <p class="MsoNormal">$ clang
                                              -O3 -g -fopenmp toy13.cpp
                                              -o toy13 -lm</p>
                                          </div>
                                          <div>
                                            <p class="MsoNormal"> </p>
                                          </div>
                                          <div>
                                            <div>
                                              <p class="MsoNormal">$ nm
                                                toy13 | grep kmpc</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">U
                                                __kmpc_cancel_barrier@@VERSION</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">U
                                                __kmpc_end_single@@VERSION</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">U
                                                __kmpc_fork_call@@VERSION</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">U
                                                __kmpc_omp_task_alloc@@VERSION</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">U
                                                __kmpc_omp_task_with_deps@@VERSION</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">U
                                                __kmpc_single@@VERSION</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal"> </p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">$ ldd
                                                toy13 </p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">linux-vdso.so.1
                                                =>
                                                 (0x00007fff9805d000)</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">libm.so.6
                                                =>
                                                /lib/x86_64-linux-gnu/libm.so.6
                                                (0x00007fc00e3cc000)</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">libiomp5.so
                                                =>
                                                /usr/lib/libiomp5.so
                                                (0x00007fc00e121000)</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">libpthread.so.0
                                                =>
                                                /lib/x86_64-linux-gnu/libpthread.so.0
                                                (0x00007fc00df03000)</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">libc.so.6
                                                =>
                                                /lib/x86_64-linux-gnu/libc.so.6
                                                (0x00007fc00db3e000)</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">libdl.so.2
                                                =>
                                                /lib/x86_64-linux-gnu/libdl.so.2
                                                (0x00007fc00d939000)</p>
                                            </div>
                                            <div>
                                              <p class="MsoNormal">/lib64/ld-linux-x86-64.so.2
                                                (0x00007fc00e6fc000)</p>
                                            </div>
                                          </div>
                                          <div>
                                            <p class="MsoNormal"> </p>
                                          </div>
                                          <div>
                                            <p class="MsoNormal"> </p>
                                          </div>
                                          <div>
                                            <p class="MsoNormal"><br
                                                clear="all">
                                            </p>
                                            <div>
                                              <div>
                                                <p class="MsoNormal"><br>
                                                  <br>
                                                  César.</p>
                                              </div>
                                            </div>
                                            <p class="MsoNormal"> </p>
                                            <div>
                                              <p class="MsoNormal">On
                                                Tue, Aug 18, 2015 at
                                                6:15 PM, <<a
                                                  moz-do-not-send="true"
href="mailto:andreybokhanko@gmail.com" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:andreybokhanko@gmail.com">andreybokhanko@gmail.com</a></a>>
                                                wrote:</p>
                                              <blockquote
                                                style="border:none;border-left:solid
                                                #cccccc
                                                1.0pt;padding:0in 0in
                                                0in
                                                6.0pt;margin-left:4.8pt;margin-right:0in">
                                                <div>
                                                  <div>
                                                    <p class="MsoNormal">César,</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal"> </p>
                                                  </div>
                                                  <div>
                                                    <blockquote
                                                      style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                      <div>
                                                        <div>
                                                          <p
                                                          class="MsoNormal"><span
style="color:black">- The version of Clang that I am using to compile
                                                          the
                                                          benchmarks:
                                                          3.5.0</span></p>
                                                        </div>
                                                      </div>
                                                    </blockquote>
                                                    <div>
                                                      <p
                                                        class="MsoNormal"> </p>
                                                    </div>
                                                    <p class="MsoNormal">Clang
                                                      3.5 doesn't
                                                      support OpenMP --
                                                      it simply ignores
                                                      the pragmas.</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal"> </p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal">Please
                                                      use version from
                                                      trunk or from 3_7
                                                      release branch.
                                                      Also, please
                                                      supply -fopenmp=
                                                      libomp option.</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal"> </p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal">Yours,</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal">Andrey
                                                      Bokhanko</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal">=============</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal">Software
                                                      Engineer</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal">Intel
                                                      Compiler Team</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal">Intel</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal"><br>
                                                      Отправлено с iPad</p>
                                                  </div>
                                                  <div>
                                                    <p class="MsoNormal"
style="margin-bottom:12.0pt"><br>
                                                      18 авг. 2015 г., в
                                                      21:14, César via
                                                      Openmp-dev <<a
moz-do-not-send="true" href="mailto:openmp-dev@lists.llvm.org"
                                                        target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:openmp-dev@lists.llvm.org">openmp-dev@lists.llvm.org</a></a>>
                                                      написал(а):</p>
                                                  </div>
                                                  <div>
                                                    <div>
                                                      <blockquote
                                                        style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                        <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">Hello,
                                                          </p>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">I
                                                          don't know if
                                                          this is the
                                                          correct list
                                                          to talk about
                                                          this - I did
                                                          not find a
                                                          better place..</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">I
                                                          am doing
                                                          performance
                                                          experiments
                                                          with a few
                                                          OpenMP
                                                          implementations
                                                          (IOMP, GOMP
                                                          and our
                                                          private impl.)
                                                          and I am
                                                          seeing a
                                                          severe
                                                          slowdown when
                                                          I use IOMP
                                                          (GOMP and
                                                          others are
                                                          performing
                                                          well). </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">The
                                                          benchmarks I
                                                          am using are
                                                          these ones: <a
moz-do-not-send="true"
                                                          href="http://kastors.gforge.inria.fr/#%21index.md"
target="_blank"><a class="moz-txt-link-freetext" href="http://kastors.gforge.inria.fr/#!index.md">http://kastors.gforge.inria.fr/#!index.md</a></a><br
                                                          clear="all">
                                                          </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">Really,
                                                          the slowdown
                                                          is huge. For
                                                          one of the
                                                          programs
                                                          (plasma/dpotrf_taskdep
                                                          -n 8192 -b 64
                                                          -i 1 -c) the
                                                          serial version
                                                          executes in
                                                          ~28s and the
                                                          parallel one
                                                          executes in
                                                          ~110s. I did
                                                          some profiling
                                                          and found that
                                                          most of the
                                                          time is being
                                                          spent on
                                                          synchronization
                                                          barriers and
                                                          dependence
                                                          tracking (see
                                                          attached
                                                          image). Before
                                                          digging deeper
                                                          I would like
                                                          to hear back
                                                          from you if I
                                                          am doing
                                                          something
                                                          wrong here:</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"> </p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">-
                                                          I tested with
                                                          the last
                                                          version of the
                                                          repository:  <a
moz-do-not-send="true"
                                                          href="http://llvm.org/svn/llvm-project/openmp/trunk"
target="_blank"><a class="moz-txt-link-freetext" href="http://llvm.org/svn/llvm-project/openmp/trunk">http://llvm.org/svn/llvm-project/openmp/trunk</a></a></p>
                                                          </div>
                                                          <div>
                                                          <div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">-
                                                          I am using
                                                          Ubuntu 14.10.</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">-
                                                          I have tested
                                                          on more than
                                                          one machine,
                                                          the results
                                                          above are from
                                                          a Intel
                                                          i7-3770</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">-
                                                          The runtime
                                                          itself is
                                                          compiled
                                                          using: make
                                                          compiler=gcc
                                                          os_omp=linux
                                                          arch=32e</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">-
                                                          The version of
                                                          GCC that I am
                                                          using is:
                                                          4.9.1</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal">-
                                                          The version of
                                                          Clang that I
                                                          am using to
                                                          compile the
                                                          benchmarks:
                                                          3.5.0</p>
                                                          </div>
                                                          <div>
                                                          <p
                                                          class="MsoNormal"><br>
                                                          <br>
                                                          César.</p>
                                                          </div>
                                                          </div>
                                                          </div>
                                                          </div>
                                                        </div>
                                                      </blockquote>
                                                    </div>
                                                  </div>
                                                  <blockquote
                                                    style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                    <div>
                                                      <p
                                                        class="MsoNormal"><pic1.png></p>
                                                    </div>
                                                  </blockquote>
                                                  <blockquote
                                                    style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                    <div>
                                                      <p
                                                        class="MsoNormal"><pic2.png></p>
                                                    </div>
                                                  </blockquote>
                                                  <blockquote
                                                    style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                    <div>
                                                      <p
                                                        class="MsoNormal"><pic3.png></p>
                                                    </div>
                                                  </blockquote>
                                                  <blockquote
                                                    style="margin-top:5.0pt;margin-bottom:5.0pt">
                                                    <div>
                                                      <p
                                                        class="MsoNormal">_______________________________________________<br>
                                                        Openmp-dev
                                                        mailing list<br>
                                                        <a
                                                          moz-do-not-send="true"
href="mailto:Openmp-dev@lists.llvm.org" target="_blank"><a class="moz-txt-link-abbreviated" href="mailto:Openmp-dev@lists.llvm.org">Openmp-dev@lists.llvm.org</a></a><br>
                                                        <a
                                                          moz-do-not-send="true"
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev"
                                                          target="_blank"><a class="moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a></a></p>
                                                    </div>
                                                  </blockquote>
                                                </div>
                                              </blockquote>
                                            </div>
                                            <p class="MsoNormal"> </p>
                                          </div>
                                        </div>
                                        <p class="MsoNormal"
                                          style="margin-bottom:12.0pt"> </p>
                                        <pre>_______________________________________________</pre>
                                        <pre>Openmp-dev mailing list</pre>
                                        <pre><a moz-do-not-send="true" href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a></pre>
                                        <pre><a moz-do-not-send="true" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a></pre>
                                      </blockquote>
                                      <p class="MsoNormal"> </p>
                                    </div>
                                  </blockquote>
                                </div>
                              </div>
                            </div>
                          </blockquote>
                        </div>
                      </div>
                    </div>
                    <p class="MsoNormal"> </p>
                  </div>
                </div>
              </div>
            </div>
          </blockquote>
        </div>
        <br>
      </div>
      <br>
      <fieldset class="mimeAttachmentHeader"></fieldset>
      <br>
      <pre wrap="">_______________________________________________
Openmp-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:Openmp-dev@lists.llvm.org">Openmp-dev@lists.llvm.org</a>
<a class="moz-txt-link-freetext" href="http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev">http://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a>
</pre>
    </blockquote>
    <br>
  </body>
</html>