<div dir="ltr">Alexey,<div>Thanks for checking the codebase. Do you feel that the C++ program is hard to support in the</div><div>current framework, without a great deal of code rewrite? </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Feb 14, 2020 at 6:19 AM Alexey Bataev <<a href="mailto:a.bataev@outlook.com">a.bataev@outlook.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div>
    <p>Itaru, there are really a lot of problems with the codebase.</p>
    <p>1. Mapped classes use standard data types, like std::stream,
      std::string etc. NVPTX does not have implementation for these
      classes and all member function for these classes wont be resolved
      (they are part of libstdc++, NVPTX does not have implementation
      for it, I assume).</p>
    <p>2. Virtual member functions and RTTI. You cannot rely on this,
      especially on RTTI (dynamic_cast, for example, is not supported by
      NVPTX since, again, RTTI requires libstdc++).</p>
    <p>3. Some static data members are also marked as not resolved. They
      must be explicitly marked as declare target, if the data type is
      mapped.<br>
    </p>
    <pre cols="72">-------------
Best regards,
Alexey Bataev</pre>
    <div>13.02.2020 3:20 PM, Itaru Kitayama
      пишет:<br>
    </div>
    <blockquote type="cite">
      
      <div>
        <div dir="auto">Right </div>
      </div>
      <div dir="auto"><br>
      </div>
      <div dir="auto">Its under my GotHub account ikitayama</div>
      <div><br>
        <div class="gmail_quote">
          <div dir="ltr" class="gmail_attr">On Fri, Feb 14, 2020 at 5:19
            Alexey Bataev <<a href="mailto:a.bataev@outlook.com" target="_blank">a.bataev@outlook.com</a>> wrote:<br>
          </div>
          <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
            <div>
              <p>Hmm, I don't see this branch in the NEST repo<br>
              </p>
              <pre cols="72">-------------
Best regards,
Alexey Bataev</pre>
              <div>13.02.2020 3:16 PM, Itaru Kitayama пишет:<br>
              </div>
            </div>
            <div>
              <blockquote type="cite">
                <div>
                  <div dir="auto">Alexey,</div>
                </div>
                <div dir="auto">The branch I’m working on is </div>
                <div dir="auto"><br>
                </div>
                <div dir="auto"><span>clang-openmp-offloading</span><br>
                </div>
                <div dir="auto"><span><br>
                  </span></div>
                <div><br>
                  <div class="gmail_quote">
                    <div dir="ltr" class="gmail_attr">On Fri, Feb 14,
                      2020 at 4:56 Alexey Bataev <<a href="mailto:a.bataev@outlook.com" target="_blank">a.bataev@outlook.com</a>>
                      wrote:<br>
                    </div>
                    <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                      <div>
                        <p>Hi Itaru, tried to rebuild the latest NEST
                          with latest trunk clang compiler. Everything
                          linked correctly with the offloading flags.<br>
                        </p>
                        <pre cols="72">-------------
Best regards,
Alexey Bataev</pre>
                        <div>12.02.2020 11:55 PM, Itaru Kitayama пишет:<br>
                        </div>
                      </div>
                      <div>
                        <blockquote type="cite">
                          <div dir="ltr">Alexey,
                            <div><br>
                            </div>
                            <div>In my app, NEST, the code which creates
                              an object on the device does not build if
                              offloading flags are enabled but</div>
                            <div>does build without it. It will likely
                              be difficult to come up with a compact
                              reproducer, would it be ok for you to ask
                              to take</div>
                            <div>a look at the code I am working on? </div>
                          </div>
                          <br>
                          <div class="gmail_quote">
                            <div dir="ltr" class="gmail_attr">On Tue,
                              Feb 4, 2020 at 8:01 PM Alexey Bataev <<a href="mailto:a.bataev@outlook.com" target="_blank">a.bataev@outlook.com</a>>
                              wrote:<br>
                            </div>
                            <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                              <div dir="auto"> It must be fixed. But you
                                need to add #include <new> to your
                                program.<br>
                                <br>
                                <div dir="ltr">Best regards,
                                  <div>Alexey Bataev</div>
                                </div>
                                <div dir="ltr"><br>
                                  <blockquote type="cite">3 февр. 2020
                                    г., в 23:34, Itaru Kitayama <<a href="mailto:itaru.kitayama@gmail.com" target="_blank">itaru.kitayama@gmail.com</a>>
                                    написал(а):<br>
                                    <br>
                                  </blockquote>
                                </div>
                                <blockquote type="cite">
                                  <div dir="ltr">
                                    <div dir="ltr">Alexey,
                                      <div><br>
                                      </div>
                                      <div>Am still seeing a build time
                                        error like below:</div>
                                      <div>$ clang++ -g -fopenmp
                                        -fopenmp-targets=nvptx64 new.cpp<br>
                                        nvlink error   : Undefined
                                        reference to '_Znam' in
                                        '/tmp/new-a4b234.cubin'<br>
                                        clang-11: error: nvlink command
                                        failed with exit code 255 (use
                                        -v to see invocation)<br>
                                        $ cat new.cpp<br>
                                        int main() {<br>
                                        #pragma omp target parallel for<br>
                                        for (int i=0;i<10;i++) {<br>
                                        int *a = new int[100];<br>
                                        }<br>
                                        }<br>
                                      </div>
                                      <div><br>
                                      </div>
                                      <div>Are you still working on
                                        this? My Clang(-11) is the
                                        latest one built from Trunk.</div>
                                      <div><br>
                                      </div>
                                    </div>
                                    <br>
                                    <div class="gmail_quote">
                                      <div dir="ltr" class="gmail_attr">On
                                        Wed, Jan 22, 2020 at 4:34 AM
                                        Alexey Bataev <<a href="mailto:a.bataev@outlook.com" target="_blank">a.bataev@outlook.com</a>>
                                        wrote:<br>
                                      </div>
                                      <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                        <div>
                                          <p>Ok, it compiles with <a href="https://reviews.llvm.org/D73128" target="_blank">
                                              https://reviews.llvm.org/D73128</a>
                                            applied and additional
                                            include of stdlib.h. Still,
                                            cannot be executed, though,
                                            most probably beacuse of the
                                            virtual functions.<br>
                                          </p>
                                          <p><br>
                                          </p>
                                          <pre cols="72">-------------
Best regards,
Alexey Bataev</pre>
                                          <div>20.01.2020 6:21 PM, Itaru
                                            Kitayama пишет:<br>
                                          </div>
                                          <blockquote type="cite">
                                            <div dir="ltr">This
                                              reproducer attached, while
                                              it may have irrelevant
                                              code, shows the issue.
                                              <div><br>
                                              </div>
                                              <div>clang++ -g -fopenmp
                                                -fopenmp-targets=nvptx64
                                                my1.cpp</div>
                                              <div><br>
                                              </div>
                                            </div>
                                            <br>
                                            <div class="gmail_quote">
                                              <div dir="ltr" class="gmail_attr">On
                                                Fri, Jan 17, 2020 at
                                                3:16 PM Alexey Bataev
                                                <<a href="mailto:a.bataev@outlook.com" target="_blank">a.bataev@outlook.com</a>>
                                                wrote:<br>
                                              </div>
                                              <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                                                <div>
                                                  <p>No idea what does
                                                    it mean. What 
                                                    object? Where
                                                    locally? Provide
                                                    code and steps to
                                                    reproduce.<br>
                                                  </p>
                                                  <pre cols="72">-------------
Best regards,
Alexey Bataev</pre>
                                                  <div>16.01.2020 6:50
                                                    PM, Itaru Kitayama
                                                    via Openmp-dev
                                                    пишет:<br>
                                                  </div>
                                                  <blockquote type="cite">
                                                    <div dir="ltr">Hi,
                                                      <div><br>
                                                      </div>
                                                      <div>Locally
                                                        creating an
                                                        object under the
                                                        target region
                                                        causes nvlink</div>
                                                      <div>errors (same
                                                        code builds fine
                                                        without
                                                        offloading
                                                        support flags):</div>
                                                      <div><br>
                                                      </div>
                                                      <div> nvlink error
                                                          : Undefined
                                                        reference to
'_ZNK14AggregateDatumIN4nest12ConnectionIDEXadL_ZNS0_10NestModule14ConnectionTypeEEEE5printERSo'
                                                        in
                                                        '/tmp/event_delivery_manager-656e32.cubin'</div>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
'_ZNK14AggregateDatumIN4nest12ConnectionIDEXadL_ZNS0_10NestModule14ConnectionTypeEEEE6pprintERSo'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNK5Datum4infoERSo'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '__assert_fail' in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest12ConnectionIDC2ERKS0_'
                                                      in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_Znam' in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEaSEOS4_'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED1Ev'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
'_ZStlsIcSt11char_traitsIcESaIcEERSt13basic_ostreamIT_T0_ES7_RKNSt7__cxx1112basic_stringIS4_S5_T1_EE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest19WeightRecorderEventclEv'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN3sli4pool4growEm'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZSt20__throw_length_errorPKc'
                                                      in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZSt17__throw_bad_allocv'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_Znwm' in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      'strlen' in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN5TokenC1El' in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZdlPv' in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      'rand' in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN5TokenC1Ed' in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest19WeightRecorderEventC2Ev'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZSt18_Rb_tree_decrementPSt18_Rb_tree_node_base'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZSt29_Rb_tree_insert_and_rebalancebPSt18_Rb_tree_node_baseS0_RS_'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_Z8getValueIdET_RK5Token'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest12ConnectionIDC1Elllll'
                                                      in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_Z8ld_roundd' in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNK4nest11SourceTable7get_gidEijm'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '__dynamic_cast'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNK4nest12ConnectionIDeqERKS0_'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest17ConnectionManager17get_delay_checkerEv'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest12DelayChecker21assert_valid_delay_msEd'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC1Ev'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE7reserveEm'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4sizeEv'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE6appendEPKcm'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE6appendERKS4_'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4Name6insertERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZSt18_Rb_tree_incrementPSt18_Rb_tree_node_base'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest4Time5Range12STEPS_PER_MSE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest5names6targetE'
                                                      in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest5names6weightE'
                                                      in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest5names7size_ofE'
                                                      in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest5names5delayE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest4Time5Range11MS_PER_STEPE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest5names5rportE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZStL19piecewise_construct'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest13KernelManager24kernel_manager_instance_E'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN10Dictionary9VoidTokenE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZN4nest10NestModule14ConnectionTypeE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZTV5Datum' in
'/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
'_ZN14AggregateDatumIN4nest12ConnectionIDEXadL_ZNS0_10NestModule14ConnectionTypeEEEE6memoryE'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      nvlink error   :
                                                      Undefined
                                                      reference to
                                                      '_ZL14max_block_size'
                                                      in
                                                      '/tmp/event_delivery_manager-656e32.cubin'<br>
                                                      clang-11: error:
                                                      nvlink command
                                                      failed with exit
                                                      code 255 (use -v
                                                      to see invocation)<br>
                                                      make[2]: ***
                                                      [nestkernel/libnestkernel.so]
                                                      Error 255<br>
                                                      make[1]: ***
                                                      [nestkernel/CMakeFiles/nestkernel.dir/all]
                                                      Error 2<br>
                                                      make: *** [all]
                                                      Error 2
                                                      <div><br>
                                                      </div>
                                                      <div>What should I
                                                        do?</div>
                                                    </div>
                                                    <br>
                                                    <fieldset></fieldset>
                                                    <pre>_______________________________________________
Openmp-dev mailing list
<a href="mailto:Openmp-dev@lists.llvm.org" target="_blank">Openmp-dev@lists.llvm.org</a>
<a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/openmp-dev</a>
</pre>
                                                  </blockquote>
                                                </div>
                                              </blockquote>
                                            </div>
                                          </blockquote>
                                        </div>
                                      </blockquote>
                                    </div>
                                  </div>
                                </blockquote>
                              </div>
                            </blockquote>
                          </div>
                        </blockquote>
                      </div>
                    </blockquote>
                  </div>
                </div>
              </blockquote>
            </div>
          </blockquote>
        </div>
      </div>
    </blockquote>
  </div>

</blockquote></div>