[Openmp-commits] [PATCH] D131089: [Libomptarget] Explicitly init / deinit libomptarget from the user

[Joseph Huber via Phabricator via Openmp-commits]

jhuber6 added a comment.

In D131089#3699902 <https://reviews.llvm.org/D131089#3699902>, @JonChesterfield wrote:

> What sort of trace do you get out of the ad hoc print statements, with the __tgt_register_requires calls still present? Ideally have the thread id in the print statement and optionally more than two threads. If something is nicely linearising them then great, but only if we can work out what that is and can rely on it being around on other systems. Maybe dlopen does have some internal locking after all.

here's my test program

  #include <dlfcn.h>
  #include <omp.h>
  #include <stdio.h>
  
  int foo;
  int bar;
  
  int main() {
  #pragma omp parallel
    {
      if (omp_get_thread_num() == 0) {
        void *h = dlopen("./libfoo.so", RTLD_NOW);
        int (*ptr)() = dlsym(h, "foo");
        foo = ptr();
      } else if (omp_get_thread_num() == 1) {
        void *h = dlopen("./libbar.so", RTLD_NOW);
        int (*ptr)() = dlsym(h, "bar");
        bar = ptr();
      }
    }
  
    printf("%d %d\n", foo, bar);
  }

With libfoo and libbar being

  #include <omp.h>
  #include <stdio.h>
  
  int foo() {
    printf("foo %p\n", &foo);
    int isDevice = 0;
  #pragma omp target map(from : isDevice)
    { isDevice = omp_is_initial_device(); }
  
    return isDevice;
  }

This outputs the following debug output (Without a mutex and with the sleep).

  Libomptarget --> RegisterLib
  Libomptarget --> Init target library!
  Libomptarget --> Loading RTLs...
  Libomptarget --> Loading library 'libomptarget.rtl.ppc64.so'...
  Libomptarget --> Unable to load library 'libomptarget.rtl.ppc64.so': libomptarget.rtl.ppc64.so: cannot open shared object file: No such file or directory!
  Libomptarget --> Loading library 'libomptarget.rtl.x86_64.so'...
  Libomptarget --> Successfully loaded library 'libomptarget.rtl.x86_64.so'!
  Libomptarget --> Registering RTL libomptarget.rtl.x86_64.so supporting 4 devices!
  Libomptarget --> Loading library 'libomptarget.rtl.cuda.so'...
  Target CUDA RTL --> Start initializing CUDA
  Target CUDA RTL --> There are no devices supporting CUDA.
  Libomptarget --> Successfully loaded library 'libomptarget.rtl.cuda.so'!
  Libomptarget --> No devices supported in this RTL
  Libomptarget --> Loading library 'libomptarget.rtl.aarch64.so'...
  Libomptarget --> Unable to load library 'libomptarget.rtl.aarch64.so': libomptarget.rtl.aarch64.so: cannot open shared object file: No such file or directory!
  Libomptarget --> Loading library 'libomptarget.rtl.ve.so'...
  Libomptarget --> Unable to load library 'libomptarget.rtl.ve.so': libomptarget.rtl.ve.so: cannot open shared object file: No such file or directory!
  Libomptarget --> Loading library 'libomptarget.rtl.amdgpu.so'...
  Libomptarget --> Successfully loaded library 'libomptarget.rtl.amdgpu.so'!
  Target AMDGPU RTL --> Start initializing AMDGPU
  Target AMDGPU RTL --> There are 1 devices supporting HSA.
  Target AMDGPU RTL --> Alloc allowed in memory pool check failed: HSA_STATUS_ERROR: A generic error has occurred.
  Target AMDGPU RTL --> Device 0: Initial groupsPerDevice 128 & threadsPerGroup 256
  Libomptarget --> Registering RTL libomptarget.rtl.amdgpu.so supporting 1 devices!
  Libomptarget --> Loading library 'libomptarget.rtl.rpc.so'...
  Libomptarget --> Unable to load library 'libomptarget.rtl.rpc.so': libomptarget.rtl.rpc.so: cannot open shared object file: No such file or directory!
  Libomptarget --> RTLs loaded!
  Libomptarget --> Image 0x00007fb2981190d8 is compatible with RTL libomptarget.rtl.x86_64.so!
  Libomptarget --> RTL 0x000055f4bf749e80 has index 0!
  Libomptarget --> Registering image 0x00007fb2981190d8 with RTL libomptarget.rtl.x86_64.so!
  Libomptarget --> Done registering entries!
  foo 0x7fb298118180
  Libomptarget --> Entering target region with entry point 0x00007fb29811901f and device Id -1
  Libomptarget --> Call to omp_get_num_devices returning 4
  Libomptarget --> Default TARGET OFFLOAD policy is now mandatory (devices were found)
  Libomptarget --> Use default device id 0
  Libomptarget --> Call to omp_get_num_devices returning 4
  Libomptarget --> Call to omp_get_num_devices returning 4
  Libomptarget --> Call to omp_get_initial_device returning 4
  Libomptarget --> Checking whether device 0 is ready.
  Libomptarget --> Is the device 0 (local ID 0) initialized? 0
  Libomptarget --> Device 0 is ready to use.
  TARGET x86_64 RTL --> Dev 0: load binary from 0x00007fb2981190d8 image
  TARGET x86_64 RTL --> Expecting to have 1 entries defined.
  TARGET x86_64 RTL --> Offset of entries section is (0x0000000000004000).
  Libomptarget --> RegisterLib
  Libomptarget --> Image 0x00007fb2981190d8 is compatible with RTL libomptarget.rtl.x86_64.so!

After which point it hangs indefinitely.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D131089/new/

https://reviews.llvm.org/D131089