<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/69318>69318</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [OMPT] Runtime can segfault during at `ompt_start_tool` when offloading and calling `std::` functions
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            new issue
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          Thyre
      </td>
    </tr>
</table>

<pre>
    ## Description

This issue is a bit hard for me to understand, but I'll try to describe it as best as I can. 

While investigating how our HIP & OMPT adapters interact when one uses OpenMP target code, I ran into the issue that the program would just end in a segmentation fault out of nowhere. Upon further investigation, I opened two issues for both `rocm_smi_lib` and `HIP`, which both showed unexpected behavior when functions of their interface were being called during `ompt_start_tool`.

Here are the corresponding issues:
- https://github.com/ROCm-Developer-Tools/HIP/issues/3330
- https://github.com/RadeonOpenCompute/rocm_smi_lib/issues/129

Upon further investigation, this seems to affect LLVM in general as well. Just using `std::cout` will cause the program to crash if called during `ompt_start_tool` or the initialization function. Trying to do the same with offloading disabled, everything works as expected.

In the HIP issue, I took a look at the order of the libraries being loaded. Here, I noticed differences when the OMPT interface is initialized. For host-only OpenMP, OMPT seems to be initialized before the first OpenMP directive / function. For host & target OpenMP, the initialization seems to occur upon loading the shared libraries, which is before entering `main`. _Maybe_ this has something to do with the issue, but I'm just guessing.

## Reproducer

One can use this small code to reproduce the issue

```cpp
#include <omp-tools.h>
#include <iostream>

int ompt_initialize(ompt_function_lookup_t lookup, int initial_device_num, ompt_data_t *tool_data) {
    return 1; //success
}

void ompt_finalize(ompt_data_t *tool_data) {}

#ifdef __cplusplus
extern "C" {
#endif
ompt_start_tool_result_t *ompt_start_tool(unsigned int omp_version, const char *runtime_version) {
    static ompt_start_tool_result_t ompt_start_tool_result = {&ompt_initialize, &ompt_finalize, 0};

    std::cout << "Hello World!" << std::endl;

    return &ompt_start_tool_result;
}
#ifdef __cplusplus
}
#endif

int main(void)
{
    #pragma omp parallel
    {}
    return 0;
}
```

Compiling with a recent trunk version of LLVM, we can see the following behavior:

```console
$ clang++ --version
clang version 18.0.0 (https://github.com/llvm/llvm-project.git e483673246bdee06e54ec06fd04236bc9fee7f63)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/software/software/LLVM/git/bin
$ clang++ -fopenmp --offload-arch=sm_70 -gdwarf-4 reproducer.cpp
$ gdb ./a.out 
[...]
(gdb) run
Starting program: /home/jreuter/tmp/Error/ompt_cpp_init/a.out 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7d3bf8a in std::ostream::sentry::sentry(std::ostream&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
(gdb) bt
#0  0x00007ffff7d3bf8a in std::ostream::sentry::sentry(std::ostream&) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#1  0x00007ffff7d3ca0c in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#2  0x00007ffff7d3cebb in std::basic_ostream<char, std::char_traits<char> >& std::operator<< <std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*) () from /lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x0000555555555252 in ompt_start_tool (omp_version=201611, 
 runtime_version=0x7ffff7bd2c66 <__kmp_version_lib_ver+6> "LLVM OMP version: 5.0.20140926") at reproducer.cpp:16
#4  0x00007ffff7bc3e13 in ompt_pre_init () from /opt/software/software/LLVM/git/lib/libomp.so
#5 0x00007ffff7b3cb8d in __kmp_do_serial_initialize() () from /opt/software/software/LLVM/git/lib/libomp.so
#6  0x00007ffff7b47c8c in __kmp_serial_initialize () from /opt/software/software/LLVM/git/lib/libomp.so
#7  0x00007ffff7bc4a1f in ompt_libomp_connect () from /opt/software/software/LLVM/git/lib/libomp.so
#8  0x00007ffff7e9aeb1 in llvm::omp::target::ompt::connectLibrary() () from /opt/software/software/LLVM/git/lib/libomptarget.so.18git
#9 0x00007ffff7e9bc95 in init() () from /opt/software/software/LLVM/git/lib/libomptarget.so.18git
#10 0x00007ffff7fc947e in call_init (l=<optimized out>, argc=argc@entry=1, argv=argv@entry=0x7fffffffd2c8, env=env@entry=0x7fffffffd2d8)
    at ./elf/dl-init.c:70
#11 0x00007ffff7fc9568 in call_init (env=0x7fffffffd2d8, argv=0x7fffffffd2c8, argc=1, l=<optimized out>) at ./elf/dl-init.c:33
#12 _dl_init (main_map=0x7ffff7ffe2e0, argc=1, argv=0x7fffffffd2c8, env=0x7fffffffd2d8) at ./elf/dl-init.c:117
#13 0x00007ffff7fe32ca in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#14 0x0000000000000001 in ?? ()
#15 0x00007fffffffd70f in ?? ()
#16 0x0000000000000000 in ?? ()
```

Without offloading enabled, everything works:

```console
$ clang++ -fopenmp -gdwarf-4 reproducer.cpp
$ ./a.out
Hello World!
```
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzMWVtz27by_zTwy440FChR0oMfbCtq8p9mkmnS9lEDAksRDQlwANCy_5_-zAIUdbGdtuekc44nES8Adn97X4DCe703iLdscc8WmxvRh9q626_1s8Ob0qrnW8ZzxnPYoJdOd0Fbw7INy-7S79dae9De9wjag4BSB6iFU1BZBy1CsNAbhc4HYRTjD1D2AT4wvmwaCO6ZxlWkXCLoAMJDiT5eP4AUZgrnzH6vdYOgzSP6oPciaLOH2h7A9g7ef_gMjBfw6ePnryCU6AI6D9oEdEIGONRowBqE3qOHTx2aj58hCLfHANIqJGgfwAlDSyyEGgepQi1CfOyc3TvRwsH2jYI_eh8AjQJtQIDHfYsmCNIOVKJvAtg-gK3A2EONDqfwa0dDvQs1unMRrEmsbYcGFYSDTYx91GBpQw2syJyV7c63etfokhUZCKPo9fsPn1mREYFDrWWdpvvaHlBBb_CpQxlQQYm1eNTWJS1UvZHE1xO-UKN2SU2VkAgHdAglkmalaBpUoHpHT6zIbNuFnQ_ChV2wtmFFNj23zntaKhxGbUnrHPrOGkWLk0QsH2ZOoA6hi898y_h2r0Pdl1NpW8a3v3x6aCcbfMTGdugmX61tPONbEpVvB0J8m-d59leICYXWkLkfbNv1ARnfXujyjOSMr8_F-b7BAvm9R2w9ubCoKpQBfv75t4_kEHs06ERDXnzAppnC_5G39H7Qow-K0OZ30vaBrHnQTQNS9B4vPC1YkE74GnT1V4wB1iW_NTpo0ej_H9xxMPcUvrpnWkwhlzzcixbhoEMNtqoaK6KtlPaibDAGKz6iew41vT5Y982TREevurD9BxMJUhBGfSafDtZ-AwFNvKQosk6hGxwPGl064TT6weMIAqopkCclCsYGLUluXVXo0Ej0yYlpeQz1k-tSIjqKTlS21kFtfZhY0zwPIU9U47LRdCWer4ISKzu4cKWdD8dUobRDGfQjAuPbM50emcTcM-STE6tXzDEytlL2DnrysqPqo01q4VCdVHMKbu2P6JBkHvygFdpQJMLuo3gucZc8sxYevG0x2S5ZPBp6TGznubhN6WzfoycfvbDskP5_wc5Z1Ut054OfDFKWhuS5FBGtIFe2KmZ-d1x0xvacdJGlf7LrRmbayKZXCCyniJ2Qb_tpzfJ3r83Q1geHoj0Nx19tAsT4OFmW8VV8c7Tcjpyy73YB0g2pg5YNK3YKH7XEnelbGolLlQhiR4a-I1DxkfE1sOV94goA4DD0zsCM5feQ0pHvpUTvB3jLzTnOR6tVol1pcwHzO7wuSZA-KoUV7Haya3pP_9MIPgV0BhjnD4zzE0zGczRKV-npKo3sHPq-CYn1dYrhq97EbkHBoOHdIzo_ZERpjQ8ga-ForetN0C2eJlwpylOxlPAm-9cHgOWbSIcXL-z7AMe3J2U-QEb6yu_PVZbYn6Vg8iSW03r-HpvGwu_WNYrxWdRbGhsXoFHNKwQHwx8hvEB-WjLa703LnU85M9XJt2PM8xW5DxuL1rl2Gc87J_atID1CJxxVj-Zs_MyPztBnr6A8xug5BopM3cSqQDlFgEOJJkBwvfkGg8kpyVNBjAkspQmPQ2a1TWMPtP7YmYytwXVmsMbb5pg1-BxkI8ye8XvG72EyOXpXHI5DI_fZappNM2B89Z32oGkej5dJ5-wfKMN0rwPgfJUXy5zPi1IhZgUu5iizolLZnOdFKdcV4rIq8lH7X2PmZ_kdPK2KXTGf9OabsQczabTpnyZ70x_bZYdCQWsVNjS7s14_HUuoD7HIbzSpg9KH7QJlEFuFg6CKeH6bVEviML4ttXlDRRX1lW0Hk8lQ4ifCyZrlG9_ulhlM9uogXDWZn3K1m56l4znsVQlTxrdiGiMlvV_cT6dTthjddLVXJYW46wccX8j_ycBDLzMIVNuWsP_hsA_oGN-GtmN8-845S08xdGTXxbh-yXNQnsKy3--JeOqoGl2GOLJTJaBJzcsR268-bRJ8uJyX6uszxTw5QOwEB9OdTBYHxjVTb6eUEi7K4-ehV6MI0I-ogBKkaODLh5--vPvpN3L-Ly82BwOF7CnLsmxZVVW1VHlZrQQ1j2OmGWsbPXg0wT1f3PPVi6m8iImWr-hSOdvCnwjng5LJVUi64tqeZRgTUQbwv4s3n12jkyKTF-hK4bXcjRgfqFCRdU6VoBZuF5zQwR-H83dAnQUvTrN2RxI7bTy68DcprX4MnlhvqdLGosv4Hb1oLIX9D1Enf6FOLMt_Qp20xROB8n-qwPnDf1eLP0R7-aC8xfGPLzgp76ozgNTsjS1SvuHZrJjNYiuTSvN1F5VvsqdkklJxWRSksN3u24kI7WnpnvH7IgrKedyUfvr4GUYid7CYZlOezebZmrTASWARrktAfjc7yTS_9IhS5jjLR6E6hzFnX2vvLxewpOZGl7btpt6ObBeXXHNZruKRSxJa2Z1HR-36RaP_ihn_YyDFlfzzpVzJE5IXMH44gOW1AeZiVo0GSPN30hqD8sebYXXJHNcCyxkxjw1UiuS2Szfh2Aqll-HYZ0dkP6ey-wONlNjF2ryi0SPk9RXiUq4XhDi1Fv84-1l2wb-S6_kSib8UTTPGSsPyDW10u6DbePxg-xBz0wMIt5cs38TLPBuq6WY2DD2mocezoSEzVFWluFzF0xtD0-j39VlqNfavtAkQITZ62FSMb1UzIZBTyfK7ZXYSa3Yt1qJYvRArMb5mNQJ_iXQQNkr3pk7WbyPM8xNCDjt1gkJbpV0rurPUWVXIMbtm-ya214X5DpjZbHlCk1_qC3MuY9dEGFMp6D26V6pOMaerGmrO06qYFHNyM36iPR9on_3FoGT5luXbgeZp-kUuJUGWWfWd6cVL6tlb01_bJP6uQ50OwcejxWNz_srJ4r-3ARx3N3-6jRm3MMez6ot9_iXPG3Wbq3W-Fjd4OyvWy0W-Xhb5TX2Laz5byUwovlDrhcJqXc7XOF-o9WpWFby60bc8o9ifLWfZfM3zqSxXYlFyUeCSlyjnFImt0M2UEufUuv1NPBW7Ldb5bHXTiBIbHz_FcG7wcDyp42yxuXG3cZta9nvP5lmjffAnKkGHJn7D-fTx81e22MAvqXMYtt379E1iOD0W4Y0D5PSR5GQvYVQM7Otza5o7fkW46V1z-_f22Oen7lHyfwUAAP__ZTBTIQ">