<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/137579>137579</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [BOLT] Instrumentation of LLVM driver binary segfaults
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            BOLT
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          beakthoven
      </td>
    </tr>
</table>

<pre>
    I have compiled LLVM with the following configuration:
- **Build flag**: `-DLLVM_TOOL_LLVM_DRIVER_BUILD=ON`
- **Static Linking**: Everything statically linked except glibc

### Binary Details:
```bash
file ./install-min/bin/llvm
./install-min/bin/llvm: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 4.4.0, not stripped

ldd ./install-min/bin/llvm
        linux-vdso.so.1 (0x000077f0a44ea000)
 libm.so.6 => /usr/lib/libm.so.6 (0x000077f09e308000)
        libc.so.6 => /usr/lib/libc.so.6 (0x000077f09e116000)
        /lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x000077f0a44ec000)
```

### Problem:
When I attempt to run the BOLT-instrumented version of the LLVM driver binary, it **segfaults** upon execution. Additionally, the `file` command outputs an error related to the **NOTE section size** being too large.

### BOLT Command:
```bash
LD_PRELOAD=/usr/lib/libjemalloc.so ./sysroot/bin/llvm-bolt --instrument --instrumentation-file=/home/dakkshesh/tc-playground/bolt-profiles/llvm.fdata --instrumentation-file-append-pid ./install-min/bin/llvm -o ./install-min/bin/llvm.inst
```

### BOLT Log Output:
```bash
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: x86_64
BOLT-INFO: BOLT version: c2d6c7cea7d857f7a9268703e30a7d1ffed7d64b
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x6200000, offset 0x6200000
BOLT-INFO: enabling relocation mode
BOLT-INFO: forcing -jump-tables=move for instrumentation
BOLT-INFO: enabling lite mode
BOLT-WARNING: 14 collisions detected while hashing binary objects. Use -v=1 to see the list.
BOLT-INFO: 0 out of 138060 functions in the binary (0.0%) have non-empty execution profile
BOLT-INFO: validate-mem-refs updated 3 object references
BOLT-INSTRUMENTER: Number of indirect call site descriptors: 72728
BOLT-INSTRUMENTER: Number of indirect call target descriptors: 137150
BOLT-INSTRUMENTER: Number of function descriptors: 137140
BOLT-INSTRUMENTER: Number of branch counters: 1805192
BOLT-INSTRUMENTER: Number of ST leaf node counters: 874477
BOLT-INSTRUMENTER: Number of direct call counters: 2524
BOLT-INSTRUMENTER: Total number of counters: 2682193
BOLT-INSTRUMENTER: Total size of counters: 21457544 bytes (static alloc memory)
BOLT-INSTRUMENTER: Total size of string table emitted: 15507201 bytes in file
BOLT-INSTRUMENTER: Total size of descriptors: 179504596 bytes in file
BOLT-INSTRUMENTER: Profile will be saved to file /home/dakkshesh/tc-playground/bolt-profiles/llvm.fdata
BOLT-INFO: 867351 instructions were shortened
BOLT-INFO: removed 9544 empty blocks
BOLT-INFO: UCE removed 97245 blocks and 5895716 bytes of code
BOLT-INFO: padding code to 0xfa00000 to accommodate hot text
BOLT-INFO: output linked against instrumentation runtime library, lib entry point is 0x11e47860
BOLT-INFO: clear procedure is 0x11e43300
BOLT-INFO: setting __bolt_runtime_start to 0x11e47860
BOLT-INFO: setting __bolt_runtime_fini to 0x11e478a0
BOLT-INFO: setting __hot_start to 0x6400000
BOLT-INFO: setting __hot_end to 0xf808bf8
```

### BOLT Instrumented Binary Segfault:
```bash
./install-min/bin/llvm.inst --help
[1]    212952 segmentation fault (core dumped)  ./install-min/bin/llvm.inst --help
```

### Error from `file` Command:
```bash
file ./install-min/bin/llvm.inst
./install-min/bin/llvm.inst: ERROR: , dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 4.4.0 Note section size too big (195011832 > 134217728) (Invalid argument)
```

### Readelf output regarding note section
```bash
llvm-readelf -SW ./install-min/bin/llvm.inst | grep '\.note'
llvm-readelf: warning: './install-min/bin/llvm.inst': DT_SYMENT value of 0xf5c1140 is not the size of a symbol (0x18)
  [ 2] .note.ABI-tag NOTE            00000000000002fc 0002fc 000020 00   A  0   0  4
  [49] .note.bolt_info   NOTE            0000000000000000 1ef8c793 000124 00      0 0  1
```
</pre>
<img width="1" height="1" alt="" src="http://email.email.llvm.org/o/eJy0WF1P68rO_jXhxko1mXz2gouyCltI3bAFrL21r9Ak4zSz1iQTzUz4eH_9K08KlMKi6-joIARNYj92PPZju8I5tR0QT6P8LMrXJ2LynbGnNYqfvjMPOJzURj6fXkInHhAa049Ko4TN5u8_4VH5DnyH0BqtzaMattCYoVXbyQqvzBClq4itYoj4KuKrs0lpCa0W2_k6SlcQFSxeE9b93fX15j58Wt9c_n1-c3_2_XKzjtL19VVUsD2YWy-8amCjhp9q2IM6f0D77DtywgURofUzaDX8RAn41ODoYatV3USMvIp4Ov_CmRqEfYY1eqG0m30mk-G3Fq6L2KpVGmER8Qs1OC-0jns1RPyiDn-1fugjtvryMTm4uYAii2vlYXN7BqNCwCdsJi9qjRH_Bk9VERcZfXpA65QZIIGIV7f_3v4d8SXdl8-D6N-9Gd1Vg0c7WvRogeypmlAutIy1GqaneMZdOLPgJN4aC39cfY_4xYYeQ7bIFoweDMaD81aNI8o5SFrKY28Nu5_Z1IN0hgwFx9kTY4yVZctElqFgjNFrsBVoVfckVUCUrqP0nLyenJ19n_--PN9HWWLKqjeUV8N1cwSs-RQsSYoPYEfC95mJL4QPQ9C82nvNr8Nk_MuaWmM_Z-E_HQ5wCcJ77EcP3oCdhlBwZ9ebu5hOxU49Dh7la8qYNgiE-pRWPaCFOiR4yBS_KyKH21ZM2rv5EqbRDLtsVGZYwEpKRZ8o00iRIKOCURlEBSMa6MUgwUx-nLwDMQBaayxY1IK88WZWCehX13fn4LAhQHDq_3BntEYqVm8MaGG3uPhQmNebO_g2m_q8LDfr-79uzjfXKyKKj-f-A3uhtaHjD2nsnp01xr9L4bg22kO8F8x3F4HH4vDewUJneoz4hRQ_f7oOXRfxC9_EoxbPW2umQRK20T4erSElt7OyaKXw4hfIsRhHHGQ8qq-rDWLz5fMFPfgquUJAN2YL1-HcPo9pSK3Lq4tr4izXCYsSTP0DGw_GwmhcSIxYDRLJawrYG42BRI-ND_zxDuiOTtiDsE2nSGKySLefquK-yA6Fg5-7hKbrhsuiKRsUpazysi3FkhdVyVJMmShl0rYoS1lk9SFOq6zzEFIAhJQWnQPlgD2xQ8nGovCUjQM-wmjN1ooeOhQSLcwvJvwrBHsqOJV1IE3Ttg793r0DZBxErQnZojZNOHTojcQPvhrbkFj8Y-rHONh0UbruzQMGxj5InF-a0crjOwP_rG6uLq_-IKEkg8ZorSiu7vWo4LGj7tYJF5rnzBa7I3cL-O4Q4ocoXSdU1Q4xVLZWzi8OnWBECMRASVqxgkE7DaHoHaiZtnbYRIzUcvKIL-e5YjBDTBz3_EZCsCuhQyMPQispPMY99rHF1sE0ykA66UueWmzR4tCge1O-vbv5_uf51d35DYFcTX2NllxVg1SWlKirgqPoSXSNVaM3lqYBKHnJq_8UyM_5fgCVpGWSs-NYL4H7TD_7Df3aiqHpoDETjQazasXyZMmP697egUbRwmAkvkOoyiwry-MA-2HY1-c5z36lfWe80DC8YrzTKyqeLNOvVamtfFBMsrzMswzqZ4-Osm4eDHeM0GNvqC0ufw-ZJiNqV4ENsFeeWI4Cm-es5CzZWVEDvM_arzAPj7dc5izLl8XvYf01Fwg8Kq2hRnDiYW694e5_26wO664qyjRPdjy0K-tHtAiuM9bj8JHzLRJ7SVjSGczVXWvT_HSHgt-_nb8JlzzLd3JAM0ZeLfMyeQlJOOKP7DkKKef1QyJFgD21ItAxXYiG5hVDJAGd8eDxyR8CzJPMy7ogtoLe85B0afzyqif2q-1upNKqBhy8fYbRqMHP_SVJMCur4mOb0SgsEVuDcrL4JpymHzuHQx9a0v09HdL9zva988L6-R1_ZeYXmq0a1L6i-EKxM37fUpF92tzey-Mgd7GvWFW31dFR5HJ_gN2tYbe7yfTz4eTo-ANx3KEeSTU_S6J8TVM9T_gy5-Bw-3aUwQhxQmMsgpx6Wnr4Eo5PWHsmfvV252EYbq3p96fmL0fZoxvmy3R3XIZ2zZub68AR_9utEa6Mx3ejfRjna7WlyCbLnCVJldLadA5JmvGkpFbKl_T0cgidHITdhhQ4shjd0DCm25dCtbgVNpT8sOfCJ3ENM77dKce3__zG-UblN9haHCHiZZR_W5AF-vgejKL7KOyghu0c6PL4yfCSRNd397f_Eo3TLDOFRsCe2rxJkowRJdAeTtPSS5cQ4J772uh5oUyql5U1ys-AU4oHDxers8vYiy2EdWvvh-3_8LaBt3-MM2AMAFYA9I8BZC_Q2fINOtCIGloD8DU8MW6CbdWUy5SeJDybDQRwBpDsH9GJPE3lMl2KEzxNyiyvkpJn1Ul3WmSZFJnAoslE3mZlxQRv8yLLpeQyFXiiTjnjOcs4TTUZqxZymYimbSteZrnkDY8yhr1QehGCb-z2RDk34WmSlnm5PNGiRu3Cd16cExdFnCJ5Yk_nnXDauihjNOS6NwSvvA7fkwWFfL3HXzOnmPaTvRted-2TyerTzvsxfMnELyJ-sVW-m-pFY_qXb1R2W-loDY2ylFHkN7XmnesPp_z_AwAA__92kRdP">