<table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Issue</th>
<td>
<a href=https://github.com/llvm/llvm-project/issues/93151>93151</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>
BOLT instrumentation fails on aarch64 both on linux_intel-x86 and MacOs_M2(compatible?)
</td>
</tr>
<tr>
<th>Labels</th>
<td>
BOLT
</td>
</tr>
<tr>
<th>Assignees</th>
<td>
</td>
</tr>
<tr>
<th>Reporter</th>
<td>
cirt-os
</td>
</tr>
</table>
<pre>
I have made attempts to perform bolt optimization on an aarch64 binary，but come accross with such many problems.
1. First I did bolt on on linux_intel-x86, and failed
```
$ ./build/bin/llvm-bolt test2 -instrument -o t2_instr
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: <unknown>
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x200000, offset 0x200000
BOLT-WARNING: debug info will be stripped from the binary. Use -update-debug-sections to keep it.
BOLT-INFO: enabling relocation mode
BOLT-INFO: forcing -jump-tables=move for instrumentation
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
./build/bin/llvm-bolt: 'test2': No relocation for first DT_FINI_ARRAY slot.
```
And the error comes from:
```
Error RewriteInstance::discoverRtFiniAddress() {
...
if (const Relocation *Reloc = FiniArraySection->getDynamicRelocationAt(0)) {
BC->FiniFunctionAddress = Reloc->Addend;
return Error::success();
}
if (const Relocation *Reloc = FiniArraySection->getRelocationAt(0)) {
BC->FiniFunctionAddress = Reloc->Value;
return Error::success();
}
return createStringError(std::errc::not_supported,
"No relocation for first DT_FINI_ARRAY slot");
}
```
I checked that the Offset of fini_function in this binary did not start with 0x0 but with 0x8. Interesting, but why would threre be a deviation of offset calculation.
```
DynamicRelocations: Type size:213 ,Type:1027, Offset:0x8, Addend:0x1460, Value:0x0
DynamicRelocations: Type size:213 ,Type:1027, Offset:0x10, Addend:0x1454, Value:0x0
```
2. Then I changed the offset manually and recompiled llvm-bolt，And failed again:
```
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: <unknown>
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x200000, offset 0x200000
BOLT-WARNING: debug info will be stripped from the binary. Use -update-debug-sections to keep it.
BOLT-INFO: enabling relocation mode
BOLT-INFO: forcing -jump-tables=move for instrumentation
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 0 out of 9 functions in the binary (0.0%) have non-empty execution profile
BOLT-INSTRUMENTER: Number of indirect call site descriptors: 5
BOLT-INSTRUMENTER: Number of indirect call target descriptors: 8
BOLT-INSTRUMENTER: Number of function descriptors: 8
BOLT-INSTRUMENTER: Number of branch counters: 90
BOLT-INSTRUMENTER: Number of ST leaf node counters: 0
BOLT-INSTRUMENTER: Number of direct call counters: 40
BOLT-INSTRUMENTER: Total number of counters: 130
BOLT-INSTRUMENTER: Total size of counters: 1040 bytes (static alloc memory)
BOLT-INSTRUMENTER: Total size of string table emitted: 922 bytes in file
BOLT-INSTRUMENTER: Total size of descriptors: 4224 bytes in file
BOLT-INSTRUMENTER: Profile will be saved to file t1.fdata
BOLT-INFO: Starting stub-insertion pass
BOLT-INFO: Inserted 0 stubs in the hot area and 0 stubs in the cold area. Shared 0 times, iterated 1 times.
BOLT-INFO: padding code to 0x600000 to accommodate hot text
BOLT-ERROR: linking object with arch x86_64 into context with arch aarch64
```
Bolt cross-compiled on x86 machine seems not supported. The library `libbolt_rt_instr.a` seems to be generated dynamicly while performing
> cmake -G Ninja ../llvm-project/llvm -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="bolt"
It seems that, `libbolt_rt_instr.a` was built on x86, and can't be linked with aarch64 binary, too bad.
3. At last, I used a Mac with M2 chip based on ARM, hoping that it would be compatible with aarch64, and then had a try on MacOs, but unfortunately it turned out to be like this:
```
BOLT-INFO: shared object or position-independent executable detected
BOLT-INFO: Target architecture: aarch64
BOLT-INFO: BOLT version: 058e4454e8965f0d96ecbec61395d5a788c37f7c
BOLT-INFO: first alloc address is 0x0
BOLT-INFO: creating new program header table at address 0x200000, offset 0x200000
BOLT-WARNING: debug info will be stripped from the binary. Use -update-debug-sections to keep it.
BOLT-INFO: enabling relocation mode
BOLT-INFO: forcing -jump-tables=move for instrumentation
BOLT-INFO: disabling -align-macro-fusion on non-x86 platform
BOLT-INFO: number of removed linker-inserted veneers: 0
BOLT-INFO: 0 out of 9 functions in the binary (0.0%) have non-empty execution profile
BOLT-INSTRUMENTER: Number of indirect call site descriptors: 5
BOLT-INSTRUMENTER: Number of indirect call target descriptors: 8
BOLT-INSTRUMENTER: Number of function descriptors: 8
BOLT-INSTRUMENTER: Number of branch counters: 30
BOLT-INSTRUMENTER: Number of ST leaf node counters: 18
BOLT-INSTRUMENTER: Number of direct call counters: 0
BOLT-INSTRUMENTER: Total number of counters: 48
BOLT-INSTRUMENTER: Total size of counters: 384 bytes (static alloc memory)
BOLT-INSTRUMENTER: Total size of string table emitted: 922 bytes in file
BOLT-INSTRUMENTER: Total size of descriptors: 4368 bytes in file
BOLT-INSTRUMENTER: Profile will be saved to file /tmp/prof.fdata
BOLT-INFO: Starting stub-insertion pass
BOLT-INFO: Inserted 0 stubs in the hot area and 0 stubs in the cold area. Shared 0 times, iterated 1 times.
BOLT-INFO: padding code to 0x600000 to accommodate hot text
BOLT-ERROR: library not found: ./build/lib/libbolt_rt_instr.a
```
So confusing，`libbolt_rt_instr.a` was not generated. Moreover, the directory `build/tools/bolt/bolt_rt-bins` did not exist as well.
**Over all, are these known problem? Did i miss something, or are there any way to fix them. Thanks.**
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzsWU1v4zgS_TXMpWBBpvx5yEGJ7YF3O8nA8czunAyKKlmcSKRAUok9v35RlJyvdvcEvX0ZIEGQWBLrFatYfPVEC-fUXiNesvEVGy8uROtLYy-lsn5g3EVm8uPlGkrxiFCLHEF4j3XjHXgDDdrC2BoyU3kwjVe1-kt4ZTQYDUKDEFaWkxFkSgt7ZMsVu7pms-us9SBNjSCktMY5eFK-BNfKEmqhj9BYk1VYu4jFCxan3d9hBCtlnYc15CrvXQZHldLtYae0x2pwmE0YvwahcyiEqjCHHmMS97_dJR9BxPgqa1WV03-lGV9V1WM9CMAenecwUNp529aoPQwMeL4LNzqIq7sv28H6dnXHkhRcKSzmYLI_UXowFhrjFCVioHSODeqcMPCAsvUiqxBy9Cg95mewtsLu0QOlTtGg1iLd7nN5xoAu4BGtU0bTNUuuW_2gzZNmyfLM-CKkUVSVkSDy3KJzoBzEh_jMYGlReKX3oPGJFmZvRQ0lihwtdKEI_4wSH3hMP7QEpigc-pdbL9D_STe369tfCD3HrN2D0oWBJ1VVkCE4b1XTYA6FNTX4EvvqieA3hzBom1x4HATDgUNJSQ61-IDYgPLRmRhQi6yiGCxWRnYFWpscz-XGWEkjB3-2dTMIATqWLGrziPQMXioiwJxByJXrvQ1EpfZ6UAtpzaBoXb8vtNFUptBUwtPu6TC-U41hTfk01CTjU7q8Na9joYl1i7rY7lbr2_Uu3WzSP8BV5pSPd_Wf6jykFq01NuxFF_LNkvTs-GUYt8EnqzyutfNCS6pKlqS5ctI8ot34ldIq7SqB8Rnjc2DTq84eIIqi00dVAOMzabTzsHmJgvE0XAFLFhCwrBXH-26JByxZ7tEvjlrUSr5YpZ7xWcz4_J07gKtrsiGcVasDRj-3gB8QaECa56hzlryytOhbqyHE3MXoWimfo3o1lk0Xrynq_wrtJ8b0u6ha_Fkh9aaBB_DeW6X3HQyfOZ93YGit7D5p43eubRpjidz49csUvvHDOP94MTPOX8_2Za5vq3UNskT5gFTjwodCv-vYyBRQKK12RZ8-UBp8qVzPMaGzaOPBeWF915biQwzUsPqLWQRr7dGiI1YkogsPyyM8mbYijxYtEpEJyPFR9e2wOPGhFJVsq3D3_Nb8qsRdaArHBsGpv2jT8WECjF_TLZakw5hPaRpdhCxJ48OMrk-FncaH4WgSKLmvi_SZ6X-Cr2H8lbPx6Kyz5zj7lswj2JaogVZL6D12jNSnqRa6FVV1DJ3cojR1E7r5CymexET60urFXij9LQr77Nef_fon9us3GLqtM7S0yy2S65w06QNaEpBITAiPqBFt2F7nVi0G0wZ2msOJmlzHTad0Ul-Jo5jxMXWFIMdpYiTFj32h0nwbawpVvc3U_Xbz283ydrvcBOnwPFelc2Wp_qWoKnDKU5k7aVXjTTfV8Q_g-G47vEOafQjpmZZ_yDqzQssSpGmJoIPhPP6Q5f0WKhQFaJPjG_uPmb8O_7X16HvmW-NF9apyXhsOk7-3JIL-yi4exZAdPToI7Vl4JXviqLE29kjd86PALvT6njSwVp7oj3LKee9DafibYnuL-G5VR5yPPoz0a1fYL7QjaJt5E-zAD6MiF16c2Vr31MopEOfbrN-QYacI584MX592bBwMnndhaYjiUYSG9O6ZNFUeHkZw3zWWGLyq0RGvKo9WEOCwu3eO8BqR5zRFSfXnDcSHSSBh-iykNHVtiEXDJDwe_CuI5WZzFxJEjEMYfU8LcoVaEBxmk91kBEp7A9Josn_19E2bet8y6VU4vKEPnhuw0QQItZCl0ggOsXadZDqpvtDWoVKZDbQ1iSuVUcfeWd-9P0eCREBn6Q0t5h51n6S8UyTVEZ5KWtj-hIGkVjfDZAmyFg8Ig1_gVuk_BUTR6WWpsYZi7y9hsPjy5feb3Tbd_LLc3u-2d7ur39ZfFixZMM7_O5uw5CpNu-g5h8Hi-ib997Ibs9v-8euSJYsNViiojXVQy9v06styl97fLzfb9d3tPUsWd7fvnv66ufvX8np73_kJWoVzgNeqeu1P4ZfCU5F8M0tPwgG9Fvo-8afDDSk0vRJS8kKryfslfXvgwq_BGwOZyN8cpSQRQOqhEi44X0PrSD3BjZAdzA0HWaoGMuG6JU83NzSyNE2gBFLVyveKN6MNUDfCqyzsz5dpnCbrSeaVglx4eyS8GyHv3Ek8t7ow1rdaeKyOhEsvHOS39X19VOoBg07_54m7eDzD0Wg8wtl8Mi7ifD5BmaGcDJP5OB-L6Wwmk2kxlZ-671P3feq-n6X7vqufPqD7hh_z_C3h96O6b_Q9t9-Wfcls9A9Sfclk9pNUH-MrXzeMr6j6PwVgr7hIjBWmDccxb77kqFTW_X2vM84ez9wHtUh0qPfPhy3f1Snk-FnJRXBjLJpHtEGFlNhvFtNpwtOcvDGVozkGmbTqkQeZ0o5gnw_k8KCoFzp4wqqK3ogpxun37hEtFX3QHJbUAjqEcKpy-j6JJStYqBwU1Mo5cKZGX_aHeMaerCyC0Ed4Eseuzg50syZRK_SDizpvF_llks-TubjAy-F0OBmNk9FwflFeilGRTIukmI7mEy6LeD6fFbN5MZ_KAuec44W65DEfxWOexLNkMp5HYjofYiFmMsbxJC_GbBRjLVQVkYaNjN1fKOdavJwnw_HwohIZVi58V8c5rT_jnI0XF_ayOxlr946N4ko5714AvPIVXgZl8q4vhpMzF76tOylH48szX6qFqg-qbXfDwxH3SfGxZMX4_KK11WXpfRMkGl8xvtorX7ZZJE3dC_KvZXqIjJa_C-7xkv8vAAD__zKvFow">