<table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Issue</th>
<td>
<a href=https://github.com/llvm/llvm-project/issues/84743>84743</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>
[RISCV][LLD] relaxTlsLe doesn't account for (p_vaddr % p_align) adjustment
</td>
</tr>
<tr>
<th>Labels</th>
<td>
lld
</td>
</tr>
<tr>
<th>Assignees</th>
<td>
</td>
</tr>
<tr>
<th>Reporter</th>
<td>
rprichard
</td>
</tr>
</table>
<pre>
For RISC-V, getTlsTpOffset adds the PT_TLS segment's (p_vaddr % p_align) to the symbol VA: https://github.com/llvm/llvm-project/blob/0f501c30b9601627c236f9abca8a3befba5dc161/lld/ELF/InputSection.cpp#L653-L654
I _think_ this makes sense, because when (p_vaddr % p_align) is non-zero, it would be inappropriate for the TP to point directly at the executable's segment, which would generally prevent libc from aligning TP for the TCB. Instead, the TP would be a multiple of p_align, and there would be an alignment gap of (p_vaddr % p_align) bytes between the TP and the start of the allocated segment. I'm not 100% sure this is the correct behavior, but a simple experiment seemed to confirm that glibc added this padding.
However, relaxTlsLe (and other callers?) use getVA directly rather than call getTlsTpOffset, and this results in broken relocation processing.
It's rare to hit this situation, because ordinarily LLD ensures that (p_vaddr % p_align) is zero. However, I can hit it if a variable has alignment greater than max-page-size. Obviously, it can happen easily if this code is commented out: https://github.com/llvm/llvm-project/blob/0f501c30b9601627c236f9abca8a3befba5dc161/lld/ELF/Writer.cpp#L2620-L2631
In general, non-zero (p_vaddr % p_align) happens with a low-alignment .tdata and a high-alignment .tbss.
To see the problem without modifying LLD, compile this C file:
repro.c:
```
__thread char tdata1 = 13;
__thread char tbss1[0x10000] __attribute__((aligned(0x10000)));
void* get_tdata1() { return &tdata1; }
void* get_tbss1() { return &tbss1; }
```
Results:
```
$ clang -fuse-ld=lld -target riscv64-linux-gnu -save-temps -nostdlib repro.c -o repro -Os
$ llvm-readelf -l repro
...
Type Offset VirtAddr PhysAddr FileSiz MemSiz Flg Align
LOAD 0x000000 0x0000000000000000 0x0000000000000000 0x00036c 0x00036c R 0x1000
LOAD 0x00036c 0x000000000000136c 0x000000000000136c 0x000014 0x000014 R E 0x1000
LOAD 0x001000 0x0000000000012000 0x0000000000012000 0x000088 0x001000 RW 0x1000
TLS 0x001000 0x0000000000012000 0x0000000000012000 0x000001 0x01e000 R 0x10000
...
$ llvm-readelf -Ss repro
...
[Nr] Name Type Address Off Size ES Flg Lk Inf Al
[ 8] .tdata PROGBITS 0000000000012000 001000 000001 00 WAT 0 0 1
[ 9] .tbss NOBITS 0000000000020000 001001 010000 00 WAT 0 0 65536
...
Num: Value Size Type Bind Vis Ndx Name
14: 0000000000000000 1 TLS GLOBAL DEFAULT 8 tdata1
16: 000000000000e000 65536 TLS GLOBAL DEFAULT 9 tbss1
...
$ llvm-objdump -dr repro.o
repro.o: file format elf64-littleriscv
Disassembly of section .text:
0000000000000000 <get_tdata1>:
0: 37 05 00 00 lui a0, 0x0
0000000000000000: R_RISCV_TPREL_HI20 tdata1
0000000000000000: R_RISCV_RELAX *ABS*
4: 33 05 45 00 add a0, a0, tp
0000000000000004: R_RISCV_TPREL_ADD tdata1
0000000000000004: R_RISCV_RELAX *ABS*
8: 13 05 05 00 mv a0, a0
0000000000000008: R_RISCV_TPREL_LO12_I tdata1
0000000000000008: R_RISCV_RELAX *ABS*
c: 82 80 ret
000000000000000e <get_tbss1>:
e: 37 05 00 00 lui a0, 0x0
000000000000000e: R_RISCV_TPREL_HI20 tbss1
000000000000000e: R_RISCV_RELAX *ABS*
12: 33 05 45 00 add a0, a0, tp
0000000000000012: R_RISCV_TPREL_ADD tbss1
0000000000000012: R_RISCV_RELAX *ABS*
16: 13 05 05 00 mv a0, a0
0000000000000016: R_RISCV_TPREL_LO12_I tbss1
0000000000000016: R_RISCV_RELAX *ABS*
1a: 82 80 ret
$ llvm-objdump -dR repro
repro: file format elf64-littleriscv
Disassembly of section .text:
000000000000136c <get_tdata1>:
136c: 13 05 02 00 mv a0, tp
1370: 82 80 ret
0000000000001372 <get_tbss1>:
1372: 37 05 01 00 lui a0, 0x10
1376: 33 05 45 00 add a0, a0, tp
137a: 13 05 05 00 mv a0, a0
137e: 82 80 ret
```
The assembly dump for the `repro` binary shows that get_tdata1 is returning tp+0, but get_tbss1 is returning tp+0x10000. tbss1 has a size of 0x10000, so it ends at tp+0x20000. That can't be right, though, because the PT_TLS segment's size is only 0x1e000.
I added some debugging prints to getTlsTpOffset, and this was the output:
```
getTlsTpOffset: s.getVA(0)=0x0 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x2000
getTlsTpOffset: s.getVA(0)=0x0 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x2000
getTlsTpOffset: s.getVA(0)=0xe000 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x10000
getTlsTpOffset: s.getVA(0)=0xe000 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x10000
```
tdata1 has an s.getVA(0) of 0, but adding (p_vaddr % p_align) produces a getTlsTpOffset of 0x2000. I think get_tdata1 should return (tp+0x2000) instead of tp+0.
It looks like relaxTlsLe in RISCV.cpp does the wrong thing because the high 20 bits of s.getVA(0) is 0. Disabling it fixes things:
```diff
diff --git a/lld/ELF/Arch/RISCV.cpp b/lld/ELF/Arch/RISCV.cpp
index 4798c86f7d1b..11f4ea31af60 100644
--- a/lld/ELF/Arch/RISCV.cpp
+++ b/lld/ELF/Arch/RISCV.cpp
@@ -762,7 +762,7 @@ static void relaxCall(const InputSection &sec, size_t i, uint64_t loc,
static void relaxTlsLe(const InputSection &sec, size_t i, uint64_t loc,
Relocation &r, uint32_t &remove) {
uint64_t val = r.sym->getVA(r.addend);
- if (hi20(val) != 0)
+ if (true || hi20(val) != 0)
return;
uint32_t insn = read32le(sec.content().data() + r.offset);
switch (r.type) {
```
Now get_tdata1 returns (tp + 0x2000):
```
000000000000136c <get_tdata1>:
136c: 37 25 00 00 lui a0, 0x2
1370: 33 05 45 00 add a0, a0, tp
1374: 13 05 05 00 mv a0, a0
1378: 82 80 ret
```
I also tried calling getTlsTpOffset from relaxTlsLe, but it didn't work because Out::tlsPhdr->p_vaddr was 0. I assume it gets initialized later.
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzMWd9y4jizfxrlpsuULIOBi1xASHZTxZlMJTmz546SrQZrR7ZckkzCPP0pyQYcEpjJTtV-n4sEY6n_6tfdUptbKzcV4jUZzcloccUbV2hzbWoj84IbcZVpsbu-0wYe759uom-E3cAG3bOyz_XDem3RARfCgisQvj6vnpdPYHFTYuUIG1sgbFKvtlwIA4SNoF5xJTcVYVNwOtDYXZlpBd9mJJlB4VxtSTIj7I6wu410RZMNcl0SdqfUdv8V1Ub_jbkj7C5TOiPsjq5HNM4Tmk1TGqdsnLMkXU95lvMJTzJcZ3wk8jiNAwNB2N3t0gu4r-rGPWHupK4GeV0TlizTURIt09GQ0AWhs_b_PaxcIavvK3CFtFDy72jBYmXReyPDnDcW4aXA6oK90kKlq-gHGu2ppIMX3SgBGYKseF0bXRvJHcJam-CZ56_eR7WWlQMhDeZO7YC7MIavmDeOZwqDlw8ev4GXQuZFx3qDFRqu1A5qg1usHCiZ5bA2uoSgl6w2XsxB4s18APeVdciF59VpcdCTQ9koJ2uFoNdH226AV8JPNtibW7UivF6w4bWnOO-cbOfQQobuBbHay-24gnXcOE_vf3CldM4dir3RA7gnbFxCpR3ElHrGtjHYLpVsgZlr4_0HGRZ8K7UJy9Y44GBl6c3B1xqNDLpaxBKFd32uq7U0JbiCO9gE13Eh_JhnXXMhZLUZ9IHyp37BLQb2BhV_fVZ2id5sb4r2HoKcK4XGkuTO2-1xs0H3bXZcYcPDPFfwKkw-ibaju6UFg7ZRzoKsIDP6O1ZerHeP1BXURudo7amO921gGu59pKGQruVlpWsCYR_T2ghZcSPVDpbLBWDlXWtbj1yEuof5AHr-uIecV0Ga_6yBw5Yb6REMBbd9rBjkbu-Akr9GNd9gZOUPHMBDtpW6sWrXRVBgyesaK0BuvZZy3VqTa4EQvkvPFQXoxv0HUsxfRjo0--TCUkajJUuT-M2KVPtI9Wbtk8QF_7YmW3iRrgAOSr9ER_8NnOCOB4xwKOSmeDOWWfsGDc_aAz4ESW10prAMXHXjoNRCrnc-RSyXC69Zrstaqi6ybmAtFXpH9rgZrI0e5MenKe0-4edq5QqDXICvLBD0jIEkC4gTksw_nJNZG5PRnL7GlFJKRgtYrbhzRmaNw9WKsImPLm8gCsIm-3ls2n32bLdaCsJmPphWreBAOgUynoNB1xifu9NuKJkDGS8-oAzqfEgYRvp0J7a3_x_bgD1x2-lUNoRc8WoD0bqxGClBkoVSAiLHzQYdGGnzbTqMlKya12hTNRBZvsXIYVlbiCptnVAyg245INLtLUQP9o1cNoSAdu9wVGuIVDuxHR4MOqgAPO9qhOPVFX6Ab9K4mUfo12Jnw83xupMKn-QPgP_BMnzDndrALIC4Y7t8mC16FPSVhutwc7zOPkrS_HjzGJh4CLyR8Hbi4YovPoqHx5tHuP2I71vN43dqxuzio8nkSPf416nmfif1uxJo7G9iDBIOvqEny3sODE_2DBrIaP7F-Fj8wss-LOAdUMBjAq31iPE_n-QPP3z7FLCw_A731RpmqscYJp5xl8R619fHhz_m989P8N7Uzi-dvRT-mj0DUAh_cZ_1tGWdWQtfHgKzo3uPF6MHrjEE3vSEazoaJempV740pa8vPii4anpOCDbv_TKXlQhxY7vRL-I1uHGvJ0A89HzeoT0MHVDxx_JhPlvC4vZu9r_L5_Bo0mXUHqf0lFNAQlD_Eqdpl3cvwkRnf4umrCESpks0-n0p0CHTTX2p8LvMkjtAtQ6pyzmFIZH1qRbScmuxzNTOb_hsuzeHgcNXd5I03zmIJDe97J7cHubvl9g7IxkDHUG7ooROVSMJnXLqyxt9PaTqKaHTU_6e-nHlD0HfVs9fH2-Xqz_vGSV02vf6edID7ePtcvZ_fh6bzeZPhO2VDKueJF69YdDQ68eFOOjX_nf1BUnD90rOFotf0XH4SzpO_Kw46EgPOpbbvornRUzeK7d8iNnq_lf0e0N8Tr328tsPmDCY0H0EEjo16C6ABw_gacv4O-zgKXY-Bx48D55joJ2n_KnhMft98LQ8PgTPZR1bwp_rmP4GeFric-D5iX7pL2Mn5r-AnQ8y4GO_UPby37-T_cKm5WL28zN63mdd9us5f4-MdvaYfjKG4mTMLsaQn9CLofjjGIrpG4r0c6juqPgv4qybjpdN_WgX_1wgHBYqQGDfQSEpbRc-pZD5k_MObKFfuhPzcYEgHN794cGfr1xN2JzumxIHH340qd3ADdoC3Z6cwZ-NPV4Oh58bsNofj7ESNnSMWlLWkj57VXJeETZ2kCEYuSlc2-7RzaboH__PNPWCQGlBV2rnpfpdxdseQ9cosbpEEJg1m403oTaychacvtTTeOFtz0Y3rm5OYX-yHCdskhnYQWin-KNgOP4t6CsFp2xEktvuNB0etlvHw0B7tA4D3kldYyU8YIcN83-hNHzD958I7B0I_n2JH0ZXFyEB29WJCgHmh-5daMBd6JPURosmRx8jJy3rEC0shMM9hN5uPzhtEbqYh9P9pBdAob_VtkhDSzKMnDTYQGn93YKS37HfB5RV6KF_G-R1DUJji_MXo310F96UftwVclMA82nE2VAN3jpCWqAD8BUjU55UOljL18BTVpuzHQYh1-v2kb-DKNpIB_ykaTUzeUHY3VHZ7CcTWo6yEvgKw_F0kk_S9VjE2WAQx-sh8iTm65RCTGk67DrrURT9VO6-3s7bzy-qQYaUDClE45QRdjMGwuaH23bIOu5kDlstRbs-N1wpwia5rqyD_lsBICy1mIeUKn_gyoH0942sXDpc-YX2Y109ecc2LPvv83089nQJS81-YsJWLjzAUm-xa0kdStuB1Zar0GMzA7srfXDuYWQGPklXot8miwBk6NMXklHCJluuAmMWexYhCezXZD_TmQaBjG_I-AZ-QhWqaxtVB4mdqsEYWdmq1RW5SJjyvrOYD3JduVB7JoRNBz5G9z04Ngcz0F0RmfaZ2hfp8gKCnW5Xv_XPh3nni37pJ4FWT9uGf5B0SADn-pv_bFOWjIGdPZKy99uyT2-Ihp_bEE0-vyG6B66sBmckivDSwmekk5Qb3jr1w6JN4tKBkKLdjLxo8_2QAx_a4p_MnLJfC2F6BSfsEULq5tY2JXomGwzvQaSTXMkfKEBxh6ZLzFfiOhHTZMqv8DoexzQeDRMWXxXXo5xm45ylYp2P2WSaTsfZFOl0zPk0jgXmV_KaUTakSRzHdMRoPEhYmg6HuJ5O80k8HU7IkGLJpRr4I8FAm82VtLbB68lwPEyuFM9Q2fBulbGQuhgZLa7MdThAZM3GkiFV0jp7pHfSqfA6NuQ1MlqQ0Xy5XJDRol9MfP1ovcbzXDeVC3vQ87WQi78b6_wW7qox6vrTr0GCVZawu2DY_wcAAP__9Ll3aw">