<table border="1" cellspacing="0" cellpadding="8">
    <tr>
        <th>Issue</th>
        <td>
            <a href=https://github.com/llvm/llvm-project/issues/84743>84743</a>
        </td>
    </tr>

    <tr>
        <th>Summary</th>
        <td>
            [RISCV][LLD] relaxTlsLe doesn't account for (p_vaddr % p_align) adjustment
        </td>
    </tr>

    <tr>
      <th>Labels</th>
      <td>
            lld
      </td>
    </tr>

    <tr>
      <th>Assignees</th>
      <td>
      </td>
    </tr>

    <tr>
      <th>Reporter</th>
      <td>
          rprichard
      </td>
    </tr>
</table>

<pre>
    For RISC-V, getTlsTpOffset adds the PT_TLS segment's (p_vaddr % p_align) to the symbol VA: https://github.com/llvm/llvm-project/blob/0f501c30b9601627c236f9abca8a3befba5dc161/lld/ELF/InputSection.cpp#L653-L654

I _think_ this makes sense, because when (p_vaddr % p_align) is non-zero, it would be inappropriate for the TP to point directly at the executable's segment, which would generally prevent libc from aligning TP for the TCB. Instead, the TP would be a multiple of p_align, and there would be an alignment gap of (p_vaddr % p_align) bytes between the TP and the start of the allocated segment. I'm not 100% sure this is the correct behavior, but a simple experiment seemed to confirm that glibc added this padding.

However, relaxTlsLe (and other callers?) use getVA directly rather than call getTlsTpOffset, and this results in broken relocation processing.

It's rare to hit this situation, because ordinarily LLD ensures that (p_vaddr % p_align) is zero. However, I can hit it if a variable has alignment greater than max-page-size. Obviously, it can happen easily if this code is commented out: https://github.com/llvm/llvm-project/blob/0f501c30b9601627c236f9abca8a3befba5dc161/lld/ELF/Writer.cpp#L2620-L2631

In general, non-zero (p_vaddr % p_align) happens with a low-alignment .tdata and a high-alignment .tbss.

To see the problem without modifying LLD, compile this C file:

repro.c:
```
__thread char tdata1 = 13;
__thread char tbss1[0x10000] __attribute__((aligned(0x10000)));
void* get_tdata1() { return &tdata1; }
void* get_tbss1() { return &tbss1; }
```

Results:

```
$ clang -fuse-ld=lld -target riscv64-linux-gnu -save-temps -nostdlib repro.c -o repro -Os

$ llvm-readelf -l repro
...
  Type           Offset   VirtAddr PhysAddr           FileSiz  MemSiz   Flg Align
  LOAD           0x000000 0x0000000000000000 0x0000000000000000 0x00036c 0x00036c R   0x1000
  LOAD 0x00036c 0x000000000000136c 0x000000000000136c 0x000014 0x000014 R E 0x1000
  LOAD           0x001000 0x0000000000012000 0x0000000000012000 0x000088 0x001000 RW  0x1000
  TLS            0x001000 0x0000000000012000 0x0000000000012000 0x000001 0x01e000 R   0x10000
...

$ llvm-readelf -Ss repro
...
  [Nr] Name              Type            Address Off    Size   ES Flg Lk Inf Al
  [ 8] .tdata            PROGBITS 0000000000012000 001000 000001 00 WAT  0   0  1
  [ 9] .tbss NOBITS          0000000000020000 001001 010000 00 WAT  0   0 65536
...
 Num:    Value          Size Type    Bind   Vis       Ndx Name
    14: 0000000000000000     1 TLS     GLOBAL DEFAULT     8 tdata1
    16: 000000000000e000 65536 TLS     GLOBAL DEFAULT     9 tbss1
...

$ llvm-objdump -dr repro.o

repro.o:        file format elf64-littleriscv

Disassembly of section .text:

0000000000000000 <get_tdata1>:
       0: 37 05 00 00   lui     a0, 0x0
                0000000000000000: R_RISCV_TPREL_HI20    tdata1
                0000000000000000:  R_RISCV_RELAX        *ABS*
 4: 33 05 45 00         add     a0, a0, tp
                0000000000000004: R_RISCV_TPREL_ADD     tdata1
                0000000000000004:  R_RISCV_RELAX        *ABS*
 8: 13 05 05 00         mv      a0, a0
                0000000000000008: R_RISCV_TPREL_LO12_I  tdata1
                0000000000000008: R_RISCV_RELAX *ABS*
       c: 82 80         ret

000000000000000e <get_tbss1>:
       e: 37 05 00 00   lui     a0, 0x0
                000000000000000e: R_RISCV_TPREL_HI20    tbss1
                000000000000000e:  R_RISCV_RELAX        *ABS*
 12: 33 05 45 00        add     a0, a0, tp
                0000000000000012: R_RISCV_TPREL_ADD     tbss1
                0000000000000012:  R_RISCV_RELAX        *ABS*
 16: 13 05 05 00        mv      a0, a0
                0000000000000016: R_RISCV_TPREL_LO12_I  tbss1
                0000000000000016: R_RISCV_RELAX *ABS*
      1a: 82 80         ret

$ llvm-objdump -dR repro

repro:  file format elf64-littleriscv

Disassembly of section .text:

000000000000136c <get_tdata1>:
    136c: 13 05 02 00   mv      a0, tp
    1370: 82 80         ret

0000000000001372 <get_tbss1>:
    1372: 37 05 01 00   lui     a0, 0x10
    1376: 33 05 45 00   add     a0, a0, tp
    137a: 13 05 05 00   mv      a0, a0
    137e: 82 80         ret
```

The assembly dump for the `repro` binary shows that get_tdata1 is returning tp+0, but get_tbss1 is returning tp+0x10000. tbss1 has a size of 0x10000, so it ends at tp+0x20000. That can't be right, though, because the PT_TLS segment's size is only 0x1e000.

I added some debugging prints to getTlsTpOffset, and this was the output:

```
getTlsTpOffset: s.getVA(0)=0x0 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x2000
getTlsTpOffset: s.getVA(0)=0x0 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x2000
getTlsTpOffset: s.getVA(0)=0xe000 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x10000
getTlsTpOffset: s.getVA(0)=0xe000 tls->p_vaddr=0x12000 tls->p_align=0x10000 result=0x10000
```

tdata1 has an s.getVA(0) of 0, but adding (p_vaddr % p_align) produces a getTlsTpOffset of 0x2000. I think get_tdata1 should return (tp+0x2000) instead of tp+0.

It looks like relaxTlsLe in RISCV.cpp does the wrong thing because the high 20 bits of s.getVA(0) is 0. Disabling it fixes things:

```diff
diff --git a/lld/ELF/Arch/RISCV.cpp b/lld/ELF/Arch/RISCV.cpp
index 4798c86f7d1b..11f4ea31af60 100644
--- a/lld/ELF/Arch/RISCV.cpp
+++ b/lld/ELF/Arch/RISCV.cpp
@@ -762,7 +762,7 @@ static void relaxCall(const InputSection &sec, size_t i, uint64_t loc,
 static void relaxTlsLe(const InputSection &sec, size_t i, uint64_t loc,
 Relocation &r, uint32_t &remove) {
   uint64_t val = r.sym->getVA(r.addend);
-  if (hi20(val) != 0)
+  if (true || hi20(val) != 0)
     return;
   uint32_t insn = read32le(sec.content().data() + r.offset);
   switch (r.type) {
```

Now get_tdata1 returns (tp + 0x2000):
```
000000000000136c <get_tdata1>:
    136c: 37 25 00 00   lui     a0, 0x2
    1370: 33 05 45 00   add     a0, a0, tp
    1374: 13 05 05 00   mv      a0, a0
    1378: 82 80         ret
```

I also tried calling getTlsTpOffset from relaxTlsLe, but it didn't work because Out::tlsPhdr->p_vaddr was 0. I assume it gets initialized later.

</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzMWd9y4jizfxrlpsuULIOBi1xASHZTxZlMJTmz546SrQZrR7ZckkzCPP0pyQYcEpjJTtV-n4sEY6n_6tfdUptbKzcV4jUZzcloccUbV2hzbWoj84IbcZVpsbu-0wYe759uom-E3cAG3bOyz_XDem3RARfCgisQvj6vnpdPYHFTYuUIG1sgbFKvtlwIA4SNoF5xJTcVYVNwOtDYXZlpBd9mJJlB4VxtSTIj7I6wu410RZMNcl0SdqfUdv8V1Ub_jbkj7C5TOiPsjq5HNM4Tmk1TGqdsnLMkXU95lvMJTzJcZ3wk8jiNAwNB2N3t0gu4r-rGPWHupK4GeV0TlizTURIt09GQ0AWhs_b_PaxcIavvK3CFtFDy72jBYmXReyPDnDcW4aXA6oK90kKlq-gHGu2ppIMX3SgBGYKseF0bXRvJHcJam-CZ56_eR7WWlQMhDeZO7YC7MIavmDeOZwqDlw8ev4GXQuZFx3qDFRqu1A5qg1usHCiZ5bA2uoSgl6w2XsxB4s18APeVdciF59VpcdCTQ9koJ2uFoNdH226AV8JPNtibW7UivF6w4bWnOO-cbOfQQobuBbHay-24gnXcOE_vf3CldM4dir3RA7gnbFxCpR3ElHrGtjHYLpVsgZlr4_0HGRZ8K7UJy9Y44GBl6c3B1xqNDLpaxBKFd32uq7U0JbiCO9gE13Eh_JhnXXMhZLUZ9IHyp37BLQb2BhV_fVZ2id5sb4r2HoKcK4XGkuTO2-1xs0H3bXZcYcPDPFfwKkw-ibaju6UFg7ZRzoKsIDP6O1ZerHeP1BXURudo7amO921gGu59pKGQruVlpWsCYR_T2ghZcSPVDpbLBWDlXWtbj1yEuof5AHr-uIecV0Ga_6yBw5Yb6REMBbd9rBjkbu-Akr9GNd9gZOUPHMBDtpW6sWrXRVBgyesaK0BuvZZy3VqTa4EQvkvPFQXoxv0HUsxfRjo0--TCUkajJUuT-M2KVPtI9Wbtk8QF_7YmW3iRrgAOSr9ER_8NnOCOB4xwKOSmeDOWWfsGDc_aAz4ESW10prAMXHXjoNRCrnc-RSyXC69Zrstaqi6ybmAtFXpH9rgZrI0e5MenKe0-4edq5QqDXICvLBD0jIEkC4gTksw_nJNZG5PRnL7GlFJKRgtYrbhzRmaNw9WKsImPLm8gCsIm-3ls2n32bLdaCsJmPphWreBAOgUynoNB1xifu9NuKJkDGS8-oAzqfEgYRvp0J7a3_x_bgD1x2-lUNoRc8WoD0bqxGClBkoVSAiLHzQYdGGnzbTqMlKya12hTNRBZvsXIYVlbiCptnVAyg245INLtLUQP9o1cNoSAdu9wVGuIVDuxHR4MOqgAPO9qhOPVFX6Ab9K4mUfo12Jnw83xupMKn-QPgP_BMnzDndrALIC4Y7t8mC16FPSVhutwc7zOPkrS_HjzGJh4CLyR8Hbi4YovPoqHx5tHuP2I71vN43dqxuzio8nkSPf416nmfif1uxJo7G9iDBIOvqEny3sODE_2DBrIaP7F-Fj8wss-LOAdUMBjAq31iPE_n-QPP3z7FLCw_A731RpmqscYJp5xl8R619fHhz_m989P8N7Uzi-dvRT-mj0DUAh_cZ_1tGWdWQtfHgKzo3uPF6MHrjEE3vSEazoaJempV740pa8vPii4anpOCDbv_TKXlQhxY7vRL-I1uHGvJ0A89HzeoT0MHVDxx_JhPlvC4vZu9r_L5_Bo0mXUHqf0lFNAQlD_Eqdpl3cvwkRnf4umrCESpks0-n0p0CHTTX2p8LvMkjtAtQ6pyzmFIZH1qRbScmuxzNTOb_hsuzeHgcNXd5I03zmIJDe97J7cHubvl9g7IxkDHUG7ooROVSMJnXLqyxt9PaTqKaHTU_6e-nHlD0HfVs9fH2-Xqz_vGSV02vf6edID7ePtcvZ_fh6bzeZPhO2VDKueJF69YdDQ68eFOOjX_nf1BUnD90rOFotf0XH4SzpO_Kw46EgPOpbbvornRUzeK7d8iNnq_lf0e0N8Tr328tsPmDCY0H0EEjo16C6ABw_gacv4O-zgKXY-Bx48D55joJ2n_KnhMft98LQ8PgTPZR1bwp_rmP4GeFric-D5iX7pL2Mn5r-AnQ8y4GO_UPby37-T_cKm5WL28zN63mdd9us5f4-MdvaYfjKG4mTMLsaQn9CLofjjGIrpG4r0c6juqPgv4qybjpdN_WgX_1wgHBYqQGDfQSEpbRc-pZD5k_MObKFfuhPzcYEgHN794cGfr1xN2JzumxIHH340qd3ADdoC3Z6cwZ-NPV4Oh58bsNofj7ESNnSMWlLWkj57VXJeETZ2kCEYuSlc2-7RzaboH__PNPWCQGlBV2rnpfpdxdseQ9cosbpEEJg1m403oTaychacvtTTeOFtz0Y3rm5OYX-yHCdskhnYQWin-KNgOP4t6CsFp2xEktvuNB0etlvHw0B7tA4D3kldYyU8YIcN83-hNHzD958I7B0I_n2JH0ZXFyEB29WJCgHmh-5daMBd6JPURosmRx8jJy3rEC0shMM9hN5uPzhtEbqYh9P9pBdAob_VtkhDSzKMnDTYQGn93YKS37HfB5RV6KF_G-R1DUJji_MXo310F96UftwVclMA82nE2VAN3jpCWqAD8BUjU55UOljL18BTVpuzHQYh1-v2kb-DKNpIB_ykaTUzeUHY3VHZ7CcTWo6yEvgKw_F0kk_S9VjE2WAQx-sh8iTm65RCTGk67DrrURT9VO6-3s7bzy-qQYaUDClE45QRdjMGwuaH23bIOu5kDlstRbs-N1wpwia5rqyD_lsBICy1mIeUKn_gyoH0942sXDpc-YX2Y109ecc2LPvv83089nQJS81-YsJWLjzAUm-xa0kdStuB1Zar0GMzA7srfXDuYWQGPklXot8miwBk6NMXklHCJluuAmMWexYhCezXZD_TmQaBjG_I-AZ-QhWqaxtVB4mdqsEYWdmq1RW5SJjyvrOYD3JduVB7JoRNBz5G9z04Ngcz0F0RmfaZ2hfp8gKCnW5Xv_XPh3nni37pJ4FWT9uGf5B0SADn-pv_bFOWjIGdPZKy99uyT2-Ihp_bEE0-vyG6B66sBmckivDSwmekk5Qb3jr1w6JN4tKBkKLdjLxo8_2QAx_a4p_MnLJfC2F6BSfsEULq5tY2JXomGwzvQaSTXMkfKEBxh6ZLzFfiOhHTZMqv8DoexzQeDRMWXxXXo5xm45ylYp2P2WSaTsfZFOl0zPk0jgXmV_KaUTakSRzHdMRoPEhYmg6HuJ5O80k8HU7IkGLJpRr4I8FAm82VtLbB68lwPEyuFM9Q2fBulbGQuhgZLa7MdThAZM3GkiFV0jp7pHfSqfA6NuQ1MlqQ0Xy5XJDRol9MfP1ovcbzXDeVC3vQ87WQi78b6_wW7qox6vrTr0GCVZawu2DY_wcAAP__9Ll3aw">