[llvm] [RISCV] Fix crash when trying to remove segment (PR #146524)

[Luke Lau via llvm-commits]

lukel97 wrote:

I think it would still be good to have an MIR test if you can add one

https://github.com/llvm/llvm-project/pull/146524