[llvm] [RISCV] Fix crash when trying to remove segment (PR #146524)

[Luke Lau via llvm-commits]

lukel97 wrote:

Thanks for fixing this, I've gone ahead and committed it if that's ok.

https://github.com/llvm/llvm-project/pull/146524