[compiler-rt] [sanitizer_common] Fix TgKill on Solaris (PR #98000)

[Rainer Orth via llvm-commits]

https://github.com/rorth created https://github.com/llvm/llvm-project/pull/98000

While working on safestack on Solaris, I noticed that the `TgKill` implementation is wrong here: `TgKill` is supposed to return `-1` on error, while `thr_kill` returns `errno` instead.  This patch compensates for that.

This went unnoticed so far since `TgKill` has been unused.

Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11` together with a subsequent patch to make safestack actually work on Solaris.

>From a7ef2d4a012fef72c1419c76b2f0c41a08e19bb2 Mon Sep 17 00:00:00 2001
From: Rainer Orth <ro at gcc.gnu.org>
Date: Mon, 8 Jul 2024 10:11:03 +0200
Subject: [PATCH] [sanitizer_common] Fix TgKill on Solaris

While working on safestack on Solaris, I noticed that the `TgKill`
implementation is wrong here: `TgKill` is supposed to return `-1` on error,
while `thr_kill` returns `errno` instead.  This patch compensates for that.

This went unnoticed so far since `TgKill` has been unused.

Tested on `amd64-pc-solaris2.11` and `sparcv9-sun-solaris2.11` together
with a subsequent patch to make safestack actually work on Solaris.
---
 compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp b/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp
index 12df3ef73da4bc..eec9d652630f95 100644
--- a/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp
+++ b/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp
@@ -563,7 +563,9 @@ int TgKill(pid_t pid, tid_t tid, int sig) {
   return internal_syscall(SYSCALL(thr_kill2), pid, tid, sig);
 #    elif SANITIZER_SOLARIS
   (void)pid;
-  return thr_kill(tid, sig);
+  errno = thr_kill(tid, sig);
+  // TgKill is expected to return -1 on error, not an errno.
+  return errno != 0 ? -1 : 0;
 #    endif
 }
 #  endif