[llvm] [llvm][Support][Memory] Add memfd based fallback for strict W^X Linux systems (PR #98538)

[via llvm-commits]

================
@@ -251,5 +320,50 @@ void Memory::InvalidateInstructionCache(const void *Addr, size_t Len) {
   ValgrindDiscardTranslations(Addr, Len);
 }
 
+static inline bool isPermissionError(int err) {
+  // PaX uses EPERM, SELinux uses EACCES
+  return err == EPERM || err == EACCES;
+}
+
+bool Memory::execProtectionChangeNeedsNewMapping() {
+#if defined(__linux__)
+  static int status = -1;
+
+  if (status != -1)
+    return status;
+
+  // Try to get the status from /proc/self/status, looking for PaX flags.
+  if (auto file = MemoryBuffer::getFileAsStream("/proc/self/status")) {
+    auto pax_flags =
+        (*file)->getBuffer().rsplit("PaX:").second.split('\n').first.trim();
+    if (!pax_flags.empty())
+      // 'M' indicates MPROTECT is enabled
+      return (status = pax_flags.find('M') != StringRef::npos);
+  }
+
+  // Create a temporary writable mapping and try to make it executable.  If
+  // this fails, test 'errno' to ensure it failed because we were not allowed
+  // to create such a mapping and not because of some transient error.
+  size_t size = Process::getPageSizeEstimate();
+  void *addr = ::mmap(NULL, size, PROT_READ | PROT_WRITE,
+                      MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
+  if (addr == MAP_FAILED) {
+    // Must be low on memory or have too many mappings already, not much we can
+    // do here.
+    status = 0;
+  } else {
+    if (::mprotect(addr, size, PROT_READ | PROT_EXEC) < 0)
+      status = isPermissionError(errno);
----------------
minipli-oss wrote:

> hmmph, I didn't think of the multithreaded issue of "one thread is the consumer setting a selinux callback, the other is the JIT doing it's thing". I'm not sure how to best solve this then, as I'd like to avoid having llvm print selinux log messages.

Yeah, me too. Having a library printing stuff at "arbitrary" times isn't nice ;)

> 
> I guess at worst we could simply not check for `execmem` specifically and instead just bail out when we find selinux enabled? I'd argue that any sane selinux instance disabled `execmem` anyways :P

>From my experience, quite the opposite. You won't find a current desktop system that has SELinux's "deny_execmem" enabled because there's just so many JITs out there all trying to either directly `mmap(RWX)` or do the transition `RW-` -> `R-X`.

So, no, just because SELinux is enabled, doesn't mean its policy will deny generating code at runtime. That's why the test in `Memory::execProtectionChangeNeedsNewMapping()` has a generic fallback if a specific test (for PaX only, so far) fails.

https://github.com/llvm/llvm-project/pull/98538