[PATCH] D151025: [llvm-exegesis] Add support for using memory annotations

Thu Jun 29 01:45:47 PDT 2023

foad added a comment.

In D151025#4456730 <https://reviews.llvm.org/D151025#4456730>, @aidengrossman wrote:

> I'm not sure what could be going wrong. I saw segmentation fault failures on one of the builders but after I added a check to the lit config to ensure that the subprocess mode was actually supported it went away. It might have something to do with security policies since this patch does some things that probably seem quite abnormal in regards to the process memory, but I might be completely wrong there with that assumption.
>
> To debug you need to modify the code slightly as this patch uses `ptrace`, so `lldb` isn't able to grab the child process because there's already another process tracing it. You'll need to comment out the `ptrace` calls and then you should be able to get into the child process. I had a flag to do that on one of my development branches that maybe I should upstream at some point.
>
> Once you have that setup, I've found it easiest to debug by breaking on a line right before the function call into the MCJITed test/test harness. Then I would single step through the assembly to see what was happening.
>
> However, I understand this might be a pain to debug, so if you want me to disable the tests or add something to the lit config to ensure that these tests only run if memory annotations don't cause seg-faults while we can work on reproducing it without being too interruptive, I'm fine with doing that.

strace on the JITted part of the child process shows:

  munmap(NULL, 140344954515456)           = 0
  munmap(0x7fa49b29d000, 392533778432)    = 0
  mmap(0x7fffffffe000, 4096, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED_NOREPLACE, 8, 0) = 0x7fffffffe000
  mmap(0x2000, 4096, PROT_READ|PROT_WRITE, MAP_SHARED|MAP_FIXED_NOREPLACE, 9, 0) = -1 EPERM (Operation not permitted)
  ioctl(7, PERF_EVENT_IOC_RESET, PERF_IOC_FLAG_GROUP|0x2) = 0
  --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=0x2000} ---
  +++ killed by SIGSEGV (core dumped) +++

i.e. the access to address 0x2000 fails because the mmap call failed with EPERM.

There is some discussion here about why you might get the "nonsensical" error EPERM if you try to map an address lower than mmap_min_addr: https://lore.kernel.org/all/20230418214009.1142926-1-Liam.Howlett@oracle.com/T/#u

And sure enough my system has mmap_min_addr set higher than 0x2000:

  $ sysctl vm.mmap_min_addr
  vm.mmap_min_addr = 65536

This setting comes from `/etc/sysctl.d/10-zeropage.conf` which seems to be an Ubuntu-specific change to the Debain procps package.

I have verified that `sed -i s/8192/65536/ test/tools/llvm-exegesis/X86/latency/memory-annotations*.s` fixes the tests for me.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D151025/new/

https://reviews.llvm.org/D151025