[llvm] [Support] Fix Process::PreventCoreFiles() when coredumps are piped (PR #83703)

[Alexander Richardson via llvm-commits]

arichardson wrote:

> > I considered that, but according to the manpage it means we can't ptrace the process afterwards:
> > > Processes that are not dumpable can not be attached via [ptrace(2)](https://man7.org/linux/man-pages/man2/ptrace.2.html) PTRACE_ATTACH; see [ptrace(2)](https://man7.org/linux/man-pages/man2/ptrace.2.html) for further details.
> > > If a process is not dumpable, the ownership of files in the process's /proc/pid directory is affected as described in [proc(5)](https://man7.org/linux/man-pages/man5/proc.5.html).
> > 
> > 
> > It does look like it will add to dmesg (but it already seems flooded with lots of other messages):
> > ```
> > [  +0.218577] traps: bounds.cpp.tmp[244068] trap invalid opcode ip:561028d05dc9 sp:7ffddaacc640 error:0 in bounds.cpp.tmp[561028c32000+d4000]
> > [  +0.013130] Process 244068(bounds.cpp.tmp) has RLIMIT_CORE set to 1
> > [  +0.006593] Aborting core
> > ```
> > 
> > 
> >     
> >       
> >     
> > 
> >       
> >     
> > 
> >     
> >   
> > Is this a dealbreak? I can see if another value avoids the coredump without spamming dmesg.
> 
> It isn't a dealbreaker. Since `PR_SET_DUMPABLE 0` would disable `PTRACE_ATTACH`, a feature which I consider more important, it seems that we should just tolerate it... This seems a Linux API issue that doesn't offer desired granularity....

Sounds good! I just tried reading through the systemd coredump code and it seems like they don't write the coredump if the rlimit is set to a value smaller than one PAGE: https://github.com/systemd/systemd/blob/1500b656cd6e4500dd0514d90c27fe0e16b14c77/src/coredump/coredump.c#L438

However, it seems like the kernel is still streaming many gigabytes of data to the file descriptor. I'm not sure if that is an issue on the systemd side or the kernel side.

https://github.com/llvm/llvm-project/pull/83703