[PATCH] D155290: [PGO] Use Unique Profile Files when New Processes are Forked

Mon Aug 14 09:49:29 PDT 2023

qiongsiwu1 added a comment.

In D155290#4582825 <https://reviews.llvm.org/D155290#4582825>, @MaskRay wrote:

> Using `%p` should not magically change the behavior and I am unsure the result is better for PGO. 
> If we decide to provide such a mode, it needs to be opt-in by adding a new format specifier or API.
> Can you elaborate separate profile files are going to be useful?

Thanks again for taking a look at this patch!

Sure! We are generating separate profiles through `%p` for two reasons. First, we think that `%p` implies a profile per process, even if `exec` is not called after `fork`. It is unexpected that with `%p`, a program that forks child processes only create one profile file from the parent. This is why we did not create a new API or create a new pattern. Second, we are trying to catch non-gracefully terminated processes during profile generate. We observed a scenario where a hot function having all 0 counters. The reasons was the child process was created by a fork (but no `exec` followed), and then was terminated probably by a signal, instead of going through `exit()`. Such a process did not have a chance to write back the profile for merging. The parent process took a different call path and never called the hot function. We would like to detect such a situation for a particular process. Hence this patch creates an empty profile file if a process is not terminated gracefully. Using `%p` was probably the most natural choice when debugging such issues.

One can argue that we may look into the continuous mode to manage abnormal terminations during profile generate. I have not looked too closely at that, but we think it is still useful for a user to tell that some processes did not write any profile back, for debugging, or for diagnostics.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D155290/new/

https://reviews.llvm.org/D155290