<div dir="ltr">Thanks for the info. <br><div><br></div><div>I just tried __llvm_profile_set_file_object(..., EnableMerge = true) in the case when "%m" is in LLVM_PROFILE_FILE in non-continuous mode. It turns out __llvm_profile_set_file_object changes the profile file for the instrumented binary and doesn't change profile files for dynamic libraries, which is expected.</div><div><br></div><div>Zequan</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Jul 28, 2021 at 1:48 PM Vedant Kumar <<a href="mailto:vsk@apple.com" target="_blank">vsk@apple.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><br><div><br><blockquote type="cite"><div>On Jul 28, 2021, at 1:23 PM, Zequan Wu <<a href="mailto:zequanwu@google.com" target="_blank">zequanwu@google.com</a>> wrote:</div><br><div><div dir="ltr"><div dir="ltr">Hi Vedant,<br></div><div><div class="gmail_quote"><div dir="ltr" class="gmail_attr"><br></div><div dir="ltr" class="gmail_attr">On Tue, Jul 27, 2021 at 10:42 PM Vedant Kumar <<a href="mailto:vsk@apple.com" target="_blank">vsk@apple.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div>Hi Zequan,<div><div><br><blockquote type="cite"><div>On Jul 27, 2021, at 6:50 PM, Zequan Wu <<a href="mailto:zequanwu@google.com" target="_blank">zequanwu@google.com</a>> wrote:</div><br><div><div dir="ltr">Hi,<div><br></div><div>When merging is disabled, a possible solution is to unmap the mapped file region, copy the old profile file to the new file, and then mmap the CounterBegin to the new file region. Does this sound correct?</div></div></div></blockquote><div><br></div><div>If the instrumented program is multithreaded, it might be possible for a thread to racily update a counter in the un-<span>mmap'd region.</span></div><div><span><br></span></div></div></div></div></blockquote><div>I thought that the instrumented program can only update counters in the mmapped regions which are setup during the initialization. How does it access un-mmaped regions?</div></div></div></div></div></blockquote><div><br></div><div>If the __llvm_prf_cnts section is mmap'd to a file (either directly, or from a standalone buffer as is done for Fuchsia), a possible sequence might be:</div><div><br></div><div>Thread 1: __llvm_profile_set_file_object(...)</div><div>Thread 1: munmap(countersRegion)</div><div>Thread 2: ++countersRegion[idx] <= Write to __llvm_prf_cnts after munmap(), not clear what this memory region contains.</div><div>Thread 1: Copy old profile...</div><div>Thread 1: countersRegion = mmap(new_fd, ...)</div><div><br></div><div>What I meant to express earlier is that I'm certain about what the contents of __llvm_prf_cnts are after the munmap(). If it turns out that the munmap() doesn't alter the in-memory counters [*], there should be no race. The subsequent mmap(new_fd) should write up-to-date counters to the new profile. It could be worth checking in a proof of concept (similar to darwin-proof-of-concept.c) to determine the behavior of mmap(__llvm_prf_cnts, fd1) -> munmap() -> mmap(__llvm_prf_cnts, fd2).</div><div><br></div><div>[*] The BSD manual for munmap(2) suggests that this is what happens:</div><div><br></div><div>```</div><div> [File mapping] If the mapping maps data from a file (MAP_SHARED), then the memory will eventually be written back to disk if it's<br> dirty. This will happen automatically at some point in the future (implementation dependent). Note: to force the<br> memory to be written back to the disk, use msync(2).<br><br> If there are still other references to the memory when the munmap is done, then nothing is done to the memory itself and it may be<br> swapped out if need be.</div>```</div><div><br><blockquote type="cite"><div><div dir="ltr"><div><div class="gmail_quote"><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><div><div><span></span></div><div><font><span>Do you think it's viable to sidestep the issue by restricting how/when __llvm_profile_set_file_object can be called, e.g. either by documenting or requiring that no other threads are active at the time of the call?</span></font></div><div><br></div></div></div></div></blockquote><div> This looks viable.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div><div><div></div><blockquote type="cite"><div><div dir="ltr"><div>When merging is enabled, multiple profile files will be created. I'm not sure how to set the file object in this case as only one file descriptor is passed to __llvm_profile_set_file_object.</div></div></div></blockquote><div><br></div><div>Can you share how __llvm_profile_set_file_object(..., EnableMerge = true) works in non-continuous mode? It's not something I'm familiar with.</div><div><br></div></div></div></div></blockquote><div>By looking at existing tests, __llvm_profile_set_file_object(..., EnableMerge = true) allows instrumented programs to accumulate profile files from different runs. My guess is that when running two instrumented programs, this function allows them to write to the same profile file in appending mode, but I'm not sure how it handles the case when multiple profile files are created during initialization for non-continuous mode.<br></div></div></div></div>
</div></blockquote></div><br><div>For the merging + continuous mode case, it sounds like it'd be necessary to lock a profile, update the in-memory __llvm_prf_cnts section with its contents (the merging step), and then set up a fresh mmap().</div><div><br></div><div>vedant</div></div></blockquote></div>