[PATCH] D14470: [asan] Fix the deadlocks introduced by "On OS X, log reports to syslog and os_trace" commit

[Anna Zaks via llvm-commits]

zaks.anna created this revision.
zaks.anna added reviewers: samsonov, kubabrecka.
zaks.anna added a subscriber: llvm-commits.

[asan] On OS X, log reports to syslog and os_trace, has been reverted in r252076 due to deadlocks on earlier versions of OS X. Alexey has also noticed deadlocks in some corner cases on Linux. This patch, if applied on top of the logging patch (http://reviews.llvm.org/D13452), addresses the known deadlock issues.

(This also proactively removes the color escape sequences from the error report buffer since we have to copy the buffer anyway.)

More background:

Alexey Samsonov <vonosmas at gmail.com>:
While I'm not exactly sure what caused these issues on Mac buildbot, I've also observed several self-deadlocks caused by this change.
Essentially, they are caused by AppendToErrorMessageBuffer function, which now takes a lock, but can be re-entrant:

* AppendToErrorMessageBuffer is called and takes a lock
* AppendToErrorMessageBuffer tries to allocate error message buffer and calls MmapOrDie()
* MmapOrDie() fails, because memory limit is exceeded, and calls Report() to report that
* Report calls AppendToErrorMessageBuffer, which attempts to take a lock, and hangs.

The same will happen if CHECK() call in AppendToErrorMessageBuffer fails. Sadly, I can't suggest a good solution for that: probably
you just need to try harder to make sure the function will never be called recursively: use RAW_CHECK instead of CHECK and
call (some variation of) MmapOrDieQuietly(), which will just call RawWrite()+Die() on failure.

Anna:
I suspect the issue with the earlier versions of OS X is that logging the error report results in ASan calling Printf under high verbosity level (and Printf calls AppendToErrorMessageBuffer). We hold the lock that is guarding error_message_buf when reporting inside ScopedInErrorReport. We can work around that issue by copying the buffer under the lock but releasing it before calling the logging routines.


http://reviews.llvm.org/D14470

Files:
  lib/asan/asan_report.cc
  lib/sanitizer_common/sanitizer_common.cc
  lib/sanitizer_common/sanitizer_common.h
  lib/sanitizer_common/sanitizer_mac.cc
  lib/sanitizer_common/sanitizer_posix.cc
  lib/sanitizer_common/sanitizer_win.cc

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D14470.39611.patch
Type: text/x-patch
Size: 5828 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20151107/e17efd7a/attachment.bin>