[PATCH] [Support] Disable assertion dialogs from the MSVC debug CRT

[Reid Kleckner]

Hi Bigcheese,

Sets a report hook that emulates pressing "retry" in the "abort, retry,
ignore" dialog box that _CrtDbgReport normally raises.  There are many
other ways to disable assertion reports, but this was the only way I
could find that still calls our exception handler.

http://llvm-reviews.chandlerc.com/D625

Files:
  lib/Support/Windows/Signals.inc

Index: lib/Support/Windows/Signals.inc
===================================================================
--- lib/Support/Windows/Signals.inc
+++ lib/Support/Windows/Signals.inc
@@ -178,6 +178,19 @@
 //===----------------------------------------------------------------------===//
 
 #ifdef _MSC_VER
+/// AvoidMessageBoxHook - Emulates hitting "retry" from an "abort, retry,
+/// ignore" CRT debug report dialog.  "retry" raises an exception which
+/// ultimately triggers our stack dumper.
+static int AvoidMessageBoxHook(int ReportType, char *Message, int *Return) {
+  // Set *Return to the retry code for the return value of _CrtDbgReport:
+  // http://msdn.microsoft.com/en-us/library/8hyw4sy7(v=vs.71).aspx
+  // This may also trigger just-in-time debugging via DebugBreak().
+  if (Return)
+    *Return = 1;
+  // Don't call _CrtDbgReport.
+  return TRUE;
+}
+
 /// CRTReportHook - Function called on a CRT debugging event.
 static int CRTReportHook(int ReportType, char *Message, int *Return) {
   // Don't cause a DebugBreak() on return.
@@ -238,6 +251,12 @@
   OldFilter = SetUnhandledExceptionFilter(LLVMUnhandledExceptionFilter);
   SetConsoleCtrlHandler(LLVMConsoleCtrlHandler, TRUE);
 
+#ifdef _MSC_VER
+  // Setting a report hook overrides the default behavior of popping an "abort,
+  // retry, or ignore" dialog.
+  _CrtSetReportHook(AvoidMessageBoxHook);
+#endif
+
   // Environment variable to disable any kind of crash dialog.
   if (getenv("LLVM_DISABLE_CRASH_REPORT")) {
 #ifdef _MSC_VER
-------------- next part --------------
A non-text attachment was scrubbed...
Name: D625.1.patch
Type: text/x-patch
Size: 1511 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130404/10ccadcb/attachment.bin>