[llvm-commits] [PATCH] Program to aid in automated testing on windows.

Francois Pichet pichet2000 at gmail.com
Sat Oct 2 14:13:22 PDT 2010 

If I understand correctly the goal of this patch is to get rid of the
default Windows error dialog when an application crashes. This is
necessary for running BugPoint in automated mode without user
intervention right?

In that case you can take look at this registry key:
HKEY_CURRENT_USER\Software\Microsoft\Windows\Windows Error Reporting\DontShowUI

Setting this key to 1 will disable any popup when an application crashes.
We could arrange to set this flag to 1 when we expect a crash and put
it back to 0 right after.
The gotcha is that this flag is global to Windows and could disable
all errors popup if we fail to put it back to 0 after. But I think we
can warn users about this.


On Sat, Oct 2, 2010 at 4:00 PM, Daniel Dunbar <daniel at zuster.org> wrote:
> Hi Michael,
>
> I don't know where to go with this.
>
> On the one hand, I agree we have to solve the automated popup problem.
> However, I am worried that introducing something like this will cause
> lots of hard to understand behavior.
>
> Is it possible to avoid this problem in the llvm-test-suite by just
> linking against the non-Debug runtime?
>
>  - Daniel
>
> On Tue, Sep 21, 2010 at 11:30 AM, Michael Spencer <bigcheesegs at gmail.com> wrote:
>> While porting test-suite over to lit so I could run it on Windows I
>> ran into severe issues due to Dr. Watson and the C runtime.
>>
>> Windows uses Structured Exception Handling (SEH) to notify
>> applications about program faults such as read/write access errors
>> (segfaults), unaligned access, divide by 0, etc... Each application
>> has a SEH filter chain similar to catch blocks on the stack in C++.
>> When an exception occurs, Windows walks the filter chain calling the
>> filters until one handles it. By default the top level filter calls
>> Dr. Watson, which then searches the internet and pops up a dialog box
>> informing the user that something exploded. This makes it very
>> difficult to to automated testing.
>>
>> The other automation blocker is the way the Microsoft C runtime
>> handles asserts. By default, when an assert is fired, the C Runtime
>> writes the message out to stderr, and then loads user32.dll and
>> presents a message box to the user.
>>
>> This program provides an extremely hacky way to stop Dr. Watson from
>> starting due to unhandled exceptions in child processes.
>>
>> This simply starts the process arg[1] with the arguments in arg[2:]
>> under a debugger. All this debugger does is catch any unhandled
>> exceptions thrown in the child process and close the program (and
>> hopefully tells someone about it).
>>
>> This also provides another really hacky method to prevent assert
>> dialog boxes from popping up. When --no-user32 is passed, if any
>> process loads user32.dll, we assume it is trying to call MessageBoxEx
>> and so we terminate it. The proper way to do this would be to actually
>> set a break point, but there's quite a bit of code involved to get the
>> address of MessageBoxEx in the remote process's address space. This
>> can be added if it's ever actually needed.
>>
>> OVERVIEW: Dr. Watson Assassin.
>>
>> USAGE: KillTheDoctor [options] <program to run> <program arguments>...
>>
>> OPTIONS:
>>  -help      - Display available options (-help-hidden for more)
>>  -no-user32 - Terminate process if it loads user32.dll.
>>  -t=<uint>  - Set maximum runtime in seconds. Defaults to infinite.
>>  -version   - Display the version of this program
>>  -x         - Print detailed output about what is being run to stderr.
>>
>> Oh, and:
>>
>> ********************
>> Testing Time: 296.32s
>> ********************
>> Failing Tests (144):
>>    llvm-test-suite :: SingleSource/Benchmarks/BenchmarkGame/Large/fasta.c
>>    llvm-test-suite :: SingleSource/Benchmarks/BenchmarkGame/partialsums.c
>>    llvm-test-suite :: SingleSource/Benchmarks/BenchmarkGame/puzzle.c
>>    llvm-test-suite :: SingleSource/Benchmarks/BenchmarkGame/spectral-norm.c
>>    llvm-test-suite :: SingleSource/Benchmarks/CoyoteBench/almabench.c
>>    llvm-test-suite :: SingleSource/Benchmarks/CoyoteBench/huffbench.c
>>    llvm-test-suite :: SingleSource/Benchmarks/CoyoteBench/lpbench.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Dhrystone/dry.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Dhrystone/fldry.c
>>    llvm-test-suite :: SingleSource/Benchmarks/McGill/chomp.c
>>    llvm-test-suite :: SingleSource/Benchmarks/McGill/exptree.c
>>    llvm-test-suite :: SingleSource/Benchmarks/McGill/misr.c
>>    llvm-test-suite :: SingleSource/Benchmarks/McGill/queens.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/ReedSolomon.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/dt.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/fbench.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/ffbench.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/flops-1.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/flops-2.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/flops-5.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/flops-7.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/flops-8.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/flops.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/fp-convert.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/himenobmtxpa.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/lowercase.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/mandel-2.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/mandel.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/oourafft.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/perlin.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/pi.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/richards_benchmark.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/salsa20.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Misc/whetstone.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Shootout/hash.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Shootout/objinst.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Shootout/sieve.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Stanford/Bubblesort.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Stanford/IntMM.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Stanford/Oscar.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Stanford/Puzzle.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Stanford/Quicksort.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Stanford/RealMM.c
>>    llvm-test-suite :: SingleSource/Benchmarks/Stanford/Treesort.c
>>    llvm-test-suite :: SingleSource/Regression/C/2003-05-21-BitfieldHandling.c
>>    llvm-test-suite :: SingleSource/Regression/C/2003-05-21-UnionTest.c
>>    llvm-test-suite :: SingleSource/Regression/C/2003-05-22-VarSizeArray.c
>>    llvm-test-suite :: SingleSource/Regression/C/2003-05-23-TransparentUnion.c
>>    llvm-test-suite ::
>> SingleSource/Regression/C/2003-10-12-GlobalVarInitializers.c
>>    llvm-test-suite :: SingleSource/Regression/C/2004-02-03-AggregateCopy.c
>>    llvm-test-suite :: SingleSource/Regression/C/2004-03-15-IndirectGoto.c
>>    llvm-test-suite :: SingleSource/Regression/C/2004-08-12-InlinerAndAllocas.c
>>    llvm-test-suite :: SingleSource/Regression/C/2008-01-07-LongDouble.c
>>    llvm-test-suite ::
>> SingleSource/Regression/C/ConstructorDestructorAttributes.c
>>    llvm-test-suite :: SingleSource/Regression/C/PR1386.c
>>    llvm-test-suite :: SingleSource/Regression/C/PR491.c
>>    llvm-test-suite :: SingleSource/Regression/C/casts.c
>>    llvm-test-suite :: SingleSource/Regression/C/globalrefs.c
>>    llvm-test-suite :: SingleSource/Regression/C/matrixTranspose.c
>>    llvm-test-suite :: SingleSource/UnitTests/2002-05-19-DivTest.c
>>    llvm-test-suite :: SingleSource/UnitTests/2002-10-09-ArrayResolution.c
>>    llvm-test-suite :: SingleSource/UnitTests/2003-05-07-VarArgs.c
>>    llvm-test-suite :: SingleSource/UnitTests/2003-05-26-Shorts.c
>>    llvm-test-suite :: SingleSource/UnitTests/2003-05-31-CastToBool.c
>>    llvm-test-suite :: SingleSource/UnitTests/2003-07-09-LoadShorts.c
>>    llvm-test-suite :: SingleSource/UnitTests/2003-07-09-SignedArgs.c
>>    llvm-test-suite :: SingleSource/UnitTests/2003-07-10-SignConversions.c
>>    llvm-test-suite :: SingleSource/UnitTests/2003-08-11-VaListArg.c
>>    llvm-test-suite :: SingleSource/UnitTests/2004-11-28-GlobalBoolLayout.c
>>    llvm-test-suite :: SingleSource/UnitTests/2005-05-11-Popcount-ffs-fls.c
>>    llvm-test-suite :: SingleSource/UnitTests/2005-05-12-Int64ToFP.c
>>    llvm-test-suite :: SingleSource/UnitTests/2005-07-17-INT-To-FP.c
>>    llvm-test-suite :: SingleSource/UnitTests/2006-01-23-UnionInit.c
>>    llvm-test-suite :: SingleSource/UnitTests/2007-03-02-VaCopy.c
>>    llvm-test-suite :: SingleSource/UnitTests/2007-04-25-weak.c
>>    llvm-test-suite :: SingleSource/UnitTests/2008-04-18-LoopBug.c
>>    llvm-test-suite :: SingleSource/UnitTests/2008-04-20-LoopBug2.c
>>    llvm-test-suite ::
>> SingleSource/UnitTests/2009-04-16-BitfieldInitialization.c
>>    llvm-test-suite :: SingleSource/UnitTests/2009-12-07-StructReturn.c
>>    llvm-test-suite :: SingleSource/UnitTests/AtomicOps.c
>>    llvm-test-suite :: SingleSource/UnitTests/FloatPrecision.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/SSAtest.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/arith.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/array.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/big_bit_concat.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/big_part_set.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/bigint.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/bit_concat.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/bit_select.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/bit_set.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/bitbit.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/bitlogic.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/convert.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/extern-inline-redef.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/field.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/folding.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/general-test.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/global.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/integer_all_onesp.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/large-array.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/list.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/local-array.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/local-union.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/matrix.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/memory.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/multiple_assign.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/negConst.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/offset.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/part_select.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/part_select2.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/part_set.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/pointer.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/reduce_xor.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/reductions.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/sign.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/sign2.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/static.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/struct1.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/struct2.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/structInit.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/switch.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/test4.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/test_part_set.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/trunc.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/union-init.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/union-struct.c
>>    llvm-test-suite :: SingleSource/UnitTests/Integer/union2.c
>>    llvm-test-suite :: SingleSource/UnitTests/SignlessTypes/cast-bug.c
>>    llvm-test-suite :: SingleSource/UnitTests/SignlessTypes/ccc.c
>>    llvm-test-suite :: SingleSource/UnitTests/SignlessTypes/rem.c
>>    llvm-test-suite :: SingleSource/UnitTests/Threads/tls.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/SSE/sse.expandfft.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/SSE/sse.isamax.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/SSE/sse.shift.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/SSE/sse.stepfft.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/build.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/build2.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/divides.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/multiplies.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/simple.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/sumarray-dbl.c
>>    llvm-test-suite :: SingleSource/UnitTests/Vector/sumarray.c
>>    llvm-test-suite :: SingleSource/UnitTests/byval-alignment.c
>>    llvm-test-suite :: SingleSource/UnitTests/conditional-gnu-ext.c
>>
>>  Expected Passes    : 86
>>  Unsupported Tests  : 14
>>  Unexpected Failures: 144
>>
>>
>> - Michael Spencer
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>>
>>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>

More information about the llvm-commits mailing list