[lldb-dev] tests xfailed globally with radar references

[Vedant Kumar via lldb-dev]

I took a first cut at this in r327463.

vedant

> On Mar 13, 2018, at 2:39 PM, Vedant Kumar <vsk at apple.com> wrote:
> 
> Hi Ted,
> 
> Thanks for bringing this up. I'll volunteer to take a look at these tests and file PRs where appropriate. I should get to this within a day or two.
> 
> vedant
> 
>> On Mar 13, 2018, at 10:31 AM, Ted Woodward via lldb-dev <lldb-dev at lists.llvm.org> wrote:
>> 
>> I was investigating TestWatchedVarHitWhenInScope.py, which tests that a
>> watchpoint is hit in scope and not hit when out of scope. It's xfailed due
>> to a radar:
>>   @unittest2.expectedFailure("rdar://problem/18685649")
>> 
>> This is an actual bug - watchpoints are hit when the address is modified but
>> the variable is out of scope. Which can happen a lot if the variable being
>> watched is a local on the stack. I manually ran the testcase on Linux, and
>> it failed when it hit the watchpoint out of scope:
>> Process 18652 stopped
>> * thread #1, name = 'a.out', stop reason = watchpoint 1
>>   frame #0: 0x00007ffff722d0d2 libc.so.6`__run_exit_handlers(status=0,
>> listp=0x00007ffff75b36c8, run_list_atexit=true) at exit.c:35
>> 
>> A google search found this message, posted by Ed Maste to llvm-dev:
>> http://lists.llvm.org/pipermail/llvm-dev/2017-August/116471.html .
>> 
>> I echo Ed's request - could someone at Apple look at these radars, and open
>> llvm bugs for the ones that aren't Darwin-only?
>> 
>> 
>> Ed's message:
>> 
>> I was recently looking at an unexpected pass test result on FreeBSD[1]
>> which is decorated with expectedFailureAll referencing
>> rdar://problem/24599697. I'm looking for more details on this specific
>> radar bug report in order to determine if the test is not exercising
>> the original bug on FreeBSD, or if it's really an Apple-only issue.
>> 
>> As it's rather unfortunate to have a bug report reference for which no
>> further public details are available I looked at other radar
>> references in the test suite, and found 118 of them. Many of these are
>> in comments, in macosx-specific cases (e.g.
>> @expectedFailureAll(oslist=["macosx"], bugnumber="rdar://28805064")),
>> or in tests decorated with skipUnlessDarwin, and I've ignored those.
>> 
>> There are 10 tests with expected failure decorators, and one skipped,
>> that apply beyond macosx and reference a radar (and do not
>> additionally reference an LLVM PR). Can someone at Apple check these
>> radar references and submit LLVM PRs, reference existing PRs, add a
>> reasonable description in the test source, etc. as appropriate?
>> 
>> functionalities/watchpoint/variable_out_of_scope/TestWatchedVarHitWhenInScop
>> e.py
>> unittest2.expectedFailure("rdar://problem/18685649")
>> 
>> functionalities/asan/TestReportData.py
>> @expectedFailureAll(archs=['i386'], bugnumber="rdar://28658860")
>> 
>> api/multiple-debuggers/TestMultipleDebuggers.py
>> @expectedFailureAll(bugnumber="rdar://30564102")
>> 
>> lang/objc/bitfield_ivars/TestBitfieldIvars.py
>> decorators.expectedFailureAll(bugnumber="rdar://problem/17990991")])
>> 
>> lang/c/shared_lib/TestSharedLib.py
>> @unittest2.expectedFailure("rdar://problem/10704639")
>> 
>> lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
>> @unittest2.expectedFailure("rdar://problem/10381325")
>> 
>> lang/cpp/stl/TestSTL.py
>> @expectedFailureAll(bugnumber="rdar://problem/10400981")
>> 
>> lang/cpp/dynamic-value/TestCppValueCast.py
>> @unittest2.expectedFailure("rdar://problem/10808472 SBValue::Cast test
>> case is failing (virtual inheritance)")
>> 
>> lang/cpp/printf/TestPrintf.py
>> decorators.expectedFailureAll(bugnumber="rdar://problem/24599697")])
>> 
>> lang/cpp/function-template-parameter-pack/TestFunctionTemplateParameterPack.
>> py
>> decorators.expectedFailureAll(bugnumber="rdar://problem/32096064")])
>> 
>> lang/cpp/unique-types/TestUniqueTypes.py
>> self.skipTest("rdar://problem/9173060 lldb hangs while running
>> unique-types for clang version < 3")
>> 
>> [1]
>> http://lists.llvm.org/pipermail/lldb-commits/Week-of-Mon-20170807/036634.htm
>> l
>> 
>> --
>> Qualcomm Innovation Center, Inc.
>> The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
>> Linux Foundation Collaborative Project
>> 
>> 
>> _______________________________________________
>> lldb-dev mailing list
>> lldb-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>