[lldb-dev] tests xfailed globally with radar references

[Ted Woodward via lldb-dev]

I was investigating TestWatchedVarHitWhenInScope.py, which tests that a
watchpoint is hit in scope and not hit when out of scope. It's xfailed due
to a radar:
    @unittest2.expectedFailure("rdar://problem/18685649")

This is an actual bug - watchpoints are hit when the address is modified but
the variable is out of scope. Which can happen a lot if the variable being
watched is a local on the stack. I manually ran the testcase on Linux, and
it failed when it hit the watchpoint out of scope:
Process 18652 stopped
* thread #1, name = 'a.out', stop reason = watchpoint 1
    frame #0: 0x00007ffff722d0d2 libc.so.6`__run_exit_handlers(status=0,
listp=0x00007ffff75b36c8, run_list_atexit=true) at exit.c:35

A google search found this message, posted by Ed Maste to llvm-dev:
http://lists.llvm.org/pipermail/llvm-dev/2017-August/116471.html .

I echo Ed's request - could someone at Apple look at these radars, and open
llvm bugs for the ones that aren't Darwin-only?


Ed's message:

I was recently looking at an unexpected pass test result on FreeBSD[1]
which is decorated with expectedFailureAll referencing
rdar://problem/24599697. I'm looking for more details on this specific
radar bug report in order to determine if the test is not exercising
the original bug on FreeBSD, or if it's really an Apple-only issue.

As it's rather unfortunate to have a bug report reference for which no
further public details are available I looked at other radar
references in the test suite, and found 118 of them. Many of these are
in comments, in macosx-specific cases (e.g.
@expectedFailureAll(oslist=["macosx"], bugnumber="rdar://28805064")),
or in tests decorated with skipUnlessDarwin, and I've ignored those.

There are 10 tests with expected failure decorators, and one skipped,
that apply beyond macosx and reference a radar (and do not
additionally reference an LLVM PR). Can someone at Apple check these
radar references and submit LLVM PRs, reference existing PRs, add a
reasonable description in the test source, etc. as appropriate?

functionalities/watchpoint/variable_out_of_scope/TestWatchedVarHitWhenInScop
e.py
unittest2.expectedFailure("rdar://problem/18685649")

functionalities/asan/TestReportData.py
@expectedFailureAll(archs=['i386'], bugnumber="rdar://28658860")

api/multiple-debuggers/TestMultipleDebuggers.py
@expectedFailureAll(bugnumber="rdar://30564102")

lang/objc/bitfield_ivars/TestBitfieldIvars.py
decorators.expectedFailureAll(bugnumber="rdar://problem/17990991")])

lang/c/shared_lib/TestSharedLib.py
@unittest2.expectedFailure("rdar://problem/10704639")

lang/c/shared_lib_stripped_symbols/TestSharedLibStrippedSymbols.py
@unittest2.expectedFailure("rdar://problem/10381325")

lang/cpp/stl/TestSTL.py
@expectedFailureAll(bugnumber="rdar://problem/10400981")

lang/cpp/dynamic-value/TestCppValueCast.py
@unittest2.expectedFailure("rdar://problem/10808472 SBValue::Cast test
case is failing (virtual inheritance)")

lang/cpp/printf/TestPrintf.py
decorators.expectedFailureAll(bugnumber="rdar://problem/24599697")])

lang/cpp/function-template-parameter-pack/TestFunctionTemplateParameterPack.
py
decorators.expectedFailureAll(bugnumber="rdar://problem/32096064")])

lang/cpp/unique-types/TestUniqueTypes.py
self.skipTest("rdar://problem/9173060 lldb hangs while running
unique-types for clang version < 3")

[1]
http://lists.llvm.org/pipermail/lldb-commits/Week-of-Mon-20170807/036634.htm
l

--
Qualcomm Innovation Center, Inc.
The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a
Linux Foundation Collaborative Project