[Lldb-commits] [PATCH] D37651: Fix for bug 34532 - A few rough corners related to post-mortem debugging (core/minidump)

[Jason Molenda via lldb-commits]

Or another way, I don't object to asserts because we hit them so often today.  I object to asserts in a shipping debugger fundamentally - it's not an error mode we should encourage or depend on for an interactive tool.

> On Sep 12, 2017, at 2:56 PM, Jason Molenda <jmolenda at apple.com> wrote:
> 
> I think debug-build asserts & more testing is something everyone can agree on.  I don't want to speak for Greg, but I don't think anyone is going to sign on to the idea that the lldb source base becomes super well tested & bug free and then we start using asserts liberally.  There are always going to be scenarios that our customers hit, with compilers we don't have access to, with programs we don't have access to, with targets we don't have access to, that we don't cover in our testsuite.  We can work to expand our testsuite, and expand the scope of what / how we test the debugger, but we'll never match the behavior someone running lldb on an AutoCAD built with a three year old gcc or whatever.
> 
>> On Sep 12, 2017, at 2:53 PM, Zachary Turner <zturner at google.com> wrote:
>> 
>> So let's say that in a hypothetical future we had very good test coverage.  Would there still be resistance to asserts?  Because if your test coverage is good, then your bots should be catching most stuff.
>> 
>> On Tue, Sep 12, 2017 at 2:42 PM Greg Clayton <clayborg at gmail.com> wrote:
>> Agreed. We need to do better on the test coverage part, don't think you'll get any pushback on that front.
>> 
>>> On Sep 12, 2017, at 2:36 PM, Zachary Turner <zturner at google.com> wrote:
>>> 
>>> Right, and the only reason it's a bigger problem is because of....   test coverage.
>>> 
>>> On Tue, Sep 12, 2017 at 2:34 PM Jason Molenda <jmolenda at apple.com> wrote:
>>> 
>>> 
>>>> On Sep 12, 2017, at 11:19 AM, Zachary Turner <zturner at google.com> wrote:
>>>> 
>>>> 
>>>> 
>>>> On Tue, Sep 12, 2017 at 11:07 AM Greg Clayton <clayborg at gmail.com> wrote:
>>>> 
>>>>> On Sep 12, 2017, at 10:10 AM, Zachary Turner <zturner at google.com> wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> On Tue, Sep 12, 2017 at 10:03 AM Greg Clayton <clayborg at gmail.com> wrote:
>>>>>> On Sep 12, 2017, at 9:53 AM, Zachary Turner <zturner at google.com> wrote:
>>>>>> 
>>>>>> If you had just logged it, the bug would still not be fixed because nobody would know about it.  I also can't believe we have to keep saying this :-/
>>>>> 
>>>>> By log, I mean Host::SystemLog(...) which would come out in the command line. Not "log enable ...". So users would see the issue and report the bug. Crashing doesn't mean people always report the bug.
>>>>> I mentioned earlier in the thread that I assumed Xcode had an automatic crash that would handle the crash and automatically upload it to Apple.  Is this really not the case?  If core dumps are too big, why not just a stack trace?  Surely the Xcode team must have some kind of internal metrics system to track stability.
>>>> 
>>>> They do just upload text crash logs. It doesn't tell us what expression triggered the issue though. It shows a crash in an expression, but doesn't show the expression text as this violates privacy.
>>>> 
>>>> So, you do get a bug report when a crash occurs then.  In contrast to the case where you simply log something, and don't get a crash report.
>>>> 
>>>> In some cases, you can look at the code and figure out why it crashed.  In other cases the bug occurs extremely infrequently (you can build heuristic matching of call-stacks into your infrastructure that processes the crash logs).  If it's a high incidence crasher then you do some investigation.  And the good news is, once you fix it, you've *actually* fixed it.  Now instead of hundreds of thousands of people using something that doesn't work quite right for presumably the rest of the software's life (since nobody knows about the bug), they have something that actually works.
>>>> 
>>>> There's probably some initial pain associated with this approach since the test coverage is so low right now (I came up with about ~25% code coverage in a test I ran a while back).  When you get this higher, your tests start catching all of the high incidence stuff, and then you're left with only occasional crashes.
>>>> 
>>>> And since you have the out of process stuff, it doesn't even bring down Xcode anymore.  Just a debugging session.  That's an amazing price to pay for having instant visibility into a huge class of bugs that LLDB is currently willfully ignoring.
>>> 
>>> 
>>> I guess I'm speaking at someone who is supporting a debugger used by tens of thousands of people.  The debugger crashing is a vastly bigger problem to us than bugs that didn't announce themselves by taking down the debugger.  lldb_asserts that activate when we're running debug builds are fine, that's exactly when I want to see the debugger crash on my desktop or when it's running the testsuite.  Asserting when it's on the customer's desktop is a no-go.
>> 
>