[lldb-dev] The lit test driver gets killed because of SIGHUP

[Jonas Devlieghere via lldb-dev]

On Tue, Dec 4, 2018 at 22:03 Zachary Turner <zturner at google.com> wrote:

> Do you know if it’s Darwin specific? If so, maybe someone internally can
> offer guidance on how to diagnose (like on the kernel team)?
>

Finding that out is part of the reason I sent this mail. We’ve only seen it
on Mac Pros and iMac Pros. I haven’t tried on Linux yet. If someone with a
specced machine would have a go at the the command I sent earlier that
would be really appreciated. I can try myself in a VM tomorrow, but if it
doesn’t reproduce it’ll be bad to tell whether it was because of the VM or
not.

When you aren’t using the lit driver, does the signal still get delivered
> (and we just handle it better), or does it not get delivered at all?


I wasn’t able to reproduce with dotest.py or the lldb-dotest wrapper. I
installed a signal handler for SIGHUP and it never triggered. Since the
signal doesn’t show up in my DTrace output I can’t be sure if it actually
reproduces, but the handler doesn’t execute. It also doesn’t fire in the
lit case, which adds to my confusion.


> On Tue, Dec 4, 2018 at 9:12 PM Jonas Devlieghere <jonas at devlieghere.com>
> wrote:
>
>>
>>
>> On Tue, Dec 4, 2018 at 19:11 Zachary Turner via lldb-dev <
>> lldb-dev at lists.llvm.org> wrote:
>>
>>> Have you tried an strace to see if it tells you who is sending the
>>> signal?
>>
>>
>> I used DTrace with the default kill.d script. It shows who sends what
>> signal and there was nothing interesting other than debugserver sending
>> signal 17 (SIGSTOP) to the inferior. This makes me think that the signal
>> might be coming from the kernel?
>>
>>
>>> On Tue, Dec 4, 2018 at 6:49 PM Jonas Devlieghere via lldb-dev <
>>> lldb-dev at lists.llvm.org> wrote:
>>>
>>>> Hi everyone,
>>>>
>>>> Since we switched to lit as the test driver we've been seeing it
>>>> getting killed as the result of a SIGHUP signal. The problem doesn't
>>>> reproduce on every machine and there seems to be a correlation between
>>>> number of occurrences and thread count.
>>>>
>>>> Davide and Raphael spent some time narrowing down what particular test
>>>> is causing this and it seems that TestChangeProcessGroup.py is always
>>>> involved. However it never reproduces when running just this test. I was
>>>> able to reproduce pretty consistently with the following filter:
>>>>
>>>> ./bin/llvm-lit ../llvm/tools/lldb/lit/Suite/ --filter="process"
>>>>
>>>> Bisecting the test itself didn't help much, the problem reproduces as
>>>> soon as we attach to the inferior.
>>>>
>>>> At this point it is still not clear who is sending the SIGHUP and why
>>>> it's reaching the lit test driver. Fred suggested that it might have
>>>> something to do with process groups (which would be an interesting
>>>> coincidence given the previously mentioned test) and he suggested having
>>>> the test run in different process groups. Indeed, adding a call to
>>>> os.setpgrp() in lit's executeCommand and having a different process group
>>>> per test prevent us from seeing this. Regardless of this issue I think it's
>>>> reasonable to have tests run in their process group, so if nobody objects I
>>>> propose adding this to lit in llvm.
>>>>
>>>> Still, I'd like to understand where the signal is coming from and fix
>>>> the root cause in addition to the symptom. Maybe someone here has an idea
>>>> of what might be going on?
>>>>
>>>> Thanks,
>>>> Jonas
>>>>
>>>> PS
>>>>
>>>> 1. There's two places where we send a SIGHUP ourself, with that code
>>>> removed we still receive the signal, which suggests that it might be coming
>>>> from Python or the OS.
>>>> 2. If you're able to reproduce you'll see that adding an early return
>>>> before the attach in TestChangeProcessGroup.py hides/prevents the problem.
>>>> Moving the return down one line and it pops up again.
>>>> _______________________________________________
>>>> lldb-dev mailing list
>>>> lldb-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>>
>>> _______________________________________________
>>> lldb-dev mailing list
>>> lldb-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>
>> --
>> Sent from my iPhone
>>
> --
Sent from my iPhone
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20181204/c8438d17/attachment.html>