[llvm-dev] lld: sigbus error handling

[Rui Ueyama via llvm-dev]

On Mon, Oct 23, 2017 at 7:49 PM, Andrew Kelley <superjoe30 at gmail.com> wrote:

>
>
> On Mon, Oct 23, 2017 at 9:48 PM, Rui Ueyama <ruiu at google.com> wrote:
>
>> On Mon, Oct 23, 2017 at 6:40 PM, Andrew Kelley <superjoe30 at gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Mon, Oct 23, 2017 at 9:30 PM, Rui Ueyama <ruiu at google.com> wrote:
>>>
>>>> On Mon, Oct 23, 2017 at 5:28 PM, Andrew Kelley <superjoe30 at gmail.com>
>>>> wrote:
>>>>
>>>>> For Zig we use LLD as a library. So for us it would be better to avoid
>>>>> global state such as SIGBUS (or any other signal handlers), instead
>>>>> returning an error from the link function when linking fails. If lld can
>>>>> encapsulate this signal handling and prevent the application using lld from
>>>>> getting the signal directly, instead carefully handling the signal in LLD
>>>>> itself and translating it into a proper error code or message, this would
>>>>> be reasonable.
>>>>>
>>>>
>>>> Signal handlers and signal masks are inherently process-wide, so
>>>> there's no way to encapsulate them to lld functions. So my plan is to
>>>> change the name of lld::{coff,elf}::link's `ExitEarly` parameter to
>>>> `IsStandalone`, and we (not only call exit(2) but also) set a signal
>>>> handler only when the argument is true. Since library users pass false to
>>>> the parameter, it shouldn't change the behavior of lld for the library use
>>>> case.
>>>>
>>>
>>> This sounds good to me.
>>>
>>> And then if an application wants to handle the SIGBUS correctly, it
>>> would have to register this signal handler to report the error?
>>>
>>
>> I could export a function that sets a signal handler as part of the
>> library interface, but I'm reluctant to do that because you can do the same
>> thing in a few lines of C code.
>>
>> Can I ask you a question? I wonder if there's a reason to not call fork
>> before calling lld's main function.
>>
>
> It's starting to look to me like that might be necessary to integrate with
> LLD, if I want to handle this SIGBUS error. I'd like to handle out of disk
> space gracefully and not crash.
>
> Here are my concerns:
>
>  * My frontend is cross-platform, so I would also need to figure out how
> this would work on Windows, and eventually on other OSes too. Installation
> of my frontend is simpler if there is not more than 1 executable to
> distribute.
>  * This isn't working right now, but I want the errors of linking to
> provide meaningful error codes and other metadata in a format the frontend
> can associate with  its own state and and handle/render the errors in its
> own way. If this is done via IPC it has to go through a
> serialization/deserialization step.
>  * One of the use cases for my frontend is as a server where it may invoke
> the linker repeatedly. I haven't tested the difference in performance or
> resource usage yet, but it seems to me that LLD as a library / no forking
> would be more efficient.
>  * I would rather compile against LLVM  and clang statically for
> performance and ease of installation (at least on Windows). If LLD is a
> separate binary it would additionally need LLVM/Clang linked in statically,
> and this is a duplicate copy of LLVM/Clang, doubling the size of my
> compiler releases. Further the LLVM initialization code can be shared
> between my frontend and LLD when linked into the same binary.
>

Thanks Andrew for writing this up. Your feedback as an actual user of
lld-as-a-library is very valuable to me. I think I understand that you want
to avoid using fork for the various reasons, and I agree with all these
points. Here are my random thoughts.

 - As to SIGBUS, we can call posix_fallocate to allocate disk space on any
file system, so that no SIGBUS will be raised later. We can do that only
when "IsStandalone" is false.
 - Orthogonal to that, you can execute lld as a separate process, while
keeping your distribution a single binary. You can add a hidden flag to
your command line interface, and when that specific flag is given you can
call lld's main function so that the process acts as lld.
 - For a long running server process, I might still want to run lld as a
separate process, so that lld's bug wouldn't crash the entire server
process.


>
>
>>
>>
>>>
>>>>
>>>> On Mon, Oct 23, 2017 at 6:21 PM, Rui Ueyama via llvm-dev <
>>>>> llvm-dev at lists.llvm.org> wrote:
>>>>>
>>>>>> If your system does not support fallocate(2), we use ftruncate(2) to
>>>>>> create an output file. fallocate(2) succeeds even if your disk have less
>>>>>> space than the requested size, because it creates a sparse file. If you
>>>>>> mmap such sparse file, you'll receive a SIGBUS when the disk actually
>>>>>> becomes full.
>>>>>>
>>>>>> So, lld can die suddenly with SIGBUS when your disk becomes full, and
>>>>>> currently we are not doing anything about it. It's sometimes hard to notice
>>>>>> that that was caused by the lack of disk space.
>>>>>>
>>>>>> I wonder if we should print out a hint (e.g. "Bus error -- disk
>>>>>> full?") when we receive a SIGBUS. Any opinions?
>>>>>>
>>>>>> _______________________________________________
>>>>>> LLVM Developers mailing list
>>>>>> llvm-dev at lists.llvm.org
>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20171023/47ef061f/attachment.html>