[cfe-dev] Why clang needs to fork into itself?

[Manuel Klimek]

On Wed, Jan 29, 2014 at 8:58 AM, Richard Smith <richard at metafoo.co.uk>wrote:

> On Tue, Jan 28, 2014 at 11:40 PM, Manuel Klimek <klimek at google.com> wrote:
>
>> On Wed, Jan 29, 2014 at 8:05 AM, Ted Kremenek <kremenek at apple.com> wrote:
>>
>>> If this can be done, then great.
>>>
>>> Yury Gribov’s point about stack smashing is a good one.  We implemented
>>> such crash recovery mechanisms in libclang and libclang still takes down
>>> Xcode sometimes because of stack overflows due to unbounded recursion, etc.
>>>  We’ve also noticed that when libclang “crashes” (and recovers) the overall
>>> process can be in an undefined state.  Our experience is that such
>>> histrionics can provide an 80% solution, but we’ve never been all that
>>> satisfied with them.  It may, however, be good enough for generating crash
>>> reports, but it seems like a lot of work to replace something we already
>>> have that works very well in practice.
>>>
>>
>> One thing I'd be curious about:
>> What would be the downside of using the parent clang *just* as crash
>> reporter - and push all driver logic into the spawned process. The parent
>> clang would then forward all command line arguments literally, and then
>> just sit there, waiting for the crash, and if there is a crash scrap up all
>> it needs for the bug report.
>> That way, we could easily turn off forking and have equivalent behavior
>> minus crash reporting.
>> Is there a downside to that that I'm missing (apart from the time needed
>> to implement it)?
>>
>
> We still lose all the simplicity benefits of having the driver run one
> subprocess per compilation, and we'd need to turn off -disable-free in the
> multiple-files case, and we'd need to worry about having a large VMSize if
> we fork to spawn another process (for instance, the linker). Having a
> separate process for the compiler from the driver is a much cleaner model.
>
> That said, most users of the compiler don't actually *want* a driver. They
> want just a .c* -> .o compiler, not a multi-source-file cross-language
> compiler + assembler + linker + kitchen sink, and it makes sense to me to
> optimize for the single-source, -c case.
>

Reiterating my question from earlier: how does that model match with the
recursive compilation step for modules in CompilerInstance.cpp? Will the
-disable-free basically kill our memory if we are in a larger module
dependency chain that needs to be rebuilt? Do we somehow disable recursive
modules compilation in that mode? Or do you think nobody will hit that
case? (I actually hit it "by chance" the first time I was playing around
with modules, without actively wanting it, and was quite surprised by what
happened ;)

I agree that most people nowadays just want .c* -> .o compiler, but if I
look at the complexity of what Java grew as part of their module system
(where many people also then basically dump it down to a simple .java ->
.class compiler), I'd be interested to see a plan of how we imagine this to
look down the road.


>
>  On Jan 28, 2014, at 10:28 PM, Yuri <yuri at rawbw.com> wrote:
>>>
>>> > On 01/28/2014 22:04, Yury Gribov wrote:
>>> >>
>>> >> Makes sense but what if some important bits (say argv) are trashed by
>>> stack overflow
>>> >
>>> > All information needed for crash reporting should be copied into the
>>> fixed memory area, and it should be made read-only for the duration of run.
>>> >
>>> > Yuri
>>> > _______________________________________________
>>> > cfe-dev mailing list
>>> > cfe-dev at cs.uiuc.edu
>>> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>
>>>
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at cs.uiuc.edu
>>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>>
>>
>>
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140129/ff1df904/attachment.html>