[cfe-dev] Why clang needs to fork into itself?

[Richard Smith]

On Wed Jan 29 2014 at 11:37:22 AM, Alp Toker <alp at nuanti.com> wrote:

>
> On 29/01/2014 18:15, Ted Kremenek wrote:
> > On Jan 29, 2014, at 9:55 AM, Richard Smith <metafoo at gmail.com
> > <mailto:metafoo at gmail.com>> wrote:
> >
> >> On Wed Jan 29 2014 at 12:35:23 AM, Ted Kremenek <kremenek at apple.com
> >> <mailto:kremenek at apple.com>> wrote:
> >>
> >>     On Jan 28, 2014, at 11:58 PM, Richard Smith
> >>     <richard at metafoo.co.uk <mailto:richard at metafoo.co.uk>> wrote:
> >>
> >>>     That said, most users of the compiler don't actually *want* a
> >>>     driver. They want just a .c* -> .o compiler, not a
> >>>     multi-source-file cross-language compiler + assembler + linker +
> >>>     kitchen sink, and it makes sense to me to optimize for the
> >>>     single-source, -c case.
> >>
> >>     I don't really know if it matters what most users "want." We
> >>     still need to support the kitchen sink model; it's never going to
> >>     go away.
> >>
> >>
> >> I think it does matter, because it helps us set our priorities. In
> >> particular, if we can make single-source -c builds measurably faster,
> >> that has benefits for a lot of users, even if we leave all other
> >> modes alone.
> >>
> >>     By "optimize", what specifically do you propose/suggest?
> >>
> >>
> >> Nothing concrete, merely that we shouldn't dismiss this out of hand.
> >> For instance, if we can tell people to run -c --disable-fork for a
> >> measurable speedup (with a corresponding degradation in crash
> >> recovery), that might be valuable to some.
> >
> > I agree that making single source builds faster is a case worth
> > optimizing.
>
> Hi Richard, Ted,
>
> I have a very simple solution to get started with:
>
> + bool TryInProcess = getArgs().hasArg(options::OPT_
> fno_crash_diagnostics);
> + if (TryInProcess && C.getExecutable() ==
> getDriver().getClangProgramPath()) {
> + const ArgStringList &Arguments = C.getArguments();
> ...
> + Res = main(Argv.size() - 1, Argv.data());
> + } else
>
> This passes all but one of the tests (which appears to be a genuine bug
> in the preprocessor that I'm looking to fix now).
>
> All -cc1, -cc1as, -E and a couple of other invocations get run
> in-process with no apparent regressions.
>
> This setup parallels the way we skip expensive checks in Parse and Sema
> when we know a diagnostic is ignored and it shouldn't have any
> observable effect to the user other than fewer PIDs and invocations.
>
> I'd like to move forward with this so we can start to experiment with
> in-process operation focusing on the single input case Richard mentioned.
>
> It's not particularly hi-tech, but that's what makes it kind of neat.
> What do you think?
>

I'd be surprised if it works as-is (I'd expect we have globals that
misbehave when treated this way). Also, invoking 'main' in this way is
ill-formed, so we'd need to extract a 'clang_recursive_main' or similar
(and mark it __attribute__((weak)) so we don't require tools to provide it).

This seems like a good way forward for extracting performance numbers, and
on the basis of those we can decide if this is worth pursuing. I don't
think it's worth upstreaming until you have such numbers (maybe you could
mail out a patch so interested parties can profile it?).


> Alp.
>
>
> >
> >>     Even the .c -> .o compiler benefits from process isolation for
> >>     the purpose of crash reporting. Perhaps there are alternate
> >>     solutions for reporting crashes, but they are a bit speculative
> >>     at this point.
> >>
> >>     I also haven't seen any numbers mentioned on this thread that
> >>     justify "optimizing" anything, at least from a performance
> >>     perspective. I'm not saying there isn't a possible performance
> >>     win here by eliminating the posix_spawn call, but that hasn't
> >>     been justified yet (or if it has, I apologize that I haven't seen
> >>     it). Or perhaps this discussion isn't about optimizing
> >>     performance, but refining the driver design. In that case, it's
> >>     not clear to me what specific problems are being tackled here
> >>     other than possibly making the compiler a little easier to debug
> >>     or profile.
> >>
> >>
> >> I think the changes proposed on this thread will make the driver
> >> design more complex. I'd also like to see some numbers.
> >
> > Agreed.
> >
> >
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/cfe-dev
>
> --
> http://www.nuanti.com
> the browser experts
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-dev/attachments/20140129/c7a002b5/attachment.html>