r261774 - Bail on compilation as soon as a job fails.
Nico Weber via cfe-commits
cfe-commits at lists.llvm.org
Mon Feb 29 10:43:45 PST 2016
On Sun, Feb 28, 2016 at 3:40 PM, Justin Lebar <jlebar at google.com> wrote:
> On Sun, Feb 28, 2016 at 1:46 PM, Nico Weber <thakis at chromium.org> wrote:
> > Do you think something like the implicit inputs thing in
> > http://reviews.llvm.org/D17695 could work for you as well, instead of
> this
> > patch?
>
> Having read just the patch description, I think it would be workable,
> although it might not be particularly clean. I think you'd have to
> make most (all?) of the intermediate actions associated with a
> compilation be implicit inputs to all later actions in all other
> compilations. Otherwise things like saving vs. not saving temps would
> give you different behavior.
>
Can you expand on this? The idea is that those implicit deps are in
addition to regular dependencies, not a replacement.
> It seems like kind of a fragile way to get this behavior.
>
> > Then we don't have to forever promise to compile all .cc input files
> > serially.
>
> I've thought about this some, since we don't really want this even for
> CUDA. With CUDA, you commonly want to compile the device code for
> different architectures, and it would be reasonable to do those
> compilations in parallel.
>
> What I think may be a reasonable promise, at least for CUDA, is that
> we will behave *as if* we're compiling in series. Which just means
> not interleaving diagnostics from different compilations, and not
> showing diagnostics from other subcompilations after we hit an error.
>
> I think basically nobody wants randomly-interleaved diagnostics.
> Whether or there's a good use-case where we continue after we hit an
> error is a harder question, I'm not sure. But in any case, stopping
> after an error shouldn't mean we're forced to serialize, I think.
>
One problem I have with this change is that it breaks part of my patch ;-)
I'd like to run two commands, one to build a pch, and another to compile a
source file using the just-built pch, and if the first fails I don't want
to run the second: `make_pch && compile`. This part still works after your
patch. But we also have a flag /fallback that says "if compilation falls,
try again with this other compiler. Ideally I want `(make_pch && compile)
|| fallback_compile`, but since it's a bit of a corner case and arguably
good enough, my patch did `make_pch ; compile || fallback`. Your change
makes it impossible to run commands after one another if one fails, so now
this is `make_pch && (compile || fallback)`, i.e. if compilation of the pch
fails the fallback compiler won't be invoked.
I can try looking at maybe making FallbackCommand a FallbackAction instead
or something. But since CUDA seems to not fit the internal model of the
driver super well (lots of isCUDA() calls in many places), maybe it'd make
sense to discuss what your requirements are and if it's possible to extend
the Action / Job / Command abstractions in a way that support CUDA without
as many special cases. I happen to be in Mountain View – if you're in the
office tomorrow, maybe we could discuss this for half an hour or so in
person?
>
> -Justin
>
> > On Wed, Feb 24, 2016 at 4:49 PM, Justin Lebar via cfe-commits
> > <cfe-commits at lists.llvm.org> wrote:
> >>
> >> Author: jlebar
> >> Date: Wed Feb 24 15:49:28 2016
> >> New Revision: 261774
> >>
> >> URL: http://llvm.org/viewvc/llvm-project?rev=261774&view=rev
> >> Log:
> >> Bail on compilation as soon as a job fails.
> >>
> >> Summary:
> >> (Re-land of r260448, which was reverted in r260522 due to a test failure
> >> in Driver/output-file-cleanup.c that only showed up in fresh builds.)
> >>
> >> Previously we attempted to be smart; if one job failed, we'd run all
> >> jobs that didn't depend on the failing job.
> >>
> >> Problem is, this doesn't work well for e.g. CUDA compilation without
> >> -save-temps. In this case, the device-side and host-side Assemble
> >> actions (which actually are responsible for preprocess, compile,
> >> backend, and assemble, since we're not saving temps) are necessarily
> >> distinct. So our clever heuristic doesn't help us, and we repeat every
> >> error message once for host and once for each device arch.
> >>
> >> The main effect of this change, other than fixing CUDA, is that if you
> >> pass multiple cc files to one instance of clang and you get a compile
> >> error, we'll stop when the first cc1 job fails.
> >>
> >> Reviewers: echristo
> >>
> >> Subscribers: cfe-commits, jhen, echristo, tra, rafael
> >>
> >> Differential Revision: http://reviews.llvm.org/D17217
> >>
> >> Modified:
> >> cfe/trunk/lib/Driver/Compilation.cpp
> >> cfe/trunk/test/Driver/output-file-cleanup.c
> >>
> >> Modified: cfe/trunk/lib/Driver/Compilation.cpp
> >> URL:
> >>
> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Compilation.cpp?rev=261774&r1=261773&r2=261774&view=diff
> >>
> >>
> ==============================================================================
> >> --- cfe/trunk/lib/Driver/Compilation.cpp (original)
> >> +++ cfe/trunk/lib/Driver/Compilation.cpp Wed Feb 24 15:49:28 2016
> >> @@ -163,39 +163,17 @@ int Compilation::ExecuteCommand(const Co
> >> return ExecutionFailed ? 1 : Res;
> >> }
> >>
> >> -typedef SmallVectorImpl< std::pair<int, const Command *> >
> >> FailingCommandList;
> >> -
> >> -static bool ActionFailed(const Action *A,
> >> - const FailingCommandList &FailingCommands) {
> >> -
> >> - if (FailingCommands.empty())
> >> - return false;
> >> -
> >> - for (FailingCommandList::const_iterator CI = FailingCommands.begin(),
> >> - CE = FailingCommands.end(); CI != CE; ++CI)
> >> - if (A == &(CI->second->getSource()))
> >> - return true;
> >> -
> >> - for (const Action *AI : A->inputs())
> >> - if (ActionFailed(AI, FailingCommands))
> >> - return true;
> >> -
> >> - return false;
> >> -}
> >> -
> >> -static bool InputsOk(const Command &C,
> >> - const FailingCommandList &FailingCommands) {
> >> - return !ActionFailed(&C.getSource(), FailingCommands);
> >> -}
> >> -
> >> -void Compilation::ExecuteJobs(const JobList &Jobs,
> >> - FailingCommandList &FailingCommands)
> const
> >> {
> >> +void Compilation::ExecuteJobs(
> >> + const JobList &Jobs,
> >> + SmallVectorImpl<std::pair<int, const Command *>> &FailingCommands)
> >> const {
> >> for (const auto &Job : Jobs) {
> >> - if (!InputsOk(Job, FailingCommands))
> >> - continue;
> >> const Command *FailingCommand = nullptr;
> >> - if (int Res = ExecuteCommand(Job, FailingCommand))
> >> + if (int Res = ExecuteCommand(Job, FailingCommand)) {
> >> FailingCommands.push_back(std::make_pair(Res, FailingCommand));
> >> + // Bail as soon as one command fails, so we don't output
> duplicate
> >> error
> >> + // messages if we die on e.g. the same file.
> >> + return;
> >> + }
> >> }
> >> }
> >>
> >>
> >> Modified: cfe/trunk/test/Driver/output-file-cleanup.c
> >> URL:
> >>
> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/output-file-cleanup.c?rev=261774&r1=261773&r2=261774&view=diff
> >>
> >>
> ==============================================================================
> >> --- cfe/trunk/test/Driver/output-file-cleanup.c (original)
> >> +++ cfe/trunk/test/Driver/output-file-cleanup.c Wed Feb 24 15:49:28 2016
> >> @@ -38,6 +38,9 @@ invalid C code
> >> // RUN: test -f %t1.s
> >> // RUN: test ! -f %t2.s
> >>
> >> +// When given multiple .c files to compile, clang compiles them in
> order
> >> until
> >> +// it hits an error, at which point it stops.
> >> +//
> >> // RUN: touch %t1.c
> >> // RUN: echo "invalid C code" > %t2.c
> >> // RUN: touch %t3.c
> >> @@ -46,6 +49,6 @@ invalid C code
> >> // RUN: cd %T && not %clang -S %t1.c %t2.c %t3.c %t4.c %t5.c
> >> // RUN: test -f %t1.s
> >> // RUN: test ! -f %t2.s
> >> -// RUN: test -f %t3.s
> >> +// RUN: test ! -f %t3.s
> >> // RUN: test ! -f %t4.s
> >> -// RUN: test -f %t5.s
> >> +// RUN: test ! -f %t5.s
> >>
> >>
> >> _______________________________________________
> >> cfe-commits mailing list
> >> cfe-commits at lists.llvm.org
> >> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20160229/1074f4ea/attachment-0001.html>
More information about the cfe-commits
mailing list