r261774 - Bail on compilation as soon as a job fails.

Sun Feb 28 15:40:46 PST 2016

On Sun, Feb 28, 2016 at 1:46 PM, Nico Weber <thakis at chromium.org> wrote:
> Do you think something like the implicit inputs thing in
> http://reviews.llvm.org/D17695 could work for you as well, instead of this
> patch?

Having read just the patch description, I think it would be workable,
although it might not be particularly clean.  I think you'd have to
make most (all?) of the intermediate actions associated with a
compilation be implicit inputs to all later actions in all other
compilations.  Otherwise things like saving vs. not saving temps would
give you different behavior.

It seems like kind of a fragile way to get this behavior.

> Then we don't have to forever promise to compile all .cc input files
> serially.

I've thought about this some, since we don't really want this even for
CUDA.  With CUDA, you commonly want to compile the device code for
different architectures, and it would be reasonable to do those
compilations in parallel.

What I think may be a reasonable promise, at least for CUDA, is that
we will behave *as if* we're compiling in series.  Which just means
not interleaving diagnostics from different compilations, and not
showing diagnostics from other subcompilations after we hit an error.

I think basically nobody wants randomly-interleaved diagnostics.
Whether or there's a good use-case where we continue after we hit an
error is a harder question, I'm not sure.  But in any case, stopping
after an error shouldn't mean we're forced to serialize, I think.

-Justin

> On Wed, Feb 24, 2016 at 4:49 PM, Justin Lebar via cfe-commits
> <cfe-commits at lists.llvm.org> wrote:
>>
>> Author: jlebar
>> Date: Wed Feb 24 15:49:28 2016
>> New Revision: 261774
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=261774&view=rev
>> Log:
>> Bail on compilation as soon as a job fails.
>>
>> Summary:
>> (Re-land of r260448, which was reverted in r260522 due to a test failure
>> in Driver/output-file-cleanup.c that only showed up in fresh builds.)
>>
>> Previously we attempted to be smart; if one job failed, we'd run all
>> jobs that didn't depend on the failing job.
>>
>> Problem is, this doesn't work well for e.g. CUDA compilation without
>> -save-temps.  In this case, the device-side and host-side Assemble
>> actions (which actually are responsible for preprocess, compile,
>> backend, and assemble, since we're not saving temps) are necessarily
>> distinct.  So our clever heuristic doesn't help us, and we repeat every
>> error message once for host and once for each device arch.
>>
>> The main effect of this change, other than fixing CUDA, is that if you
>> pass multiple cc files to one instance of clang and you get a compile
>> error, we'll stop when the first cc1 job fails.
>>
>> Reviewers: echristo
>>
>> Subscribers: cfe-commits, jhen, echristo, tra, rafael
>>
>> Differential Revision: http://reviews.llvm.org/D17217
>>
>> Modified:
>>     cfe/trunk/lib/Driver/Compilation.cpp
>>     cfe/trunk/test/Driver/output-file-cleanup.c
>>
>> Modified: cfe/trunk/lib/Driver/Compilation.cpp
>> URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/Compilation.cpp?rev=261774&r1=261773&r2=261774&view=diff
>>
>> ==============================================================================
>> --- cfe/trunk/lib/Driver/Compilation.cpp (original)
>> +++ cfe/trunk/lib/Driver/Compilation.cpp Wed Feb 24 15:49:28 2016
>> @@ -163,39 +163,17 @@ int Compilation::ExecuteCommand(const Co
>>    return ExecutionFailed ? 1 : Res;
>>  }
>>
>> -typedef SmallVectorImpl< std::pair<int, const Command *> >
>> FailingCommandList;
>> -
>> -static bool ActionFailed(const Action *A,
>> -                         const FailingCommandList &FailingCommands) {
>> -
>> -  if (FailingCommands.empty())
>> -    return false;
>> -
>> -  for (FailingCommandList::const_iterator CI = FailingCommands.begin(),
>> -         CE = FailingCommands.end(); CI != CE; ++CI)
>> -    if (A == &(CI->second->getSource()))
>> -      return true;
>> -
>> -  for (const Action *AI : A->inputs())
>> -    if (ActionFailed(AI, FailingCommands))
>> -      return true;
>> -
>> -  return false;
>> -}
>> -
>> -static bool InputsOk(const Command &C,
>> -                     const FailingCommandList &FailingCommands) {
>> -  return !ActionFailed(&C.getSource(), FailingCommands);
>> -}
>> -
>> -void Compilation::ExecuteJobs(const JobList &Jobs,
>> -                              FailingCommandList &FailingCommands) const
>> {
>> +void Compilation::ExecuteJobs(
>> +    const JobList &Jobs,
>> +    SmallVectorImpl<std::pair<int, const Command *>> &FailingCommands)
>> const {
>>    for (const auto &Job : Jobs) {
>> -    if (!InputsOk(Job, FailingCommands))
>> -      continue;
>>      const Command *FailingCommand = nullptr;
>> -    if (int Res = ExecuteCommand(Job, FailingCommand))
>> +    if (int Res = ExecuteCommand(Job, FailingCommand)) {
>>        FailingCommands.push_back(std::make_pair(Res, FailingCommand));
>> +      // Bail as soon as one command fails, so we don't output duplicate
>> error
>> +      // messages if we die on e.g. the same file.
>> +      return;
>> +    }
>>    }
>>  }
>>
>>
>> Modified: cfe/trunk/test/Driver/output-file-cleanup.c
>> URL:
>> http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/output-file-cleanup.c?rev=261774&r1=261773&r2=261774&view=diff
>>
>> ==============================================================================
>> --- cfe/trunk/test/Driver/output-file-cleanup.c (original)
>> +++ cfe/trunk/test/Driver/output-file-cleanup.c Wed Feb 24 15:49:28 2016
>> @@ -38,6 +38,9 @@ invalid C code
>>  // RUN: test -f %t1.s
>>  // RUN: test ! -f %t2.s
>>
>> +// When given multiple .c files to compile, clang compiles them in order
>> until
>> +// it hits an error, at which point it stops.
>> +//
>>  // RUN: touch %t1.c
>>  // RUN: echo "invalid C code" > %t2.c
>>  // RUN: touch %t3.c
>> @@ -46,6 +49,6 @@ invalid C code
>>  // RUN: cd %T && not %clang -S %t1.c %t2.c %t3.c %t4.c %t5.c
>>  // RUN: test -f %t1.s
>>  // RUN: test ! -f %t2.s
>> -// RUN: test -f %t3.s
>> +// RUN: test ! -f %t3.s
>>  // RUN: test ! -f %t4.s
>> -// RUN: test -f %t5.s
>> +// RUN: test ! -f %t5.s
>>
>>
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>
>