[llvm-dev] Ninja build (on Windows anyway) may be doing redundant work

[Nico Weber via llvm-dev]

It depends on if ninja is writing to a interactive terminal or no. If it's
writing to an interactive terminal, it prints the "[N/M] building foobar"
line when it starts running it, and then it prints it again followed by the
subprocesses output when it's done running it (usually with a different
value for N). If it's writing to a buildbot log, then it only prints the
line when it's done running the command, followed by the command's output.

On Tue, Nov 20, 2018 at 1:32 PM <paul.robinson at sony.com> wrote:

> So when ninja says "[N/M] Building foobar" that means it's actually *all
> done* building foobar.  Excellent.
>
> Re. the double message, "blame Microsoft" as I'm not motivated to dig into
> it any further.
>
> Thanks for the info!  It did help.
>
> --paulr
>
>
>
> *From:* Nico Weber [mailto:thakis at chromium.org]
> *Sent:* Tuesday, November 20, 2018 9:49 AM
> *To:* Robinson, Paul
> *Cc:* llvm-dev
> *Subject:* Re: [llvm-dev] Ninja build (on Windows anyway) may be doing
> redundant work
>
>
>
> Ninja makes subprocesses write to a pipe and prints their output preceded
> by the [N/M] line before printing their output the process is done, to not
> get interleaving output from parallel processes and to make it easy to see
> which process a given output belongs to. So link.exe definitely prints this
> line twice.
>
>
>
> On Tue, Nov 20, 2018 at 9:28 AM <paul.robinson at sony.com> wrote:
>
> (resend to the list)
>
> And of course, just as I say that, my next ninja build shows the line only
> once.
>
> On reflection I am less sure that the lack of a [N/M] line means they are
> from the same invocation. Surely ninja could spawn two links, which then
> independently report "Creating library" after ninja emits the [N/M] lines.
>
> --paulr
>
>
>
>
>
> *From:* llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] *On Behalf Of *via
> llvm-dev
> *Sent:* Tuesday, November 20, 2018 9:01 AM
> *To:* thakis at chromium.org
> *Cc:* llvm-dev at lists.llvm.org
> *Subject:* Re: [llvm-dev] Ninja build (on Windows anyway) may be doing
> redundant work
>
>
>
> OK, worth knowing.  Thanks.
>
> --paulr
>
>
>
> *From:* Nico Weber [mailto:thakis at chromium.org]
> *Sent:* Monday, November 19, 2018 7:11 PM
> *To:* Robinson, Paul
> *Cc:* Zachary Turner; llvm-dev
> *Subject:* Re: [llvm-dev] Ninja build (on Windows anyway) may be doing
> redundant work
>
>
>
> Since there's no "[2663/3121] " line between the two messages, the two
> lines are from the same link.exe invocation. I don't know why link.exe
> thinks it needs to print this line twice, ninja doesn't have anything to do
> with it.
>
>
>
> On Mon, Nov 19, 2018 at 6:57 PM <paul.robinson at sony.com> wrote:
>
> I'm more concerned about seeing the message come out twice, which implies
> Ninja is executing a rule twice when it doesn't need to.
>
> --paulr
>
>
>
> *From:* Nico Weber [mailto:thakis at chromium.org]
> *Sent:* Monday, November 19, 2018 6:09 PM
> *To:* Zachary Turner
> *Cc:* Robinson, Paul; llvm-dev
> *Subject:* Re: [llvm-dev] Ninja build (on Windows anyway) may be doing
> redundant work
>
>
>
> Do you still see this if you use lld-link for linking?
>
>
>
> The "corrupt obj file" is something we saw on chrome's bots every now and
> then before we switched to lld.
>
>
>
> On Mon, Nov 19, 2018 at 5:27 PM Zachary Turner <zturner at google.com> wrote:
>
> +Nico Weber <thakis at google.com>
>
> On Mon, Nov 19, 2018 at 12:25 PM via llvm-dev <llvm-dev at lists.llvm.org>
> wrote:
>
> This afternoon I did a clean build of llvm using ninja and MSVC 2015 on
> Windows 10. I saw something curious and wondered if any of the ninja or
> CMake experts out there can explain/fix it.
>
> [2663/3121] Linking CXX shared library bin\LTO.dll
>    Creating library lib\LTO.dll and object lib\LTO.exp
>    Creating library lib\LTO.dll and object lib\LTO.exp
> [3120/3121] Running the LLVM regression tests
> [[[ etc ]]]
>
> I'm curious about the "creating library" message popping out twice.
> Sometimes ninja builds will die on me, complaining about a corrupted
> object file, usually related to LTO. Rerunning ninja usually completes
> successfully.  So I wonder if there's some mishap in the dependencies
> somewhere that actually causes the build to try to link the DLL twice,
> which of course might corrupt a file if the links are overlapping in
> time.
>
> Thought I'd throw this out there in case someone wants to take a look.
> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181120/d7c6c4f3/attachment.html>