[llvm-dev] Ninja build (on Windows anyway) may be doing redundant work

[via llvm-dev]

So when ninja says "[N/M] Building foobar" that means it's actually *all done* building foobar.  Excellent.
Re. the double message, "blame Microsoft" as I'm not motivated to dig into it any further.
Thanks for the info!  It did help.
--paulr

From: Nico Weber [mailto:thakis at chromium.org]
Sent: Tuesday, November 20, 2018 9:49 AM
To: Robinson, Paul
Cc: llvm-dev
Subject: Re: [llvm-dev] Ninja build (on Windows anyway) may be doing redundant work

Ninja makes subprocesses write to a pipe and prints their output preceded by the [N/M] line before printing their output the process is done, to not get interleaving output from parallel processes and to make it easy to see which process a given output belongs to. So link.exe definitely prints this line twice.

On Tue, Nov 20, 2018 at 9:28 AM <paul.robinson at sony.com<mailto:paul.robinson at sony.com>> wrote:
(resend to the list)
And of course, just as I say that, my next ninja build shows the line only once.
On reflection I am less sure that the lack of a [N/M] line means they are from the same invocation. Surely ninja could spawn two links, which then independently report "Creating library" after ninja emits the [N/M] lines.
--paulr


From: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org<mailto:llvm-dev-bounces at lists.llvm.org>] On Behalf Of via llvm-dev
Sent: Tuesday, November 20, 2018 9:01 AM
To: thakis at chromium.org<mailto:thakis at chromium.org>
Cc: llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
Subject: Re: [llvm-dev] Ninja build (on Windows anyway) may be doing redundant work

OK, worth knowing.  Thanks.
--paulr

From: Nico Weber [mailto:thakis at chromium.org<mailto:thakis at chromium.org>]
Sent: Monday, November 19, 2018 7:11 PM
To: Robinson, Paul
Cc: Zachary Turner; llvm-dev
Subject: Re: [llvm-dev] Ninja build (on Windows anyway) may be doing redundant work

Since there's no "[2663/3121] " line between the two messages, the two lines are from the same link.exe invocation. I don't know why link.exe thinks it needs to print this line twice, ninja doesn't have anything to do with it.

On Mon, Nov 19, 2018 at 6:57 PM <paul.robinson at sony.com<mailto:paul.robinson at sony.com>> wrote:
I'm more concerned about seeing the message come out twice, which implies Ninja is executing a rule twice when it doesn't need to.
--paulr

From: Nico Weber [mailto:thakis at chromium.org<mailto:thakis at chromium.org>]
Sent: Monday, November 19, 2018 6:09 PM
To: Zachary Turner
Cc: Robinson, Paul; llvm-dev
Subject: Re: [llvm-dev] Ninja build (on Windows anyway) may be doing redundant work

Do you still see this if you use lld-link for linking?

The "corrupt obj file" is something we saw on chrome's bots every now and then before we switched to lld.

On Mon, Nov 19, 2018 at 5:27 PM Zachary Turner <zturner at google.com<mailto:zturner at google.com>> wrote:
+Nico Weber<mailto:thakis at google.com>
On Mon, Nov 19, 2018 at 12:25 PM via llvm-dev <llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>> wrote:
This afternoon I did a clean build of llvm using ninja and MSVC 2015 on
Windows 10. I saw something curious and wondered if any of the ninja or
CMake experts out there can explain/fix it.

[2663/3121] Linking CXX shared library bin\LTO.dll
   Creating library lib\LTO.dll and object lib\LTO.exp
   Creating library lib\LTO.dll and object lib\LTO.exp
[3120/3121] Running the LLVM regression tests
[[[ etc ]]]

I'm curious about the "creating library" message popping out twice.
Sometimes ninja builds will die on me, complaining about a corrupted
object file, usually related to LTO. Rerunning ninja usually completes
successfully.  So I wonder if there's some mishap in the dependencies
somewhere that actually causes the build to try to link the DLL twice,
which of course might corrupt a file if the links are overlapping in
time.

Thought I'd throw this out there in case someone wants to take a look.
--paulr

_______________________________________________
LLVM Developers mailing list
llvm-dev at lists.llvm.org<mailto:llvm-dev at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20181120/bb59467b/attachment.html>