[PATCH] [lld][Driver] Parallelize reading initial object files.

[kledzik at apple.com]

================
Comment at: lib/Driver/Driver.cpp:53-67
@@ +52,17 @@
+
+    tg.spawn([&, index] {
+      auto buffer = input.getBuffer();
+      if (error_code ec = buffer) {
+        diagnostics << "Failed to read file: " << input.getPath()
+                    << ": " << ec.message() << "\n";
+        fail = true;
+        return;
+      }
+
+      std::unique_ptr<MemoryBuffer> buff(MemoryBuffer::getMemBuffer(
+          buffer->getBuffer(), buffer->getBufferIdentifier()));
+      if (error_code ec = targetInfo.parseFile(buff, files[index])) {
+        diagnostics << "Failed to read file: " << input.getPath()
+                    << ": " << ec.message() << "\n";
+        fail = true;
+        return;
----------------
Michael Spencer wrote:
> kledzik at apple.com wrote:
> > Is there a reason why the high level readFile() in the original was split up into its two low level steps of getBuffer() and parseFile()? Using the low level steps makes this code more complicated.
> > 
> > Also, if multiple files have errors, you can get intermixed error message since you have lots of threads writing to diagnostics at once.  Another way would be to have a parallel array of error_codes and each thread/lambda just sets its error code in ec[index] and returns.  Then after the tg.sync(), you walk the ec array in order and if there is an error, you write it out.  That way every run results in the same error messages in the same order, and you don't need the atomic "fail" anymore (unless  you want it to terminate early).
> It uses getBuffer and parseFile so that we don't mmap the file multiple times.
> 
> Good idea for synchronizing the errors.
The multiple mappings is independent of this parallelization (i.e. could be separate patch). We should overload readFile() to take a LinkerInput or a path.  

Sometimes the reader needs to take ownership of the buffer and sometimes it does not.  Which is why parseFile() takes the unique_ptr<MemoryBuffer> by reference.  This intermediate getMemBuffer() object muddies the water. 


http://llvm-reviews.chandlerc.com/D664