[PATCH] [lld][Driver] Parallelize reading initial object files.

[Shankar Kalpathi Easwaran]

================
Comment at: lib/Driver/Driver.cpp:62-67
@@ +61,8 @@
+
+      std::unique_ptr<MemoryBuffer> buff(MemoryBuffer::getMemBuffer(
+          buffer->getBuffer(), buffer->getBufferIdentifier()));
+      if (error_code ec = targetInfo.parseFile(buff, files[index])) {
+        diagnostics << "Failed to read file: " << input.getPath()
+                    << ": " << ec.message() << "\n";
+        fail = true;
+        return;
----------------
Michael Spencer wrote:
> Shankar Kalpathi Easwaran wrote:
> > Isnt using index a problem with archive files.
> > 
> > --force-load-archive option / multiple archive files getting pulled in ?
> files is a std::vector<std::vector<std::unique_ptr<File>>>. Each input argument gets a std::vector.
ah ok.

================
Comment at: lib/Driver/Driver.cpp:47
@@ +46,3 @@
+  size_t index = 0;
+  std::atomic<bool> fail(false);
+  TaskGroup tg;
----------------
Michael Spencer wrote:
> Shankar Kalpathi Easwaran wrote:
> > Why you want this to be atomic bool ? because you are only setting the value to be true and the check/return is done only after all the threads have got synced ? 
> > 
> > Also it would be essential to check if there was a failure reading one of the files before reading the next set of files.
> > 
> > For example:
> > 
> > Say thread concurrency = 2
> > 
> > TG created 2 threads and there were 4 files
> > 
> > Say there was an error reading file 1 (or) file 2. You could stop file 3/4 from being read.
> It has to be atomic because it could be written from multiple threads.
> 
> I don't see why it's essential to early exit the other tasks. It is indeed possible though. Just check the value of fail.
But all threads would write only a value true. Atomic would be needed only if one thread is writing true and other false, and you need to sync between them right ?

Say that 3.o and 4.o were very huge files. Linker is going to anyways exit with an error, I see no point of reading 3.o / 4.o.


http://llvm-reviews.chandlerc.com/D664