[llvm] r269684 - ThinLTO: sort inputs and schedule by decreasing size

[Mehdi Amini via llvm-commits]

Author: mehdi_amini
Date: Mon May 16 14:33:07 2016
New Revision: 269684

URL: http://llvm.org/viewvc/llvm-project?rev=269684&view=rev
Log:
ThinLTO: sort inputs and schedule by decreasing size

This is a compile time optimization: keeping a large file to process
at the end hurts parallelism.
The heurisitic used right now is the input buffer size, however we
may want to consider the number of functions to import or the
different number of files to load for importing as well.

From: Mehdi Amini <mehdi.amini at apple.com>

Modified:
    llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp

Modified: llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp?rev=269684&r1=269683&r2=269684&view=diff
==============================================================================
--- llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp (original)
+++ llvm/trunk/lib/LTO/ThinLTOCodeGenerator.cpp Mon May 16 14:33:07 2016
@@ -52,6 +52,8 @@
 #include "llvm/Transforms/ObjCARC.h"
 #include "llvm/Transforms/Utils/FunctionImportUtils.h"
 
+#include <numeric>
+
 using namespace llvm;
 
 #define DEBUG_TYPE "thinlto"
@@ -898,11 +900,24 @@ void ThinLTOCodeGenerator::run() {
   for (auto &DefinedGVSummaries : ModuleToDefinedGVSummaries)
     ExportLists[DefinedGVSummaries.first()];
 
+  // Compute the ordering we will process the inputs: the rough heuristic here
+  // is to sort them per size so that the largest module get schedule as soon as
+  // possible. This is purely a compile-time optimization.
+  std::vector<int> ModulesOrdering;
+  ModulesOrdering.resize(Modules.size());
+  std::iota(ModulesOrdering.begin(), ModulesOrdering.end(), 0);
+  std::sort(ModulesOrdering.begin(), ModulesOrdering.end(),
+            [&](int LeftIndex, int RightIndex) {
+              auto LSize = Modules[LeftIndex].getBufferSize();
+              auto RSize = Modules[RightIndex].getBufferSize();
+              return LSize > RSize;
+            });
+
   // Parallel optimizer + codegen
   {
     ThreadPool Pool(ThreadCount);
-    int count = 0;
-    for (auto &ModuleBuffer : Modules) {
+    for (auto IndexCount : ModulesOrdering) {
+      auto &ModuleBuffer = Modules[IndexCount];
       Pool.async([&](int count) {
         auto ModuleIdentifier = ModuleBuffer.getBufferIdentifier();
         auto &ExportList = ExportLists[ModuleIdentifier];
@@ -954,8 +969,7 @@ void ThinLTOCodeGenerator::run() {
 
         OutputBuffer = CacheEntry.write(std::move(OutputBuffer));
         ProducedBinaries[count] = std::move(OutputBuffer);
-      }, count);
-      count++;
+      }, IndexCount);
     }
   }