[lld] r296702 - ELF: Add ThinLTO caching support.
Rui Ueyama via llvm-commits
llvm-commits at lists.llvm.org
Wed Mar 1 18:39:45 PST 2017
On Wed, Mar 1, 2017 at 6:36 PM, Peter Collingbourne <peter at pcc.me.uk> wrote:
> On Wed, Mar 1, 2017 at 3:31 PM, Sean Silva <chisophugis at gmail.com> wrote:
>
>>
>>
>> On Wed, Mar 1, 2017 at 3:00 PM, Peter Collingbourne via llvm-commits <
>> llvm-commits at lists.llvm.org> wrote:
>>
>>> Author: pcc
>>> Date: Wed Mar 1 17:00:10 2017
>>> New Revision: 296702
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=296702&view=rev
>>> Log:
>>> ELF: Add ThinLTO caching support.
>>>
>>> This patch adds an option named --thinlto-cache-dir, which specifies the
>>> path to a directory in which to cache native object files for ThinLTO
>>> incremental builds.
>>>
>>> Differential Revision: https://reviews.llvm.org/D30509
>>>
>>> Added:
>>> lld/trunk/test/ELF/lto/Inputs/cache.ll
>>> lld/trunk/test/ELF/lto/cache.ll
>>> Modified:
>>> lld/trunk/ELF/Config.h
>>> lld/trunk/ELF/Driver.cpp
>>> lld/trunk/ELF/LTO.cpp
>>> lld/trunk/ELF/LTO.h
>>> lld/trunk/ELF/Options.td
>>>
>>> Modified: lld/trunk/ELF/Config.h
>>> URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Config.h?r
>>> ev=296702&r1=296701&r2=296702&view=diff
>>> ============================================================
>>> ==================
>>> --- lld/trunk/ELF/Config.h (original)
>>> +++ lld/trunk/ELF/Config.h Wed Mar 1 17:00:10 2017
>>> @@ -86,6 +86,7 @@ struct Configuration {
>>> llvm::StringRef OptRemarksFilename;
>>> llvm::StringRef SoName;
>>> llvm::StringRef Sysroot;
>>> + llvm::StringRef ThinLTOCacheDir;
>>> std::string RPath;
>>> std::vector<VersionDefinition> VersionDefinitions;
>>> std::vector<llvm::StringRef> AuxiliaryList;
>>>
>>> Modified: lld/trunk/ELF/Driver.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Driver.cpp
>>> ?rev=296702&r1=296701&r2=296702&view=diff
>>> ============================================================
>>> ==================
>>> --- lld/trunk/ELF/Driver.cpp (original)
>>> +++ lld/trunk/ELF/Driver.cpp Wed Mar 1 17:00:10 2017
>>> @@ -583,6 +583,7 @@ void LinkerDriver::readConfigs(opt::Inpu
>>> Config->Sysroot = getString(Args, OPT_sysroot);
>>> Config->Target1Rel = getArg(Args, OPT_target1_rel, OPT_target1_abs,
>>> false);
>>> Config->Target2 = getTarget2(Args);
>>> + Config->ThinLTOCacheDir = getString(Args, OPT_thinlto_cache_dir);
>>> Config->ThinLTOJobs = getInteger(Args, OPT_thinlto_jobs, -1u);
>>> Config->Threads = getArg(Args, OPT_threads, OPT_no_threads, true);
>>> Config->Trace = Args.hasArg(OPT_trace);
>>>
>>> Modified: lld/trunk/ELF/LTO.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/LTO.cpp?re
>>> v=296702&r1=296701&r2=296702&view=diff
>>> ============================================================
>>> ==================
>>> --- lld/trunk/ELF/LTO.cpp (original)
>>> +++ lld/trunk/ELF/LTO.cpp Wed Mar 1 17:00:10 2017
>>> @@ -18,6 +18,7 @@
>>> #include "llvm/ADT/StringRef.h"
>>> #include "llvm/ADT/Twine.h"
>>> #include "llvm/IR/DiagnosticPrinter.h"
>>> +#include "llvm/LTO/Caching.h"
>>> #include "llvm/LTO/Config.h"
>>> #include "llvm/LTO/LTO.h"
>>> #include "llvm/Object/SymbolicFile.h"
>>> @@ -142,11 +143,24 @@ std::vector<InputFile *> BitcodeCompiler
>>> std::vector<InputFile *> Ret;
>>> unsigned MaxTasks = LTOObj->getMaxTasks();
>>> Buff.resize(MaxTasks);
>>> + Files.resize(MaxTasks);
>>>
>>> - checkError(LTOObj->run([&](size_t Task) {
>>> - return llvm::make_unique<lto::NativeObjectStream>(
>>> - llvm::make_unique<raw_svector_ostream>(Buff[Task]));
>>> - }));
>>> + // The --thinlto-cache-dir option specifies the path to a directory
>>> in which
>>> + // to cache native object files for ThinLTO incremental builds. If a
>>> path was
>>> + // specified, configure LTO to use it as the cache directory.
>>> + lto::NativeObjectCache Cache;
>>> + if (!Config->ThinLTOCacheDir.empty())
>>> + Cache = lto::localCache(Config->ThinLTOCacheDir,
>>> + [&](size_t Task, StringRef Path) {
>>> + Files[Task] =
>>> check(MemoryBuffer::getFile(Path));
>>> + });
>>> +
>>> + checkError(LTOObj->run(
>>> + [&](size_t Task) {
>>> + return llvm::make_unique<lto::NativeObjectStream>(
>>> + llvm::make_unique<raw_svector_ostream>(Buff[Task]));
>>> + },
>>> + Cache));
>>>
>>> for (unsigned I = 0; I != MaxTasks; ++I) {
>>> if (Buff[I].empty())
>>> @@ -160,5 +174,10 @@ std::vector<InputFile *> BitcodeCompiler
>>> InputFile *Obj = createObjectFile(MemoryBufferRef(Buff[I],
>>> "lto.tmp"));
>>> Ret.push_back(Obj);
>>> }
>>> +
>>> + for (std::unique_ptr<MemoryBuffer> &File : Files)
>>> + if (File)
>>> + Ret.push_back(createObjectFile(*File));
>>> +
>>> return Ret;
>>> }
>>>
>>> Modified: lld/trunk/ELF/LTO.h
>>> URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/LTO.h?rev=
>>> 296702&r1=296701&r2=296702&view=diff
>>> ============================================================
>>> ==================
>>> --- lld/trunk/ELF/LTO.h (original)
>>> +++ lld/trunk/ELF/LTO.h Wed Mar 1 17:00:10 2017
>>> @@ -49,6 +49,7 @@ public:
>>> private:
>>> std::unique_ptr<llvm::lto::LTO> LTOObj;
>>> std::vector<SmallString<0>> Buff;
>>> + std::vector<std::unique_ptr<MemoryBuffer>> Files;
>>> };
>>> }
>>> }
>>>
>>> Modified: lld/trunk/ELF/Options.td
>>> URL: http://llvm.org/viewvc/llvm-project/lld/trunk/ELF/Options.td
>>> ?rev=296702&r1=296701&r2=296702&view=diff
>>> ============================================================
>>> ==================
>>> --- lld/trunk/ELF/Options.td (original)
>>> +++ lld/trunk/ELF/Options.td Wed Mar 1 17:00:10 2017
>>> @@ -392,4 +392,6 @@ def opt_remarks_filename: S<"opt-remarks
>>> def opt_remarks_with_hotness: F<"opt-remarks-with-hotness">,
>>> HelpText<"Include hotness informations in the optimization remarks
>>> file">;
>>> def save_temps: F<"save-temps">;
>>> +def thinlto_cache_dir: J<"thinlto-cache-dir=">,
>>> + HelpText<"Path to ThinLTO cached object file directory">;
>>> def thinlto_jobs: J<"thinlto-jobs=">, HelpText<"Number of ThinLTO
>>> jobs">;
>>>
>>> Added: lld/trunk/test/ELF/lto/Inputs/cache.ll
>>> URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/lto/I
>>> nputs/cache.ll?rev=296702&view=auto
>>> ============================================================
>>> ==================
>>> --- lld/trunk/test/ELF/lto/Inputs/cache.ll (added)
>>> +++ lld/trunk/test/ELF/lto/Inputs/cache.ll Wed Mar 1 17:00:10 2017
>>> @@ -0,0 +1,10 @@
>>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>> +target triple = "x86_64-unknown-linux-gnu"
>>> +
>>> +define i32 @main() {
>>> +entry:
>>> + call void (...) @globalfunc()
>>> + ret i32 0
>>> +}
>>> +
>>> +declare void @globalfunc(...)
>>>
>>> Added: lld/trunk/test/ELF/lto/cache.ll
>>> URL: http://llvm.org/viewvc/llvm-project/lld/trunk/test/ELF/lto/c
>>> ache.ll?rev=296702&view=auto
>>> ============================================================
>>> ==================
>>> --- lld/trunk/test/ELF/lto/cache.ll (added)
>>> +++ lld/trunk/test/ELF/lto/cache.ll Wed Mar 1 17:00:10 2017
>>> @@ -0,0 +1,15 @@
>>> +; RUN: opt -module-hash -module-summary %s -o %t.o
>>> +; RUN: opt -module-hash -module-summary %p/Inputs/cache.ll -o %t2.o
>>> +
>>> +; RUN: rm -Rf %t.cache && mkdir %t.cache
>>> +; RUN: ld.lld --thinlto-cache-dir=%t.cache -o %t3.o %t2.o %t.o
>>>
>>
>> I find this a bit confusing. The argument to `-o` is a .o file, but -r is
>> not passed.
>>
>
> Renamed to %t3 in r296728.
>
>
>> Also, there doesn't seem to be a definition of _start in any of the input
>> files?
>>
>
> I believe that we will accept an undefined entry symbol in executables
> with a warning (that is what bfd and gold do). I don't know how important
> it is to accept that, though. Rui, Rafael?
>
IIRC, there is a program that doesn't pass the -e parameter, expecting the
default behavior. That's not important for most user applications, but
there's no reason not to support that too. But it prints out a warning, so
it is better to pass -e or define _start in tests.
For now I've renamed the main symbol to _start.
>
> Thanks,
> --
> --
> Peter
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20170301/363bc0ec/attachment.html>
More information about the llvm-commits
mailing list