[llvm] r278338 - Restore "Resolution-based LTO API."
Teresa Johnson via llvm-commits
llvm-commits at lists.llvm.org
Tue Aug 30 07:11:03 PDT 2016
+mehdi for his thoughts.
On Mon, Aug 29, 2016 at 11:55 AM, Rafael Espíndola
<rafael.espindola at gmail.com> wrote:
> I have just finished reading this patch with an eye on seeing if it
> would make sense to use it in lld.
That's for reviewing that.
>
> I think it should work (and Davide has a wip patch),
Great!
> I think the only
> thing that would be a bit odd is the GlobalResolutions StringMap. In
> lld we try to have a single symbol table. Any objections to removing
> GlobalResolutions and expecting each linker to track GlobalResolution?
That would mean essentially moving LTO::addSymbolToGlobalRes into each
client (which would also have to invoke collectUsedGlobalVariables to
get the Used set), right? Would the lld symbol table track info that
is current in the GlobalResolutions e.g. the partition which is
detecting whether there are cross-module references for ThinLTO?
I am wary of duplicating a bunch of logic into every client. But I
suppose we could move the ownership of GlobalResolution info into the
clients and use a callback in addSymbolToGlobalRes to provide the
resulting partition info to the client. Then the current consumers of
GlobalResolution would instead need a way to get a list of symbols and
their partition info from the clients.
>
> Cheers,
> Rafael
>
> On 11 August 2016 at 10:58, Teresa Johnson via llvm-commits
> <llvm-commits at lists.llvm.org> wrote:
>> Author: tejohnson
>> Date: Thu Aug 11 09:58:12 2016
>> New Revision: 278338
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=278338&view=rev
>> Log:
>> Restore "Resolution-based LTO API."
>>
>> This restores commit r278330, with fixes for a few bot failures:
>> - Fix a late change I had made to the save temps output file that I
>> missed due to existing files sitting on my disk
>> - Fix a bunch of Windows bot failures with "ambiguous call to overloaded
>> function" due to confusion between llvm::make_unique vs
>> std::make_unique (preface the new make_unique calls with "llvm::")
>> - Attempt to fix a modules bot failure by adding a missing include
>> to LTO/Config.h.
>>
>> Original change:
>>
>> Resolution-based LTO API.
>>
>> Summary:
>> This introduces a resolution-based LTO API. The main advantage of this API over
>> existing APIs is that it allows the linker to supply a resolution for each
>> symbol in each object, rather than the combined object as a whole. This will
>> become increasingly important for use cases such as ThinLTO which require us
>> to process symbol resolutions in a more complicated way than just adjusting
>> linkage.
>>
>> Patch by Peter Collingbourne.
>>
>> Reviewers: rafael, tejohnson, mehdi_amini
>>
>> Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits
>>
>> Differential Revision: https://reviews.llvm.org/D20268
>>
>> Added:
>> llvm/trunk/include/llvm/LTO/Config.h
>> llvm/trunk/include/llvm/LTO/LTOBackend.h
>> llvm/trunk/lib/LTO/LTOBackend.cpp
>> llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll
>> llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll
>> llvm/trunk/test/LTO/Resolution/X86/alias.ll
>> llvm/trunk/test/LTO/Resolution/X86/comdat.ll
>> - copied, changed from r278331, llvm/trunk/test/tools/gold/X86/comdat.ll
>> llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg
>> llvm/trunk/test/tools/llvm-lto2/errors.ll
>> llvm/trunk/tools/llvm-lto2/CMakeLists.txt
>> llvm/trunk/tools/llvm-lto2/LLVMBuild.txt
>> - copied, changed from r278331, llvm/trunk/lib/LTO/LLVMBuild.txt
>> llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp
>> Modified:
>> llvm/trunk/include/llvm/LTO/LTO.h
>> llvm/trunk/lib/LTO/CMakeLists.txt
>> llvm/trunk/lib/LTO/LLVMBuild.txt
>> llvm/trunk/lib/LTO/LTO.cpp
>> llvm/trunk/lib/Object/IRObjectFile.cpp
>> llvm/trunk/test/CMakeLists.txt
>> llvm/trunk/test/lit.cfg
>> llvm/trunk/test/tools/gold/X86/coff.ll
>> llvm/trunk/test/tools/gold/X86/comdat.ll
>> llvm/trunk/test/tools/gold/X86/common.ll
>> llvm/trunk/test/tools/gold/X86/emit-llvm.ll
>> llvm/trunk/test/tools/gold/X86/opt-level.ll
>> llvm/trunk/test/tools/gold/X86/parallel.ll
>> llvm/trunk/test/tools/gold/X86/slp-vectorize.ll
>> llvm/trunk/test/tools/gold/X86/start-lib-common.ll
>> llvm/trunk/test/tools/gold/X86/strip_names.ll
>> llvm/trunk/test/tools/gold/X86/thinlto.ll
>> llvm/trunk/test/tools/gold/X86/thinlto_alias.ll
>> llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll
>> llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll
>> llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll
>> llvm/trunk/test/tools/gold/X86/type-merge2.ll
>> llvm/trunk/test/tools/gold/X86/vectorize.ll
>> llvm/trunk/test/tools/gold/X86/visibility.ll
>> llvm/trunk/tools/gold/gold-plugin.cpp
>>
>> Added: llvm/trunk/include/llvm/LTO/Config.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LTO/Config.h?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/include/llvm/LTO/Config.h (added)
>> +++ llvm/trunk/include/llvm/LTO/Config.h Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,172 @@
>> +//===-Config.h - LLVM Link Time Optimizer Configuration -------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// This file defines the lto::Config data structure, which allows clients to
>> +// configure LTO.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef LLVM_LTO_CONFIG_H
>> +#define LLVM_LTO_CONFIG_H
>> +
>> +#include "llvm/IR/DiagnosticInfo.h"
>> +#include "llvm/Support/CodeGen.h"
>> +#include "llvm/Target/TargetOptions.h"
>> +
>> +#include <functional>
>> +
>> +namespace llvm {
>> +
>> +class Error;
>> +class Module;
>> +class ModuleSummaryIndex;
>> +class raw_pwrite_stream;
>> +
>> +namespace lto {
>> +
>> +/// LTO configuration. A linker can configure LTO by setting fields in this data
>> +/// structure and passing it to the lto::LTO constructor.
>> +struct Config {
>> + std::string CPU;
>> + std::string Features;
>> + TargetOptions Options;
>> + std::vector<std::string> MAttrs;
>> + Reloc::Model RelocModel = Reloc::PIC_;
>> + CodeModel::Model CodeModel = CodeModel::Default;
>> + CodeGenOpt::Level CGOptLevel = CodeGenOpt::Default;
>> + unsigned OptLevel = 2;
>> + bool DisableVerify = false;
>> +
>> + /// Setting this field will replace target triples in input files with this
>> + /// triple.
>> + std::string OverrideTriple;
>> +
>> + /// Setting this field will replace unspecified target triples in input files
>> + /// with this triple.
>> + std::string DefaultTriple;
>> +
>> + bool ShouldDiscardValueNames = true;
>> + DiagnosticHandlerFunction DiagHandler;
>> +
>> + /// If this field is set, LTO will write input file paths and symbol
>> + /// resolutions here in llvm-lto2 command line flag format. This can be
>> + /// used for testing and for running the LTO pipeline outside of the linker
>> + /// with llvm-lto2.
>> + std::unique_ptr<raw_ostream> ResolutionFile;
>> +
>> + /// The following callbacks deal with tasks, which normally represent the
>> + /// entire optimization and code generation pipeline for what will become a
>> + /// single native object file. Each task has a unique identifier between 0 and
>> + /// getMaxTasks()-1, which is supplied to the callback via the Task parameter.
>> + /// A task represents the entire pipeline for ThinLTO and regular
>> + /// (non-parallel) LTO, but a parallel code generation task will be split into
>> + /// N tasks before code generation, where N is the parallelism level.
>> + ///
>> + /// LTO may decide to stop processing a task at any time, for example if the
>> + /// module is empty or if a module hook (see below) returns false. For this
>> + /// reason, the client should not expect to receive exactly getMaxTasks()
>> + /// native object files.
>> +
>> + /// A module hook may be used by a linker to perform actions during the LTO
>> + /// pipeline. For example, a linker may use this function to implement
>> + /// -save-temps, or to add its own resolved symbols to the module. If this
>> + /// function returns false, any further processing for that task is aborted.
>> + ///
>> + /// Module hooks must be thread safe with respect to the linker's internal
>> + /// data structures. A module hook will never be called concurrently from
>> + /// multiple threads with the same task ID, or the same module.
>> + ///
>> + /// Note that in out-of-process backend scenarios, none of the hooks will be
>> + /// called for ThinLTO tasks.
>> + typedef std::function<bool(size_t Task, Module &)> ModuleHookFn;
>> +
>> + /// This module hook is called after linking (regular LTO) or loading
>> + /// (ThinLTO) the module, before modifying it.
>> + ModuleHookFn PreOptModuleHook;
>> +
>> + /// This hook is called after promoting any internal functions
>> + /// (ThinLTO-specific).
>> + ModuleHookFn PostPromoteModuleHook;
>> +
>> + /// This hook is called after internalizing the module.
>> + ModuleHookFn PostInternalizeModuleHook;
>> +
>> + /// This hook is called after importing from other modules (ThinLTO-specific).
>> + ModuleHookFn PostImportModuleHook;
>> +
>> + /// This module hook is called after optimization is complete.
>> + ModuleHookFn PostOptModuleHook;
>> +
>> + /// This module hook is called before code generation. It is similar to the
>> + /// PostOptModuleHook, but for parallel code generation it is called after
>> + /// splitting the module.
>> + ModuleHookFn PreCodeGenModuleHook;
>> +
>> + /// A combined index hook is called after all per-module indexes have been
>> + /// combined (ThinLTO-specific). It can be used to implement -save-temps for
>> + /// the combined index.
>> + ///
>> + /// If this function returns false, any further processing for ThinLTO tasks
>> + /// is aborted.
>> + ///
>> + /// It is called regardless of whether the backend is in-process, although it
>> + /// is not called from individual backend processes.
>> + typedef std::function<bool(const ModuleSummaryIndex &Index)>
>> + CombinedIndexHookFn;
>> + CombinedIndexHookFn CombinedIndexHook;
>> +
>> + /// This is a convenience function that configures this Config object to write
>> + /// temporary files named after the given OutputFileName for each of the LTO
>> + /// phases to disk. A client can use this function to implement -save-temps.
>> + ///
>> + /// FIXME: Temporary files derived from ThinLTO backends are currently named
>> + /// after the input file name, rather than the output file name, when
>> + /// UseInputModulePath is set to true.
>> + ///
>> + /// Specifically, it (1) sets each of the above module hooks and the combined
>> + /// index hook to a function that calls the hook function (if any) that was
>> + /// present in the appropriate field when the addSaveTemps function was
>> + /// called, and writes the module to a bitcode file with a name prefixed by
>> + /// the given output file name, and (2) creates a resolution file whose name
>> + /// is prefixed by the given output file name and sets ResolutionFile to its
>> + /// file handle.
>> + Error addSaveTemps(std::string OutputFileName,
>> + bool UseInputModulePath = false);
>> +};
>> +
>> +/// This type defines a stream callback. A stream callback is used to add a
>> +/// native object that is generated on the fly. The callee must set up and
>> +/// return a output stream to write the native object to.
>> +///
>> +/// Stream callbacks must be thread safe.
>> +typedef std::function<std::unique_ptr<raw_pwrite_stream>(size_t Task)>
>> + AddStreamFn;
>> +
>> +/// A derived class of LLVMContext that initializes itself according to a given
>> +/// Config object. The purpose of this class is to tie ownership of the
>> +/// diagnostic handler to the context, as opposed to the Config object (which
>> +/// may be ephemeral).
>> +struct LTOLLVMContext : LLVMContext {
>> + static void funcDiagHandler(const DiagnosticInfo &DI, void *Context) {
>> + auto *Fn = static_cast<DiagnosticHandlerFunction *>(Context);
>> + (*Fn)(DI);
>> + }
>> +
>> + LTOLLVMContext(const Config &C) : DiagHandler(C.DiagHandler) {
>> + setDiscardValueNames(C.ShouldDiscardValueNames);
>> + enableDebugTypeODRUniquing();
>> + setDiagnosticHandler(funcDiagHandler, &DiagHandler, true);
>> + }
>> + DiagnosticHandlerFunction DiagHandler;
>> +};
>> +
>> +}
>> +}
>> +
>> +#endif
>>
>> Modified: llvm/trunk/include/llvm/LTO/LTO.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LTO/LTO.h?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/include/llvm/LTO/LTO.h (original)
>> +++ llvm/trunk/include/llvm/LTO/LTO.h Thu Aug 11 09:58:12 2016
>> @@ -16,14 +16,27 @@
>> #ifndef LLVM_LTO_LTO_H
>> #define LLVM_LTO_LTO_H
>>
>> +#include "llvm/ADT/MapVector.h"
>> #include "llvm/ADT/StringMap.h"
>> +#include "llvm/ADT/StringSet.h"
>> +#include "llvm/CodeGen/Analysis.h"
>> +#include "llvm/IR/DiagnosticInfo.h"
>> #include "llvm/IR/ModuleSummaryIndex.h"
>> +#include "llvm/LTO/Config.h"
>> +#include "llvm/Linker/IRMover.h"
>> +#include "llvm/Object/IRObjectFile.h"
>> +#include "llvm/Support/thread.h"
>> +#include "llvm/Target/TargetOptions.h"
>> +#include "llvm/Transforms/IPO/FunctionImport.h"
>>
>> namespace llvm {
>>
>> +class Error;
>> class LLVMContext;
>> class MemoryBufferRef;
>> class Module;
>> +class Target;
>> +class raw_pwrite_stream;
>>
>> /// Helper to load a module from bitcode.
>> std::unique_ptr<Module> loadModuleFromBuffer(const MemoryBufferRef &Buffer,
>> @@ -69,6 +82,319 @@ void thinLTOResolveWeakForLinkerInIndex(
>> void thinLTOInternalizeAndPromoteInIndex(
>> ModuleSummaryIndex &Index,
>> function_ref<bool(StringRef, GlobalValue::GUID)> isExported);
>> -}
>> +
>> +namespace lto {
>> +
>> +class LTO;
>> +struct SymbolResolution;
>> +class ThinBackendProc;
>> +
>> +/// An input file. This is a wrapper for IRObjectFile that exposes only the
>> +/// information that an LTO client should need in order to do symbol resolution.
>> +class InputFile {
>> + // FIXME: Remove LTO class friendship once we have bitcode symbol tables.
>> + friend LTO;
>> + InputFile() = default;
>> +
>> + // FIXME: Remove the LLVMContext once we have bitcode symbol tables.
>> + LLVMContext Ctx;
>> + std::unique_ptr<object::IRObjectFile> Obj;
>> +
>> +public:
>> + /// Create an InputFile.
>> + static Expected<std::unique_ptr<InputFile>> create(MemoryBufferRef Object);
>> +
>> + class symbol_iterator;
>> +
>> + /// This is a wrapper for object::basic_symbol_iterator that exposes only the
>> + /// information that an LTO client should need in order to do symbol
>> + /// resolution.
>> + ///
>> + /// This object is ephemeral; it is only valid as long as an iterator obtained
>> + /// from symbols() refers to it.
>> + class Symbol {
>> + friend symbol_iterator;
>> + friend LTO;
>> +
>> + object::basic_symbol_iterator I;
>> + const GlobalValue *GV;
>> + uint32_t Flags;
>> + SmallString<64> Name;
>> +
>> + bool shouldSkip() {
>> + return !(Flags & object::BasicSymbolRef::SF_Global) ||
>> + (Flags & object::BasicSymbolRef::SF_FormatSpecific);
>> + }
>> +
>> + void skip() {
>> + const object::SymbolicFile *Obj = I->getObject();
>> + auto E = Obj->symbol_end();
>> + while (I != E) {
>> + Flags = I->getFlags();
>> + if (!shouldSkip())
>> + break;
>> + ++I;
>> + }
>> + if (I == E)
>> + return;
>> +
>> + Name.clear();
>> + {
>> + raw_svector_ostream OS(Name);
>> + I->printName(OS);
>> + }
>> + GV = cast<object::IRObjectFile>(Obj)->getSymbolGV(I->getRawDataRefImpl());
>> + }
>> +
>> + public:
>> + Symbol(object::basic_symbol_iterator I) : I(I) { skip(); }
>> +
>> + StringRef getName() const { return Name; }
>> + StringRef getIRName() const {
>> + if (GV)
>> + return GV->getName();
>> + return StringRef();
>> + }
>> + uint32_t getFlags() const { return Flags; }
>> + GlobalValue::VisibilityTypes getVisibility() const {
>> + if (GV)
>> + return GV->getVisibility();
>> + return GlobalValue::DefaultVisibility;
>> + }
>> + bool canBeOmittedFromSymbolTable() const {
>> + return GV && llvm::canBeOmittedFromSymbolTable(GV);
>> + }
>> + Expected<const Comdat *> getComdat() const {
>> + const GlobalObject *GO;
>> + if (auto *GA = dyn_cast<GlobalAlias>(GV)) {
>> + GO = GA->getBaseObject();
>> + if (!GO)
>> + return make_error<StringError>("Unable to determine comdat of alias!",
>> + inconvertibleErrorCode());
>> + } else {
>> + GO = cast<GlobalObject>(GV);
>> + }
>> + if (GV)
>> + return GV->getComdat();
>> + return nullptr;
>> + }
>> + size_t getCommonSize() const {
>> + assert(Flags & object::BasicSymbolRef::SF_Common);
>> + if (!GV)
>> + return 0;
>> + return GV->getParent()->getDataLayout().getTypeAllocSize(
>> + GV->getType()->getElementType());
>> + }
>> + unsigned getCommonAlignment() const {
>> + assert(Flags & object::BasicSymbolRef::SF_Common);
>> + if (!GV)
>> + return 0;
>> + return GV->getAlignment();
>> + }
>> + };
>> +
>> + class symbol_iterator {
>> + Symbol Sym;
>> +
>> + public:
>> + symbol_iterator(object::basic_symbol_iterator I) : Sym(I) {}
>> +
>> + symbol_iterator &operator++() {
>> + ++Sym.I;
>> + Sym.skip();
>> + return *this;
>> + }
>> +
>> + symbol_iterator operator++(int) {
>> + symbol_iterator I = *this;
>> + ++*this;
>> + return I;
>> + }
>> +
>> + const Symbol &operator*() const { return Sym; }
>> + const Symbol *operator->() const { return &Sym; }
>> +
>> + bool operator!=(const symbol_iterator &Other) const {
>> + return Sym.I != Other.Sym.I;
>> + }
>> + };
>> +
>> + /// A range over the symbols in this InputFile.
>> + iterator_range<symbol_iterator> symbols() {
>> + return llvm::make_range(symbol_iterator(Obj->symbol_begin()),
>> + symbol_iterator(Obj->symbol_end()));
>> + }
>> +
>> + StringRef getSourceFileName() const {
>> + return Obj->getModule().getSourceFileName();
>> + }
>> +};
>> +
>> +/// A ThinBackend defines what happens after the thin-link phase during ThinLTO.
>> +/// The details of this type definition aren't important; clients can only
>> +/// create a ThinBackend using one of the create*ThinBackend() functions below.
>> +typedef std::function<std::unique_ptr<ThinBackendProc>(
>> + Config &C, ModuleSummaryIndex &CombinedIndex,
>> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
>> + AddStreamFn AddStream)>
>> + ThinBackend;
>> +
>> +/// This ThinBackend runs the individual backend jobs in-process.
>> +ThinBackend createInProcessThinBackend(unsigned ParallelismLevel);
>> +
>> +/// This ThinBackend writes individual module indexes to files, instead of
>> +/// running the individual backend jobs. This backend is for distributed builds
>> +/// where separate processes will invoke the real backends.
>> +///
>> +/// To find the path to write the index to, the backend checks if the path has a
>> +/// prefix of OldPrefix; if so, it replaces that prefix with NewPrefix. It then
>> +/// appends ".thinlto.bc" and writes the index to that path. If
>> +/// ShouldEmitImportsFiles is true it also writes a list of imported files to a
>> +/// similar path with ".imports" appended instead.
>> +ThinBackend createWriteIndexesThinBackend(std::string OldPrefix,
>> + std::string NewPrefix,
>> + bool ShouldEmitImportsFiles,
>> + std::string LinkedObjectsFile);
>> +
>> +/// This class implements a resolution-based interface to LLVM's LTO
>> +/// functionality. It supports regular LTO, parallel LTO code generation and
>> +/// ThinLTO. You can use it from a linker in the following way:
>> +/// - Set hooks and code generation options (see lto::Config struct defined in
>> +/// Config.h), and use the lto::Config object to create an lto::LTO object.
>> +/// - Create lto::InputFile objects using lto::InputFile::create(), then use
>> +/// the symbols() function to enumerate its symbols and compute a resolution
>> +/// for each symbol (see SymbolResolution below).
>> +/// - After the linker has visited each input file (and each regular object
>> +/// file) and computed a resolution for each symbol, take each lto::InputFile
>> +/// and pass it and an array of symbol resolutions to the add() function.
>> +/// - Call the getMaxTasks() function to get an upper bound on the number of
>> +/// native object files that LTO may add to the link.
>> +/// - Call the run() function. This function will use the supplied AddStream
>> +/// function to add up to getMaxTasks() native object files to the link.
>> +class LTO {
>> + friend InputFile;
>> +
>> +public:
>> + /// Create an LTO object. A default constructed LTO object has a reasonable
>> + /// production configuration, but you can customize it by passing arguments to
>> + /// this constructor.
>> + /// FIXME: We do currently require the DiagHandler field to be set in Conf.
>> + /// Until that is fixed, a Config argument is required.
>> + LTO(Config Conf, ThinBackend Backend = nullptr,
>> + unsigned ParallelCodeGenParallelismLevel = 1);
>> +
>> + /// Add an input file to the LTO link, using the provided symbol resolutions.
>> + /// The symbol resolutions must appear in the enumeration order given by
>> + /// InputFile::symbols().
>> + Error add(std::unique_ptr<InputFile> Obj, ArrayRef<SymbolResolution> Res);
>> +
>> + /// Returns an upper bound on the number of tasks that the client may expect.
>> + /// This may only be called after all IR object files have been added. For a
>> + /// full description of tasks see LTOBackend.h.
>> + size_t getMaxTasks() const;
>> +
>> + /// Runs the LTO pipeline. This function calls the supplied AddStream function
>> + /// to add native object files to the link.
>> + Error run(AddStreamFn AddStream);
>> +
>> +private:
>> + Config Conf;
>> +
>> + struct RegularLTOState {
>> + RegularLTOState(unsigned ParallelCodeGenParallelismLevel, Config &Conf);
>> +
>> + unsigned ParallelCodeGenParallelismLevel;
>> + LTOLLVMContext Ctx;
>> + bool HasModule = false;
>> + std::unique_ptr<Module> CombinedModule;
>> + IRMover Mover;
>> + } RegularLTO;
>> +
>> + struct ThinLTOState {
>> + ThinLTOState(ThinBackend Backend);
>> +
>> + ThinBackend Backend;
>> + ModuleSummaryIndex CombinedIndex;
>> + MapVector<StringRef, MemoryBufferRef> ModuleMap;
>> + DenseMap<GlobalValue::GUID, StringRef> PrevailingModuleForGUID;
>> + } ThinLTO;
>> +
>> + // The global resolution for a particular (mangled) symbol name. This is in
>> + // particular necessary to track whether each symbol can be internalized.
>> + // Because any input file may introduce a new cross-partition reference, we
>> + // cannot make any final internalization decisions until all input files have
>> + // been added and the client has called run(). During run() we apply
>> + // internalization decisions either directly to the module (for regular LTO)
>> + // or to the combined index (for ThinLTO).
>> + struct GlobalResolution {
>> + /// The unmangled name of the global.
>> + std::string IRName;
>> +
>> + bool UnnamedAddr = true;
>> +
>> + /// This field keeps track of the partition number of this global. The
>> + /// regular LTO object is partition 0, while each ThinLTO object has its own
>> + /// partition number from 1 onwards.
>> + ///
>> + /// Any global that is defined or used by more than one partition, or that
>> + /// is referenced externally, may not be internalized.
>> + ///
>> + /// Partitions generally have a one-to-one correspondence with tasks, except
>> + /// that we use partition 0 for all parallel LTO code generation partitions.
>> + /// Any partitioning of the combined LTO object is done internally by the
>> + /// LTO backend.
>> + size_t Partition = Unknown;
>> +
>> + /// Special partition numbers.
>> + enum {
>> + /// A partition number has not yet been assigned to this global.
>> + Unknown = -1ull,
>> +
>> + /// This global is either used by more than one partition or has an
>> + /// external reference, and therefore cannot be internalized.
>> + External = -2ull,
>> + };
>> + };
>> +
>> + // Global mapping from mangled symbol names to resolutions.
>> + StringMap<GlobalResolution> GlobalResolutions;
>> +
>> + void writeToResolutionFile(InputFile *Input, ArrayRef<SymbolResolution> Res);
>> +
>> + void addSymbolToGlobalRes(object::IRObjectFile *Obj,
>> + SmallPtrSet<GlobalValue *, 8> &Used,
>> + const InputFile::Symbol &Sym, SymbolResolution Res,
>> + size_t Partition);
>> +
>> + Error addRegularLTO(std::unique_ptr<InputFile> Input,
>> + ArrayRef<SymbolResolution> Res);
>> + Error addThinLTO(std::unique_ptr<InputFile> Input,
>> + ArrayRef<SymbolResolution> Res);
>> +
>> + Error runRegularLTO(AddStreamFn AddStream);
>> + Error runThinLTO(AddStreamFn AddStream);
>> +
>> + mutable bool CalledGetMaxTasks = false;
>> +};
>> +
>> +/// The resolution for a symbol. The linker must provide a SymbolResolution for
>> +/// each global symbol based on its internal resolution of that symbol.
>> +struct SymbolResolution {
>> + SymbolResolution()
>> + : Prevailing(0), FinalDefinitionInLinkageUnit(0), VisibleToRegularObj(0) {
>> + }
>> + /// The linker has chosen this definition of the symbol.
>> + unsigned Prevailing : 1;
>> +
>> + /// The definition of this symbol is unpreemptable at runtime and is known to
>> + /// be in this linkage unit.
>> + unsigned FinalDefinitionInLinkageUnit : 1;
>> +
>> + /// The definition of this symbol is visible outside of the LTO unit.
>> + unsigned VisibleToRegularObj : 1;
>> +};
>> +
>> +} // namespace lto
>> +} // namespace llvm
>>
>> #endif
>>
>> Added: llvm/trunk/include/llvm/LTO/LTOBackend.h
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LTO/LTOBackend.h?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/include/llvm/LTO/LTOBackend.h (added)
>> +++ llvm/trunk/include/llvm/LTO/LTOBackend.h Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,51 @@
>> +//===-LTOBackend.h - LLVM Link Time Optimizer Backend ---------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// This file implements the "backend" phase of LTO, i.e. it performs
>> +// optimization and code generation on a loaded module. It is generally used
>> +// internally by the LTO class but can also be used independently, for example
>> +// to implement a standalone ThinLTO backend.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef LLVM_LTO_LTOBACKEND_H
>> +#define LLVM_LTO_LTOBACKEND_H
>> +
>> +#include "llvm/ADT/MapVector.h"
>> +#include "llvm/IR/DiagnosticInfo.h"
>> +#include "llvm/IR/ModuleSummaryIndex.h"
>> +#include "llvm/LTO/Config.h"
>> +#include "llvm/Support/MemoryBuffer.h"
>> +#include "llvm/Target/TargetOptions.h"
>> +#include "llvm/Transforms/IPO/FunctionImport.h"
>> +
>> +namespace llvm {
>> +
>> +class Error;
>> +class Module;
>> +class Target;
>> +
>> +namespace lto {
>> +
>> +/// Runs a regular LTO backend.
>> +Error backend(Config &C, AddStreamFn AddStream,
>> + unsigned ParallelCodeGenParallelismLevel,
>> + std::unique_ptr<Module> M);
>> +
>> +/// Runs a ThinLTO backend.
>> +Error thinBackend(Config &C, size_t Task, AddStreamFn AddStream, Module &M,
>> + ModuleSummaryIndex &CombinedIndex,
>> + const FunctionImporter::ImportMapTy &ImportList,
>> + const GVSummaryMapTy &DefinedGlobals,
>> + MapVector<StringRef, MemoryBufferRef> &ModuleMap);
>> +
>> +}
>> +}
>> +
>> +#endif
>>
>> Modified: llvm/trunk/lib/LTO/CMakeLists.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/CMakeLists.txt?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/LTO/CMakeLists.txt (original)
>> +++ llvm/trunk/lib/LTO/CMakeLists.txt Thu Aug 11 09:58:12 2016
>> @@ -49,6 +49,7 @@ endif()
>>
>> add_llvm_library(LLVMLTO
>> LTO.cpp
>> + LTOBackend.cpp
>> LTOModule.cpp
>> LTOCodeGenerator.cpp
>> UpdateCompilerUsed.cpp
>>
>> Modified: llvm/trunk/lib/LTO/LLVMBuild.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/LLVMBuild.txt?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/LTO/LLVMBuild.txt (original)
>> +++ llvm/trunk/lib/LTO/LLVMBuild.txt Thu Aug 11 09:58:12 2016
>> @@ -34,4 +34,4 @@ required_libraries =
>> Scalar
>> Support
>> Target
>> - TransformUtils
>> \ No newline at end of file
>> + TransformUtils
>>
>> Modified: llvm/trunk/lib/LTO/LTO.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/LTO.cpp?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/LTO/LTO.cpp (original)
>> +++ llvm/trunk/lib/LTO/LTO.cpp Thu Aug 11 09:58:12 2016
>> @@ -12,16 +12,39 @@
>> //===----------------------------------------------------------------------===//
>>
>> #include "llvm/LTO/LTO.h"
>> +#include "llvm/Analysis/TargetLibraryInfo.h"
>> +#include "llvm/Analysis/TargetTransformInfo.h"
>> #include "llvm/Bitcode/ReaderWriter.h"
>> +#include "llvm/CodeGen/Analysis.h"
>> +#include "llvm/IR/AutoUpgrade.h"
>> +#include "llvm/IR/DiagnosticPrinter.h"
>> +#include "llvm/IR/LegacyPassManager.h"
>> +#include "llvm/LTO/LTOBackend.h"
>> +#include "llvm/Linker/IRMover.h"
>> +#include "llvm/Object/ModuleSummaryIndexObjectFile.h"
>> +#include "llvm/Support/ManagedStatic.h"
>> #include "llvm/Support/MemoryBuffer.h"
>> +#include "llvm/Support/Path.h"
>> #include "llvm/Support/SourceMgr.h"
>> +#include "llvm/Support/TargetRegistry.h"
>> +#include "llvm/Support/ThreadPool.h"
>> #include "llvm/Support/raw_ostream.h"
>> +#include "llvm/Target/TargetMachine.h"
>> +#include "llvm/Target/TargetOptions.h"
>> +#include "llvm/Transforms/IPO.h"
>> +#include "llvm/Transforms/IPO/PassManagerBuilder.h"
>> +#include "llvm/Transforms/Utils/SplitModule.h"
>>
>> -namespace llvm {
>> +#include <set>
>> +
>> +using namespace llvm;
>> +using namespace lto;
>> +using namespace object;
>>
>> // Simple helper to load a module from bitcode
>> -std::unique_ptr<Module> loadModuleFromBuffer(const MemoryBufferRef &Buffer,
>> - LLVMContext &Context, bool Lazy) {
>> +std::unique_ptr<Module>
>> +llvm::loadModuleFromBuffer(const MemoryBufferRef &Buffer, LLVMContext &Context,
>> + bool Lazy) {
>> SMDiagnostic Err;
>> ErrorOr<std::unique_ptr<Module>> ModuleOrErr(nullptr);
>> if (Lazy) {
>> @@ -76,7 +99,7 @@ static void thinLTOResolveWeakForLinkerG
>> // current module. However there is a chance that another module is still
>> // referencing them because of the import. We make sure we always emit at least
>> // one copy.
>> -void thinLTOResolveWeakForLinkerInIndex(
>> +void llvm::thinLTOResolveWeakForLinkerInIndex(
>> ModuleSummaryIndex &Index,
>> function_ref<bool(GlobalValue::GUID, const GlobalValueSummary *)>
>> isPrevailing,
>> @@ -110,10 +133,513 @@ static void thinLTOInternalizeAndPromote
>>
>> // Update the linkages in the given \p Index to mark exported values
>> // as external and non-exported values as internal.
>> -void thinLTOInternalizeAndPromoteInIndex(
>> +void llvm::thinLTOInternalizeAndPromoteInIndex(
>> ModuleSummaryIndex &Index,
>> function_ref<bool(StringRef, GlobalValue::GUID)> isExported) {
>> for (auto &I : Index)
>> thinLTOInternalizeAndPromoteGUID(I.second, I.first, isExported);
>> }
>> +
>> +Expected<std::unique_ptr<InputFile>> InputFile::create(MemoryBufferRef Object) {
>> + std::unique_ptr<InputFile> File(new InputFile);
>> + std::string Msg;
>> + auto DiagHandler = [](const DiagnosticInfo &DI, void *MsgP) {
>> + auto *Msg = reinterpret_cast<std::string *>(MsgP);
>> + raw_string_ostream OS(*Msg);
>> + DiagnosticPrinterRawOStream DP(OS);
>> + DI.print(DP);
>> + };
>> + File->Ctx.setDiagnosticHandler(DiagHandler, static_cast<void *>(&Msg));
>> +
>> + ErrorOr<std::unique_ptr<object::IRObjectFile>> IRObj =
>> + IRObjectFile::create(Object, File->Ctx);
>> + if (!Msg.empty())
>> + return make_error<StringError>(Msg, inconvertibleErrorCode());
>> + if (!IRObj)
>> + return errorCodeToError(IRObj.getError());
>> + File->Obj = std::move(*IRObj);
>> +
>> + File->Ctx.setDiagnosticHandler(nullptr, nullptr);
>> +
>> + return std::move(File);
>> +}
>> +
>> +LTO::RegularLTOState::RegularLTOState(unsigned ParallelCodeGenParallelismLevel,
>> + Config &Conf)
>> + : ParallelCodeGenParallelismLevel(ParallelCodeGenParallelismLevel),
>> + Ctx(Conf), CombinedModule(llvm::make_unique<Module>("ld-temp.o", Ctx)),
>> + Mover(*CombinedModule) {}
>> +
>> +LTO::ThinLTOState::ThinLTOState(ThinBackend Backend) : Backend(Backend) {
>> + if (!Backend)
>> + this->Backend = createInProcessThinBackend(thread::hardware_concurrency());
>> +}
>> +
>> +LTO::LTO(Config Conf, ThinBackend Backend,
>> + unsigned ParallelCodeGenParallelismLevel)
>> + : Conf(std::move(Conf)),
>> + RegularLTO(ParallelCodeGenParallelismLevel, this->Conf),
>> + ThinLTO(Backend) {}
>> +
>> +// Add the given symbol to the GlobalResolutions map, and resolve its partition.
>> +void LTO::addSymbolToGlobalRes(IRObjectFile *Obj,
>> + SmallPtrSet<GlobalValue *, 8> &Used,
>> + const InputFile::Symbol &Sym,
>> + SymbolResolution Res, size_t Partition) {
>> + GlobalValue *GV = Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
>> +
>> + auto &GlobalRes = GlobalResolutions[Sym.getName()];
>> + if (GV) {
>> + GlobalRes.UnnamedAddr &= GV->hasGlobalUnnamedAddr();
>> + if (Res.Prevailing)
>> + GlobalRes.IRName = GV->getName();
>> + }
>> + if (Res.VisibleToRegularObj || (GV && Used.count(GV)) ||
>> + (GlobalRes.Partition != GlobalResolution::Unknown &&
>> + GlobalRes.Partition != Partition))
>> + GlobalRes.Partition = GlobalResolution::External;
>> + else
>> + GlobalRes.Partition = Partition;
>> +}
>> +
>> +void LTO::writeToResolutionFile(InputFile *Input,
>> + ArrayRef<SymbolResolution> Res) {
>> + StringRef Path = Input->Obj->getMemoryBufferRef().getBufferIdentifier();
>> + *Conf.ResolutionFile << Path << '\n';
>> + auto ResI = Res.begin();
>> + for (const InputFile::Symbol &Sym : Input->symbols()) {
>> + assert(ResI != Res.end());
>> + SymbolResolution Res = *ResI++;
>> +
>> + *Conf.ResolutionFile << "-r=" << Path << ',' << Sym.getName() << ',';
>> + if (Res.Prevailing)
>> + *Conf.ResolutionFile << 'p';
>> + if (Res.FinalDefinitionInLinkageUnit)
>> + *Conf.ResolutionFile << 'l';
>> + if (Res.VisibleToRegularObj)
>> + *Conf.ResolutionFile << 'x';
>> + *Conf.ResolutionFile << '\n';
>> + }
>> + assert(ResI == Res.end());
>> +}
>> +
>> +Error LTO::add(std::unique_ptr<InputFile> Input,
>> + ArrayRef<SymbolResolution> Res) {
>> + assert(!CalledGetMaxTasks);
>> +
>> + if (Conf.ResolutionFile)
>> + writeToResolutionFile(Input.get(), Res);
>> +
>> + Module &M = Input->Obj->getModule();
>> + SmallPtrSet<GlobalValue *, 8> Used;
>> + collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
>> +
>> + if (!Conf.OverrideTriple.empty())
>> + M.setTargetTriple(Conf.OverrideTriple);
>> + else if (M.getTargetTriple().empty())
>> + M.setTargetTriple(Conf.DefaultTriple);
>> +
>> + MemoryBufferRef MBRef = Input->Obj->getMemoryBufferRef();
>> + bool HasThinLTOSummary = hasGlobalValueSummary(MBRef, Conf.DiagHandler);
>> +
>> + if (HasThinLTOSummary)
>> + return addThinLTO(std::move(Input), Res);
>> + else
>> + return addRegularLTO(std::move(Input), Res);
>> +}
>> +
>> +// Add a regular LTO object to the link.
>> +Error LTO::addRegularLTO(std::unique_ptr<InputFile> Input,
>> + ArrayRef<SymbolResolution> Res) {
>> + RegularLTO.HasModule = true;
>> +
>> + ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
>> + IRObjectFile::create(Input->Obj->getMemoryBufferRef(), RegularLTO.Ctx);
>> + if (!ObjOrErr)
>> + return errorCodeToError(ObjOrErr.getError());
>> + std::unique_ptr<object::IRObjectFile> Obj = std::move(*ObjOrErr);
>> +
>> + Module &M = Obj->getModule();
>> + M.materializeMetadata();
>> + UpgradeDebugInfo(M);
>> +
>> + SmallPtrSet<GlobalValue *, 8> Used;
>> + collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
>> +
>> + std::vector<GlobalValue *> Keep;
>> +
>> + for (GlobalVariable &GV : M.globals())
>> + if (GV.hasAppendingLinkage())
>> + Keep.push_back(&GV);
>> +
>> + auto ResI = Res.begin();
>> + for (const InputFile::Symbol &Sym :
>> + make_range(InputFile::symbol_iterator(Obj->symbol_begin()),
>> + InputFile::symbol_iterator(Obj->symbol_end()))) {
>> + assert(ResI != Res.end());
>> + SymbolResolution Res = *ResI++;
>> + addSymbolToGlobalRes(Obj.get(), Used, Sym, Res, 0);
>> +
>> + GlobalValue *GV = Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
>> + if (Res.Prevailing && GV) {
>> + Keep.push_back(GV);
>> + switch (GV->getLinkage()) {
>> + default:
>> + break;
>> + case GlobalValue::LinkOnceAnyLinkage:
>> + GV->setLinkage(GlobalValue::WeakAnyLinkage);
>> + break;
>> + case GlobalValue::LinkOnceODRLinkage:
>> + GV->setLinkage(GlobalValue::WeakODRLinkage);
>> + break;
>> + }
>> + }
>> +
>> + // FIXME: use proposed local attribute for FinalDefinitionInLinkageUnit.
>> + }
>> + assert(ResI == Res.end());
>> +
>> + return RegularLTO.Mover.move(Obj->takeModule(), Keep,
>> + [](GlobalValue &, IRMover::ValueAdder) {});
>> +}
>> +
>> +// Add a ThinLTO object to the link.
>> +Error LTO::addThinLTO(std::unique_ptr<InputFile> Input,
>> + ArrayRef<SymbolResolution> Res) {
>> + Module &M = Input->Obj->getModule();
>> + SmallPtrSet<GlobalValue *, 8> Used;
>> + collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
>> +
>> + // We need to initialize the target info for the combined regular LTO module
>> + // in case we have no regular LTO objects. In that case we still need to build
>> + // it as usual because the client may want to add symbol definitions to it.
>> + if (RegularLTO.CombinedModule->getTargetTriple().empty()) {
>> + RegularLTO.CombinedModule->setTargetTriple(M.getTargetTriple());
>> + RegularLTO.CombinedModule->setDataLayout(M.getDataLayout());
>> + }
>> +
>> + MemoryBufferRef MBRef = Input->Obj->getMemoryBufferRef();
>> + ErrorOr<std::unique_ptr<object::ModuleSummaryIndexObjectFile>>
>> + SummaryObjOrErr =
>> + object::ModuleSummaryIndexObjectFile::create(MBRef, Conf.DiagHandler);
>> + if (!SummaryObjOrErr)
>> + return errorCodeToError(SummaryObjOrErr.getError());
>> + ThinLTO.CombinedIndex.mergeFrom((*SummaryObjOrErr)->takeIndex(),
>> + ThinLTO.ModuleMap.size());
>> +
>> + auto ResI = Res.begin();
>> + for (const InputFile::Symbol &Sym : Input->symbols()) {
>> + assert(ResI != Res.end());
>> + SymbolResolution Res = *ResI++;
>> + addSymbolToGlobalRes(Input->Obj.get(), Used, Sym, Res,
>> + ThinLTO.ModuleMap.size() + 1);
>> +
>> + GlobalValue *GV = Input->Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
>> + if (Res.Prevailing && GV)
>> + ThinLTO.PrevailingModuleForGUID[GV->getGUID()] =
>> + MBRef.getBufferIdentifier();
>> + }
>> + assert(ResI == Res.end());
>> +
>> + ThinLTO.ModuleMap[MBRef.getBufferIdentifier()] = MBRef;
>> + return Error();
>> +}
>> +
>> +size_t LTO::getMaxTasks() const {
>> + CalledGetMaxTasks = true;
>> + return RegularLTO.ParallelCodeGenParallelismLevel + ThinLTO.ModuleMap.size();
>> +}
>> +
>> +Error LTO::run(AddStreamFn AddStream) {
>> + // Invoke regular LTO if there was a regular LTO module to start with,
>> + // or if there are any hooks that the linker may have used to add
>> + // its own resolved symbols to the combined module.
>> + if (RegularLTO.HasModule || Conf.PreOptModuleHook ||
>> + Conf.PostInternalizeModuleHook || Conf.PostOptModuleHook ||
>> + Conf.PreCodeGenModuleHook)
>> + if (auto E = runRegularLTO(AddStream))
>> + return E;
>> + return runThinLTO(AddStream);
>> +}
>> +
>> +Error LTO::runRegularLTO(AddStreamFn AddStream) {
>> + if (Conf.PreOptModuleHook &&
>> + !Conf.PreOptModuleHook(0, *RegularLTO.CombinedModule))
>> + return Error();
>> +
>> + for (const auto &R : GlobalResolutions) {
>> + if (R.second.IRName.empty())
>> + continue;
>> + if (R.second.Partition != 0 &&
>> + R.second.Partition != GlobalResolution::External)
>> + continue;
>> +
>> + GlobalValue *GV = RegularLTO.CombinedModule->getNamedValue(R.second.IRName);
>> + // Ignore symbols defined in other partitions.
>> + if (!GV || GV->hasLocalLinkage())
>> + continue;
>> + GV->setUnnamedAddr(R.second.UnnamedAddr ? GlobalValue::UnnamedAddr::Global
>> + : GlobalValue::UnnamedAddr::None);
>> + if (R.second.Partition == 0)
>> + GV->setLinkage(GlobalValue::InternalLinkage);
>> + }
>> +
>> + if (Conf.PostInternalizeModuleHook &&
>> + !Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))
>> + return Error();
>> +
>> + return backend(Conf, AddStream, RegularLTO.ParallelCodeGenParallelismLevel,
>> + std::move(RegularLTO.CombinedModule));
>> +}
>> +
>> +/// This class defines the interface to the ThinLTO backend.
>> +class lto::ThinBackendProc {
>> +protected:
>> + Config &Conf;
>> + ModuleSummaryIndex &CombinedIndex;
>> + AddStreamFn AddStream;
>> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries;
>> +
>> +public:
>> + ThinBackendProc(Config &Conf, ModuleSummaryIndex &CombinedIndex,
>> + AddStreamFn AddStream,
>> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries)
>> + : Conf(Conf), CombinedIndex(CombinedIndex), AddStream(AddStream),
>> + ModuleToDefinedGVSummaries(ModuleToDefinedGVSummaries) {}
>> +
>> + virtual ~ThinBackendProc() {}
>> + virtual Error start(size_t Task, MemoryBufferRef MBRef,
>> + StringMap<FunctionImporter::ImportMapTy> &ImportLists,
>> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) = 0;
>> + virtual Error wait() = 0;
>> +};
>> +
>> +class InProcessThinBackend : public ThinBackendProc {
>> + ThreadPool BackendThreadPool;
>> +
>> + Optional<Error> Err;
>> + std::mutex ErrMu;
>> +
>> +public:
>> + InProcessThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
>> + unsigned ThinLTOParallelismLevel,
>> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
>> + AddStreamFn AddStream)
>> + : ThinBackendProc(Conf, CombinedIndex, AddStream,
>> + ModuleToDefinedGVSummaries),
>> + BackendThreadPool(ThinLTOParallelismLevel) {}
>> +
>> + Error
>> + runThinLTOBackendThread(AddStreamFn AddStream, size_t Task,
>> + MemoryBufferRef MBRef,
>> + ModuleSummaryIndex &CombinedIndex,
>> + const FunctionImporter::ImportMapTy &ImportList,
>> + const GVSummaryMapTy &DefinedGlobals,
>> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
>> + LLVMContext BackendContext;
>> +
>> + ErrorOr<std::unique_ptr<Module>> MOrErr =
>> + parseBitcodeFile(MBRef, BackendContext);
>> + assert(MOrErr && "Unable to load module in thread?");
>> +
>> + return thinBackend(Conf, Task, AddStream, **MOrErr, CombinedIndex,
>> + ImportList, DefinedGlobals, ModuleMap);
>> + }
>> +
>> + Error start(size_t Task, MemoryBufferRef MBRef,
>> + StringMap<FunctionImporter::ImportMapTy> &ImportLists,
>> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) override {
>> + StringRef ModulePath = MBRef.getBufferIdentifier();
>> + BackendThreadPool.async(
>> + [=](MemoryBufferRef MBRef, ModuleSummaryIndex &CombinedIndex,
>> + const FunctionImporter::ImportMapTy &ImportList,
>> + GVSummaryMapTy &DefinedGlobals,
>> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
>> + Error E =
>> + runThinLTOBackendThread(AddStream, Task, MBRef, CombinedIndex,
>> + ImportList, DefinedGlobals, ModuleMap);
>> + if (E) {
>> + std::unique_lock<std::mutex> L(ErrMu);
>> + if (Err)
>> + Err = joinErrors(std::move(*Err), std::move(E));
>> + else
>> + Err = std::move(E);
>> + }
>> + },
>> + MBRef, std::ref(CombinedIndex), std::ref(ImportLists[ModulePath]),
>> + std::ref(ModuleToDefinedGVSummaries[ModulePath]), std::ref(ModuleMap));
>> + return Error();
>> + }
>> +
>> + Error wait() override {
>> + BackendThreadPool.wait();
>> + if (Err)
>> + return std::move(*Err);
>> + else
>> + return Error();
>> + }
>> +};
>> +
>> +ThinBackend lto::createInProcessThinBackend(unsigned ParallelismLevel) {
>> + return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
>> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
>> + AddStreamFn AddStream) {
>> + return llvm::make_unique<InProcessThinBackend>(
>> + Conf, CombinedIndex, ParallelismLevel, ModuleToDefinedGVSummaries,
>> + AddStream);
>> + };
>> +}
>> +
>> +class WriteIndexesThinBackend : public ThinBackendProc {
>> + std::string OldPrefix, NewPrefix;
>> + bool ShouldEmitImportsFiles;
>> +
>> + std::string LinkedObjectsFileName;
>> + std::unique_ptr<llvm::raw_fd_ostream> LinkedObjectsFile;
>> +
>> +public:
>> + WriteIndexesThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
>> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
>> + AddStreamFn AddStream, std::string OldPrefix,
>> + std::string NewPrefix, bool ShouldEmitImportsFiles,
>> + std::string LinkedObjectsFileName)
>> + : ThinBackendProc(Conf, CombinedIndex, AddStream,
>> + ModuleToDefinedGVSummaries),
>> + OldPrefix(OldPrefix), NewPrefix(NewPrefix),
>> + ShouldEmitImportsFiles(ShouldEmitImportsFiles),
>> + LinkedObjectsFileName(LinkedObjectsFileName) {}
>> +
>> + /// Given the original \p Path to an output file, replace any path
>> + /// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
>> + /// resulting directory if it does not yet exist.
>> + std::string getThinLTOOutputFile(const std::string &Path,
>> + const std::string &OldPrefix,
>> + const std::string &NewPrefix) {
>> + if (OldPrefix.empty() && NewPrefix.empty())
>> + return Path;
>> + SmallString<128> NewPath(Path);
>> + llvm::sys::path::replace_path_prefix(NewPath, OldPrefix, NewPrefix);
>> + StringRef ParentPath = llvm::sys::path::parent_path(NewPath.str());
>> + if (!ParentPath.empty()) {
>> + // Make sure the new directory exists, creating it if necessary.
>> + if (std::error_code EC = llvm::sys::fs::create_directories(ParentPath))
>> + llvm::errs() << "warning: could not create directory '" << ParentPath
>> + << "': " << EC.message() << '\n';
>> + }
>> + return NewPath.str();
>> + }
>> +
>> + Error start(size_t Task, MemoryBufferRef MBRef,
>> + StringMap<FunctionImporter::ImportMapTy> &ImportLists,
>> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) override {
>> + StringRef ModulePath = MBRef.getBufferIdentifier();
>> + std::string NewModulePath =
>> + getThinLTOOutputFile(ModulePath, OldPrefix, NewPrefix);
>> +
>> + std::error_code EC;
>> + if (!LinkedObjectsFileName.empty()) {
>> + if (!LinkedObjectsFile) {
>> + LinkedObjectsFile = llvm::make_unique<raw_fd_ostream>(
>> + LinkedObjectsFileName, EC, sys::fs::OpenFlags::F_None);
>> + if (EC)
>> + return errorCodeToError(EC);
>> + }
>> + *LinkedObjectsFile << NewModulePath << '\n';
>> + }
>> +
>> + std::map<std::string, GVSummaryMapTy> ModuleToSummariesForIndex;
>> + gatherImportedSummariesForModule(ModulePath, ModuleToDefinedGVSummaries,
>> + ImportLists, ModuleToSummariesForIndex);
>> +
>> + raw_fd_ostream OS(NewModulePath + ".thinlto.bc", EC,
>> + sys::fs::OpenFlags::F_None);
>> + if (EC)
>> + return errorCodeToError(EC);
>> + WriteIndexToFile(CombinedIndex, OS, &ModuleToSummariesForIndex);
>> +
>> + if (ShouldEmitImportsFiles)
>> + return errorCodeToError(EmitImportsFiles(
>> + ModulePath, NewModulePath + ".imports", ImportLists));
>> + return Error();
>> + }
>> +
>> + Error wait() override { return Error(); }
>> +};
>> +
>> +ThinBackend lto::createWriteIndexesThinBackend(std::string OldPrefix,
>> + std::string NewPrefix,
>> + bool ShouldEmitImportsFiles,
>> + std::string LinkedObjectsFile) {
>> + return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
>> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
>> + AddStreamFn AddStream) {
>> + return llvm::make_unique<WriteIndexesThinBackend>(
>> + Conf, CombinedIndex, ModuleToDefinedGVSummaries, AddStream, OldPrefix,
>> + NewPrefix, ShouldEmitImportsFiles, LinkedObjectsFile);
>> + };
>> +}
>> +
>> +Error LTO::runThinLTO(AddStreamFn AddStream) {
>> + if (ThinLTO.ModuleMap.empty())
>> + return Error();
>> +
>> + if (Conf.CombinedIndexHook && !Conf.CombinedIndexHook(ThinLTO.CombinedIndex))
>> + return Error();
>> +
>> + // Collect for each module the list of function it defines (GUID ->
>> + // Summary).
>> + StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
>> + ModuleToDefinedGVSummaries(ThinLTO.ModuleMap.size());
>> + ThinLTO.CombinedIndex.collectDefinedGVSummariesPerModule(
>> + ModuleToDefinedGVSummaries);
>> +
>> + StringMap<FunctionImporter::ImportMapTy> ImportLists(
>> + ThinLTO.ModuleMap.size());
>> + StringMap<FunctionImporter::ExportSetTy> ExportLists(
>> + ThinLTO.ModuleMap.size());
>> + ComputeCrossModuleImport(ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries,
>> + ImportLists, ExportLists);
>> +
>> + std::set<GlobalValue::GUID> ExportedGUIDs;
>> + for (auto &Res : GlobalResolutions) {
>> + if (!Res.second.IRName.empty() &&
>> + Res.second.Partition == GlobalResolution::External)
>> + ExportedGUIDs.insert(GlobalValue::getGUID(Res.second.IRName));
>> + }
>> +
>> + auto isPrevailing = [&](GlobalValue::GUID GUID, const GlobalValueSummary *S) {
>> + return ThinLTO.PrevailingModuleForGUID[GUID] == S->modulePath();
>> + };
>> + auto isExported = [&](StringRef ModuleIdentifier, GlobalValue::GUID GUID) {
>> + const auto &ExportList = ExportLists.find(ModuleIdentifier);
>> + return (ExportList != ExportLists.end() &&
>> + ExportList->second.count(GUID)) ||
>> + ExportedGUIDs.count(GUID);
>> + };
>> + thinLTOInternalizeAndPromoteInIndex(ThinLTO.CombinedIndex, isExported);
>> + thinLTOResolveWeakForLinkerInIndex(
>> + ThinLTO.CombinedIndex, isPrevailing,
>> + [](StringRef, GlobalValue::GUID, GlobalValue::LinkageTypes) {});
>> +
>> + std::unique_ptr<ThinBackendProc> BackendProc = ThinLTO.Backend(
>> + Conf, ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries, AddStream);
>> +
>> + // Partition numbers for ThinLTO jobs start at 1 (see comments for
>> + // GlobalResolution in LTO.h). Task numbers, however, start at
>> + // ParallelCodeGenParallelismLevel, as tasks 0 through
>> + // ParallelCodeGenParallelismLevel-1 are reserved for parallel code generation
>> + // partitions.
>> + size_t Task = RegularLTO.ParallelCodeGenParallelismLevel;
>> + size_t Partition = 1;
>> +
>> + for (auto &Mod : ThinLTO.ModuleMap) {
>> + if (Error E = BackendProc->start(Task, Mod.second, ImportLists,
>> + ThinLTO.ModuleMap))
>> + return E;
>> +
>> + ++Task;
>> + ++Partition;
>> + }
>> +
>> + return BackendProc->wait();
>> }
>>
>> Added: llvm/trunk/lib/LTO/LTOBackend.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/LTOBackend.cpp?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/lib/LTO/LTOBackend.cpp (added)
>> +++ llvm/trunk/lib/LTO/LTOBackend.cpp Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,284 @@
>> +//===-LTOBackend.cpp - LLVM Link Time Optimizer Backend -------------------===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// This file implements the "backend" phase of LTO, i.e. it performs
>> +// optimization and code generation on a loaded module. It is generally used
>> +// internally by the LTO class but can also be used independently, for example
>> +// to implement a standalone ThinLTO backend.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "llvm/LTO/LTOBackend.h"
>> +#include "llvm/Analysis/TargetLibraryInfo.h"
>> +#include "llvm/Analysis/TargetTransformInfo.h"
>> +#include "llvm/Bitcode/ReaderWriter.h"
>> +#include "llvm/IR/LegacyPassManager.h"
>> +#include "llvm/MC/SubtargetFeature.h"
>> +#include "llvm/Support/Error.h"
>> +#include "llvm/Support/FileSystem.h"
>> +#include "llvm/Support/TargetRegistry.h"
>> +#include "llvm/Support/ThreadPool.h"
>> +#include "llvm/Target/TargetMachine.h"
>> +#include "llvm/Transforms/IPO.h"
>> +#include "llvm/Transforms/IPO/PassManagerBuilder.h"
>> +#include "llvm/Transforms/Utils/FunctionImportUtils.h"
>> +#include "llvm/Transforms/Utils/SplitModule.h"
>> +
>> +using namespace llvm;
>> +using namespace lto;
>> +
>> +Error Config::addSaveTemps(std::string OutputFileName,
>> + bool UseInputModulePath) {
>> + ShouldDiscardValueNames = false;
>> +
>> + std::error_code EC;
>> + ResolutionFile = llvm::make_unique<raw_fd_ostream>(
>> + OutputFileName + ".resolution.txt", EC, sys::fs::OpenFlags::F_Text);
>> + if (EC)
>> + return errorCodeToError(EC);
>> +
>> + auto setHook = [&](std::string PathSuffix, ModuleHookFn &Hook) {
>> + // Keep track of the hook provided by the linker, which also needs to run.
>> + ModuleHookFn LinkerHook = Hook;
>> + Hook = [=](size_t Task, Module &M) {
>> + // If the linker's hook returned false, we need to pass that result
>> + // through.
>> + if (LinkerHook && !LinkerHook(Task, M))
>> + return false;
>> +
>> + std::string PathPrefix;
>> + // If this is the combined module (not a ThinLTO backend compile) or the
>> + // user hasn't requested using the input module's path, emit to a file
>> + // named from the provided OutputFileName with the Task ID appended.
>> + if (M.getModuleIdentifier() == "ld-temp.o" || !UseInputModulePath) {
>> + PathPrefix = OutputFileName;
>> + if (Task != 0)
>> + PathPrefix += "." + utostr(Task);
>> + } else
>> + PathPrefix = M.getModuleIdentifier();
>> + std::string Path = PathPrefix + "." + PathSuffix + ".bc";
>> + std::error_code EC;
>> + raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
>> + if (EC) {
>> + // Because -save-temps is a debugging feature, we report the error
>> + // directly and exit.
>> + llvm::errs() << "failed to open " << Path << ": " << EC.message()
>> + << '\n';
>> + exit(1);
>> + }
>> + WriteBitcodeToFile(&M, OS, /*ShouldPreserveUseListOrder=*/false);
>> + return true;
>> + };
>> + };
>> +
>> + setHook("0.preopt", PreOptModuleHook);
>> + setHook("1.promote", PostPromoteModuleHook);
>> + setHook("2.internalize", PostInternalizeModuleHook);
>> + setHook("3.import", PostImportModuleHook);
>> + setHook("4.opt", PostOptModuleHook);
>> + setHook("5.precodegen", PreCodeGenModuleHook);
>> +
>> + CombinedIndexHook = [=](const ModuleSummaryIndex &Index) {
>> + std::string Path = OutputFileName + ".index.bc";
>> + std::error_code EC;
>> + raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
>> + if (EC) {
>> + // Because -save-temps is a debugging feature, we report the error
>> + // directly and exit.
>> + llvm::errs() << "failed to open " << Path << ": " << EC.message() << '\n';
>> + exit(1);
>> + }
>> + WriteIndexToFile(Index, OS);
>> + return true;
>> + };
>> +
>> + return Error();
>> +}
>> +
>> +namespace {
>> +
>> +std::unique_ptr<TargetMachine>
>> +createTargetMachine(Config &C, StringRef TheTriple, const Target *TheTarget) {
>> + SubtargetFeatures Features;
>> + Features.getDefaultSubtargetFeatures(Triple(TheTriple));
>> + for (const std::string &A : C.MAttrs)
>> + Features.AddFeature(A);
>> +
>> + return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
>> + TheTriple, C.CPU, Features.getString(), C.Options, C.RelocModel,
>> + C.CodeModel, C.CGOptLevel));
>> +}
>> +
>> +bool opt(Config &C, TargetMachine *TM, size_t Task, Module &M, bool IsThinLto) {
>> + M.setDataLayout(TM->createDataLayout());
>> +
>> + legacy::PassManager passes;
>> + passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
>> +
>> + PassManagerBuilder PMB;
>> + PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
>> + PMB.Inliner = createFunctionInliningPass();
>> + // Unconditionally verify input since it is not verified before this
>> + // point and has unknown origin.
>> + PMB.VerifyInput = true;
>> + PMB.VerifyOutput = !C.DisableVerify;
>> + PMB.LoopVectorize = true;
>> + PMB.SLPVectorize = true;
>> + PMB.OptLevel = C.OptLevel;
>> + if (IsThinLto)
>> + PMB.populateThinLTOPassManager(passes);
>> + else
>> + PMB.populateLTOPassManager(passes);
>> + passes.run(M);
>> +
>> + if (C.PostOptModuleHook && !C.PostOptModuleHook(Task, M))
>> + return false;
>> +
>> + return true;
>> +}
>> +
>> +void codegen(Config &C, TargetMachine *TM, AddStreamFn AddStream, size_t Task,
>> + Module &M) {
>> + if (C.PreCodeGenModuleHook && !C.PreCodeGenModuleHook(Task, M))
>> + return;
>> +
>> + std::unique_ptr<raw_pwrite_stream> OS = AddStream(Task);
>> + legacy::PassManager CodeGenPasses;
>> + if (TM->addPassesToEmitFile(CodeGenPasses, *OS,
>> + TargetMachine::CGFT_ObjectFile))
>> + report_fatal_error("Failed to setup codegen");
>> + CodeGenPasses.run(M);
>> +}
>> +
>> +void splitCodeGen(Config &C, TargetMachine *TM, AddStreamFn AddStream,
>> + unsigned ParallelCodeGenParallelismLevel,
>> + std::unique_ptr<Module> M) {
>> + ThreadPool CodegenThreadPool(ParallelCodeGenParallelismLevel);
>> + unsigned ThreadCount = 0;
>> + const Target *T = &TM->getTarget();
>> +
>> + SplitModule(
>> + std::move(M), ParallelCodeGenParallelismLevel,
>> + [&](std::unique_ptr<Module> MPart) {
>> + // We want to clone the module in a new context to multi-thread the
>> + // codegen. We do it by serializing partition modules to bitcode
>> + // (while still on the main thread, in order to avoid data races) and
>> + // spinning up new threads which deserialize the partitions into
>> + // separate contexts.
>> + // FIXME: Provide a more direct way to do this in LLVM.
>> + SmallString<0> BC;
>> + raw_svector_ostream BCOS(BC);
>> + WriteBitcodeToFile(MPart.get(), BCOS);
>> +
>> + // Enqueue the task
>> + CodegenThreadPool.async(
>> + [&](const SmallString<0> &BC, unsigned ThreadId) {
>> + LTOLLVMContext Ctx(C);
>> + ErrorOr<std::unique_ptr<Module>> MOrErr = parseBitcodeFile(
>> + MemoryBufferRef(StringRef(BC.data(), BC.size()), "ld-temp.o"),
>> + Ctx);
>> + if (!MOrErr)
>> + report_fatal_error("Failed to read bitcode");
>> + std::unique_ptr<Module> MPartInCtx = std::move(MOrErr.get());
>> +
>> + std::unique_ptr<TargetMachine> TM =
>> + createTargetMachine(C, MPartInCtx->getTargetTriple(), T);
>> + codegen(C, TM.get(), AddStream, ThreadId, *MPartInCtx);
>> + },
>> + // Pass BC using std::move to ensure that it get moved rather than
>> + // copied into the thread's context.
>> + std::move(BC), ThreadCount++);
>> + },
>> + false);
>> +}
>> +
>> +Expected<const Target *> initAndLookupTarget(Config &C, Module &M) {
>> + if (!C.OverrideTriple.empty())
>> + M.setTargetTriple(C.OverrideTriple);
>> + else if (M.getTargetTriple().empty())
>> + M.setTargetTriple(C.DefaultTriple);
>> +
>> + std::string Msg;
>> + const Target *T = TargetRegistry::lookupTarget(M.getTargetTriple(), Msg);
>> + if (!T)
>> + return make_error<StringError>(Msg, inconvertibleErrorCode());
>> + return T;
>> +}
>> +
>> +}
>> +
>> +Error lto::backend(Config &C, AddStreamFn AddStream,
>> + unsigned ParallelCodeGenParallelismLevel,
>> + std::unique_ptr<Module> M) {
>> + Expected<const Target *> TOrErr = initAndLookupTarget(C, *M);
>> + if (!TOrErr)
>> + return TOrErr.takeError();
>> +
>> + std::unique_ptr<TargetMachine> TM =
>> + createTargetMachine(C, M->getTargetTriple(), *TOrErr);
>> +
>> + if (!opt(C, TM.get(), 0, *M, /*IsThinLto=*/false))
>> + return Error();
>> +
>> + if (ParallelCodeGenParallelismLevel == 1)
>> + codegen(C, TM.get(), AddStream, 0, *M);
>> + else
>> + splitCodeGen(C, TM.get(), AddStream, ParallelCodeGenParallelismLevel,
>> + std::move(M));
>> + return Error();
>> +}
>> +
>> +Error lto::thinBackend(Config &C, size_t Task, AddStreamFn AddStream, Module &M,
>> + ModuleSummaryIndex &CombinedIndex,
>> + const FunctionImporter::ImportMapTy &ImportList,
>> + const GVSummaryMapTy &DefinedGlobals,
>> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
>> + Expected<const Target *> TOrErr = initAndLookupTarget(C, M);
>> + if (!TOrErr)
>> + return TOrErr.takeError();
>> +
>> + std::unique_ptr<TargetMachine> TM =
>> + createTargetMachine(C, M.getTargetTriple(), *TOrErr);
>> +
>> + if (C.PreOptModuleHook && !C.PreOptModuleHook(Task, M))
>> + return Error();
>> +
>> + thinLTOResolveWeakForLinkerModule(M, DefinedGlobals);
>> +
>> + renameModuleForThinLTO(M, CombinedIndex);
>> +
>> + if (C.PostPromoteModuleHook && !C.PostPromoteModuleHook(Task, M))
>> + return Error();
>> +
>> + if (!DefinedGlobals.empty())
>> + thinLTOInternalizeModule(M, DefinedGlobals);
>> +
>> + if (C.PostInternalizeModuleHook && !C.PostInternalizeModuleHook(Task, M))
>> + return Error();
>> +
>> + auto ModuleLoader = [&](StringRef Identifier) {
>> + return std::move(getLazyBitcodeModule(MemoryBuffer::getMemBuffer(
>> + ModuleMap[Identifier], false),
>> + M.getContext(),
>> + /*ShouldLazyLoadMetadata=*/true)
>> + .get());
>> + };
>> +
>> + FunctionImporter Importer(CombinedIndex, ModuleLoader);
>> + Importer.importFunctions(M, ImportList);
>> +
>> + if (C.PostImportModuleHook && !C.PostImportModuleHook(Task, M))
>> + return Error();
>> +
>> + if (!opt(C, TM.get(), Task, M, /*IsThinLto=*/true))
>> + return Error();
>> +
>> + codegen(C, TM.get(), AddStream, Task, M);
>> + return Error();
>> +}
>>
>> Modified: llvm/trunk/lib/Object/IRObjectFile.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Object/IRObjectFile.cpp?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/Object/IRObjectFile.cpp (original)
>> +++ llvm/trunk/lib/Object/IRObjectFile.cpp Thu Aug 11 09:58:12 2016
>> @@ -324,5 +324,5 @@ llvm::object::IRObjectFile::create(Memor
>> return EC;
>>
>> std::unique_ptr<Module> &M = MOrErr.get();
>> - return llvm::make_unique<IRObjectFile>(Object, std::move(M));
>> + return llvm::make_unique<IRObjectFile>(BCOrErr.get(), std::move(M));
>> }
>>
>> Modified: llvm/trunk/test/CMakeLists.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CMakeLists.txt?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/CMakeLists.txt (original)
>> +++ llvm/trunk/test/CMakeLists.txt Thu Aug 11 09:58:12 2016
>> @@ -43,6 +43,7 @@ set(LLVM_TEST_DEPENDS
>> llvm-extract
>> llvm-lib
>> llvm-link
>> + llvm-lto2
>> llvm-mc
>> llvm-mcmarkup
>> llvm-nm
>>
>> Added: llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll (added)
>> +++ llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,4 @@
>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>> +target triple = "x86_64-unknown-linux-gnu"
>> +
>> + at a = global i32 42
>>
>> Added: llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll (added)
>> +++ llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,28 @@
>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>> +target triple = "x86_64-unknown-linux-gnu"
>> +
>> +$c2 = comdat any
>> +$c1 = comdat any
>> +
>> +; This is only present in this file. The linker will keep $c1 from the first
>> +; file and this will be undefined.
>> + at will_be_undefined = global i32 1, comdat($c1)
>> +
>> + at v1 = weak_odr global i32 41, comdat($c2)
>> +define weak_odr protected i32 @f1(i8* %this) comdat($c2) {
>> +bb20:
>> + store i8* %this, i8** null
>> + br label %bb21
>> +bb21:
>> + ret i32 41
>> +}
>> +
>> + at r21 = global i32* @v1
>> + at r22 = global i32(i8*)* @f1
>> +
>> + at a21 = alias i32, i32* @v1
>> + at a22 = alias i16, bitcast (i32* @v1 to i16*)
>> +
>> + at a23 = alias i32(i8*), i32(i8*)* @f1
>> + at a24 = alias i16, bitcast (i32(i8*)* @f1 to i16*)
>> + at a25 = alias i16, i16* @a24
>>
>> Added: llvm/trunk/test/LTO/Resolution/X86/alias.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/alias.ll?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/test/LTO/Resolution/X86/alias.ll (added)
>> +++ llvm/trunk/test/LTO/Resolution/X86/alias.ll Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,22 @@
>> +; RUN: llvm-as %s -o %t1.o
>> +; RUN: llvm-as %p/Inputs/alias-1.ll -o %t2.o
>> +; RUN: llvm-lto2 -o %t3.o %t2.o %t1.o -r %t2.o,a,px -r %t1.o,a, -r %t1.o,b,px -save-temps
>> +; RUN: llvm-dis < %t3.o.0.preopt.bc -o - | FileCheck %s
>> +; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
>> +
>> +; CHECK-NOT: alias
>> +; CHECK: @a = global i32 42
>> +; CHECK-NEXT: @b = global i32 1
>> +; CHECK-NOT: alias
>> +
>> +; RES: 2.o{{$}}
>> +; RES: {{^}}-r={{.*}}2.o,a,px{{$}}
>> +; RES: 1.o{{$}}
>> +; RES: {{^}}-r={{.*}}1.o,b,px{{$}}
>> +; RES: {{^}}-r={{.*}}1.o,a,{{$}}
>> +
>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>> +target triple = "x86_64-unknown-linux-gnu"
>> +
>> + at a = weak alias i32, i32* @b
>> + at b = global i32 1
>>
>> Copied: llvm/trunk/test/LTO/Resolution/X86/comdat.ll (from r278331, llvm/trunk/test/tools/gold/X86/comdat.ll)
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/comdat.ll?p2=llvm/trunk/test/LTO/Resolution/X86/comdat.ll&p1=llvm/trunk/test/tools/gold/X86/comdat.ll&r1=278331&r2=278338&rev=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/comdat.ll (original)
>> +++ llvm/trunk/test/LTO/Resolution/X86/comdat.ll Thu Aug 11 09:58:12 2016
>> @@ -1,8 +1,29 @@
>> ; RUN: llvm-as %s -o %t.o
>> ; RUN: llvm-as %p/Inputs/comdat.ll -o %t2.o
>> -; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t.o %t2.o \
>> -; RUN: -plugin-opt=save-temps
>> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
>> +; RUN: llvm-lto2 -save-temps -o %t3.o %t.o %t2.o \
>> +; RUN: -r=%t.o,f1,plx \
>> +; RUN: -r=%t.o,v1,px \
>> +; RUN: -r=%t.o,r11,px \
>> +; RUN: -r=%t.o,r12,px \
>> +; RUN: -r=%t.o,a11,px \
>> +; RUN: -r=%t.o,a12,px \
>> +; RUN: -r=%t.o,a13,px \
>> +; RUN: -r=%t.o,a14,px \
>> +; RUN: -r=%t.o,a15,px \
>> +; RUN: -r=%t2.o,f1,l \
>> +; RUN: -r=%t2.o,will_be_undefined, \
>> +; RUN: -r=%t2.o,v1, \
>> +; RUN: -r=%t2.o,r21,px \
>> +; RUN: -r=%t2.o,r22,px \
>> +; RUN: -r=%t2.o,a21,px \
>> +; RUN: -r=%t2.o,a22,px \
>> +; RUN: -r=%t2.o,a23,px \
>> +; RUN: -r=%t2.o,a24,px \
>> +; RUN: -r=%t2.o,a25,px
>> +; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
>> +
>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>> +target triple = "x86_64-unknown-linux-gnu"
>>
>> $c1 = comdat any
>>
>> @@ -49,7 +70,7 @@ bb11:
>> ; CHECK-DAG: @a23 = alias i32 (i8*), i32 (i8*)* @f1.2{{$}}
>> ; CHECK-DAG: @a24 = alias i16, bitcast (i32 (i8*)* @f1.2 to i16*)
>>
>> -; CHECK: define weak_odr protected i32 @f1(i8*) comdat($c1) {
>> +; CHECK: define weak_odr i32 @f1(i8*) comdat($c1) {
>> ; CHECK-NEXT: bb10:
>> ; CHECK-NEXT: br label %bb11{{$}}
>> ; CHECK: bb11:
>>
>> Added: llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg (added)
>> +++ llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,2 @@
>> +if not 'X86' in config.root.targets:
>> + config.unsupported = True
>>
>> Modified: llvm/trunk/test/lit.cfg
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/lit.cfg?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/lit.cfg (original)
>> +++ llvm/trunk/test/lit.cfg Thu Aug 11 09:58:12 2016
>> @@ -295,6 +295,7 @@ for pattern in [r"\bbugpoint\b(?!-)",
>> r"\bllvm-lib\b",
>> r"\bllvm-link\b",
>> r"\bllvm-lto\b",
>> + r"\bllvm-lto2\b",
>> r"\bllvm-mc\b",
>> r"\bllvm-mcmarkup\b",
>> r"\bllvm-nm\b",
>>
>> Modified: llvm/trunk/test/tools/gold/X86/coff.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/coff.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/coff.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/coff.ll Thu Aug 11 09:58:12 2016
>> @@ -16,7 +16,7 @@ define hidden void @g() {
>> ret void
>> }
>>
>> -; CHECK: define internal void @h() local_unnamed_addr {
>> +; CHECK: define internal void @h() {
>> define linkonce_odr void @h() local_unnamed_addr {
>> ret void
>> }
>>
>> Modified: llvm/trunk/test/tools/gold/X86/comdat.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/comdat.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/comdat.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/comdat.ll Thu Aug 11 09:58:12 2016
>> @@ -1,8 +1,9 @@
>> -; RUN: llvm-as %s -o %t.o
>> +; RUN: llvm-as %s -o %t1.o
>> ; RUN: llvm-as %p/Inputs/comdat.ll -o %t2.o
>> -; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t.o %t2.o \
>> +; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t1.o %t2.o \
>> ; RUN: -plugin-opt=save-temps
>> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
>> +; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
>> +; RUN: llvm-readobj -t %t3.o | FileCheck --check-prefix=OBJ %s
>>
>> $c1 = comdat any
>>
>> @@ -24,42 +25,37 @@ bb11:
>> @a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
>> @a15 = alias i16, i16* @a14
>>
>> -; CHECK: $c1 = comdat any
>> -; CHECK: $c2 = comdat any
>> +; gold's resolutions should tell us that our $c1 wins, and the other input's $c2
>> +; wins. f1 is also local due to having protected visibility in the other object.
>>
>> -; CHECK-DAG: @v1 = weak_odr global i32 42, comdat($c1)
>> -
>> -; CHECK-DAG: @r11 = global i32* @v1{{$}}
>> -; CHECK-DAG: @r12 = global i32 (i8*)* @f1{{$}}
>> -
>> -; CHECK-DAG: @r21 = global i32* @v1{{$}}
>> -; CHECK-DAG: @r22 = global i32 (i8*)* @f1{{$}}
>> -
>> -; CHECK-DAG: @v1.1 = internal global i32 41, comdat($c2)
>> -
>> -; CHECK-DAG: @a11 = alias i32, i32* @v1{{$}}
>> -; CHECK-DAG: @a12 = alias i16, bitcast (i32* @v1 to i16*)
>> -
>> -; CHECK-DAG: @a13 = alias i32 (i8*), i32 (i8*)* @f1{{$}}
>> -; CHECK-DAG: @a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
>> -
>> -; CHECK-DAG: @a21 = alias i32, i32* @v1.1{{$}}
>> -; CHECK-DAG: @a22 = alias i16, bitcast (i32* @v1.1 to i16*)
>> -
>> -; CHECK-DAG: @a23 = alias i32 (i8*), i32 (i8*)* @f1.2{{$}}
>> -; CHECK-DAG: @a24 = alias i16, bitcast (i32 (i8*)* @f1.2 to i16*)
>> -
>> -; CHECK: define weak_odr protected i32 @f1(i8*) comdat($c1) {
>> -; CHECK-NEXT: bb10:
>> -; CHECK-NEXT: br label %bb11{{$}}
>> -; CHECK: bb11:
>> -; CHECK-NEXT: ret i32 42
>> -; CHECK-NEXT: }
>> -
>> -; CHECK: define internal i32 @f1.2(i8* %this) comdat($c2) {
>> -; CHECK-NEXT: bb20:
>> -; CHECK-NEXT: store i8* %this, i8** null
>> -; CHECK-NEXT: br label %bb21
>> -; CHECK: bb21:
>> -; CHECK-NEXT: ret i32 41
>> -; CHECK-NEXT: }
>> +; RES: 1.o,f1,plx{{$}}
>> +; RES: 1.o,v1,px{{$}}
>> +; RES: 1.o,r11,px{{$}}
>> +; RES: 1.o,r12,px{{$}}
>> +; RES: 1.o,a11,px{{$}}
>> +; RES: 1.o,a12,px{{$}}
>> +; RES: 1.o,a13,px{{$}}
>> +; RES: 1.o,a14,px{{$}}
>> +; RES: 1.o,a15,px{{$}}
>> +
>> +; RES: 2.o,f1,l{{$}}
>> +; RES: 2.o,will_be_undefined,{{$}}
>> +; RES: 2.o,v1,{{$}}
>> +; RES: 2.o,r21,px{{$}}
>> +; RES: 2.o,r22,px{{$}}
>> +; RES: 2.o,a21,px{{$}}
>> +; RES: 2.o,a22,px{{$}}
>> +; RES: 2.o,a23,px{{$}}
>> +; RES: 2.o,a24,px{{$}}
>> +; RES: 2.o,a25,px{{$}}
>> +
>> +; f1's protected visibility should be reflected in the DSO.
>> +
>> +; OBJ: Name: f1 (
>> +; OBJ-NEXT: Value:
>> +; OBJ-NEXT: Size:
>> +; OBJ-NEXT: Binding:
>> +; OBJ-NEXT: Type:
>> +; OBJ-NEXT: Other [
>> +; OBJ-NEXT: STV_PROTECTED
>> +; OBJ-NEXT: ]
>>
>> Modified: llvm/trunk/test/tools/gold/X86/common.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/common.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/common.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/common.ll Thu Aug 11 09:58:12 2016
>> @@ -11,7 +11,7 @@
>> ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=A
>>
>> ; Shared library case, we merge @a as common and keep it for the symbol table.
>> -; A: @a = common global i32 0, align 8
>> +; A: @a = common global [4 x i8] zeroinitializer, align 8
>>
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=emit-llvm \
>> @@ -19,7 +19,7 @@
>> ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=B
>>
>> ; (i16 align 8) + (i8 align 16) = i16 align 16
>> -; B: @a = common global i16 0, align 16
>> +; B: @a = common global [2 x i8] zeroinitializer, align 16
>>
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=emit-llvm \
>> @@ -27,7 +27,7 @@
>> ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=C
>>
>> ; (i16 align 8) + (i8 align 1) = i16 align 8.
>> -; C: @a = common global i16 0, align 8
>> +; C: @a = common global [2 x i8] zeroinitializer, align 8
>>
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=emit-llvm \
>> @@ -35,7 +35,7 @@
>> ; RUN: llvm-dis %t3.o -o - | FileCheck --check-prefix=EXEC %s
>>
>> ; All IR case, we internalize a after merging.
>> -; EXEC: @a = internal global i32 0, align 8
>> +; EXEC: @a = internal global [4 x i8] zeroinitializer, align 8
>>
>> ; RUN: llc %p/Inputs/common.ll -o %t2native.o -filetype=obj
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> @@ -44,4 +44,4 @@
>> ; RUN: llvm-dis %t3.o -o - | FileCheck --check-prefix=MIXED %s
>>
>> ; Mixed ELF and IR. We keep ours as common so the linker will finish the merge.
>> -; MIXED: @a = common global i16 0, align 8
>> +; MIXED: @a = common global [2 x i8] zeroinitializer, align 8
>>
>> Modified: llvm/trunk/test/tools/gold/X86/emit-llvm.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/emit-llvm.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/emit-llvm.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/emit-llvm.ll Thu Aug 11 09:58:12 2016
>> @@ -2,17 +2,16 @@
>>
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=emit-llvm \
>> -; RUN: --plugin-opt=generate-api-file \
>> ; RUN: -shared %t.o -o %t2.o
>> ; RUN: llvm-dis %t2.o -o - | FileCheck %s
>> -; RUN: FileCheck --check-prefix=API %s < %T/../apifile.txt
>>
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: -m elf_x86_64 --plugin-opt=save-temps \
>> ; RUN: -shared %t.o -o %t3.o
>> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
>> -; RUN: llvm-dis %t3.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
>> -; RUN: llvm-dis %t3.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>> +; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
>> +; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
>> +; RUN: llvm-dis %t3.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
>> +; RUN: llvm-dis %t3.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>> ; RUN: llvm-nm %t3.o.o | FileCheck --check-prefix=NM %s
>>
>> ; RUN: rm -f %t4.o
>> @@ -25,19 +24,19 @@
>>
>> target triple = "x86_64-unknown-linux-gnu"
>>
>> -; CHECK-DAG: @g1 = linkonce_odr constant i32 32
>> +; CHECK-DAG: @g1 = weak_odr constant i32 32
>> @g1 = linkonce_odr constant i32 32
>>
>> -; CHECK-DAG: @g2 = internal local_unnamed_addr constant i32 32
>> +; CHECK-DAG: @g2 = internal constant i32 32
>> @g2 = linkonce_odr local_unnamed_addr constant i32 32
>>
>> ; CHECK-DAG: @g3 = internal unnamed_addr constant i32 32
>> @g3 = linkonce_odr unnamed_addr constant i32 32
>>
>> -; CHECK-DAG: @g4 = linkonce_odr global i32 32
>> +; CHECK-DAG: @g4 = weak_odr global i32 32
>> @g4 = linkonce_odr global i32 32
>>
>> -; CHECK-DAG: @g5 = linkonce_odr local_unnamed_addr global i32 32
>> +; CHECK-DAG: @g5 = weak_odr global i32 32
>> @g5 = linkonce_odr local_unnamed_addr global i32 32
>>
>> ; CHECK-DAG: @g6 = internal unnamed_addr global i32 32
>> @@ -75,8 +74,8 @@ define linkonce_odr void @f4() local_unn
>> ret void
>> }
>>
>> -; CHECK-DAG: define linkonce_odr void @f5()
>> -; OPT-DAG: define linkonce_odr void @f5()
>> +; CHECK-DAG: define weak_odr void @f5()
>> +; OPT-DAG: define weak_odr void @f5()
>> define linkonce_odr void @f5() {
>> ret void
>> }
>> @@ -97,15 +96,21 @@ define i32* @f8() {
>> ret i32* @g8
>> }
>>
>> -; API: f1 PREVAILING_DEF_IRONLY
>> -; API: f2 PREVAILING_DEF_IRONLY
>> -; API: f3 PREVAILING_DEF_IRONLY_EXP
>> -; API: f4 PREVAILING_DEF_IRONLY_EXP
>> -; API: f5 PREVAILING_DEF_IRONLY_EXP
>> -; API: f6 PREVAILING_DEF_IRONLY_EXP
>> -; API: f7 PREVAILING_DEF_IRONLY_EXP
>> -; API: f8 PREVAILING_DEF_IRONLY_EXP
>> -; API: g7 UNDEF
>> -; API: g8 UNDEF
>> -; API: g9 PREVAILING_DEF_IRONLY_EXP
>> -; API: g10 PREVAILING_DEF_IRONLY_EXP
>> +; RES: .o,f1,pl{{$}}
>> +; RES: .o,f2,pl{{$}}
>> +; RES: .o,f3,px{{$}}
>> +; RES: .o,f4,p{{$}}
>> +; RES: .o,f5,px{{$}}
>> +; RES: .o,f6,p{{$}}
>> +; RES: .o,f7,px{{$}}
>> +; RES: .o,f8,px{{$}}
>> +; RES: .o,g1,px{{$}}
>> +; RES: .o,g2,p{{$}}
>> +; RES: .o,g3,p{{$}}
>> +; RES: .o,g4,px{{$}}
>> +; RES: .o,g5,px{{$}}
>> +; RES: .o,g6,p{{$}}
>> +; RES: .o,g7,{{$}}
>> +; RES: .o,g8,{{$}}
>> +; RES: .o,g9,px{{$}}
>> +; RES: .o,g10,px{{$}}
>>
>> Modified: llvm/trunk/test/tools/gold/X86/opt-level.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/opt-level.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/opt-level.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/opt-level.ll Thu Aug 11 09:58:12 2016
>> @@ -1,13 +1,13 @@
>> ; RUN: llvm-as -o %t.bc %s
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
>> ; RUN: -plugin-opt=O0 -r -o %t.o %t.bc
>> -; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O0 %s
>> +; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O0 %s
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
>> ; RUN: -plugin-opt=O1 -r -o %t.o %t.bc
>> -; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O1 %s
>> +; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O1 %s
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
>> ; RUN: -plugin-opt=O2 -r -o %t.o %t.bc
>> -; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O2 %s
>> +; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O2 %s
>>
>> ; CHECK-O0: define internal void @foo(
>> ; CHECK-O1: define internal void @foo(
>>
>> Modified: llvm/trunk/test/tools/gold/X86/parallel.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/parallel.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/parallel.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/parallel.ll Thu Aug 11 09:58:12 2016
>> @@ -1,7 +1,8 @@
>> ; RUN: llvm-as -o %t.bc %s
>> +; RUN: rm -f %t.opt.bc0 %t.opt.bc1 %t.o0 %t.o1
>> ; RUN: env LD_PRELOAD=%llvmshlibdir/LLVMgold.so %gold -plugin %llvmshlibdir/LLVMgold.so -u foo -u bar -plugin-opt jobs=2 -plugin-opt save-temps -m elf_x86_64 -o %t %t.bc
>> -; RUN: llvm-dis %t.opt.bc0 -o - | FileCheck --check-prefix=CHECK-BC0 %s
>> -; RUN: llvm-dis %t.opt.bc1 -o - | FileCheck --check-prefix=CHECK-BC1 %s
>> +; RUN: llvm-dis %t.5.precodegen.bc -o - | FileCheck --check-prefix=CHECK-BC0 %s
>> +; RUN: llvm-dis %t.1.5.precodegen.bc -o - | FileCheck --check-prefix=CHECK-BC1 %s
>> ; RUN: llvm-nm %t.o0 | FileCheck --check-prefix=CHECK0 %s
>> ; RUN: llvm-nm %t.o1 | FileCheck --check-prefix=CHECK1 %s
>>
>>
>> Modified: llvm/trunk/test/tools/gold/X86/slp-vectorize.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/slp-vectorize.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/slp-vectorize.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/slp-vectorize.ll Thu Aug 11 09:58:12 2016
>> @@ -3,7 +3,7 @@
>> ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=save-temps \
>> ; RUN: -shared %t.o -o %t2.o
>> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck %s
>> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck %s
>>
>> ; test that the vectorizer is run.
>> ; CHECK: fadd <4 x float>
>>
>> Modified: llvm/trunk/test/tools/gold/X86/start-lib-common.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/start-lib-common.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/start-lib-common.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/start-lib-common.ll Thu Aug 11 09:58:12 2016
>> @@ -19,4 +19,4 @@
>>
>> ; Check that the common symbol is not dropped completely, which was a regression
>> ; in r262676.
>> -; CHECK: @x = common global i32 0
>> +; CHECK: @x = common global [4 x i8] zeroinitializer
>>
>> Modified: llvm/trunk/test/tools/gold/X86/strip_names.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/strip_names.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/strip_names.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/strip_names.ll Thu Aug 11 09:58:12 2016
>> @@ -3,7 +3,7 @@
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=save-temps \
>> ; RUN: -shared %t.o -o %t2.o
>> -; RUN: llvm-dis %t2.o.bc -o - | FileCheck %s
>> +; RUN: llvm-dis %t2.o.2.internalize.bc -o - | FileCheck %s
>>
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=emit-llvm \
>>
>> Modified: llvm/trunk/test/tools/gold/X86/thinlto.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/thinlto.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/thinlto.ll Thu Aug 11 09:58:12 2016
>> @@ -25,21 +25,23 @@
>> ; RUN: llvm-bcanalyzer -dump %t2.o.thinlto.bc | FileCheck %s --check-prefix=BACKEND2
>> ; RUN: not test -e %t3
>>
>> -; Ensure gold generates an index as well as a binary by default in ThinLTO mode.
>> +; Ensure gold generates an index as well as a binary with save-temps in ThinLTO mode.
>> ; First force single-threaded mode
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> +; RUN: --plugin-opt=save-temps \
>> ; RUN: --plugin-opt=thinlto \
>> ; RUN: --plugin-opt=jobs=1 \
>> ; RUN: -shared %t.o %t2.o -o %t4
>> -; RUN: llvm-bcanalyzer -dump %t4.thinlto.bc | FileCheck %s --check-prefix=COMBINED
>> +; RUN: llvm-bcanalyzer -dump %t4.index.bc | FileCheck %s --check-prefix=COMBINED
>> ; RUN: llvm-nm %t4 | FileCheck %s --check-prefix=NM
>>
>> ; Next force multi-threaded mode
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> +; RUN: --plugin-opt=save-temps \
>> ; RUN: --plugin-opt=thinlto \
>> ; RUN: --plugin-opt=jobs=2 \
>> ; RUN: -shared %t.o %t2.o -o %t4
>> -; RUN: llvm-bcanalyzer -dump %t4.thinlto.bc | FileCheck %s --check-prefix=COMBINED
>> +; RUN: llvm-bcanalyzer -dump %t4.index.bc | FileCheck %s --check-prefix=COMBINED
>> ; RUN: llvm-nm %t4 | FileCheck %s --check-prefix=NM
>>
>> ; Test --plugin-opt=obj-path to ensure unique object files generated.
>> @@ -48,8 +50,8 @@
>> ; RUN: --plugin-opt=jobs=2 \
>> ; RUN: --plugin-opt=obj-path=%t5.o \
>> ; RUN: -shared %t.o %t2.o -o %t4
>> -; RUN: llvm-nm %t5.o0 | FileCheck %s --check-prefix=NM2
>> ; RUN: llvm-nm %t5.o1 | FileCheck %s --check-prefix=NM2
>> +; RUN: llvm-nm %t5.o2 | FileCheck %s --check-prefix=NM2
>>
>> ; NM: T f
>> ; NM2: T {{f|g}}
>>
>> Modified: llvm/trunk/test/tools/gold/X86/thinlto_alias.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_alias.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/thinlto_alias.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/thinlto_alias.ll Thu Aug 11 09:58:12 2016
>> @@ -14,8 +14,14 @@
>> ; RUN: --plugin-opt=save-temps \
>> ; RUN: -o %t3.o %t2.o %t.o
>> ; RUN: llvm-nm %t3.o | FileCheck %s
>> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
>> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
>> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>> +
>> +; This does not currently pass because the gold plugin now uses the
>> +; combined summary rather than the IRMover to change the module's linkage
>> +; during the ThinLTO backend. The internalization step implemented by IRMover
>> +; for preempted symbols has not yet been implemented for the combined summary.
>> +; XFAIL: *
>>
>> ; CHECK-NOT: U f
>> ; OPT: define hidden void @weakfunc.llvm.0()
>>
>> Modified: llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll Thu Aug 11 09:58:12 2016
>> @@ -6,7 +6,7 @@
>> ; RUN: --plugin-opt=-import-instr-limit=0 \
>> ; RUN: --plugin-opt=save-temps \
>> ; RUN: -o %t3.o %t2.o %t.o
>> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck %s
>> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck %s
>>
>> ; f() should be internalized and eliminated after inlining
>> ; CHECK-NOT: @f()
>>
>> Modified: llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll Thu Aug 11 09:58:12 2016
>> @@ -14,13 +14,13 @@
>> ; RUN: -shared \
>> ; RUN: -o %t3.o %t2.o %t.o
>> ; RUN: llvm-nm %t3.o | FileCheck %s
>> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
>> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
>> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>>
>> ; Ensure that f() is defined in resulting object file, and also
>> ; confirm the weak linkage directly in the saved opt bitcode files.
>> ; CHECK-NOT: U f
>> -; OPT: declare hidden void @f()
>> +; OPT-NOT: @f()
>> ; OPT2: define weak_odr hidden void @f()
>>
>> target triple = "x86_64-unknown-linux-gnu"
>>
>> Modified: llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll Thu Aug 11 09:58:12 2016
>> @@ -13,12 +13,17 @@
>> ; RUN: llvm-nm %t3.o | FileCheck %s
>> ; CHECK: weakfunc
>>
>> -; All of the preempted functions should have been eliminated (the plugin will
>> -; not link them in).
>> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>> +; Most of the preempted functions should have been eliminated (the plugin will
>> +; set linkage of odr functions to available_externally and linkonce functions
>> +; are removed by globaldce). FIXME: Need to introduce combined index linkage
>> +; that means "drop this function" so we can avoid importing linkonce functions
>> +; and drop weak functions.
>> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>> +; OPT2-NOT: @
>> +; OPT2: @weakfunc
>> ; OPT2-NOT: @
>>
>> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
>> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
>>
>> target triple = "x86_64-unknown-linux-gnu"
>>
>>
>> Modified: llvm/trunk/test/tools/gold/X86/type-merge2.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/type-merge2.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/type-merge2.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/type-merge2.ll Thu Aug 11 09:58:12 2016
>> @@ -3,7 +3,7 @@
>> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=save-temps \
>> ; RUN: -shared %t.o %t2.o -o %t3.o
>> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
>> +; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
>>
>> %zed = type { i8 }
>> define void @foo() {
>>
>> Modified: llvm/trunk/test/tools/gold/X86/vectorize.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/vectorize.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/vectorize.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/vectorize.ll Thu Aug 11 09:58:12 2016
>> @@ -3,7 +3,7 @@
>> ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
>> ; RUN: --plugin-opt=save-temps \
>> ; RUN: -shared %t.o -o %t2.o
>> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck %s
>> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck %s
>>
>> ; test that the vectorizer is run.
>> ; CHECK: fadd <4 x float>
>>
>> Modified: llvm/trunk/test/tools/gold/X86/visibility.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/visibility.ll?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/test/tools/gold/X86/visibility.ll (original)
>> +++ llvm/trunk/test/tools/gold/X86/visibility.ll Thu Aug 11 09:58:12 2016
>> @@ -5,7 +5,7 @@
>> ; RUN: --plugin-opt=save-temps \
>> ; RUN: -shared %t.o %t2.o -o %t.so
>> ; RUN: llvm-readobj -t %t.so | FileCheck %s
>> -; RUN: llvm-dis %t.so.bc -o - | FileCheck --check-prefix=IR %s
>> +; RUN: llvm-dis %t.so.2.internalize.bc -o - | FileCheck --check-prefix=IR %s
>>
>> ; CHECK: Name: foo
>> ; CHECK-NEXT: Value:
>> @@ -16,7 +16,7 @@
>> ; CHECK-NEXT: STV_PROTECTED
>> ; CHECK-NEXT: ]
>>
>> -; IR: define protected void @foo
>> +; IR: define void @foo
>>
>> define weak protected void @foo() {
>> ret void
>>
>> Added: llvm/trunk/test/tools/llvm-lto2/errors.ll
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-lto2/errors.ll?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/test/tools/llvm-lto2/errors.ll (added)
>> +++ llvm/trunk/test/tools/llvm-lto2/errors.ll Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,11 @@
>> +; RUN: llvm-as %s -o %t.bc
>> +; RUN: not llvm-lto2 -o %t2.o %t.bc 2>&1 | FileCheck --check-prefix=ERR1 %s
>> +; RUN: not llvm-lto2 -o %t2.o -r %t.bc,foo,p -r %t.bc,bar,p %t.bc 2>&1 | FileCheck --check-prefix=ERR2 %s
>> +; RUN: not llvm-lto2 -o %t2.o -r %t.bc,foo,q %t.bc 2>&1 | FileCheck --check-prefix=ERR3 %s
>> +; RUN: not llvm-lto2 -o %t2.o -r foo %t.bc 2>&1 | FileCheck --check-prefix=ERR4 %s
>> +
>> +; ERR1: missing symbol resolution for {{.*}}.bc,foo
>> +; ERR2: unused symbol resolution for {{.*}}.bc,bar
>> +; ERR3: invalid character q in resolution: {{.*}}.bc,foo
>> +; ERR4: invalid resolution: foo
>> + at foo = global i32 0
>>
>> Modified: llvm/trunk/tools/gold/gold-plugin.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/gold/gold-plugin.cpp?rev=278338&r1=278337&r2=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/tools/gold/gold-plugin.cpp (original)
>> +++ llvm/trunk/tools/gold/gold-plugin.cpp Thu Aug 11 09:58:12 2016
>> @@ -12,44 +12,22 @@
>> //
>> //===----------------------------------------------------------------------===//
>>
>> -#include "llvm/ADT/StringSet.h"
>> -#include "llvm/Analysis/TargetLibraryInfo.h"
>> -#include "llvm/Analysis/TargetTransformInfo.h"
>> #include "llvm/Bitcode/ReaderWriter.h"
>> -#include "llvm/CodeGen/Analysis.h"
>> #include "llvm/CodeGen/CommandFlags.h"
>> -#include "llvm/CodeGen/ParallelCG.h"
>> #include "llvm/Config/config.h" // plugin-api.h requires HAVE_STDINT_H
>> -#include "llvm/IR/AutoUpgrade.h"
>> #include "llvm/IR/Constants.h"
>> -#include "llvm/IR/DiagnosticInfo.h"
>> #include "llvm/IR/DiagnosticPrinter.h"
>> -#include "llvm/IR/LLVMContext.h"
>> -#include "llvm/IR/LegacyPassManager.h"
>> -#include "llvm/IR/Module.h"
>> -#include "llvm/IR/Verifier.h"
>> #include "llvm/LTO/LTO.h"
>> -#include "llvm/Linker/IRMover.h"
>> -#include "llvm/MC/SubtargetFeature.h"
>> -#include "llvm/Object/IRObjectFile.h"
>> -#include "llvm/Object/ModuleSummaryIndexObjectFile.h"
>> -#include "llvm/Support/Host.h"
>> +#include "llvm/Support/CommandLine.h"
>> #include "llvm/Support/ManagedStatic.h"
>> #include "llvm/Support/MemoryBuffer.h"
>> #include "llvm/Support/Path.h"
>> -#include "llvm/Support/TargetRegistry.h"
>> #include "llvm/Support/TargetSelect.h"
>> -#include "llvm/Support/ThreadPool.h"
>> #include "llvm/Support/raw_ostream.h"
>> -#include "llvm/Support/thread.h"
>> -#include "llvm/Transforms/IPO.h"
>> -#include "llvm/Transforms/IPO/FunctionImport.h"
>> -#include "llvm/Transforms/IPO/PassManagerBuilder.h"
>> -#include "llvm/Transforms/Utils/FunctionImportUtils.h"
>> -#include "llvm/Transforms/Utils/GlobalStatus.h"
>> -#include "llvm/Transforms/Utils/ValueMapper.h"
>> #include <list>
>> +#include <map>
>> #include <plugin-api.h>
>> +#include <string>
>> #include <system_error>
>> #include <utility>
>> #include <vector>
>> @@ -61,6 +39,7 @@
>> #define LDPT_GET_SYMBOLS_V3 28
>>
>> using namespace llvm;
>> +using namespace lto;
>>
>> static ld_plugin_status discard_message(int level, const char *format, ...) {
>> // Die loudly. Recent versions of Gold pass ld_plugin_message as the first
>> @@ -106,38 +85,17 @@ struct PluginInputFile {
>> };
>>
>> struct ResolutionInfo {
>> - uint64_t CommonSize = 0;
>> - unsigned CommonAlign = 0;
>> - bool IsLinkonceOdr = true;
>> - GlobalValue::UnnamedAddr UnnamedAddr = GlobalValue::UnnamedAddr::Global;
>> - GlobalValue::VisibilityTypes Visibility = GlobalValue::DefaultVisibility;
>> - bool CommonInternal = false;
>> - bool UseCommon = false;
>> + bool CanOmitFromDynSym = true;
>> + bool DefaultVisibility = true;
>> };
>>
>> -/// Class to own information used by a task or during its cleanup for a
>> -/// ThinLTO backend instantiation.
>> -class ThinLTOTaskInfo {
>> - /// The output stream the task will codegen into.
>> - std::unique_ptr<raw_fd_ostream> OS;
>> -
>> - /// The file name corresponding to the output stream, used during cleanup.
>> - std::string Filename;
>> -
>> - /// Flag indicating whether the output file is a temp file that must be
>> - /// added to the cleanup list during cleanup.
>> - bool TempOutFile;
>> -
>> -public:
>> - ThinLTOTaskInfo(std::unique_ptr<raw_fd_ostream> OS, std::string Filename,
>> - bool TempOutFile)
>> - : OS(std::move(OS)), Filename(std::move(Filename)),
>> - TempOutFile(TempOutFile) {}
>> -
>> - /// Performs task related cleanup activities that must be done
>> - /// single-threaded (i.e. call backs to gold).
>> - void cleanup();
>> +struct CommonResolution {
>> + bool Prevailing = false;
>> + bool VisibleToRegularObj = false;
>> + uint64_t Size = 0;
>> + unsigned Align = 0;
>> };
>> +
>> }
>>
>> static ld_plugin_add_symbols add_symbols = nullptr;
>> @@ -145,14 +103,16 @@ static ld_plugin_get_symbols get_symbols
>> static ld_plugin_add_input_file add_input_file = nullptr;
>> static ld_plugin_set_extra_library_path set_extra_library_path = nullptr;
>> static ld_plugin_get_view get_view = nullptr;
>> +static bool IsExecutable = false;
>> static Optional<Reloc::Model> RelocationModel;
>> static std::string output_name = "";
>> static std::list<claimed_file> Modules;
>> static DenseMap<int, void *> FDToLeaderHandle;
>> static StringMap<ResolutionInfo> ResInfo;
>> +static std::map<std::string, CommonResolution> Commons;
>> static std::vector<std::string> Cleanup;
>> static llvm::TargetOptions TargetOpts;
>> -static std::string DefaultTriple = sys::getDefaultTargetTriple();
>> +static size_t MaxTasks;
>>
>> namespace options {
>> enum OutputType {
>> @@ -161,7 +121,6 @@ namespace options {
>> OT_BC_ONLY,
>> OT_SAVE_TEMPS
>> };
>> - static bool generate_api_file = false;
>> static OutputType TheOutputType = OT_NORMAL;
>> static unsigned OptLevel = 2;
>> // Default parallelism of 0 used to indicate that user did not specify.
>> @@ -213,8 +172,6 @@ namespace options {
>> // Additional options to pass into the code generator.
>> // Note: This array will contain all plugin options which are not claimed
>> // as plugin exclusive to pass to the code generator.
>> - // For example, "generate-api-file" and "as"options are for the plugin
>> - // use only and will not be passed.
>> static std::vector<const char *> extra;
>>
>> static void process_plugin_option(const char *opt_)
>> @@ -223,9 +180,7 @@ namespace options {
>> return;
>> llvm::StringRef opt = opt_;
>>
>> - if (opt == "generate-api-file") {
>> - generate_api_file = true;
>> - } else if (opt.startswith("mcpu=")) {
>> + if (opt.startswith("mcpu=")) {
>> mcpu = opt.substr(strlen("mcpu="));
>> } else if (opt.startswith("extra-library-path=")) {
>> extra_library_path = opt.substr(strlen("extra_library_path="));
>> @@ -307,10 +262,15 @@ ld_plugin_status onload(ld_plugin_tv *tv
>> switch (tv->tv_u.tv_val) {
>> case LDPO_REL: // .o
>> case LDPO_DYN: // .so
>> + IsExecutable = false;
>> + RelocationModel = Reloc::PIC_;
>> + break;
>> case LDPO_PIE: // position independent executable
>> + IsExecutable = true;
>> RelocationModel = Reloc::PIC_;
>> break;
>> case LDPO_EXEC: // .exe
>> + IsExecutable = true;
>> RelocationModel = Reloc::Static;
>> break;
>> default:
>> @@ -404,20 +364,6 @@ ld_plugin_status onload(ld_plugin_tv *tv
>> return LDPS_OK;
>> }
>>
>> -static const GlobalObject *getBaseObject(const GlobalValue &GV) {
>> - if (auto *GA = dyn_cast<GlobalAlias>(&GV))
>> - return GA->getBaseObject();
>> - return cast<GlobalObject>(&GV);
>> -}
>> -
>> -static bool shouldSkip(uint32_t Symflags) {
>> - if (!(Symflags & object::BasicSymbolRef::SF_Global))
>> - return true;
>> - if (Symflags & object::BasicSymbolRef::SF_FormatSpecific)
>> - return true;
>> - return false;
>> -}
>> -
>> static void diagnosticHandler(const DiagnosticInfo &DI) {
>> if (const auto *BDI = dyn_cast<BitcodeDiagnosticInfo>(&DI)) {
>> std::error_code EC = BDI->getError();
>> @@ -447,21 +393,18 @@ static void diagnosticHandler(const Diag
>> message(Level, "LLVM gold plugin: %s", ErrStorage.c_str());
>> }
>>
>> -static void diagnosticHandlerForContext(const DiagnosticInfo &DI,
>> - void *Context) {
>> - diagnosticHandler(DI);
>> +static void check(Error E, std::string Msg = "LLVM gold plugin") {
>> + handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
>> + message(LDPL_FATAL, "%s: %s", Msg.c_str(), EIB.message().c_str());
>> + return Error::success();
>> + });
>> }
>>
>> -static GlobalValue::VisibilityTypes
>> -getMinVisibility(GlobalValue::VisibilityTypes A,
>> - GlobalValue::VisibilityTypes B) {
>> - if (A == GlobalValue::HiddenVisibility)
>> - return A;
>> - if (B == GlobalValue::HiddenVisibility)
>> - return B;
>> - if (A == GlobalValue::ProtectedVisibility)
>> - return A;
>> - return B;
>> +template <typename T> static T check(Expected<T> E) {
>> + if (E)
>> + return std::move(*E);
>> + check(E.takeError());
>> + return T();
>> }
>>
>> /// Called by gold to see whether this file is one that our plugin can handle.
>> @@ -469,7 +412,6 @@ getMinVisibility(GlobalValue::Visibility
>> /// possible.
>> static ld_plugin_status claim_file_hook(const ld_plugin_input_file *file,
>> int *claimed) {
>> - LLVMContext Context;
>> MemoryBufferRef BufferRef;
>> std::unique_ptr<MemoryBuffer> Buffer;
>> if (get_view) {
>> @@ -498,22 +440,25 @@ static ld_plugin_status claim_file_hook(
>> BufferRef = Buffer->getMemBufferRef();
>> }
>>
>> - Context.setDiagnosticHandler(diagnosticHandlerForContext);
>> - ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
>> - object::IRObjectFile::create(BufferRef, Context);
>> - std::error_code EC = ObjOrErr.getError();
>> - if (EC == object::object_error::invalid_file_type ||
>> - EC == object::object_error::bitcode_section_not_found)
>> - return LDPS_OK;
>> -
>> *claimed = 1;
>>
>> - if (EC) {
>> - message(LDPL_ERROR, "LLVM gold plugin has failed to create LTO module: %s",
>> - EC.message().c_str());
>> - return LDPS_ERR;
>> + Expected<std::unique_ptr<InputFile>> ObjOrErr = InputFile::create(BufferRef);
>> + if (!ObjOrErr) {
>> + handleAllErrors(ObjOrErr.takeError(), [&](const ErrorInfoBase &EI) {
>> + std::error_code EC = EI.convertToErrorCode();
>> + if (EC == object::object_error::invalid_file_type ||
>> + EC == object::object_error::bitcode_section_not_found)
>> + *claimed = 0;
>> + else
>> + message(LDPL_ERROR,
>> + "LLVM gold plugin has failed to create LTO module: %s",
>> + EI.message().c_str());
>> + });
>> +
>> + return *claimed ? LDPS_ERR : LDPS_OK;
>> }
>> - std::unique_ptr<object::IRObjectFile> Obj = std::move(*ObjOrErr);
>> +
>> + std::unique_ptr<InputFile> Obj = std::move(*ObjOrErr);
>>
>> Modules.resize(Modules.size() + 1);
>> claimed_file &cf = Modules.back();
>> @@ -535,72 +480,52 @@ static ld_plugin_status claim_file_hook(
>> cf.name = file->name;
>> if (file->offset)
>> cf.name += ".llvm." + std::to_string(file->offset) + "." +
>> - sys::path::filename(Obj->getModule().getSourceFileName()).str();
>> + sys::path::filename(Obj->getSourceFileName()).str();
>>
>> for (auto &Sym : Obj->symbols()) {
>> uint32_t Symflags = Sym.getFlags();
>> - if (shouldSkip(Symflags))
>> - continue;
>>
>> cf.syms.push_back(ld_plugin_symbol());
>> ld_plugin_symbol &sym = cf.syms.back();
>> sym.version = nullptr;
>> + StringRef Name = Sym.getName();
>> + sym.name = strdup(Name.str().c_str());
>>
>> - SmallString<64> Name;
>> - {
>> - raw_svector_ostream OS(Name);
>> - Sym.printName(OS);
>> - }
>> - sym.name = strdup(Name.c_str());
>> -
>> - const GlobalValue *GV = Obj->getSymbolGV(Sym.getRawDataRefImpl());
>> + ResolutionInfo &Res = ResInfo[Name];
>>
>> - ResolutionInfo &Res = ResInfo[sym.name];
>> + Res.CanOmitFromDynSym &= Sym.canBeOmittedFromSymbolTable();
>>
>> sym.visibility = LDPV_DEFAULT;
>> - if (GV) {
>> - Res.UnnamedAddr =
>> - GlobalValue::getMinUnnamedAddr(Res.UnnamedAddr, GV->getUnnamedAddr());
>> - Res.IsLinkonceOdr &= GV->hasLinkOnceLinkage();
>> - Res.Visibility = getMinVisibility(Res.Visibility, GV->getVisibility());
>> - switch (GV->getVisibility()) {
>> - case GlobalValue::DefaultVisibility:
>> - break;
>> - case GlobalValue::HiddenVisibility:
>> - sym.visibility = LDPV_HIDDEN;
>> - break;
>> - case GlobalValue::ProtectedVisibility:
>> - sym.visibility = LDPV_PROTECTED;
>> - break;
>> - }
>> + GlobalValue::VisibilityTypes Vis = Sym.getVisibility();
>> + if (Vis != GlobalValue::DefaultVisibility)
>> + Res.DefaultVisibility = false;
>> + switch (Vis) {
>> + case GlobalValue::DefaultVisibility:
>> + break;
>> + case GlobalValue::HiddenVisibility:
>> + sym.visibility = LDPV_HIDDEN;
>> + break;
>> + case GlobalValue::ProtectedVisibility:
>> + sym.visibility = LDPV_PROTECTED;
>> + break;
>> }
>>
>> if (Symflags & object::BasicSymbolRef::SF_Undefined) {
>> sym.def = LDPK_UNDEF;
>> - if (GV && GV->hasExternalWeakLinkage())
>> + if (Symflags & object::BasicSymbolRef::SF_Weak)
>> sym.def = LDPK_WEAKUNDEF;
>> - } else {
>> + } else if (Symflags & object::BasicSymbolRef::SF_Common)
>> + sym.def = LDPK_COMMON;
>> + else if (Symflags & object::BasicSymbolRef::SF_Weak)
>> + sym.def = LDPK_WEAKDEF;
>> + else
>> sym.def = LDPK_DEF;
>> - if (GV) {
>> - assert(!GV->hasExternalWeakLinkage() &&
>> - !GV->hasAvailableExternallyLinkage() && "Not a declaration!");
>> - if (GV->hasCommonLinkage())
>> - sym.def = LDPK_COMMON;
>> - else if (GV->isWeakForLinker())
>> - sym.def = LDPK_WEAKDEF;
>> - }
>> - }
>>
>> sym.size = 0;
>> sym.comdat_key = nullptr;
>> - if (GV) {
>> - const GlobalObject *Base = getBaseObject(*GV);
>> - if (!Base)
>> - message(LDPL_FATAL, "Unable to determine comdat of alias!");
>> - const Comdat *C = Base->getComdat();
>> - if (C)
>> - sym.comdat_key = strdup(C->getName().str().c_str());
>> - }
>> + const Comdat *C = check(Sym.getComdat());
>> + if (C)
>> + sym.comdat_key = strdup(C->getName().str().c_str());
>>
>> sym.resolution = LDPR_UNKNOWN;
>> }
>> @@ -615,39 +540,6 @@ static ld_plugin_status claim_file_hook(
>> return LDPS_OK;
>> }
>>
>> -static void internalize(GlobalValue &GV) {
>> - if (GV.isDeclarationForLinker())
>> - return; // We get here if there is a matching asm definition.
>> - if (!GV.hasLocalLinkage())
>> - GV.setLinkage(GlobalValue::InternalLinkage);
>> -}
>> -
>> -static const char *getResolutionName(ld_plugin_symbol_resolution R) {
>> - switch (R) {
>> - case LDPR_UNKNOWN:
>> - return "UNKNOWN";
>> - case LDPR_UNDEF:
>> - return "UNDEF";
>> - case LDPR_PREVAILING_DEF:
>> - return "PREVAILING_DEF";
>> - case LDPR_PREVAILING_DEF_IRONLY:
>> - return "PREVAILING_DEF_IRONLY";
>> - case LDPR_PREEMPTED_REG:
>> - return "PREEMPTED_REG";
>> - case LDPR_PREEMPTED_IR:
>> - return "PREEMPTED_IR";
>> - case LDPR_RESOLVED_IR:
>> - return "RESOLVED_IR";
>> - case LDPR_RESOLVED_EXEC:
>> - return "RESOLVED_EXEC";
>> - case LDPR_RESOLVED_DYN:
>> - return "RESOLVED_DYN";
>> - case LDPR_PREVAILING_DEF_IRONLY_EXP:
>> - return "PREVAILING_DEF_IRONLY_EXP";
>> - }
>> - llvm_unreachable("Unknown resolution");
>> -}
>> -
>> static void freeSymName(ld_plugin_symbol &Sym) {
>> free(Sym.name);
>> free(Sym.comdat_key);
>> @@ -671,120 +563,27 @@ static const void *getSymbolsAndView(cla
>> return View;
>> }
>>
>> -static std::unique_ptr<ModuleSummaryIndex>
>> -getModuleSummaryIndexForFile(claimed_file &F) {
>> - const void *View = getSymbolsAndView(F);
>> - if (!View)
>> - return nullptr;
>> -
>> +static void addModule(LTO &Lto, claimed_file &F, const void *View) {
>> MemoryBufferRef BufferRef(StringRef((const char *)View, F.filesize), F.name);
>> + Expected<std::unique_ptr<InputFile>> ObjOrErr = InputFile::create(BufferRef);
>>
>> - // Don't bother trying to build an index if there is no summary information
>> - // in this bitcode file.
>> - if (!object::ModuleSummaryIndexObjectFile::hasGlobalValueSummaryInMemBuffer(
>> - BufferRef, diagnosticHandler))
>> - return std::unique_ptr<ModuleSummaryIndex>(nullptr);
>> -
>> - ErrorOr<std::unique_ptr<object::ModuleSummaryIndexObjectFile>> ObjOrErr =
>> - object::ModuleSummaryIndexObjectFile::create(BufferRef,
>> - diagnosticHandler);
>> -
>> - if (std::error_code EC = ObjOrErr.getError())
>> - message(LDPL_FATAL,
>> - "Could not read module summary index bitcode from file : %s",
>> - EC.message().c_str());
>> -
>> - object::ModuleSummaryIndexObjectFile &Obj = **ObjOrErr;
>> -
>> - return Obj.takeIndex();
>> -}
>> -
>> -static std::unique_ptr<Module>
>> -getModuleForFile(LLVMContext &Context, claimed_file &F, const void *View,
>> - StringRef Name, raw_fd_ostream *ApiFile,
>> - StringSet<> &Internalize, std::vector<GlobalValue *> &Keep,
>> - StringMap<unsigned> &Realign) {
>> - MemoryBufferRef BufferRef(StringRef((const char *)View, F.filesize), Name);
>> - ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
>> - object::IRObjectFile::create(BufferRef, Context);
>> -
>> - if (std::error_code EC = ObjOrErr.getError())
>> + if (!ObjOrErr)
>> message(LDPL_FATAL, "Could not read bitcode from file : %s",
>> - EC.message().c_str());
>> -
>> - object::IRObjectFile &Obj = **ObjOrErr;
>> + toString(ObjOrErr.takeError()).c_str());
>>
>> - Module &M = Obj.getModule();
>> -
>> - M.materializeMetadata();
>> - UpgradeDebugInfo(M);
>> -
>> - SmallPtrSet<GlobalValue *, 8> Used;
>> - collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
>> + InputFile &Obj = **ObjOrErr;
>>
>> unsigned SymNum = 0;
>> + std::vector<SymbolResolution> Resols(F.syms.size());
>> for (auto &ObjSym : Obj.symbols()) {
>> - GlobalValue *GV = Obj.getSymbolGV(ObjSym.getRawDataRefImpl());
>> - if (GV && GV->hasAppendingLinkage())
>> - Keep.push_back(GV);
>> -
>> - if (shouldSkip(ObjSym.getFlags()))
>> - continue;
>> ld_plugin_symbol &Sym = F.syms[SymNum];
>> + SymbolResolution &R = Resols[SymNum];
>> ++SymNum;
>>
>> ld_plugin_symbol_resolution Resolution =
>> (ld_plugin_symbol_resolution)Sym.resolution;
>>
>> - if (options::generate_api_file)
>> - *ApiFile << Sym.name << ' ' << getResolutionName(Resolution) << '\n';
>> -
>> - if (!GV) {
>> - freeSymName(Sym);
>> - continue; // Asm symbol.
>> - }
>> -
>> ResolutionInfo &Res = ResInfo[Sym.name];
>> - if (Resolution == LDPR_PREVAILING_DEF_IRONLY_EXP && !Res.IsLinkonceOdr)
>> - Resolution = LDPR_PREVAILING_DEF;
>> -
>> - GV->setUnnamedAddr(Res.UnnamedAddr);
>> - GV->setVisibility(Res.Visibility);
>> -
>> - // Override gold's resolution for common symbols. We want the largest
>> - // one to win.
>> - if (GV->hasCommonLinkage()) {
>> - if (Resolution == LDPR_PREVAILING_DEF_IRONLY)
>> - Res.CommonInternal = true;
>> -
>> - if (Resolution == LDPR_PREVAILING_DEF_IRONLY ||
>> - Resolution == LDPR_PREVAILING_DEF)
>> - Res.UseCommon = true;
>> -
>> - const DataLayout &DL = GV->getParent()->getDataLayout();
>> - uint64_t Size = DL.getTypeAllocSize(GV->getType()->getElementType());
>> - unsigned Align = GV->getAlignment();
>> -
>> - if (Res.UseCommon && Size >= Res.CommonSize) {
>> - // Take GV.
>> - if (Res.CommonInternal)
>> - Resolution = LDPR_PREVAILING_DEF_IRONLY;
>> - else
>> - Resolution = LDPR_PREVAILING_DEF;
>> - cast<GlobalVariable>(GV)->setAlignment(
>> - std::max(Res.CommonAlign, Align));
>> - } else {
>> - // Do not take GV, it's smaller than what we already have in the
>> - // combined module.
>> - Resolution = LDPR_PREEMPTED_IR;
>> - if (Align > Res.CommonAlign)
>> - // Need to raise the alignment though.
>> - Realign[Sym.name] = Align;
>> - }
>> -
>> - Res.CommonSize = std::max(Res.CommonSize, Size);
>> - Res.CommonAlign = std::max(Res.CommonAlign, Align);
>> - }
>>
>> switch (Resolution) {
>> case LDPR_UNKNOWN:
>> @@ -795,57 +594,48 @@ getModuleForFile(LLVMContext &Context, c
>> case LDPR_RESOLVED_DYN:
>> case LDPR_PREEMPTED_IR:
>> case LDPR_PREEMPTED_REG:
>> - break;
>> -
>> case LDPR_UNDEF:
>> - if (!GV->isDeclarationForLinker())
>> - assert(GV->hasComdat());
>> break;
>>
>> - case LDPR_PREVAILING_DEF_IRONLY: {
>> - Keep.push_back(GV);
>> - // The IR linker has to be able to map this value to a declaration,
>> - // so we can only internalize after linking.
>> - if (!Used.count(GV))
>> - Internalize.insert(GV->getName());
>> + case LDPR_PREVAILING_DEF_IRONLY:
>> + R.Prevailing = true;
>> break;
>> - }
>>
>> case LDPR_PREVAILING_DEF:
>> - Keep.push_back(GV);
>> - // There is a non IR use, so we have to force optimizations to keep this.
>> - switch (GV->getLinkage()) {
>> - default:
>> - break;
>> - case GlobalValue::LinkOnceAnyLinkage:
>> - GV->setLinkage(GlobalValue::WeakAnyLinkage);
>> - break;
>> - case GlobalValue::LinkOnceODRLinkage:
>> - GV->setLinkage(GlobalValue::WeakODRLinkage);
>> - break;
>> - }
>> + R.Prevailing = true;
>> + R.VisibleToRegularObj = true;
>> break;
>>
>> - case LDPR_PREVAILING_DEF_IRONLY_EXP: {
>> - Keep.push_back(GV);
>> - if (canBeOmittedFromSymbolTable(GV))
>> - Internalize.insert(GV->getName());
>> + case LDPR_PREVAILING_DEF_IRONLY_EXP:
>> + R.Prevailing = true;
>> + if (!Res.CanOmitFromDynSym)
>> + R.VisibleToRegularObj = true;
>> break;
>> }
>> +
>> + if (Resolution != LDPR_RESOLVED_DYN && Resolution != LDPR_UNDEF &&
>> + (IsExecutable || !Res.DefaultVisibility))
>> + R.FinalDefinitionInLinkageUnit = true;
>> +
>> + if (ObjSym.getFlags() & object::BasicSymbolRef::SF_Common) {
>> + // We ignore gold's resolution for common symbols. A common symbol with
>> + // the correct size and alignment is added to the module by the pre-opt
>> + // module hook if any common symbol prevailed.
>> + CommonResolution &CommonRes = Commons[ObjSym.getIRName()];
>> + if (R.Prevailing) {
>> + CommonRes.Prevailing = true;
>> + CommonRes.VisibleToRegularObj = R.VisibleToRegularObj;
>> + }
>> + CommonRes.Size = std::max(CommonRes.Size, ObjSym.getCommonSize());
>> + CommonRes.Align = std::max(CommonRes.Align, ObjSym.getCommonAlignment());
>> + R.Prevailing = false;
>> }
>>
>> freeSymName(Sym);
>> }
>>
>> - return Obj.takeModule();
>> -}
>> -
>> -static void saveBCFile(StringRef Path, Module &M) {
>> - std::error_code EC;
>> - raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
>> - if (EC)
>> - message(LDPL_FATAL, "Failed to write the output file.");
>> - WriteBitcodeToFile(&M, OS, /* ShouldPreserveUseListOrder */ false);
>> + check(Lto.add(std::move(*ObjOrErr), Resols),
>> + std::string("Failed to link module ") + F.name);
>> }
>>
>> static void recordFile(std::string Filename, bool TempOutFile) {
>> @@ -857,190 +647,6 @@ static void recordFile(std::string Filen
>> Cleanup.push_back(Filename.c_str());
>> }
>>
>> -void ThinLTOTaskInfo::cleanup() {
>> - // Close the output file descriptor before we pass it to gold.
>> - OS->close();
>> -
>> - recordFile(Filename, TempOutFile);
>> -}
>> -
>> -namespace {
>> -/// Class to manage optimization and code generation for a module, possibly
>> -/// in a thread (ThinLTO).
>> -class CodeGen {
>> - /// The module for which this will generate code.
>> - std::unique_ptr<llvm::Module> M;
>> -
>> - /// The output stream to generate code into.
>> - raw_fd_ostream *OS;
>> -
>> - /// The task ID when this was invoked in a thread (ThinLTO).
>> - int TaskID;
>> -
>> - /// The module summary index for ThinLTO tasks.
>> - const ModuleSummaryIndex *CombinedIndex;
>> -
>> - /// The target machine for generating code for this module.
>> - std::unique_ptr<TargetMachine> TM;
>> -
>> - /// Filename to use as base when save-temps is enabled, used to get
>> - /// a unique and identifiable save-temps output file for each ThinLTO backend.
>> - std::string SaveTempsFilename;
>> -
>> - /// Map from a module name to the corresponding buffer holding a view of the
>> - /// bitcode provided via the get_view gold callback.
>> - StringMap<MemoryBufferRef> *ModuleMap;
>> -
>> - // Functions to import into this module.
>> - FunctionImporter::ImportMapTy *ImportList;
>> -
>> - // Map of globals defined in this module to their summary.
>> - std::map<GlobalValue::GUID, GlobalValueSummary *> *DefinedGlobals;
>> -
>> -public:
>> - /// Constructor used by full LTO.
>> - CodeGen(std::unique_ptr<llvm::Module> M)
>> - : M(std::move(M)), OS(nullptr), TaskID(-1), CombinedIndex(nullptr),
>> - ModuleMap(nullptr) {
>> - initTargetMachine();
>> - }
>> - /// Constructor used by ThinLTO.
>> - CodeGen(std::unique_ptr<llvm::Module> M, raw_fd_ostream *OS, int TaskID,
>> - const ModuleSummaryIndex *CombinedIndex, std::string Filename,
>> - StringMap<MemoryBufferRef> *ModuleMap,
>> - FunctionImporter::ImportMapTy *ImportList,
>> - std::map<GlobalValue::GUID, GlobalValueSummary *> *DefinedGlobals)
>> - : M(std::move(M)), OS(OS), TaskID(TaskID), CombinedIndex(CombinedIndex),
>> - SaveTempsFilename(std::move(Filename)), ModuleMap(ModuleMap),
>> - ImportList(ImportList), DefinedGlobals(DefinedGlobals) {
>> - assert(options::thinlto == !!CombinedIndex &&
>> - "Expected module summary index iff performing ThinLTO");
>> - initTargetMachine();
>> - }
>> -
>> - /// Invoke LTO passes and the code generator for the module.
>> - void runAll();
>> -
>> - /// Invoke the actual code generation to emit Module's object to file.
>> - void runCodegenPasses();
>> -
>> -private:
>> - const Target *TheTarget;
>> - std::string TripleStr;
>> - std::string FeaturesString;
>> - TargetOptions Options;
>> -
>> - /// Create a target machine for the module. Must be unique for each
>> - /// module/task.
>> - void initTargetMachine();
>> -
>> - std::unique_ptr<TargetMachine> createTargetMachine();
>> -
>> - /// Run all LTO passes on the module.
>> - void runLTOPasses();
>> -
>> - /// Sets up output files necessary to perform optional multi-threaded
>> - /// split code generation, and invokes the code generation implementation.
>> - /// If BCFileName is not empty, saves bitcode for module partitions into
>> - /// {BCFileName}0 .. {BCFileName}N.
>> - void runSplitCodeGen(const SmallString<128> &BCFilename);
>> -};
>> -}
>> -
>> -static SubtargetFeatures getFeatures(Triple &TheTriple) {
>> - SubtargetFeatures Features;
>> - Features.getDefaultSubtargetFeatures(TheTriple);
>> - for (const std::string &A : MAttrs)
>> - Features.AddFeature(A);
>> - return Features;
>> -}
>> -
>> -static CodeGenOpt::Level getCGOptLevel() {
>> - switch (options::OptLevel) {
>> - case 0:
>> - return CodeGenOpt::None;
>> - case 1:
>> - return CodeGenOpt::Less;
>> - case 2:
>> - return CodeGenOpt::Default;
>> - case 3:
>> - return CodeGenOpt::Aggressive;
>> - }
>> - llvm_unreachable("Invalid optimization level");
>> -}
>> -
>> -void CodeGen::initTargetMachine() {
>> - TripleStr = M->getTargetTriple();
>> - Triple TheTriple(TripleStr);
>> -
>> - std::string ErrMsg;
>> - TheTarget = TargetRegistry::lookupTarget(TripleStr, ErrMsg);
>> - if (!TheTarget)
>> - message(LDPL_FATAL, "Target not found: %s", ErrMsg.c_str());
>> -
>> - SubtargetFeatures Features = getFeatures(TheTriple);
>> - FeaturesString = Features.getString();
>> - Options = InitTargetOptionsFromCodeGenFlags();
>> -
>> - // Disable the new X86 relax relocations since gold might not support them.
>> - // FIXME: Check the gold version or add a new option to enable them.
>> - Options.RelaxELFRelocations = false;
>> -
>> - TM = createTargetMachine();
>> -}
>> -
>> -std::unique_ptr<TargetMachine> CodeGen::createTargetMachine() {
>> - CodeGenOpt::Level CGOptLevel = getCGOptLevel();
>> -
>> - return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
>> - TripleStr, options::mcpu, FeaturesString, Options, RelocationModel,
>> - CodeModel::Default, CGOptLevel));
>> -}
>> -
>> -void CodeGen::runLTOPasses() {
>> - M->setDataLayout(TM->createDataLayout());
>> -
>> - if (CombinedIndex) {
>> - // Apply summary-based LinkOnce/Weak resolution decisions.
>> - thinLTOResolveWeakForLinkerModule(*M, *DefinedGlobals);
>> -
>> - // Apply summary-based internalization decisions. Skip if there are no
>> - // defined globals from the summary since not only is it unnecessary, but
>> - // if this module did not have a summary section the internalizer will
>> - // assert if it finds any definitions in this module that aren't in the
>> - // DefinedGlobals set.
>> - if (!DefinedGlobals->empty())
>> - thinLTOInternalizeModule(*M, *DefinedGlobals);
>> -
>> - // Create a loader that will parse the bitcode from the buffers
>> - // in the ModuleMap.
>> - ModuleLoader Loader(M->getContext(), *ModuleMap);
>> -
>> - // Perform function importing.
>> - FunctionImporter Importer(*CombinedIndex, Loader);
>> - Importer.importFunctions(*M, *ImportList);
>> - }
>> -
>> - legacy::PassManager passes;
>> - passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
>> -
>> - PassManagerBuilder PMB;
>> - PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
>> - PMB.Inliner = createFunctionInliningPass();
>> - // Unconditionally verify input since it is not verified before this
>> - // point and has unknown origin.
>> - PMB.VerifyInput = true;
>> - PMB.VerifyOutput = !options::DisableVerify;
>> - PMB.LoopVectorize = true;
>> - PMB.SLPVectorize = true;
>> - PMB.OptLevel = options::OptLevel;
>> - if (options::thinlto)
>> - PMB.populateThinLTOPassManager(passes);
>> - else
>> - PMB.populateLTOPassManager(passes);
>> - passes.run(*M);
>> -}
>> -
>> /// Open a file and return the new file descriptor given a base input
>> /// file name, a flag indicating whether a temp file should be generated,
>> /// and an optional task id. The new filename generated is
>> @@ -1066,220 +672,44 @@ static int openOutputFile(SmallString<12
>> return FD;
>> }
>>
>> -void CodeGen::runCodegenPasses() {
>> - assert(OS && "Output stream must be set before emitting to file");
>> - legacy::PassManager CodeGenPasses;
>> - if (TM->addPassesToEmitFile(CodeGenPasses, *OS,
>> - TargetMachine::CGFT_ObjectFile))
>> - report_fatal_error("Failed to setup codegen");
>> - CodeGenPasses.run(*M);
>> -}
>> -
>> -void CodeGen::runSplitCodeGen(const SmallString<128> &BCFilename) {
>> - SmallString<128> Filename;
>> - // Note that openOutputFile will append a unique ID for each task
>> - if (!options::obj_path.empty())
>> - Filename = options::obj_path;
>> - else if (options::TheOutputType == options::OT_SAVE_TEMPS)
>> - Filename = output_name + ".o";
>> -
>> - // Note that the default parallelism is 1 instead of the
>> - // hardware_concurrency, as there are behavioral differences between
>> - // parallelism levels (e.g. symbol ordering will be different, and some uses
>> - // of inline asm currently have issues with parallelism >1).
>> - unsigned int MaxThreads = options::Parallelism ? options::Parallelism : 1;
>> -
>> - std::vector<SmallString<128>> Filenames(MaxThreads);
>> - std::vector<SmallString<128>> BCFilenames(MaxThreads);
>> - bool TempOutFile = Filename.empty();
>> - {
>> - // Open a file descriptor for each backend task. This is done in a block
>> - // so that the output file descriptors are closed before gold opens them.
>> - std::list<llvm::raw_fd_ostream> OSs;
>> - std::vector<llvm::raw_pwrite_stream *> OSPtrs(MaxThreads);
>> - for (unsigned I = 0; I != MaxThreads; ++I) {
>> - int FD = openOutputFile(Filename, TempOutFile, Filenames[I],
>> - // Only append ID if there are multiple tasks.
>> - MaxThreads > 1 ? I : -1);
>> - OSs.emplace_back(FD, true);
>> - OSPtrs[I] = &OSs.back();
>> - }
>> -
>> - std::list<llvm::raw_fd_ostream> BCOSs;
>> - std::vector<llvm::raw_pwrite_stream *> BCOSPtrs;
>> - if (!BCFilename.empty() && MaxThreads > 1) {
>> - for (unsigned I = 0; I != MaxThreads; ++I) {
>> - int FD = openOutputFile(BCFilename, false, BCFilenames[I], I);
>> - BCOSs.emplace_back(FD, true);
>> - BCOSPtrs.push_back(&BCOSs.back());
>> - }
>> - }
>> -
>> - // Run backend tasks.
>> - splitCodeGen(std::move(M), OSPtrs, BCOSPtrs,
>> - [&]() { return createTargetMachine(); });
>> - }
>> -
>> - for (auto &Filename : Filenames)
>> - recordFile(Filename.c_str(), TempOutFile);
>> -}
>> -
>> -void CodeGen::runAll() {
>> - runLTOPasses();
>> -
>> - SmallString<128> OptFilename;
>> - if (options::TheOutputType == options::OT_SAVE_TEMPS) {
>> - OptFilename = output_name;
>> - // If the CodeGen client provided a filename, use it. Always expect
>> - // a provided filename if we are in a task (i.e. ThinLTO backend).
>> - assert(!SaveTempsFilename.empty() || TaskID == -1);
>> - if (!SaveTempsFilename.empty())
>> - OptFilename = SaveTempsFilename;
>> - OptFilename += ".opt.bc";
>> - saveBCFile(OptFilename, *M);
>> - }
>> -
>> - // If we are already in a thread (i.e. ThinLTO), just perform
>> - // codegen passes directly.
>> - if (TaskID >= 0)
>> - runCodegenPasses();
>> - // Otherwise attempt split code gen.
>> - else
>> - runSplitCodeGen(OptFilename);
>> -}
>> -
>> -/// Links the module in \p View from file \p F into the combined module
>> -/// saved in the IRMover \p L.
>> -static void linkInModule(LLVMContext &Context, IRMover &L, claimed_file &F,
>> - const void *View, StringRef Name,
>> - raw_fd_ostream *ApiFile, StringSet<> &Internalize,
>> - bool SetName = false) {
>> - std::vector<GlobalValue *> Keep;
>> - StringMap<unsigned> Realign;
>> - std::unique_ptr<Module> M = getModuleForFile(Context, F, View, Name, ApiFile,
>> - Internalize, Keep, Realign);
>> - if (!M.get())
>> - return;
>> - if (!options::triple.empty())
>> - M->setTargetTriple(options::triple.c_str());
>> - else if (M->getTargetTriple().empty()) {
>> - M->setTargetTriple(DefaultTriple);
>> - }
>> -
>> - // For ThinLTO we want to propagate the source file name to ensure
>> - // we can create the correct global identifiers matching those in the
>> - // original module.
>> - if (SetName)
>> - L.getModule().setSourceFileName(M->getSourceFileName());
>> -
>> - if (Error E = L.move(std::move(M), Keep,
>> - [](GlobalValue &, IRMover::ValueAdder) {})) {
>> - handleAllErrors(std::move(E), [&](const llvm::ErrorInfoBase &EIB) {
>> - message(LDPL_FATAL, "Failed to link module %s: %s", Name.str().c_str(),
>> - EIB.message().c_str());
>> - });
>> - }
>> -
>> - for (const auto &I : Realign) {
>> - GlobalValue *Dst = L.getModule().getNamedValue(I.first());
>> - if (!Dst)
>> +/// Add all required common symbols to M, which is expected to be the first
>> +/// combined module.
>> +static void addCommons(Module &M) {
>> + for (auto &I : Commons) {
>> + if (!I.second.Prevailing)
>> continue;
>> - cast<GlobalVariable>(Dst)->setAlignment(I.second);
>> + ArrayType *Ty =
>> + ArrayType::get(Type::getInt8Ty(M.getContext()), I.second.Size);
>> + GlobalVariable *OldGV = M.getNamedGlobal(I.first);
>> + auto *GV = new GlobalVariable(M, Ty, false, GlobalValue::CommonLinkage,
>> + ConstantAggregateZero::get(Ty), "");
>> + GV->setAlignment(I.second.Align);
>> + if (OldGV) {
>> + OldGV->replaceAllUsesWith(ConstantExpr::getBitCast(GV, OldGV->getType()));
>> + GV->takeName(OldGV);
>> + OldGV->eraseFromParent();
>> + } else {
>> + GV->setName(I.first);
>> + }
>> + // We may only internalize commons if there is a single LTO task because
>> + // other native object files may require the common.
>> + if (MaxTasks == 1 && !I.second.VisibleToRegularObj)
>> + GV->setLinkage(GlobalValue::InternalLinkage);
>> }
>> }
>>
>> -/// Perform the ThinLTO backend on a single module, invoking the LTO and codegen
>> -/// pipelines.
>> -static void thinLTOBackendTask(claimed_file &F, const void *View,
>> - StringRef Name, raw_fd_ostream *ApiFile,
>> - const ModuleSummaryIndex &CombinedIndex,
>> - raw_fd_ostream *OS, unsigned TaskID,
>> - StringMap<MemoryBufferRef> &ModuleMap,
>> - FunctionImporter::ImportMapTy &ImportList,
>> - std::map<GlobalValue::GUID, GlobalValueSummary *> &DefinedGlobals) {
>> - // Need to use a separate context for each task
>> - LLVMContext Context;
>> - Context.setDiscardValueNames(options::TheOutputType !=
>> - options::OT_SAVE_TEMPS);
>> - Context.enableDebugTypeODRUniquing(); // Merge debug info types.
>> - Context.setDiagnosticHandler(diagnosticHandlerForContext, nullptr, true);
>> -
>> - std::unique_ptr<llvm::Module> NewModule(new llvm::Module(Name, Context));
>> - IRMover L(*NewModule.get());
>> -
>> - StringSet<> Dummy;
>> - linkInModule(Context, L, F, View, Name, ApiFile, Dummy, true);
>> - if (renameModuleForThinLTO(*NewModule, CombinedIndex))
>> - message(LDPL_FATAL, "Failed to rename module for ThinLTO");
>> -
>> - CodeGen codeGen(std::move(NewModule), OS, TaskID, &CombinedIndex, Name,
>> - &ModuleMap, &ImportList, &DefinedGlobals);
>> - codeGen.runAll();
>> -}
>> -
>> -/// Launch each module's backend pipeline in a separate task in a thread pool.
>> -static void
>> -thinLTOBackends(raw_fd_ostream *ApiFile,
>> - const ModuleSummaryIndex &CombinedIndex,
>> - StringMap<MemoryBufferRef> &ModuleMap,
>> - StringMap<FunctionImporter::ImportMapTy> &ImportLists,
>> - StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
>> - &ModuleToDefinedGVSummaries) {
>> - unsigned TaskCount = 0;
>> - std::vector<ThinLTOTaskInfo> Tasks;
>> - Tasks.reserve(Modules.size());
>> - unsigned int MaxThreads = options::Parallelism
>> - ? options::Parallelism
>> - : thread::hardware_concurrency();
>> -
>> - // Create ThreadPool in nested scope so that threads will be joined
>> - // on destruction.
>> - {
>> - ThreadPool ThinLTOThreadPool(MaxThreads);
>> - for (claimed_file &F : Modules) {
>> - // Do all the gold callbacks in the main thread, since gold is not thread
>> - // safe by default.
>> - const void *View = getSymbolsAndView(F);
>> - if (!View)
>> - continue;
>> -
>> - SmallString<128> Filename;
>> - if (!options::obj_path.empty())
>> - // Note that openOutputFile will append a unique ID for each task
>> - Filename = options::obj_path;
>> - else if (options::TheOutputType == options::OT_SAVE_TEMPS) {
>> - // Use the input file name so that we get a unique and identifiable
>> - // output file for each ThinLTO backend task.
>> - Filename = F.name;
>> - Filename += ".thinlto.o";
>> - }
>> - bool TempOutFile = Filename.empty();
>> -
>> - SmallString<128> NewFilename;
>> - int FD = openOutputFile(Filename, TempOutFile, NewFilename,
>> - // Only append the TaskID if we will use the
>> - // non-unique obj_path.
>> - !options::obj_path.empty() ? TaskCount : -1);
>> - TaskCount++;
>> - std::unique_ptr<raw_fd_ostream> OS =
>> - llvm::make_unique<raw_fd_ostream>(FD, true);
>> -
>> - // Enqueue the task
>> - ThinLTOThreadPool.async(thinLTOBackendTask, std::ref(F), View, F.name,
>> - ApiFile, std::ref(CombinedIndex), OS.get(),
>> - TaskCount, std::ref(ModuleMap),
>> - std::ref(ImportLists[F.name]),
>> - std::ref(ModuleToDefinedGVSummaries[F.name]));
>> -
>> - // Record the information needed by the task or during its cleanup
>> - // to a ThinLTOTaskInfo instance. For information needed by the task
>> - // the unique_ptr ownership is transferred to the ThinLTOTaskInfo.
>> - Tasks.emplace_back(std::move(OS), NewFilename.c_str(), TempOutFile);
>> - }
>> +static CodeGenOpt::Level getCGOptLevel() {
>> + switch (options::OptLevel) {
>> + case 0:
>> + return CodeGenOpt::None;
>> + case 1:
>> + return CodeGenOpt::Less;
>> + case 2:
>> + return CodeGenOpt::Default;
>> + case 3:
>> + return CodeGenOpt::Aggressive;
>> }
>> -
>> - for (auto &Task : Tasks)
>> - Task.cleanup();
>> + llvm_unreachable("Invalid optimization level");
>> }
>>
>> /// Parse the thinlto_prefix_replace option into the \p OldPrefix and
>> @@ -1293,266 +723,130 @@ static void getThinLTOOldAndNewPrefix(st
>> NewPrefix = Split.second.str();
>> }
>>
>> -/// Given the original \p Path to an output file, replace any path
>> -/// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
>> -/// resulting directory if it does not yet exist.
>> -static std::string getThinLTOOutputFile(const std::string &Path,
>> - const std::string &OldPrefix,
>> - const std::string &NewPrefix) {
>> - if (OldPrefix.empty() && NewPrefix.empty())
>> - return Path;
>> - SmallString<128> NewPath(Path);
>> - llvm::sys::path::replace_path_prefix(NewPath, OldPrefix, NewPrefix);
>> - StringRef ParentPath = llvm::sys::path::parent_path(NewPath.str());
>> - if (!ParentPath.empty()) {
>> - // Make sure the new directory exists, creating it if necessary.
>> - if (std::error_code EC = llvm::sys::fs::create_directories(ParentPath))
>> - llvm::errs() << "warning: could not create directory '" << ParentPath
>> - << "': " << EC.message() << '\n';
>> - }
>> - return NewPath.str();
>> -}
>> -
>> -/// Perform ThinLTO link, which creates the combined index file.
>> -/// Also, either launch backend threads or (under thinlto-index-only)
>> -/// emit individual index files for distributed backends and exit.
>> -static ld_plugin_status thinLTOLink(raw_fd_ostream *ApiFile) {
>> - // Map from a module name to the corresponding buffer holding a view of the
>> - // bitcode provided via the get_view gold callback.
>> - StringMap<MemoryBufferRef> ModuleMap;
>> - // Map to own RAII objects that manage the file opening and releasing
>> - // interfaces with gold.
>> - DenseMap<void *, std::unique_ptr<PluginInputFile>> HandleToInputFile;
>> -
>> - // Keep track of symbols that must not be internalized because they
>> - // are referenced outside of a single IR module.
>> - DenseSet<GlobalValue::GUID> Preserve;
>> -
>> - // Keep track of the prevailing copy for each GUID, for use in resolving
>> - // weak linkages.
>> - DenseMap<GlobalValue::GUID, const GlobalValueSummary *> PrevailingCopy;
>> -
>> - ModuleSummaryIndex CombinedIndex;
>> - uint64_t NextModuleId = 0;
>> - for (claimed_file &F : Modules) {
>> - if (!HandleToInputFile.count(F.leader_handle))
>> - HandleToInputFile.insert(std::make_pair(
>> - F.leader_handle, llvm::make_unique<PluginInputFile>(F.handle)));
>> - // Pass this into getModuleSummaryIndexForFile
>> - const void *View = getSymbolsAndView(F);
>> - if (!View)
>> - continue;
>> -
>> - MemoryBufferRef ModuleBuffer(StringRef((const char *)View, F.filesize),
>> - F.name);
>> - assert(ModuleMap.find(ModuleBuffer.getBufferIdentifier()) ==
>> - ModuleMap.end() &&
>> - "Expect unique Buffer Identifier");
>> - ModuleMap[ModuleBuffer.getBufferIdentifier()] = ModuleBuffer;
>> -
>> - std::unique_ptr<ModuleSummaryIndex> Index = getModuleSummaryIndexForFile(F);
>> -
>> - // Use gold's symbol resolution information to identify symbols referenced
>> - // by more than a single IR module (i.e. referenced by multiple IR modules
>> - // or by a non-IR module). Cross references introduced by importing are
>> - // checked separately via the export lists. Also track the prevailing copy
>> - // for later symbol resolution.
>> - for (auto &Sym : F.syms) {
>> - ld_plugin_symbol_resolution Resolution =
>> - (ld_plugin_symbol_resolution)Sym.resolution;
>> - GlobalValue::GUID SymGUID = GlobalValue::getGUID(Sym.name);
>> - if (Resolution != LDPR_PREVAILING_DEF_IRONLY)
>> - Preserve.insert(SymGUID);
>> -
>> - if (Index && (Resolution == LDPR_PREVAILING_DEF ||
>> - Resolution == LDPR_PREVAILING_DEF_IRONLY ||
>> - Resolution == LDPR_PREVAILING_DEF_IRONLY_EXP))
>> - PrevailingCopy[SymGUID] = Index->getGlobalValueSummary(SymGUID);
>> - }
>> -
>> - // Skip files without a module summary.
>> - if (Index)
>> - CombinedIndex.mergeFrom(std::move(Index), ++NextModuleId);
>> - }
>> -
>> - // Collect for each module the list of function it defines (GUID ->
>> - // Summary).
>> - StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
>> - ModuleToDefinedGVSummaries(NextModuleId);
>> - CombinedIndex.collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
>> -
>> - StringMap<FunctionImporter::ImportMapTy> ImportLists(NextModuleId);
>> - StringMap<FunctionImporter::ExportSetTy> ExportLists(NextModuleId);
>> - ComputeCrossModuleImport(CombinedIndex, ModuleToDefinedGVSummaries,
>> - ImportLists, ExportLists);
>> -
>> - auto isPrevailing = [&](GlobalValue::GUID GUID, const GlobalValueSummary *S) {
>> - const auto &Prevailing = PrevailingCopy.find(GUID);
>> - assert(Prevailing != PrevailingCopy.end());
>> - return Prevailing->second == S;
>> - };
>> +static std::unique_ptr<LTO> createLTO() {
>> + Config Conf;
>> + ThinBackend Backend;
>> + unsigned ParallelCodeGenParallelismLevel = 1;
>>
>> - // Callback for internalization, to prevent internalization of symbols
>> - // that were not candidates initially, and those that are being imported
>> - // (which introduces new cross references).
>> - auto isExported = [&](StringRef ModuleIdentifier, GlobalValue::GUID GUID) {
>> - const auto &ExportList = ExportLists.find(ModuleIdentifier);
>> - return (ExportList != ExportLists.end() &&
>> - ExportList->second.count(GUID)) ||
>> - Preserve.count(GUID);
>> - };
>> + Conf.CPU = options::mcpu;
>> + Conf.Options = InitTargetOptionsFromCodeGenFlags();
>>
>> - thinLTOResolveWeakForLinkerInIndex(
>> - CombinedIndex, isPrevailing,
>> - [](StringRef ModuleIdentifier, GlobalValue::GUID GUID,
>> - GlobalValue::LinkageTypes NewLinkage) {});
>> -
>> - // Use global summary-based analysis to identify symbols that can be
>> - // internalized (because they aren't exported or preserved as per callback).
>> - // Changes are made in the index, consumed in the ThinLTO backends.
>> - thinLTOInternalizeAndPromoteInIndex(CombinedIndex, isExported);
>> -
>> - if (options::thinlto_emit_imports_files && !options::thinlto_index_only)
>> - message(LDPL_WARNING,
>> - "thinlto-emit-imports-files ignored unless thinlto-index-only");
>> + // Disable the new X86 relax relocations since gold might not support them.
>> + // FIXME: Check the gold version or add a new option to enable them.
>> + Conf.Options.RelaxELFRelocations = false;
>>
>> + Conf.MAttrs = MAttrs;
>> + Conf.RelocModel = *RelocationModel;
>> + Conf.CGOptLevel = getCGOptLevel();
>> + Conf.DisableVerify = options::DisableVerify;
>> + Conf.OptLevel = options::OptLevel;
>> + if (options::Parallelism) {
>> + if (options::thinlto)
>> + Backend = createInProcessThinBackend(options::Parallelism);
>> + else
>> + ParallelCodeGenParallelismLevel = options::Parallelism;
>> + }
>> if (options::thinlto_index_only) {
>> - // If the thinlto-prefix-replace option was specified, parse it and
>> - // extract the old and new prefixes.
>> std::string OldPrefix, NewPrefix;
>> getThinLTOOldAndNewPrefix(OldPrefix, NewPrefix);
>> + Backend = createWriteIndexesThinBackend(
>> + OldPrefix, NewPrefix, options::thinlto_emit_imports_files,
>> + options::thinlto_linked_objects_file);
>> + }
>>
>> - // If the user requested a list of objects gold included in the link,
>> - // create and open the requested file.
>> - raw_fd_ostream *ObjFileOS = nullptr;
>> - if (!options::thinlto_linked_objects_file.empty()) {
>> - std::error_code EC;
>> - ObjFileOS = new raw_fd_ostream(options::thinlto_linked_objects_file, EC,
>> - sys::fs::OpenFlags::F_None);
>> - if (EC)
>> - message(LDPL_FATAL, "Unable to open %s for writing: %s",
>> - options::thinlto_linked_objects_file.c_str(),
>> - EC.message().c_str());
>> - }
>> - // For each input bitcode file, generate an individual index that
>> - // contains summaries only for its own global values, and for any that
>> - // should be imported.
>> - for (claimed_file &F : Modules) {
>> - std::error_code EC;
>> + Conf.OverrideTriple = options::triple;
>> + Conf.DefaultTriple = sys::getDefaultTargetTriple();
>>
>> - std::string NewModulePath =
>> - getThinLTOOutputFile(F.name, OldPrefix, NewPrefix);
>> + Conf.DiagHandler = diagnosticHandler;
>>
>> - if (!options::thinlto_linked_objects_file.empty()) {
>> - // If gold included any symbols from ths file in the link, emit path
>> - // to the final object file, which should be included in the final
>> - // native link.
>> - if (get_symbols(F.handle, F.syms.size(), F.syms.data()) !=
>> - LDPS_NO_SYMS) {
>> - assert(ObjFileOS);
>> - *ObjFileOS << NewModulePath << "\n";
>> - }
>> - }
>> + Conf.PreOptModuleHook = [](size_t Task, Module &M) {
>> + if (Task == 0)
>> + addCommons(M);
>> + return true;
>> + };
>>
>> - raw_fd_ostream OS((Twine(NewModulePath) + ".thinlto.bc").str(), EC,
>> - sys::fs::OpenFlags::F_None);
>> - if (EC)
>> - message(LDPL_FATAL, "Unable to open %s.thinlto.bc for writing: %s",
>> - NewModulePath.c_str(), EC.message().c_str());
>> - // Build a map of module to the GUIDs and summary objects that should
>> - // be written to its index.
>> - std::map<std::string, GVSummaryMapTy> ModuleToSummariesForIndex;
>> - gatherImportedSummariesForModule(F.name, ModuleToDefinedGVSummaries,
>> - ImportLists, ModuleToSummariesForIndex);
>> - WriteIndexToFile(CombinedIndex, OS, &ModuleToSummariesForIndex);
>> -
>> - if (options::thinlto_emit_imports_files) {
>> - if ((EC = EmitImportsFiles(F.name,
>> - (Twine(NewModulePath) + ".imports").str(),
>> - ImportLists)))
>> - message(LDPL_FATAL, "Unable to open %s.imports",
>> - NewModulePath.c_str(), EC.message().c_str());
>> - }
>> - }
>> + switch (options::TheOutputType) {
>> + case options::OT_NORMAL:
>> + break;
>>
>> - if (ObjFileOS)
>> - ObjFileOS->close();
>> + case options::OT_DISABLE:
>> + Conf.PreOptModuleHook = [](size_t Task, Module &M) { return false; };
>> + break;
>>
>> - cleanup_hook();
>> - exit(0);
>> - }
>> + case options::OT_BC_ONLY:
>> + Conf.PostInternalizeModuleHook = [](size_t Task, Module &M) {
>> + std::error_code EC;
>> + raw_fd_ostream OS(output_name, EC, sys::fs::OpenFlags::F_None);
>> + if (EC)
>> + message(LDPL_FATAL, "Failed to write the output file.");
>> + WriteBitcodeToFile(&M, OS, /* ShouldPreserveUseListOrder */ false);
>> + return false;
>> + };
>> + break;
>>
>> - // Create OS in nested scope so that it will be closed on destruction.
>> - {
>> - std::error_code EC;
>> - raw_fd_ostream OS(output_name + ".thinlto.bc", EC,
>> - sys::fs::OpenFlags::F_None);
>> - if (EC)
>> - message(LDPL_FATAL, "Unable to open %s.thinlto.bc for writing: %s",
>> - output_name.data(), EC.message().c_str());
>> - WriteIndexToFile(CombinedIndex, OS);
>> + case options::OT_SAVE_TEMPS:
>> + check(Conf.addSaveTemps(output_name, /* UseInputModulePath */ true));
>> + break;
>> }
>>
>> - thinLTOBackends(ApiFile, CombinedIndex, ModuleMap, ImportLists,
>> - ModuleToDefinedGVSummaries);
>> - return LDPS_OK;
>> + return llvm::make_unique<LTO>(std::move(Conf), Backend,
>> + ParallelCodeGenParallelismLevel);
>> }
>>
>> /// gold informs us that all symbols have been read. At this point, we use
>> /// get_symbols to see if any of our definitions have been overridden by a
>> /// native object file. Then, perform optimization and codegen.
>> -static ld_plugin_status allSymbolsReadHook(raw_fd_ostream *ApiFile) {
>> +static ld_plugin_status allSymbolsReadHook() {
>> if (Modules.empty())
>> return LDPS_OK;
>>
>> if (unsigned NumOpts = options::extra.size())
>> cl::ParseCommandLineOptions(NumOpts, &options::extra[0]);
>>
>> - if (options::thinlto)
>> - return thinLTOLink(ApiFile);
>> + std::unique_ptr<LTO> Lto = createLTO();
>>
>> - LLVMContext Context;
>> - Context.setDiscardValueNames(options::TheOutputType !=
>> - options::OT_SAVE_TEMPS);
>> - Context.enableDebugTypeODRUniquing(); // Merge debug info types.
>> - Context.setDiagnosticHandler(diagnosticHandlerForContext, nullptr, true);
>> -
>> - std::unique_ptr<Module> Combined(new Module("ld-temp.o", Context));
>> - IRMover L(*Combined);
>> -
>> - StringSet<> Internalize;
>> for (claimed_file &F : Modules) {
>> - // RAII object to manage the file opening and releasing interfaces with
>> - // gold.
>> PluginInputFile InputFile(F.handle);
>> const void *View = getSymbolsAndView(F);
>> if (!View)
>> continue;
>> - linkInModule(Context, L, F, View, F.name, ApiFile, Internalize);
>> + addModule(*Lto, F, View);
>> }
>>
>> - for (const auto &Name : Internalize) {
>> - GlobalValue *GV = Combined->getNamedValue(Name.first());
>> - if (GV)
>> - internalize(*GV);
>> - }
>> + SmallString<128> Filename;
>> + // Note that openOutputFile will append a unique ID for each task
>> + if (!options::obj_path.empty())
>> + Filename = options::obj_path;
>> + else if (options::TheOutputType == options::OT_SAVE_TEMPS)
>> + Filename = output_name + ".o";
>> + bool SaveTemps = !Filename.empty();
>>
>> - if (options::TheOutputType == options::OT_DISABLE)
>> + MaxTasks = Lto->getMaxTasks();
>> + std::vector<uintptr_t> IsTemporary(MaxTasks);
>> + std::vector<SmallString<128>> Filenames(MaxTasks);
>> +
>> + auto AddStream = [&](size_t Task) {
>> + int FD = openOutputFile(Filename, /*TempOutFile=*/!SaveTemps,
>> + Filenames[Task], MaxTasks > 1 ? Task : -1);
>> + IsTemporary[Task] = !SaveTemps;
>> +
>> + return llvm::make_unique<llvm::raw_fd_ostream>(FD, true);
>> + };
>> +
>> + check(Lto->run(AddStream));
>> +
>> + if (options::TheOutputType == options::OT_DISABLE ||
>> + options::TheOutputType == options::OT_BC_ONLY)
>> return LDPS_OK;
>>
>> - if (options::TheOutputType != options::OT_NORMAL) {
>> - std::string path;
>> - if (options::TheOutputType == options::OT_BC_ONLY)
>> - path = output_name;
>> - else
>> - path = output_name + ".bc";
>> - saveBCFile(path, *Combined);
>> - if (options::TheOutputType == options::OT_BC_ONLY)
>> - return LDPS_OK;
>> + if (options::thinlto_index_only) {
>> + cleanup_hook();
>> + exit(0);
>> }
>>
>> - CodeGen codeGen(std::move(Combined));
>> - codeGen.runAll();
>> + for (unsigned I = 0; I != MaxTasks; ++I)
>> + if (!Filenames[I].empty())
>> + recordFile(Filenames[I].str(), IsTemporary[I]);
>>
>> if (!options::extra_library_path.empty() &&
>> set_extra_library_path(options::extra_library_path.c_str()) != LDPS_OK)
>> @@ -1562,18 +856,7 @@ static ld_plugin_status allSymbolsReadHo
>> }
>>
>> static ld_plugin_status all_symbols_read_hook(void) {
>> - ld_plugin_status Ret;
>> - if (!options::generate_api_file) {
>> - Ret = allSymbolsReadHook(nullptr);
>> - } else {
>> - std::error_code EC;
>> - raw_fd_ostream ApiFile("apifile.txt", EC, sys::fs::F_None);
>> - if (EC)
>> - message(LDPL_FATAL, "Unable to open apifile.txt for writing: %s",
>> - EC.message().c_str());
>> - Ret = allSymbolsReadHook(&ApiFile);
>> - }
>> -
>> + ld_plugin_status Ret = allSymbolsReadHook();
>> llvm_shutdown();
>>
>> if (options::TheOutputType == options::OT_BC_ONLY ||
>>
>> Added: llvm/trunk/tools/llvm-lto2/CMakeLists.txt
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-lto2/CMakeLists.txt?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/tools/llvm-lto2/CMakeLists.txt (added)
>> +++ llvm/trunk/tools/llvm-lto2/CMakeLists.txt Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,10 @@
>> +set(LLVM_LINK_COMPONENTS
>> + ${LLVM_TARGETS_TO_BUILD}
>> + LTO
>> + Object
>> + Support
>> + )
>> +
>> +add_llvm_tool(llvm-lto2
>> + llvm-lto2.cpp
>> + )
>>
>> Copied: llvm/trunk/tools/llvm-lto2/LLVMBuild.txt (from r278331, llvm/trunk/lib/LTO/LLVMBuild.txt)
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-lto2/LLVMBuild.txt?p2=llvm/trunk/tools/llvm-lto2/LLVMBuild.txt&p1=llvm/trunk/lib/LTO/LLVMBuild.txt&r1=278331&r2=278338&rev=278338&view=diff
>> ==============================================================================
>> --- llvm/trunk/lib/LTO/LLVMBuild.txt (original)
>> +++ llvm/trunk/tools/llvm-lto2/LLVMBuild.txt Thu Aug 11 09:58:12 2016
>> @@ -1,4 +1,4 @@
>> -;===- ./lib/LTO/LLVMBuild.txt ----------------------------------*- Conf -*--===;
>> +;===- ./tools/llvm-lto2/LLVMBuild.txt --------------------------*- Conf -*--===;
>> ;
>> ; The LLVM Compiler Infrastructure
>> ;
>> @@ -16,22 +16,7 @@
>> ;===------------------------------------------------------------------------===;
>>
>> [component_0]
>> -type = Library
>> -name = LTO
>> -parent = Libraries
>> -required_libraries =
>> - Analysis
>> - BitReader
>> - BitWriter
>> - CodeGen
>> - Core
>> - IPO
>> - InstCombine
>> - Linker
>> - MC
>> - ObjCARC
>> - Object
>> - Scalar
>> - Support
>> - Target
>> - TransformUtils
>> \ No newline at end of file
>> +type = Tool
>> +name = llvm-lto2
>> +parent = Tools
>> +required_libraries = LTO Object all-targets
>>
>> Added: llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp
>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp?rev=278338&view=auto
>> ==============================================================================
>> --- llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp (added)
>> +++ llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp Thu Aug 11 09:58:12 2016
>> @@ -0,0 +1,168 @@
>> +//===-- llvm-lto2: test harness for the resolution-based LTO interface ----===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// This program takes in a list of bitcode files, links them and performs
>> +// link-time optimization according to the provided symbol resolutions using the
>> +// resolution-based LTO interface, and outputs one or more object files.
>> +//
>> +// This program is intended to eventually replace llvm-lto which uses the legacy
>> +// LTO interface.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#include "llvm/LTO/LTO.h"
>> +#include "llvm/Support/CommandLine.h"
>> +#include "llvm/Support/TargetSelect.h"
>> +
>> +using namespace llvm;
>> +using namespace lto;
>> +using namespace object;
>> +
>> +static cl::list<std::string> InputFilenames(cl::Positional, cl::OneOrMore,
>> + cl::desc("<input bitcode files>"));
>> +
>> +static cl::opt<std::string> OutputFilename("o", cl::Required,
>> + cl::desc("Output filename"),
>> + cl::value_desc("filename"));
>> +
>> +static cl::opt<bool> SaveTemps("save-temps", cl::desc("Save temporary files"));
>> +
>> +static cl::list<std::string> SymbolResolutions(
>> + "r",
>> + cl::desc("Specify a symbol resolution: filename,symbolname,resolution\n"
>> + "where \"resolution\" is a sequence (which may be empty) of the\n"
>> + "following characters:\n"
>> + " p - prevailing: the linker has chosen this definition of the\n"
>> + " symbol\n"
>> + " l - local: the definition of this symbol is unpreemptable at\n"
>> + " runtime and is known to be in this linkage unit\n"
>> + " x - externally visible: the definition of this symbol is\n"
>> + " visible outside of the LTO unit\n"
>> + "A resolution for each symbol must be specified."),
>> + cl::ZeroOrMore);
>> +
>> +static void check(Error E, std::string Msg) {
>> + if (!E)
>> + return;
>> + handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
>> + errs() << "llvm-lto: " << Msg << ": " << EIB.message().c_str() << '\n';
>> + });
>> + exit(1);
>> +}
>> +
>> +template <typename T> static T check(Expected<T> E, std::string Msg) {
>> + if (E)
>> + return std::move(*E);
>> + check(E.takeError(), Msg);
>> + return T();
>> +}
>> +
>> +static void check(std::error_code EC, std::string Msg) {
>> + check(errorCodeToError(EC), Msg);
>> +}
>> +
>> +template <typename T> static T check(ErrorOr<T> E, std::string Msg) {
>> + if (E)
>> + return std::move(*E);
>> + check(E.getError(), Msg);
>> + return T();
>> +}
>> +
>> +int main(int argc, char **argv) {
>> + InitializeAllTargets();
>> + InitializeAllTargetMCs();
>> + InitializeAllAsmPrinters();
>> + InitializeAllAsmParsers();
>> +
>> + cl::ParseCommandLineOptions(argc, argv, "Resolution-based LTO test harness");
>> +
>> + std::map<std::pair<std::string, std::string>, SymbolResolution>
>> + CommandLineResolutions;
>> + for (std::string R : SymbolResolutions) {
>> + StringRef Rest = R;
>> + StringRef FileName, SymbolName;
>> + std::tie(FileName, Rest) = Rest.split(',');
>> + if (Rest.empty()) {
>> + llvm::errs() << "invalid resolution: " << R << '\n';
>> + return 1;
>> + }
>> + std::tie(SymbolName, Rest) = Rest.split(',');
>> + SymbolResolution Res;
>> + for (char C : Rest) {
>> + if (C == 'p')
>> + Res.Prevailing = true;
>> + else if (C == 'l')
>> + Res.FinalDefinitionInLinkageUnit = true;
>> + else if (C == 'x')
>> + Res.VisibleToRegularObj = true;
>> + else
>> + llvm::errs() << "invalid character " << C << " in resolution: " << R
>> + << '\n';
>> + }
>> + CommandLineResolutions[{FileName, SymbolName}] = Res;
>> + }
>> +
>> + std::vector<std::unique_ptr<MemoryBuffer>> MBs;
>> +
>> + Config Conf;
>> + Conf.DiagHandler = [](const DiagnosticInfo &) {
>> + exit(1);
>> + };
>> +
>> + if (SaveTemps)
>> + check(Conf.addSaveTemps(OutputFilename), "Config::addSaveTemps failed");
>> +
>> + LTO Lto(std::move(Conf));
>> +
>> + bool HasErrors = false;
>> + for (std::string F : InputFilenames) {
>> + std::unique_ptr<MemoryBuffer> MB = check(MemoryBuffer::getFile(F), F);
>> + std::unique_ptr<InputFile> Input =
>> + check(InputFile::create(MB->getMemBufferRef()), F);
>> +
>> + std::vector<SymbolResolution> Res;
>> + for (const InputFile::Symbol &Sym : Input->symbols()) {
>> + auto I = CommandLineResolutions.find({F, Sym.getName()});
>> + if (I == CommandLineResolutions.end()) {
>> + llvm::errs() << argv[0] << ": missing symbol resolution for " << F
>> + << ',' << Sym.getName() << '\n';
>> + HasErrors = true;
>> + } else {
>> + Res.push_back(I->second);
>> + CommandLineResolutions.erase(I);
>> + }
>> + }
>> +
>> + if (HasErrors)
>> + continue;
>> +
>> + MBs.push_back(std::move(MB));
>> + check(Lto.add(std::move(Input), Res), F);
>> + }
>> +
>> + if (!CommandLineResolutions.empty()) {
>> + HasErrors = true;
>> + for (auto UnusedRes : CommandLineResolutions)
>> + llvm::errs() << argv[0] << ": unused symbol resolution for "
>> + << UnusedRes.first.first << ',' << UnusedRes.first.second
>> + << '\n';
>> + }
>> + if (HasErrors)
>> + return 1;
>> +
>> + auto AddStream = [&](size_t Task) {
>> + std::string Path = OutputFilename + "." + utostr(Task);
>> + std::error_code EC;
>> + auto S = llvm::make_unique<raw_fd_ostream>(Path, EC, sys::fs::F_None);
>> + check(EC, Path);
>> + return S;
>> + };
>> +
>> + check(Lto.run(AddStream), "LTO::run failed");
>> +}
>>
>>
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
--
Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413
More information about the llvm-commits
mailing list