[llvm] r278338 - Restore "Resolution-based LTO API."
Rafael Espíndola via llvm-commits
llvm-commits at lists.llvm.org
Mon Aug 29 11:55:25 PDT 2016
I have just finished reading this patch with an eye on seeing if it
would make sense to use it in lld.
I think it should work (and Davide has a wip patch), I think the only
thing that would be a bit odd is the GlobalResolutions StringMap. In
lld we try to have a single symbol table. Any objections to removing
GlobalResolutions and expecting each linker to track GlobalResolution?
Cheers,
Rafael
On 11 August 2016 at 10:58, Teresa Johnson via llvm-commits
<llvm-commits at lists.llvm.org> wrote:
> Author: tejohnson
> Date: Thu Aug 11 09:58:12 2016
> New Revision: 278338
>
> URL: http://llvm.org/viewvc/llvm-project?rev=278338&view=rev
> Log:
> Restore "Resolution-based LTO API."
>
> This restores commit r278330, with fixes for a few bot failures:
> - Fix a late change I had made to the save temps output file that I
> missed due to existing files sitting on my disk
> - Fix a bunch of Windows bot failures with "ambiguous call to overloaded
> function" due to confusion between llvm::make_unique vs
> std::make_unique (preface the new make_unique calls with "llvm::")
> - Attempt to fix a modules bot failure by adding a missing include
> to LTO/Config.h.
>
> Original change:
>
> Resolution-based LTO API.
>
> Summary:
> This introduces a resolution-based LTO API. The main advantage of this API over
> existing APIs is that it allows the linker to supply a resolution for each
> symbol in each object, rather than the combined object as a whole. This will
> become increasingly important for use cases such as ThinLTO which require us
> to process symbol resolutions in a more complicated way than just adjusting
> linkage.
>
> Patch by Peter Collingbourne.
>
> Reviewers: rafael, tejohnson, mehdi_amini
>
> Subscribers: lhames, tejohnson, mehdi_amini, llvm-commits
>
> Differential Revision: https://reviews.llvm.org/D20268
>
> Added:
> llvm/trunk/include/llvm/LTO/Config.h
> llvm/trunk/include/llvm/LTO/LTOBackend.h
> llvm/trunk/lib/LTO/LTOBackend.cpp
> llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll
> llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll
> llvm/trunk/test/LTO/Resolution/X86/alias.ll
> llvm/trunk/test/LTO/Resolution/X86/comdat.ll
> - copied, changed from r278331, llvm/trunk/test/tools/gold/X86/comdat.ll
> llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg
> llvm/trunk/test/tools/llvm-lto2/errors.ll
> llvm/trunk/tools/llvm-lto2/CMakeLists.txt
> llvm/trunk/tools/llvm-lto2/LLVMBuild.txt
> - copied, changed from r278331, llvm/trunk/lib/LTO/LLVMBuild.txt
> llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp
> Modified:
> llvm/trunk/include/llvm/LTO/LTO.h
> llvm/trunk/lib/LTO/CMakeLists.txt
> llvm/trunk/lib/LTO/LLVMBuild.txt
> llvm/trunk/lib/LTO/LTO.cpp
> llvm/trunk/lib/Object/IRObjectFile.cpp
> llvm/trunk/test/CMakeLists.txt
> llvm/trunk/test/lit.cfg
> llvm/trunk/test/tools/gold/X86/coff.ll
> llvm/trunk/test/tools/gold/X86/comdat.ll
> llvm/trunk/test/tools/gold/X86/common.ll
> llvm/trunk/test/tools/gold/X86/emit-llvm.ll
> llvm/trunk/test/tools/gold/X86/opt-level.ll
> llvm/trunk/test/tools/gold/X86/parallel.ll
> llvm/trunk/test/tools/gold/X86/slp-vectorize.ll
> llvm/trunk/test/tools/gold/X86/start-lib-common.ll
> llvm/trunk/test/tools/gold/X86/strip_names.ll
> llvm/trunk/test/tools/gold/X86/thinlto.ll
> llvm/trunk/test/tools/gold/X86/thinlto_alias.ll
> llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll
> llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll
> llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll
> llvm/trunk/test/tools/gold/X86/type-merge2.ll
> llvm/trunk/test/tools/gold/X86/vectorize.ll
> llvm/trunk/test/tools/gold/X86/visibility.ll
> llvm/trunk/tools/gold/gold-plugin.cpp
>
> Added: llvm/trunk/include/llvm/LTO/Config.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LTO/Config.h?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/include/llvm/LTO/Config.h (added)
> +++ llvm/trunk/include/llvm/LTO/Config.h Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,172 @@
> +//===-Config.h - LLVM Link Time Optimizer Configuration -------------------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file defines the lto::Config data structure, which allows clients to
> +// configure LTO.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef LLVM_LTO_CONFIG_H
> +#define LLVM_LTO_CONFIG_H
> +
> +#include "llvm/IR/DiagnosticInfo.h"
> +#include "llvm/Support/CodeGen.h"
> +#include "llvm/Target/TargetOptions.h"
> +
> +#include <functional>
> +
> +namespace llvm {
> +
> +class Error;
> +class Module;
> +class ModuleSummaryIndex;
> +class raw_pwrite_stream;
> +
> +namespace lto {
> +
> +/// LTO configuration. A linker can configure LTO by setting fields in this data
> +/// structure and passing it to the lto::LTO constructor.
> +struct Config {
> + std::string CPU;
> + std::string Features;
> + TargetOptions Options;
> + std::vector<std::string> MAttrs;
> + Reloc::Model RelocModel = Reloc::PIC_;
> + CodeModel::Model CodeModel = CodeModel::Default;
> + CodeGenOpt::Level CGOptLevel = CodeGenOpt::Default;
> + unsigned OptLevel = 2;
> + bool DisableVerify = false;
> +
> + /// Setting this field will replace target triples in input files with this
> + /// triple.
> + std::string OverrideTriple;
> +
> + /// Setting this field will replace unspecified target triples in input files
> + /// with this triple.
> + std::string DefaultTriple;
> +
> + bool ShouldDiscardValueNames = true;
> + DiagnosticHandlerFunction DiagHandler;
> +
> + /// If this field is set, LTO will write input file paths and symbol
> + /// resolutions here in llvm-lto2 command line flag format. This can be
> + /// used for testing and for running the LTO pipeline outside of the linker
> + /// with llvm-lto2.
> + std::unique_ptr<raw_ostream> ResolutionFile;
> +
> + /// The following callbacks deal with tasks, which normally represent the
> + /// entire optimization and code generation pipeline for what will become a
> + /// single native object file. Each task has a unique identifier between 0 and
> + /// getMaxTasks()-1, which is supplied to the callback via the Task parameter.
> + /// A task represents the entire pipeline for ThinLTO and regular
> + /// (non-parallel) LTO, but a parallel code generation task will be split into
> + /// N tasks before code generation, where N is the parallelism level.
> + ///
> + /// LTO may decide to stop processing a task at any time, for example if the
> + /// module is empty or if a module hook (see below) returns false. For this
> + /// reason, the client should not expect to receive exactly getMaxTasks()
> + /// native object files.
> +
> + /// A module hook may be used by a linker to perform actions during the LTO
> + /// pipeline. For example, a linker may use this function to implement
> + /// -save-temps, or to add its own resolved symbols to the module. If this
> + /// function returns false, any further processing for that task is aborted.
> + ///
> + /// Module hooks must be thread safe with respect to the linker's internal
> + /// data structures. A module hook will never be called concurrently from
> + /// multiple threads with the same task ID, or the same module.
> + ///
> + /// Note that in out-of-process backend scenarios, none of the hooks will be
> + /// called for ThinLTO tasks.
> + typedef std::function<bool(size_t Task, Module &)> ModuleHookFn;
> +
> + /// This module hook is called after linking (regular LTO) or loading
> + /// (ThinLTO) the module, before modifying it.
> + ModuleHookFn PreOptModuleHook;
> +
> + /// This hook is called after promoting any internal functions
> + /// (ThinLTO-specific).
> + ModuleHookFn PostPromoteModuleHook;
> +
> + /// This hook is called after internalizing the module.
> + ModuleHookFn PostInternalizeModuleHook;
> +
> + /// This hook is called after importing from other modules (ThinLTO-specific).
> + ModuleHookFn PostImportModuleHook;
> +
> + /// This module hook is called after optimization is complete.
> + ModuleHookFn PostOptModuleHook;
> +
> + /// This module hook is called before code generation. It is similar to the
> + /// PostOptModuleHook, but for parallel code generation it is called after
> + /// splitting the module.
> + ModuleHookFn PreCodeGenModuleHook;
> +
> + /// A combined index hook is called after all per-module indexes have been
> + /// combined (ThinLTO-specific). It can be used to implement -save-temps for
> + /// the combined index.
> + ///
> + /// If this function returns false, any further processing for ThinLTO tasks
> + /// is aborted.
> + ///
> + /// It is called regardless of whether the backend is in-process, although it
> + /// is not called from individual backend processes.
> + typedef std::function<bool(const ModuleSummaryIndex &Index)>
> + CombinedIndexHookFn;
> + CombinedIndexHookFn CombinedIndexHook;
> +
> + /// This is a convenience function that configures this Config object to write
> + /// temporary files named after the given OutputFileName for each of the LTO
> + /// phases to disk. A client can use this function to implement -save-temps.
> + ///
> + /// FIXME: Temporary files derived from ThinLTO backends are currently named
> + /// after the input file name, rather than the output file name, when
> + /// UseInputModulePath is set to true.
> + ///
> + /// Specifically, it (1) sets each of the above module hooks and the combined
> + /// index hook to a function that calls the hook function (if any) that was
> + /// present in the appropriate field when the addSaveTemps function was
> + /// called, and writes the module to a bitcode file with a name prefixed by
> + /// the given output file name, and (2) creates a resolution file whose name
> + /// is prefixed by the given output file name and sets ResolutionFile to its
> + /// file handle.
> + Error addSaveTemps(std::string OutputFileName,
> + bool UseInputModulePath = false);
> +};
> +
> +/// This type defines a stream callback. A stream callback is used to add a
> +/// native object that is generated on the fly. The callee must set up and
> +/// return a output stream to write the native object to.
> +///
> +/// Stream callbacks must be thread safe.
> +typedef std::function<std::unique_ptr<raw_pwrite_stream>(size_t Task)>
> + AddStreamFn;
> +
> +/// A derived class of LLVMContext that initializes itself according to a given
> +/// Config object. The purpose of this class is to tie ownership of the
> +/// diagnostic handler to the context, as opposed to the Config object (which
> +/// may be ephemeral).
> +struct LTOLLVMContext : LLVMContext {
> + static void funcDiagHandler(const DiagnosticInfo &DI, void *Context) {
> + auto *Fn = static_cast<DiagnosticHandlerFunction *>(Context);
> + (*Fn)(DI);
> + }
> +
> + LTOLLVMContext(const Config &C) : DiagHandler(C.DiagHandler) {
> + setDiscardValueNames(C.ShouldDiscardValueNames);
> + enableDebugTypeODRUniquing();
> + setDiagnosticHandler(funcDiagHandler, &DiagHandler, true);
> + }
> + DiagnosticHandlerFunction DiagHandler;
> +};
> +
> +}
> +}
> +
> +#endif
>
> Modified: llvm/trunk/include/llvm/LTO/LTO.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LTO/LTO.h?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/include/llvm/LTO/LTO.h (original)
> +++ llvm/trunk/include/llvm/LTO/LTO.h Thu Aug 11 09:58:12 2016
> @@ -16,14 +16,27 @@
> #ifndef LLVM_LTO_LTO_H
> #define LLVM_LTO_LTO_H
>
> +#include "llvm/ADT/MapVector.h"
> #include "llvm/ADT/StringMap.h"
> +#include "llvm/ADT/StringSet.h"
> +#include "llvm/CodeGen/Analysis.h"
> +#include "llvm/IR/DiagnosticInfo.h"
> #include "llvm/IR/ModuleSummaryIndex.h"
> +#include "llvm/LTO/Config.h"
> +#include "llvm/Linker/IRMover.h"
> +#include "llvm/Object/IRObjectFile.h"
> +#include "llvm/Support/thread.h"
> +#include "llvm/Target/TargetOptions.h"
> +#include "llvm/Transforms/IPO/FunctionImport.h"
>
> namespace llvm {
>
> +class Error;
> class LLVMContext;
> class MemoryBufferRef;
> class Module;
> +class Target;
> +class raw_pwrite_stream;
>
> /// Helper to load a module from bitcode.
> std::unique_ptr<Module> loadModuleFromBuffer(const MemoryBufferRef &Buffer,
> @@ -69,6 +82,319 @@ void thinLTOResolveWeakForLinkerInIndex(
> void thinLTOInternalizeAndPromoteInIndex(
> ModuleSummaryIndex &Index,
> function_ref<bool(StringRef, GlobalValue::GUID)> isExported);
> -}
> +
> +namespace lto {
> +
> +class LTO;
> +struct SymbolResolution;
> +class ThinBackendProc;
> +
> +/// An input file. This is a wrapper for IRObjectFile that exposes only the
> +/// information that an LTO client should need in order to do symbol resolution.
> +class InputFile {
> + // FIXME: Remove LTO class friendship once we have bitcode symbol tables.
> + friend LTO;
> + InputFile() = default;
> +
> + // FIXME: Remove the LLVMContext once we have bitcode symbol tables.
> + LLVMContext Ctx;
> + std::unique_ptr<object::IRObjectFile> Obj;
> +
> +public:
> + /// Create an InputFile.
> + static Expected<std::unique_ptr<InputFile>> create(MemoryBufferRef Object);
> +
> + class symbol_iterator;
> +
> + /// This is a wrapper for object::basic_symbol_iterator that exposes only the
> + /// information that an LTO client should need in order to do symbol
> + /// resolution.
> + ///
> + /// This object is ephemeral; it is only valid as long as an iterator obtained
> + /// from symbols() refers to it.
> + class Symbol {
> + friend symbol_iterator;
> + friend LTO;
> +
> + object::basic_symbol_iterator I;
> + const GlobalValue *GV;
> + uint32_t Flags;
> + SmallString<64> Name;
> +
> + bool shouldSkip() {
> + return !(Flags & object::BasicSymbolRef::SF_Global) ||
> + (Flags & object::BasicSymbolRef::SF_FormatSpecific);
> + }
> +
> + void skip() {
> + const object::SymbolicFile *Obj = I->getObject();
> + auto E = Obj->symbol_end();
> + while (I != E) {
> + Flags = I->getFlags();
> + if (!shouldSkip())
> + break;
> + ++I;
> + }
> + if (I == E)
> + return;
> +
> + Name.clear();
> + {
> + raw_svector_ostream OS(Name);
> + I->printName(OS);
> + }
> + GV = cast<object::IRObjectFile>(Obj)->getSymbolGV(I->getRawDataRefImpl());
> + }
> +
> + public:
> + Symbol(object::basic_symbol_iterator I) : I(I) { skip(); }
> +
> + StringRef getName() const { return Name; }
> + StringRef getIRName() const {
> + if (GV)
> + return GV->getName();
> + return StringRef();
> + }
> + uint32_t getFlags() const { return Flags; }
> + GlobalValue::VisibilityTypes getVisibility() const {
> + if (GV)
> + return GV->getVisibility();
> + return GlobalValue::DefaultVisibility;
> + }
> + bool canBeOmittedFromSymbolTable() const {
> + return GV && llvm::canBeOmittedFromSymbolTable(GV);
> + }
> + Expected<const Comdat *> getComdat() const {
> + const GlobalObject *GO;
> + if (auto *GA = dyn_cast<GlobalAlias>(GV)) {
> + GO = GA->getBaseObject();
> + if (!GO)
> + return make_error<StringError>("Unable to determine comdat of alias!",
> + inconvertibleErrorCode());
> + } else {
> + GO = cast<GlobalObject>(GV);
> + }
> + if (GV)
> + return GV->getComdat();
> + return nullptr;
> + }
> + size_t getCommonSize() const {
> + assert(Flags & object::BasicSymbolRef::SF_Common);
> + if (!GV)
> + return 0;
> + return GV->getParent()->getDataLayout().getTypeAllocSize(
> + GV->getType()->getElementType());
> + }
> + unsigned getCommonAlignment() const {
> + assert(Flags & object::BasicSymbolRef::SF_Common);
> + if (!GV)
> + return 0;
> + return GV->getAlignment();
> + }
> + };
> +
> + class symbol_iterator {
> + Symbol Sym;
> +
> + public:
> + symbol_iterator(object::basic_symbol_iterator I) : Sym(I) {}
> +
> + symbol_iterator &operator++() {
> + ++Sym.I;
> + Sym.skip();
> + return *this;
> + }
> +
> + symbol_iterator operator++(int) {
> + symbol_iterator I = *this;
> + ++*this;
> + return I;
> + }
> +
> + const Symbol &operator*() const { return Sym; }
> + const Symbol *operator->() const { return &Sym; }
> +
> + bool operator!=(const symbol_iterator &Other) const {
> + return Sym.I != Other.Sym.I;
> + }
> + };
> +
> + /// A range over the symbols in this InputFile.
> + iterator_range<symbol_iterator> symbols() {
> + return llvm::make_range(symbol_iterator(Obj->symbol_begin()),
> + symbol_iterator(Obj->symbol_end()));
> + }
> +
> + StringRef getSourceFileName() const {
> + return Obj->getModule().getSourceFileName();
> + }
> +};
> +
> +/// A ThinBackend defines what happens after the thin-link phase during ThinLTO.
> +/// The details of this type definition aren't important; clients can only
> +/// create a ThinBackend using one of the create*ThinBackend() functions below.
> +typedef std::function<std::unique_ptr<ThinBackendProc>(
> + Config &C, ModuleSummaryIndex &CombinedIndex,
> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
> + AddStreamFn AddStream)>
> + ThinBackend;
> +
> +/// This ThinBackend runs the individual backend jobs in-process.
> +ThinBackend createInProcessThinBackend(unsigned ParallelismLevel);
> +
> +/// This ThinBackend writes individual module indexes to files, instead of
> +/// running the individual backend jobs. This backend is for distributed builds
> +/// where separate processes will invoke the real backends.
> +///
> +/// To find the path to write the index to, the backend checks if the path has a
> +/// prefix of OldPrefix; if so, it replaces that prefix with NewPrefix. It then
> +/// appends ".thinlto.bc" and writes the index to that path. If
> +/// ShouldEmitImportsFiles is true it also writes a list of imported files to a
> +/// similar path with ".imports" appended instead.
> +ThinBackend createWriteIndexesThinBackend(std::string OldPrefix,
> + std::string NewPrefix,
> + bool ShouldEmitImportsFiles,
> + std::string LinkedObjectsFile);
> +
> +/// This class implements a resolution-based interface to LLVM's LTO
> +/// functionality. It supports regular LTO, parallel LTO code generation and
> +/// ThinLTO. You can use it from a linker in the following way:
> +/// - Set hooks and code generation options (see lto::Config struct defined in
> +/// Config.h), and use the lto::Config object to create an lto::LTO object.
> +/// - Create lto::InputFile objects using lto::InputFile::create(), then use
> +/// the symbols() function to enumerate its symbols and compute a resolution
> +/// for each symbol (see SymbolResolution below).
> +/// - After the linker has visited each input file (and each regular object
> +/// file) and computed a resolution for each symbol, take each lto::InputFile
> +/// and pass it and an array of symbol resolutions to the add() function.
> +/// - Call the getMaxTasks() function to get an upper bound on the number of
> +/// native object files that LTO may add to the link.
> +/// - Call the run() function. This function will use the supplied AddStream
> +/// function to add up to getMaxTasks() native object files to the link.
> +class LTO {
> + friend InputFile;
> +
> +public:
> + /// Create an LTO object. A default constructed LTO object has a reasonable
> + /// production configuration, but you can customize it by passing arguments to
> + /// this constructor.
> + /// FIXME: We do currently require the DiagHandler field to be set in Conf.
> + /// Until that is fixed, a Config argument is required.
> + LTO(Config Conf, ThinBackend Backend = nullptr,
> + unsigned ParallelCodeGenParallelismLevel = 1);
> +
> + /// Add an input file to the LTO link, using the provided symbol resolutions.
> + /// The symbol resolutions must appear in the enumeration order given by
> + /// InputFile::symbols().
> + Error add(std::unique_ptr<InputFile> Obj, ArrayRef<SymbolResolution> Res);
> +
> + /// Returns an upper bound on the number of tasks that the client may expect.
> + /// This may only be called after all IR object files have been added. For a
> + /// full description of tasks see LTOBackend.h.
> + size_t getMaxTasks() const;
> +
> + /// Runs the LTO pipeline. This function calls the supplied AddStream function
> + /// to add native object files to the link.
> + Error run(AddStreamFn AddStream);
> +
> +private:
> + Config Conf;
> +
> + struct RegularLTOState {
> + RegularLTOState(unsigned ParallelCodeGenParallelismLevel, Config &Conf);
> +
> + unsigned ParallelCodeGenParallelismLevel;
> + LTOLLVMContext Ctx;
> + bool HasModule = false;
> + std::unique_ptr<Module> CombinedModule;
> + IRMover Mover;
> + } RegularLTO;
> +
> + struct ThinLTOState {
> + ThinLTOState(ThinBackend Backend);
> +
> + ThinBackend Backend;
> + ModuleSummaryIndex CombinedIndex;
> + MapVector<StringRef, MemoryBufferRef> ModuleMap;
> + DenseMap<GlobalValue::GUID, StringRef> PrevailingModuleForGUID;
> + } ThinLTO;
> +
> + // The global resolution for a particular (mangled) symbol name. This is in
> + // particular necessary to track whether each symbol can be internalized.
> + // Because any input file may introduce a new cross-partition reference, we
> + // cannot make any final internalization decisions until all input files have
> + // been added and the client has called run(). During run() we apply
> + // internalization decisions either directly to the module (for regular LTO)
> + // or to the combined index (for ThinLTO).
> + struct GlobalResolution {
> + /// The unmangled name of the global.
> + std::string IRName;
> +
> + bool UnnamedAddr = true;
> +
> + /// This field keeps track of the partition number of this global. The
> + /// regular LTO object is partition 0, while each ThinLTO object has its own
> + /// partition number from 1 onwards.
> + ///
> + /// Any global that is defined or used by more than one partition, or that
> + /// is referenced externally, may not be internalized.
> + ///
> + /// Partitions generally have a one-to-one correspondence with tasks, except
> + /// that we use partition 0 for all parallel LTO code generation partitions.
> + /// Any partitioning of the combined LTO object is done internally by the
> + /// LTO backend.
> + size_t Partition = Unknown;
> +
> + /// Special partition numbers.
> + enum {
> + /// A partition number has not yet been assigned to this global.
> + Unknown = -1ull,
> +
> + /// This global is either used by more than one partition or has an
> + /// external reference, and therefore cannot be internalized.
> + External = -2ull,
> + };
> + };
> +
> + // Global mapping from mangled symbol names to resolutions.
> + StringMap<GlobalResolution> GlobalResolutions;
> +
> + void writeToResolutionFile(InputFile *Input, ArrayRef<SymbolResolution> Res);
> +
> + void addSymbolToGlobalRes(object::IRObjectFile *Obj,
> + SmallPtrSet<GlobalValue *, 8> &Used,
> + const InputFile::Symbol &Sym, SymbolResolution Res,
> + size_t Partition);
> +
> + Error addRegularLTO(std::unique_ptr<InputFile> Input,
> + ArrayRef<SymbolResolution> Res);
> + Error addThinLTO(std::unique_ptr<InputFile> Input,
> + ArrayRef<SymbolResolution> Res);
> +
> + Error runRegularLTO(AddStreamFn AddStream);
> + Error runThinLTO(AddStreamFn AddStream);
> +
> + mutable bool CalledGetMaxTasks = false;
> +};
> +
> +/// The resolution for a symbol. The linker must provide a SymbolResolution for
> +/// each global symbol based on its internal resolution of that symbol.
> +struct SymbolResolution {
> + SymbolResolution()
> + : Prevailing(0), FinalDefinitionInLinkageUnit(0), VisibleToRegularObj(0) {
> + }
> + /// The linker has chosen this definition of the symbol.
> + unsigned Prevailing : 1;
> +
> + /// The definition of this symbol is unpreemptable at runtime and is known to
> + /// be in this linkage unit.
> + unsigned FinalDefinitionInLinkageUnit : 1;
> +
> + /// The definition of this symbol is visible outside of the LTO unit.
> + unsigned VisibleToRegularObj : 1;
> +};
> +
> +} // namespace lto
> +} // namespace llvm
>
> #endif
>
> Added: llvm/trunk/include/llvm/LTO/LTOBackend.h
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/LTO/LTOBackend.h?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/include/llvm/LTO/LTOBackend.h (added)
> +++ llvm/trunk/include/llvm/LTO/LTOBackend.h Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,51 @@
> +//===-LTOBackend.h - LLVM Link Time Optimizer Backend ---------------------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file implements the "backend" phase of LTO, i.e. it performs
> +// optimization and code generation on a loaded module. It is generally used
> +// internally by the LTO class but can also be used independently, for example
> +// to implement a standalone ThinLTO backend.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#ifndef LLVM_LTO_LTOBACKEND_H
> +#define LLVM_LTO_LTOBACKEND_H
> +
> +#include "llvm/ADT/MapVector.h"
> +#include "llvm/IR/DiagnosticInfo.h"
> +#include "llvm/IR/ModuleSummaryIndex.h"
> +#include "llvm/LTO/Config.h"
> +#include "llvm/Support/MemoryBuffer.h"
> +#include "llvm/Target/TargetOptions.h"
> +#include "llvm/Transforms/IPO/FunctionImport.h"
> +
> +namespace llvm {
> +
> +class Error;
> +class Module;
> +class Target;
> +
> +namespace lto {
> +
> +/// Runs a regular LTO backend.
> +Error backend(Config &C, AddStreamFn AddStream,
> + unsigned ParallelCodeGenParallelismLevel,
> + std::unique_ptr<Module> M);
> +
> +/// Runs a ThinLTO backend.
> +Error thinBackend(Config &C, size_t Task, AddStreamFn AddStream, Module &M,
> + ModuleSummaryIndex &CombinedIndex,
> + const FunctionImporter::ImportMapTy &ImportList,
> + const GVSummaryMapTy &DefinedGlobals,
> + MapVector<StringRef, MemoryBufferRef> &ModuleMap);
> +
> +}
> +}
> +
> +#endif
>
> Modified: llvm/trunk/lib/LTO/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/CMakeLists.txt?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/lib/LTO/CMakeLists.txt (original)
> +++ llvm/trunk/lib/LTO/CMakeLists.txt Thu Aug 11 09:58:12 2016
> @@ -49,6 +49,7 @@ endif()
>
> add_llvm_library(LLVMLTO
> LTO.cpp
> + LTOBackend.cpp
> LTOModule.cpp
> LTOCodeGenerator.cpp
> UpdateCompilerUsed.cpp
>
> Modified: llvm/trunk/lib/LTO/LLVMBuild.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/LLVMBuild.txt?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/lib/LTO/LLVMBuild.txt (original)
> +++ llvm/trunk/lib/LTO/LLVMBuild.txt Thu Aug 11 09:58:12 2016
> @@ -34,4 +34,4 @@ required_libraries =
> Scalar
> Support
> Target
> - TransformUtils
> \ No newline at end of file
> + TransformUtils
>
> Modified: llvm/trunk/lib/LTO/LTO.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/LTO.cpp?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/lib/LTO/LTO.cpp (original)
> +++ llvm/trunk/lib/LTO/LTO.cpp Thu Aug 11 09:58:12 2016
> @@ -12,16 +12,39 @@
> //===----------------------------------------------------------------------===//
>
> #include "llvm/LTO/LTO.h"
> +#include "llvm/Analysis/TargetLibraryInfo.h"
> +#include "llvm/Analysis/TargetTransformInfo.h"
> #include "llvm/Bitcode/ReaderWriter.h"
> +#include "llvm/CodeGen/Analysis.h"
> +#include "llvm/IR/AutoUpgrade.h"
> +#include "llvm/IR/DiagnosticPrinter.h"
> +#include "llvm/IR/LegacyPassManager.h"
> +#include "llvm/LTO/LTOBackend.h"
> +#include "llvm/Linker/IRMover.h"
> +#include "llvm/Object/ModuleSummaryIndexObjectFile.h"
> +#include "llvm/Support/ManagedStatic.h"
> #include "llvm/Support/MemoryBuffer.h"
> +#include "llvm/Support/Path.h"
> #include "llvm/Support/SourceMgr.h"
> +#include "llvm/Support/TargetRegistry.h"
> +#include "llvm/Support/ThreadPool.h"
> #include "llvm/Support/raw_ostream.h"
> +#include "llvm/Target/TargetMachine.h"
> +#include "llvm/Target/TargetOptions.h"
> +#include "llvm/Transforms/IPO.h"
> +#include "llvm/Transforms/IPO/PassManagerBuilder.h"
> +#include "llvm/Transforms/Utils/SplitModule.h"
>
> -namespace llvm {
> +#include <set>
> +
> +using namespace llvm;
> +using namespace lto;
> +using namespace object;
>
> // Simple helper to load a module from bitcode
> -std::unique_ptr<Module> loadModuleFromBuffer(const MemoryBufferRef &Buffer,
> - LLVMContext &Context, bool Lazy) {
> +std::unique_ptr<Module>
> +llvm::loadModuleFromBuffer(const MemoryBufferRef &Buffer, LLVMContext &Context,
> + bool Lazy) {
> SMDiagnostic Err;
> ErrorOr<std::unique_ptr<Module>> ModuleOrErr(nullptr);
> if (Lazy) {
> @@ -76,7 +99,7 @@ static void thinLTOResolveWeakForLinkerG
> // current module. However there is a chance that another module is still
> // referencing them because of the import. We make sure we always emit at least
> // one copy.
> -void thinLTOResolveWeakForLinkerInIndex(
> +void llvm::thinLTOResolveWeakForLinkerInIndex(
> ModuleSummaryIndex &Index,
> function_ref<bool(GlobalValue::GUID, const GlobalValueSummary *)>
> isPrevailing,
> @@ -110,10 +133,513 @@ static void thinLTOInternalizeAndPromote
>
> // Update the linkages in the given \p Index to mark exported values
> // as external and non-exported values as internal.
> -void thinLTOInternalizeAndPromoteInIndex(
> +void llvm::thinLTOInternalizeAndPromoteInIndex(
> ModuleSummaryIndex &Index,
> function_ref<bool(StringRef, GlobalValue::GUID)> isExported) {
> for (auto &I : Index)
> thinLTOInternalizeAndPromoteGUID(I.second, I.first, isExported);
> }
> +
> +Expected<std::unique_ptr<InputFile>> InputFile::create(MemoryBufferRef Object) {
> + std::unique_ptr<InputFile> File(new InputFile);
> + std::string Msg;
> + auto DiagHandler = [](const DiagnosticInfo &DI, void *MsgP) {
> + auto *Msg = reinterpret_cast<std::string *>(MsgP);
> + raw_string_ostream OS(*Msg);
> + DiagnosticPrinterRawOStream DP(OS);
> + DI.print(DP);
> + };
> + File->Ctx.setDiagnosticHandler(DiagHandler, static_cast<void *>(&Msg));
> +
> + ErrorOr<std::unique_ptr<object::IRObjectFile>> IRObj =
> + IRObjectFile::create(Object, File->Ctx);
> + if (!Msg.empty())
> + return make_error<StringError>(Msg, inconvertibleErrorCode());
> + if (!IRObj)
> + return errorCodeToError(IRObj.getError());
> + File->Obj = std::move(*IRObj);
> +
> + File->Ctx.setDiagnosticHandler(nullptr, nullptr);
> +
> + return std::move(File);
> +}
> +
> +LTO::RegularLTOState::RegularLTOState(unsigned ParallelCodeGenParallelismLevel,
> + Config &Conf)
> + : ParallelCodeGenParallelismLevel(ParallelCodeGenParallelismLevel),
> + Ctx(Conf), CombinedModule(llvm::make_unique<Module>("ld-temp.o", Ctx)),
> + Mover(*CombinedModule) {}
> +
> +LTO::ThinLTOState::ThinLTOState(ThinBackend Backend) : Backend(Backend) {
> + if (!Backend)
> + this->Backend = createInProcessThinBackend(thread::hardware_concurrency());
> +}
> +
> +LTO::LTO(Config Conf, ThinBackend Backend,
> + unsigned ParallelCodeGenParallelismLevel)
> + : Conf(std::move(Conf)),
> + RegularLTO(ParallelCodeGenParallelismLevel, this->Conf),
> + ThinLTO(Backend) {}
> +
> +// Add the given symbol to the GlobalResolutions map, and resolve its partition.
> +void LTO::addSymbolToGlobalRes(IRObjectFile *Obj,
> + SmallPtrSet<GlobalValue *, 8> &Used,
> + const InputFile::Symbol &Sym,
> + SymbolResolution Res, size_t Partition) {
> + GlobalValue *GV = Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
> +
> + auto &GlobalRes = GlobalResolutions[Sym.getName()];
> + if (GV) {
> + GlobalRes.UnnamedAddr &= GV->hasGlobalUnnamedAddr();
> + if (Res.Prevailing)
> + GlobalRes.IRName = GV->getName();
> + }
> + if (Res.VisibleToRegularObj || (GV && Used.count(GV)) ||
> + (GlobalRes.Partition != GlobalResolution::Unknown &&
> + GlobalRes.Partition != Partition))
> + GlobalRes.Partition = GlobalResolution::External;
> + else
> + GlobalRes.Partition = Partition;
> +}
> +
> +void LTO::writeToResolutionFile(InputFile *Input,
> + ArrayRef<SymbolResolution> Res) {
> + StringRef Path = Input->Obj->getMemoryBufferRef().getBufferIdentifier();
> + *Conf.ResolutionFile << Path << '\n';
> + auto ResI = Res.begin();
> + for (const InputFile::Symbol &Sym : Input->symbols()) {
> + assert(ResI != Res.end());
> + SymbolResolution Res = *ResI++;
> +
> + *Conf.ResolutionFile << "-r=" << Path << ',' << Sym.getName() << ',';
> + if (Res.Prevailing)
> + *Conf.ResolutionFile << 'p';
> + if (Res.FinalDefinitionInLinkageUnit)
> + *Conf.ResolutionFile << 'l';
> + if (Res.VisibleToRegularObj)
> + *Conf.ResolutionFile << 'x';
> + *Conf.ResolutionFile << '\n';
> + }
> + assert(ResI == Res.end());
> +}
> +
> +Error LTO::add(std::unique_ptr<InputFile> Input,
> + ArrayRef<SymbolResolution> Res) {
> + assert(!CalledGetMaxTasks);
> +
> + if (Conf.ResolutionFile)
> + writeToResolutionFile(Input.get(), Res);
> +
> + Module &M = Input->Obj->getModule();
> + SmallPtrSet<GlobalValue *, 8> Used;
> + collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
> +
> + if (!Conf.OverrideTriple.empty())
> + M.setTargetTriple(Conf.OverrideTriple);
> + else if (M.getTargetTriple().empty())
> + M.setTargetTriple(Conf.DefaultTriple);
> +
> + MemoryBufferRef MBRef = Input->Obj->getMemoryBufferRef();
> + bool HasThinLTOSummary = hasGlobalValueSummary(MBRef, Conf.DiagHandler);
> +
> + if (HasThinLTOSummary)
> + return addThinLTO(std::move(Input), Res);
> + else
> + return addRegularLTO(std::move(Input), Res);
> +}
> +
> +// Add a regular LTO object to the link.
> +Error LTO::addRegularLTO(std::unique_ptr<InputFile> Input,
> + ArrayRef<SymbolResolution> Res) {
> + RegularLTO.HasModule = true;
> +
> + ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
> + IRObjectFile::create(Input->Obj->getMemoryBufferRef(), RegularLTO.Ctx);
> + if (!ObjOrErr)
> + return errorCodeToError(ObjOrErr.getError());
> + std::unique_ptr<object::IRObjectFile> Obj = std::move(*ObjOrErr);
> +
> + Module &M = Obj->getModule();
> + M.materializeMetadata();
> + UpgradeDebugInfo(M);
> +
> + SmallPtrSet<GlobalValue *, 8> Used;
> + collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
> +
> + std::vector<GlobalValue *> Keep;
> +
> + for (GlobalVariable &GV : M.globals())
> + if (GV.hasAppendingLinkage())
> + Keep.push_back(&GV);
> +
> + auto ResI = Res.begin();
> + for (const InputFile::Symbol &Sym :
> + make_range(InputFile::symbol_iterator(Obj->symbol_begin()),
> + InputFile::symbol_iterator(Obj->symbol_end()))) {
> + assert(ResI != Res.end());
> + SymbolResolution Res = *ResI++;
> + addSymbolToGlobalRes(Obj.get(), Used, Sym, Res, 0);
> +
> + GlobalValue *GV = Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
> + if (Res.Prevailing && GV) {
> + Keep.push_back(GV);
> + switch (GV->getLinkage()) {
> + default:
> + break;
> + case GlobalValue::LinkOnceAnyLinkage:
> + GV->setLinkage(GlobalValue::WeakAnyLinkage);
> + break;
> + case GlobalValue::LinkOnceODRLinkage:
> + GV->setLinkage(GlobalValue::WeakODRLinkage);
> + break;
> + }
> + }
> +
> + // FIXME: use proposed local attribute for FinalDefinitionInLinkageUnit.
> + }
> + assert(ResI == Res.end());
> +
> + return RegularLTO.Mover.move(Obj->takeModule(), Keep,
> + [](GlobalValue &, IRMover::ValueAdder) {});
> +}
> +
> +// Add a ThinLTO object to the link.
> +Error LTO::addThinLTO(std::unique_ptr<InputFile> Input,
> + ArrayRef<SymbolResolution> Res) {
> + Module &M = Input->Obj->getModule();
> + SmallPtrSet<GlobalValue *, 8> Used;
> + collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
> +
> + // We need to initialize the target info for the combined regular LTO module
> + // in case we have no regular LTO objects. In that case we still need to build
> + // it as usual because the client may want to add symbol definitions to it.
> + if (RegularLTO.CombinedModule->getTargetTriple().empty()) {
> + RegularLTO.CombinedModule->setTargetTriple(M.getTargetTriple());
> + RegularLTO.CombinedModule->setDataLayout(M.getDataLayout());
> + }
> +
> + MemoryBufferRef MBRef = Input->Obj->getMemoryBufferRef();
> + ErrorOr<std::unique_ptr<object::ModuleSummaryIndexObjectFile>>
> + SummaryObjOrErr =
> + object::ModuleSummaryIndexObjectFile::create(MBRef, Conf.DiagHandler);
> + if (!SummaryObjOrErr)
> + return errorCodeToError(SummaryObjOrErr.getError());
> + ThinLTO.CombinedIndex.mergeFrom((*SummaryObjOrErr)->takeIndex(),
> + ThinLTO.ModuleMap.size());
> +
> + auto ResI = Res.begin();
> + for (const InputFile::Symbol &Sym : Input->symbols()) {
> + assert(ResI != Res.end());
> + SymbolResolution Res = *ResI++;
> + addSymbolToGlobalRes(Input->Obj.get(), Used, Sym, Res,
> + ThinLTO.ModuleMap.size() + 1);
> +
> + GlobalValue *GV = Input->Obj->getSymbolGV(Sym.I->getRawDataRefImpl());
> + if (Res.Prevailing && GV)
> + ThinLTO.PrevailingModuleForGUID[GV->getGUID()] =
> + MBRef.getBufferIdentifier();
> + }
> + assert(ResI == Res.end());
> +
> + ThinLTO.ModuleMap[MBRef.getBufferIdentifier()] = MBRef;
> + return Error();
> +}
> +
> +size_t LTO::getMaxTasks() const {
> + CalledGetMaxTasks = true;
> + return RegularLTO.ParallelCodeGenParallelismLevel + ThinLTO.ModuleMap.size();
> +}
> +
> +Error LTO::run(AddStreamFn AddStream) {
> + // Invoke regular LTO if there was a regular LTO module to start with,
> + // or if there are any hooks that the linker may have used to add
> + // its own resolved symbols to the combined module.
> + if (RegularLTO.HasModule || Conf.PreOptModuleHook ||
> + Conf.PostInternalizeModuleHook || Conf.PostOptModuleHook ||
> + Conf.PreCodeGenModuleHook)
> + if (auto E = runRegularLTO(AddStream))
> + return E;
> + return runThinLTO(AddStream);
> +}
> +
> +Error LTO::runRegularLTO(AddStreamFn AddStream) {
> + if (Conf.PreOptModuleHook &&
> + !Conf.PreOptModuleHook(0, *RegularLTO.CombinedModule))
> + return Error();
> +
> + for (const auto &R : GlobalResolutions) {
> + if (R.second.IRName.empty())
> + continue;
> + if (R.second.Partition != 0 &&
> + R.second.Partition != GlobalResolution::External)
> + continue;
> +
> + GlobalValue *GV = RegularLTO.CombinedModule->getNamedValue(R.second.IRName);
> + // Ignore symbols defined in other partitions.
> + if (!GV || GV->hasLocalLinkage())
> + continue;
> + GV->setUnnamedAddr(R.second.UnnamedAddr ? GlobalValue::UnnamedAddr::Global
> + : GlobalValue::UnnamedAddr::None);
> + if (R.second.Partition == 0)
> + GV->setLinkage(GlobalValue::InternalLinkage);
> + }
> +
> + if (Conf.PostInternalizeModuleHook &&
> + !Conf.PostInternalizeModuleHook(0, *RegularLTO.CombinedModule))
> + return Error();
> +
> + return backend(Conf, AddStream, RegularLTO.ParallelCodeGenParallelismLevel,
> + std::move(RegularLTO.CombinedModule));
> +}
> +
> +/// This class defines the interface to the ThinLTO backend.
> +class lto::ThinBackendProc {
> +protected:
> + Config &Conf;
> + ModuleSummaryIndex &CombinedIndex;
> + AddStreamFn AddStream;
> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries;
> +
> +public:
> + ThinBackendProc(Config &Conf, ModuleSummaryIndex &CombinedIndex,
> + AddStreamFn AddStream,
> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries)
> + : Conf(Conf), CombinedIndex(CombinedIndex), AddStream(AddStream),
> + ModuleToDefinedGVSummaries(ModuleToDefinedGVSummaries) {}
> +
> + virtual ~ThinBackendProc() {}
> + virtual Error start(size_t Task, MemoryBufferRef MBRef,
> + StringMap<FunctionImporter::ImportMapTy> &ImportLists,
> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) = 0;
> + virtual Error wait() = 0;
> +};
> +
> +class InProcessThinBackend : public ThinBackendProc {
> + ThreadPool BackendThreadPool;
> +
> + Optional<Error> Err;
> + std::mutex ErrMu;
> +
> +public:
> + InProcessThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
> + unsigned ThinLTOParallelismLevel,
> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
> + AddStreamFn AddStream)
> + : ThinBackendProc(Conf, CombinedIndex, AddStream,
> + ModuleToDefinedGVSummaries),
> + BackendThreadPool(ThinLTOParallelismLevel) {}
> +
> + Error
> + runThinLTOBackendThread(AddStreamFn AddStream, size_t Task,
> + MemoryBufferRef MBRef,
> + ModuleSummaryIndex &CombinedIndex,
> + const FunctionImporter::ImportMapTy &ImportList,
> + const GVSummaryMapTy &DefinedGlobals,
> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
> + LLVMContext BackendContext;
> +
> + ErrorOr<std::unique_ptr<Module>> MOrErr =
> + parseBitcodeFile(MBRef, BackendContext);
> + assert(MOrErr && "Unable to load module in thread?");
> +
> + return thinBackend(Conf, Task, AddStream, **MOrErr, CombinedIndex,
> + ImportList, DefinedGlobals, ModuleMap);
> + }
> +
> + Error start(size_t Task, MemoryBufferRef MBRef,
> + StringMap<FunctionImporter::ImportMapTy> &ImportLists,
> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) override {
> + StringRef ModulePath = MBRef.getBufferIdentifier();
> + BackendThreadPool.async(
> + [=](MemoryBufferRef MBRef, ModuleSummaryIndex &CombinedIndex,
> + const FunctionImporter::ImportMapTy &ImportList,
> + GVSummaryMapTy &DefinedGlobals,
> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
> + Error E =
> + runThinLTOBackendThread(AddStream, Task, MBRef, CombinedIndex,
> + ImportList, DefinedGlobals, ModuleMap);
> + if (E) {
> + std::unique_lock<std::mutex> L(ErrMu);
> + if (Err)
> + Err = joinErrors(std::move(*Err), std::move(E));
> + else
> + Err = std::move(E);
> + }
> + },
> + MBRef, std::ref(CombinedIndex), std::ref(ImportLists[ModulePath]),
> + std::ref(ModuleToDefinedGVSummaries[ModulePath]), std::ref(ModuleMap));
> + return Error();
> + }
> +
> + Error wait() override {
> + BackendThreadPool.wait();
> + if (Err)
> + return std::move(*Err);
> + else
> + return Error();
> + }
> +};
> +
> +ThinBackend lto::createInProcessThinBackend(unsigned ParallelismLevel) {
> + return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
> + AddStreamFn AddStream) {
> + return llvm::make_unique<InProcessThinBackend>(
> + Conf, CombinedIndex, ParallelismLevel, ModuleToDefinedGVSummaries,
> + AddStream);
> + };
> +}
> +
> +class WriteIndexesThinBackend : public ThinBackendProc {
> + std::string OldPrefix, NewPrefix;
> + bool ShouldEmitImportsFiles;
> +
> + std::string LinkedObjectsFileName;
> + std::unique_ptr<llvm::raw_fd_ostream> LinkedObjectsFile;
> +
> +public:
> + WriteIndexesThinBackend(Config &Conf, ModuleSummaryIndex &CombinedIndex,
> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
> + AddStreamFn AddStream, std::string OldPrefix,
> + std::string NewPrefix, bool ShouldEmitImportsFiles,
> + std::string LinkedObjectsFileName)
> + : ThinBackendProc(Conf, CombinedIndex, AddStream,
> + ModuleToDefinedGVSummaries),
> + OldPrefix(OldPrefix), NewPrefix(NewPrefix),
> + ShouldEmitImportsFiles(ShouldEmitImportsFiles),
> + LinkedObjectsFileName(LinkedObjectsFileName) {}
> +
> + /// Given the original \p Path to an output file, replace any path
> + /// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
> + /// resulting directory if it does not yet exist.
> + std::string getThinLTOOutputFile(const std::string &Path,
> + const std::string &OldPrefix,
> + const std::string &NewPrefix) {
> + if (OldPrefix.empty() && NewPrefix.empty())
> + return Path;
> + SmallString<128> NewPath(Path);
> + llvm::sys::path::replace_path_prefix(NewPath, OldPrefix, NewPrefix);
> + StringRef ParentPath = llvm::sys::path::parent_path(NewPath.str());
> + if (!ParentPath.empty()) {
> + // Make sure the new directory exists, creating it if necessary.
> + if (std::error_code EC = llvm::sys::fs::create_directories(ParentPath))
> + llvm::errs() << "warning: could not create directory '" << ParentPath
> + << "': " << EC.message() << '\n';
> + }
> + return NewPath.str();
> + }
> +
> + Error start(size_t Task, MemoryBufferRef MBRef,
> + StringMap<FunctionImporter::ImportMapTy> &ImportLists,
> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) override {
> + StringRef ModulePath = MBRef.getBufferIdentifier();
> + std::string NewModulePath =
> + getThinLTOOutputFile(ModulePath, OldPrefix, NewPrefix);
> +
> + std::error_code EC;
> + if (!LinkedObjectsFileName.empty()) {
> + if (!LinkedObjectsFile) {
> + LinkedObjectsFile = llvm::make_unique<raw_fd_ostream>(
> + LinkedObjectsFileName, EC, sys::fs::OpenFlags::F_None);
> + if (EC)
> + return errorCodeToError(EC);
> + }
> + *LinkedObjectsFile << NewModulePath << '\n';
> + }
> +
> + std::map<std::string, GVSummaryMapTy> ModuleToSummariesForIndex;
> + gatherImportedSummariesForModule(ModulePath, ModuleToDefinedGVSummaries,
> + ImportLists, ModuleToSummariesForIndex);
> +
> + raw_fd_ostream OS(NewModulePath + ".thinlto.bc", EC,
> + sys::fs::OpenFlags::F_None);
> + if (EC)
> + return errorCodeToError(EC);
> + WriteIndexToFile(CombinedIndex, OS, &ModuleToSummariesForIndex);
> +
> + if (ShouldEmitImportsFiles)
> + return errorCodeToError(EmitImportsFiles(
> + ModulePath, NewModulePath + ".imports", ImportLists));
> + return Error();
> + }
> +
> + Error wait() override { return Error(); }
> +};
> +
> +ThinBackend lto::createWriteIndexesThinBackend(std::string OldPrefix,
> + std::string NewPrefix,
> + bool ShouldEmitImportsFiles,
> + std::string LinkedObjectsFile) {
> + return [=](Config &Conf, ModuleSummaryIndex &CombinedIndex,
> + StringMap<GVSummaryMapTy> &ModuleToDefinedGVSummaries,
> + AddStreamFn AddStream) {
> + return llvm::make_unique<WriteIndexesThinBackend>(
> + Conf, CombinedIndex, ModuleToDefinedGVSummaries, AddStream, OldPrefix,
> + NewPrefix, ShouldEmitImportsFiles, LinkedObjectsFile);
> + };
> +}
> +
> +Error LTO::runThinLTO(AddStreamFn AddStream) {
> + if (ThinLTO.ModuleMap.empty())
> + return Error();
> +
> + if (Conf.CombinedIndexHook && !Conf.CombinedIndexHook(ThinLTO.CombinedIndex))
> + return Error();
> +
> + // Collect for each module the list of function it defines (GUID ->
> + // Summary).
> + StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
> + ModuleToDefinedGVSummaries(ThinLTO.ModuleMap.size());
> + ThinLTO.CombinedIndex.collectDefinedGVSummariesPerModule(
> + ModuleToDefinedGVSummaries);
> +
> + StringMap<FunctionImporter::ImportMapTy> ImportLists(
> + ThinLTO.ModuleMap.size());
> + StringMap<FunctionImporter::ExportSetTy> ExportLists(
> + ThinLTO.ModuleMap.size());
> + ComputeCrossModuleImport(ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries,
> + ImportLists, ExportLists);
> +
> + std::set<GlobalValue::GUID> ExportedGUIDs;
> + for (auto &Res : GlobalResolutions) {
> + if (!Res.second.IRName.empty() &&
> + Res.second.Partition == GlobalResolution::External)
> + ExportedGUIDs.insert(GlobalValue::getGUID(Res.second.IRName));
> + }
> +
> + auto isPrevailing = [&](GlobalValue::GUID GUID, const GlobalValueSummary *S) {
> + return ThinLTO.PrevailingModuleForGUID[GUID] == S->modulePath();
> + };
> + auto isExported = [&](StringRef ModuleIdentifier, GlobalValue::GUID GUID) {
> + const auto &ExportList = ExportLists.find(ModuleIdentifier);
> + return (ExportList != ExportLists.end() &&
> + ExportList->second.count(GUID)) ||
> + ExportedGUIDs.count(GUID);
> + };
> + thinLTOInternalizeAndPromoteInIndex(ThinLTO.CombinedIndex, isExported);
> + thinLTOResolveWeakForLinkerInIndex(
> + ThinLTO.CombinedIndex, isPrevailing,
> + [](StringRef, GlobalValue::GUID, GlobalValue::LinkageTypes) {});
> +
> + std::unique_ptr<ThinBackendProc> BackendProc = ThinLTO.Backend(
> + Conf, ThinLTO.CombinedIndex, ModuleToDefinedGVSummaries, AddStream);
> +
> + // Partition numbers for ThinLTO jobs start at 1 (see comments for
> + // GlobalResolution in LTO.h). Task numbers, however, start at
> + // ParallelCodeGenParallelismLevel, as tasks 0 through
> + // ParallelCodeGenParallelismLevel-1 are reserved for parallel code generation
> + // partitions.
> + size_t Task = RegularLTO.ParallelCodeGenParallelismLevel;
> + size_t Partition = 1;
> +
> + for (auto &Mod : ThinLTO.ModuleMap) {
> + if (Error E = BackendProc->start(Task, Mod.second, ImportLists,
> + ThinLTO.ModuleMap))
> + return E;
> +
> + ++Task;
> + ++Partition;
> + }
> +
> + return BackendProc->wait();
> }
>
> Added: llvm/trunk/lib/LTO/LTOBackend.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/LTO/LTOBackend.cpp?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/lib/LTO/LTOBackend.cpp (added)
> +++ llvm/trunk/lib/LTO/LTOBackend.cpp Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,284 @@
> +//===-LTOBackend.cpp - LLVM Link Time Optimizer Backend -------------------===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This file implements the "backend" phase of LTO, i.e. it performs
> +// optimization and code generation on a loaded module. It is generally used
> +// internally by the LTO class but can also be used independently, for example
> +// to implement a standalone ThinLTO backend.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "llvm/LTO/LTOBackend.h"
> +#include "llvm/Analysis/TargetLibraryInfo.h"
> +#include "llvm/Analysis/TargetTransformInfo.h"
> +#include "llvm/Bitcode/ReaderWriter.h"
> +#include "llvm/IR/LegacyPassManager.h"
> +#include "llvm/MC/SubtargetFeature.h"
> +#include "llvm/Support/Error.h"
> +#include "llvm/Support/FileSystem.h"
> +#include "llvm/Support/TargetRegistry.h"
> +#include "llvm/Support/ThreadPool.h"
> +#include "llvm/Target/TargetMachine.h"
> +#include "llvm/Transforms/IPO.h"
> +#include "llvm/Transforms/IPO/PassManagerBuilder.h"
> +#include "llvm/Transforms/Utils/FunctionImportUtils.h"
> +#include "llvm/Transforms/Utils/SplitModule.h"
> +
> +using namespace llvm;
> +using namespace lto;
> +
> +Error Config::addSaveTemps(std::string OutputFileName,
> + bool UseInputModulePath) {
> + ShouldDiscardValueNames = false;
> +
> + std::error_code EC;
> + ResolutionFile = llvm::make_unique<raw_fd_ostream>(
> + OutputFileName + ".resolution.txt", EC, sys::fs::OpenFlags::F_Text);
> + if (EC)
> + return errorCodeToError(EC);
> +
> + auto setHook = [&](std::string PathSuffix, ModuleHookFn &Hook) {
> + // Keep track of the hook provided by the linker, which also needs to run.
> + ModuleHookFn LinkerHook = Hook;
> + Hook = [=](size_t Task, Module &M) {
> + // If the linker's hook returned false, we need to pass that result
> + // through.
> + if (LinkerHook && !LinkerHook(Task, M))
> + return false;
> +
> + std::string PathPrefix;
> + // If this is the combined module (not a ThinLTO backend compile) or the
> + // user hasn't requested using the input module's path, emit to a file
> + // named from the provided OutputFileName with the Task ID appended.
> + if (M.getModuleIdentifier() == "ld-temp.o" || !UseInputModulePath) {
> + PathPrefix = OutputFileName;
> + if (Task != 0)
> + PathPrefix += "." + utostr(Task);
> + } else
> + PathPrefix = M.getModuleIdentifier();
> + std::string Path = PathPrefix + "." + PathSuffix + ".bc";
> + std::error_code EC;
> + raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
> + if (EC) {
> + // Because -save-temps is a debugging feature, we report the error
> + // directly and exit.
> + llvm::errs() << "failed to open " << Path << ": " << EC.message()
> + << '\n';
> + exit(1);
> + }
> + WriteBitcodeToFile(&M, OS, /*ShouldPreserveUseListOrder=*/false);
> + return true;
> + };
> + };
> +
> + setHook("0.preopt", PreOptModuleHook);
> + setHook("1.promote", PostPromoteModuleHook);
> + setHook("2.internalize", PostInternalizeModuleHook);
> + setHook("3.import", PostImportModuleHook);
> + setHook("4.opt", PostOptModuleHook);
> + setHook("5.precodegen", PreCodeGenModuleHook);
> +
> + CombinedIndexHook = [=](const ModuleSummaryIndex &Index) {
> + std::string Path = OutputFileName + ".index.bc";
> + std::error_code EC;
> + raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
> + if (EC) {
> + // Because -save-temps is a debugging feature, we report the error
> + // directly and exit.
> + llvm::errs() << "failed to open " << Path << ": " << EC.message() << '\n';
> + exit(1);
> + }
> + WriteIndexToFile(Index, OS);
> + return true;
> + };
> +
> + return Error();
> +}
> +
> +namespace {
> +
> +std::unique_ptr<TargetMachine>
> +createTargetMachine(Config &C, StringRef TheTriple, const Target *TheTarget) {
> + SubtargetFeatures Features;
> + Features.getDefaultSubtargetFeatures(Triple(TheTriple));
> + for (const std::string &A : C.MAttrs)
> + Features.AddFeature(A);
> +
> + return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
> + TheTriple, C.CPU, Features.getString(), C.Options, C.RelocModel,
> + C.CodeModel, C.CGOptLevel));
> +}
> +
> +bool opt(Config &C, TargetMachine *TM, size_t Task, Module &M, bool IsThinLto) {
> + M.setDataLayout(TM->createDataLayout());
> +
> + legacy::PassManager passes;
> + passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
> +
> + PassManagerBuilder PMB;
> + PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
> + PMB.Inliner = createFunctionInliningPass();
> + // Unconditionally verify input since it is not verified before this
> + // point and has unknown origin.
> + PMB.VerifyInput = true;
> + PMB.VerifyOutput = !C.DisableVerify;
> + PMB.LoopVectorize = true;
> + PMB.SLPVectorize = true;
> + PMB.OptLevel = C.OptLevel;
> + if (IsThinLto)
> + PMB.populateThinLTOPassManager(passes);
> + else
> + PMB.populateLTOPassManager(passes);
> + passes.run(M);
> +
> + if (C.PostOptModuleHook && !C.PostOptModuleHook(Task, M))
> + return false;
> +
> + return true;
> +}
> +
> +void codegen(Config &C, TargetMachine *TM, AddStreamFn AddStream, size_t Task,
> + Module &M) {
> + if (C.PreCodeGenModuleHook && !C.PreCodeGenModuleHook(Task, M))
> + return;
> +
> + std::unique_ptr<raw_pwrite_stream> OS = AddStream(Task);
> + legacy::PassManager CodeGenPasses;
> + if (TM->addPassesToEmitFile(CodeGenPasses, *OS,
> + TargetMachine::CGFT_ObjectFile))
> + report_fatal_error("Failed to setup codegen");
> + CodeGenPasses.run(M);
> +}
> +
> +void splitCodeGen(Config &C, TargetMachine *TM, AddStreamFn AddStream,
> + unsigned ParallelCodeGenParallelismLevel,
> + std::unique_ptr<Module> M) {
> + ThreadPool CodegenThreadPool(ParallelCodeGenParallelismLevel);
> + unsigned ThreadCount = 0;
> + const Target *T = &TM->getTarget();
> +
> + SplitModule(
> + std::move(M), ParallelCodeGenParallelismLevel,
> + [&](std::unique_ptr<Module> MPart) {
> + // We want to clone the module in a new context to multi-thread the
> + // codegen. We do it by serializing partition modules to bitcode
> + // (while still on the main thread, in order to avoid data races) and
> + // spinning up new threads which deserialize the partitions into
> + // separate contexts.
> + // FIXME: Provide a more direct way to do this in LLVM.
> + SmallString<0> BC;
> + raw_svector_ostream BCOS(BC);
> + WriteBitcodeToFile(MPart.get(), BCOS);
> +
> + // Enqueue the task
> + CodegenThreadPool.async(
> + [&](const SmallString<0> &BC, unsigned ThreadId) {
> + LTOLLVMContext Ctx(C);
> + ErrorOr<std::unique_ptr<Module>> MOrErr = parseBitcodeFile(
> + MemoryBufferRef(StringRef(BC.data(), BC.size()), "ld-temp.o"),
> + Ctx);
> + if (!MOrErr)
> + report_fatal_error("Failed to read bitcode");
> + std::unique_ptr<Module> MPartInCtx = std::move(MOrErr.get());
> +
> + std::unique_ptr<TargetMachine> TM =
> + createTargetMachine(C, MPartInCtx->getTargetTriple(), T);
> + codegen(C, TM.get(), AddStream, ThreadId, *MPartInCtx);
> + },
> + // Pass BC using std::move to ensure that it get moved rather than
> + // copied into the thread's context.
> + std::move(BC), ThreadCount++);
> + },
> + false);
> +}
> +
> +Expected<const Target *> initAndLookupTarget(Config &C, Module &M) {
> + if (!C.OverrideTriple.empty())
> + M.setTargetTriple(C.OverrideTriple);
> + else if (M.getTargetTriple().empty())
> + M.setTargetTriple(C.DefaultTriple);
> +
> + std::string Msg;
> + const Target *T = TargetRegistry::lookupTarget(M.getTargetTriple(), Msg);
> + if (!T)
> + return make_error<StringError>(Msg, inconvertibleErrorCode());
> + return T;
> +}
> +
> +}
> +
> +Error lto::backend(Config &C, AddStreamFn AddStream,
> + unsigned ParallelCodeGenParallelismLevel,
> + std::unique_ptr<Module> M) {
> + Expected<const Target *> TOrErr = initAndLookupTarget(C, *M);
> + if (!TOrErr)
> + return TOrErr.takeError();
> +
> + std::unique_ptr<TargetMachine> TM =
> + createTargetMachine(C, M->getTargetTriple(), *TOrErr);
> +
> + if (!opt(C, TM.get(), 0, *M, /*IsThinLto=*/false))
> + return Error();
> +
> + if (ParallelCodeGenParallelismLevel == 1)
> + codegen(C, TM.get(), AddStream, 0, *M);
> + else
> + splitCodeGen(C, TM.get(), AddStream, ParallelCodeGenParallelismLevel,
> + std::move(M));
> + return Error();
> +}
> +
> +Error lto::thinBackend(Config &C, size_t Task, AddStreamFn AddStream, Module &M,
> + ModuleSummaryIndex &CombinedIndex,
> + const FunctionImporter::ImportMapTy &ImportList,
> + const GVSummaryMapTy &DefinedGlobals,
> + MapVector<StringRef, MemoryBufferRef> &ModuleMap) {
> + Expected<const Target *> TOrErr = initAndLookupTarget(C, M);
> + if (!TOrErr)
> + return TOrErr.takeError();
> +
> + std::unique_ptr<TargetMachine> TM =
> + createTargetMachine(C, M.getTargetTriple(), *TOrErr);
> +
> + if (C.PreOptModuleHook && !C.PreOptModuleHook(Task, M))
> + return Error();
> +
> + thinLTOResolveWeakForLinkerModule(M, DefinedGlobals);
> +
> + renameModuleForThinLTO(M, CombinedIndex);
> +
> + if (C.PostPromoteModuleHook && !C.PostPromoteModuleHook(Task, M))
> + return Error();
> +
> + if (!DefinedGlobals.empty())
> + thinLTOInternalizeModule(M, DefinedGlobals);
> +
> + if (C.PostInternalizeModuleHook && !C.PostInternalizeModuleHook(Task, M))
> + return Error();
> +
> + auto ModuleLoader = [&](StringRef Identifier) {
> + return std::move(getLazyBitcodeModule(MemoryBuffer::getMemBuffer(
> + ModuleMap[Identifier], false),
> + M.getContext(),
> + /*ShouldLazyLoadMetadata=*/true)
> + .get());
> + };
> +
> + FunctionImporter Importer(CombinedIndex, ModuleLoader);
> + Importer.importFunctions(M, ImportList);
> +
> + if (C.PostImportModuleHook && !C.PostImportModuleHook(Task, M))
> + return Error();
> +
> + if (!opt(C, TM.get(), Task, M, /*IsThinLto=*/true))
> + return Error();
> +
> + codegen(C, TM.get(), AddStream, Task, M);
> + return Error();
> +}
>
> Modified: llvm/trunk/lib/Object/IRObjectFile.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Object/IRObjectFile.cpp?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/lib/Object/IRObjectFile.cpp (original)
> +++ llvm/trunk/lib/Object/IRObjectFile.cpp Thu Aug 11 09:58:12 2016
> @@ -324,5 +324,5 @@ llvm::object::IRObjectFile::create(Memor
> return EC;
>
> std::unique_ptr<Module> &M = MOrErr.get();
> - return llvm::make_unique<IRObjectFile>(Object, std::move(M));
> + return llvm::make_unique<IRObjectFile>(BCOrErr.get(), std::move(M));
> }
>
> Modified: llvm/trunk/test/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/CMakeLists.txt?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/CMakeLists.txt (original)
> +++ llvm/trunk/test/CMakeLists.txt Thu Aug 11 09:58:12 2016
> @@ -43,6 +43,7 @@ set(LLVM_TEST_DEPENDS
> llvm-extract
> llvm-lib
> llvm-link
> + llvm-lto2
> llvm-mc
> llvm-mcmarkup
> llvm-nm
>
> Added: llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll (added)
> +++ llvm/trunk/test/LTO/Resolution/X86/Inputs/alias-1.ll Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,4 @@
> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
> +target triple = "x86_64-unknown-linux-gnu"
> +
> + at a = global i32 42
>
> Added: llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll (added)
> +++ llvm/trunk/test/LTO/Resolution/X86/Inputs/comdat.ll Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,28 @@
> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
> +target triple = "x86_64-unknown-linux-gnu"
> +
> +$c2 = comdat any
> +$c1 = comdat any
> +
> +; This is only present in this file. The linker will keep $c1 from the first
> +; file and this will be undefined.
> + at will_be_undefined = global i32 1, comdat($c1)
> +
> + at v1 = weak_odr global i32 41, comdat($c2)
> +define weak_odr protected i32 @f1(i8* %this) comdat($c2) {
> +bb20:
> + store i8* %this, i8** null
> + br label %bb21
> +bb21:
> + ret i32 41
> +}
> +
> + at r21 = global i32* @v1
> + at r22 = global i32(i8*)* @f1
> +
> + at a21 = alias i32, i32* @v1
> + at a22 = alias i16, bitcast (i32* @v1 to i16*)
> +
> + at a23 = alias i32(i8*), i32(i8*)* @f1
> + at a24 = alias i16, bitcast (i32(i8*)* @f1 to i16*)
> + at a25 = alias i16, i16* @a24
>
> Added: llvm/trunk/test/LTO/Resolution/X86/alias.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/alias.ll?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/test/LTO/Resolution/X86/alias.ll (added)
> +++ llvm/trunk/test/LTO/Resolution/X86/alias.ll Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,22 @@
> +; RUN: llvm-as %s -o %t1.o
> +; RUN: llvm-as %p/Inputs/alias-1.ll -o %t2.o
> +; RUN: llvm-lto2 -o %t3.o %t2.o %t1.o -r %t2.o,a,px -r %t1.o,a, -r %t1.o,b,px -save-temps
> +; RUN: llvm-dis < %t3.o.0.preopt.bc -o - | FileCheck %s
> +; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
> +
> +; CHECK-NOT: alias
> +; CHECK: @a = global i32 42
> +; CHECK-NEXT: @b = global i32 1
> +; CHECK-NOT: alias
> +
> +; RES: 2.o{{$}}
> +; RES: {{^}}-r={{.*}}2.o,a,px{{$}}
> +; RES: 1.o{{$}}
> +; RES: {{^}}-r={{.*}}1.o,b,px{{$}}
> +; RES: {{^}}-r={{.*}}1.o,a,{{$}}
> +
> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
> +target triple = "x86_64-unknown-linux-gnu"
> +
> + at a = weak alias i32, i32* @b
> + at b = global i32 1
>
> Copied: llvm/trunk/test/LTO/Resolution/X86/comdat.ll (from r278331, llvm/trunk/test/tools/gold/X86/comdat.ll)
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/comdat.ll?p2=llvm/trunk/test/LTO/Resolution/X86/comdat.ll&p1=llvm/trunk/test/tools/gold/X86/comdat.ll&r1=278331&r2=278338&rev=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/comdat.ll (original)
> +++ llvm/trunk/test/LTO/Resolution/X86/comdat.ll Thu Aug 11 09:58:12 2016
> @@ -1,8 +1,29 @@
> ; RUN: llvm-as %s -o %t.o
> ; RUN: llvm-as %p/Inputs/comdat.ll -o %t2.o
> -; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t.o %t2.o \
> -; RUN: -plugin-opt=save-temps
> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
> +; RUN: llvm-lto2 -save-temps -o %t3.o %t.o %t2.o \
> +; RUN: -r=%t.o,f1,plx \
> +; RUN: -r=%t.o,v1,px \
> +; RUN: -r=%t.o,r11,px \
> +; RUN: -r=%t.o,r12,px \
> +; RUN: -r=%t.o,a11,px \
> +; RUN: -r=%t.o,a12,px \
> +; RUN: -r=%t.o,a13,px \
> +; RUN: -r=%t.o,a14,px \
> +; RUN: -r=%t.o,a15,px \
> +; RUN: -r=%t2.o,f1,l \
> +; RUN: -r=%t2.o,will_be_undefined, \
> +; RUN: -r=%t2.o,v1, \
> +; RUN: -r=%t2.o,r21,px \
> +; RUN: -r=%t2.o,r22,px \
> +; RUN: -r=%t2.o,a21,px \
> +; RUN: -r=%t2.o,a22,px \
> +; RUN: -r=%t2.o,a23,px \
> +; RUN: -r=%t2.o,a24,px \
> +; RUN: -r=%t2.o,a25,px
> +; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
> +
> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
> +target triple = "x86_64-unknown-linux-gnu"
>
> $c1 = comdat any
>
> @@ -49,7 +70,7 @@ bb11:
> ; CHECK-DAG: @a23 = alias i32 (i8*), i32 (i8*)* @f1.2{{$}}
> ; CHECK-DAG: @a24 = alias i16, bitcast (i32 (i8*)* @f1.2 to i16*)
>
> -; CHECK: define weak_odr protected i32 @f1(i8*) comdat($c1) {
> +; CHECK: define weak_odr i32 @f1(i8*) comdat($c1) {
> ; CHECK-NEXT: bb10:
> ; CHECK-NEXT: br label %bb11{{$}}
> ; CHECK: bb11:
>
> Added: llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg (added)
> +++ llvm/trunk/test/LTO/Resolution/X86/lit.local.cfg Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,2 @@
> +if not 'X86' in config.root.targets:
> + config.unsupported = True
>
> Modified: llvm/trunk/test/lit.cfg
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/lit.cfg?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/lit.cfg (original)
> +++ llvm/trunk/test/lit.cfg Thu Aug 11 09:58:12 2016
> @@ -295,6 +295,7 @@ for pattern in [r"\bbugpoint\b(?!-)",
> r"\bllvm-lib\b",
> r"\bllvm-link\b",
> r"\bllvm-lto\b",
> + r"\bllvm-lto2\b",
> r"\bllvm-mc\b",
> r"\bllvm-mcmarkup\b",
> r"\bllvm-nm\b",
>
> Modified: llvm/trunk/test/tools/gold/X86/coff.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/coff.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/coff.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/coff.ll Thu Aug 11 09:58:12 2016
> @@ -16,7 +16,7 @@ define hidden void @g() {
> ret void
> }
>
> -; CHECK: define internal void @h() local_unnamed_addr {
> +; CHECK: define internal void @h() {
> define linkonce_odr void @h() local_unnamed_addr {
> ret void
> }
>
> Modified: llvm/trunk/test/tools/gold/X86/comdat.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/comdat.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/comdat.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/comdat.ll Thu Aug 11 09:58:12 2016
> @@ -1,8 +1,9 @@
> -; RUN: llvm-as %s -o %t.o
> +; RUN: llvm-as %s -o %t1.o
> ; RUN: llvm-as %p/Inputs/comdat.ll -o %t2.o
> -; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t.o %t2.o \
> +; RUN: %gold -shared -o %t3.o -plugin %llvmshlibdir/LLVMgold.so %t1.o %t2.o \
> ; RUN: -plugin-opt=save-temps
> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
> +; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
> +; RUN: llvm-readobj -t %t3.o | FileCheck --check-prefix=OBJ %s
>
> $c1 = comdat any
>
> @@ -24,42 +25,37 @@ bb11:
> @a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
> @a15 = alias i16, i16* @a14
>
> -; CHECK: $c1 = comdat any
> -; CHECK: $c2 = comdat any
> +; gold's resolutions should tell us that our $c1 wins, and the other input's $c2
> +; wins. f1 is also local due to having protected visibility in the other object.
>
> -; CHECK-DAG: @v1 = weak_odr global i32 42, comdat($c1)
> -
> -; CHECK-DAG: @r11 = global i32* @v1{{$}}
> -; CHECK-DAG: @r12 = global i32 (i8*)* @f1{{$}}
> -
> -; CHECK-DAG: @r21 = global i32* @v1{{$}}
> -; CHECK-DAG: @r22 = global i32 (i8*)* @f1{{$}}
> -
> -; CHECK-DAG: @v1.1 = internal global i32 41, comdat($c2)
> -
> -; CHECK-DAG: @a11 = alias i32, i32* @v1{{$}}
> -; CHECK-DAG: @a12 = alias i16, bitcast (i32* @v1 to i16*)
> -
> -; CHECK-DAG: @a13 = alias i32 (i8*), i32 (i8*)* @f1{{$}}
> -; CHECK-DAG: @a14 = alias i16, bitcast (i32 (i8*)* @f1 to i16*)
> -
> -; CHECK-DAG: @a21 = alias i32, i32* @v1.1{{$}}
> -; CHECK-DAG: @a22 = alias i16, bitcast (i32* @v1.1 to i16*)
> -
> -; CHECK-DAG: @a23 = alias i32 (i8*), i32 (i8*)* @f1.2{{$}}
> -; CHECK-DAG: @a24 = alias i16, bitcast (i32 (i8*)* @f1.2 to i16*)
> -
> -; CHECK: define weak_odr protected i32 @f1(i8*) comdat($c1) {
> -; CHECK-NEXT: bb10:
> -; CHECK-NEXT: br label %bb11{{$}}
> -; CHECK: bb11:
> -; CHECK-NEXT: ret i32 42
> -; CHECK-NEXT: }
> -
> -; CHECK: define internal i32 @f1.2(i8* %this) comdat($c2) {
> -; CHECK-NEXT: bb20:
> -; CHECK-NEXT: store i8* %this, i8** null
> -; CHECK-NEXT: br label %bb21
> -; CHECK: bb21:
> -; CHECK-NEXT: ret i32 41
> -; CHECK-NEXT: }
> +; RES: 1.o,f1,plx{{$}}
> +; RES: 1.o,v1,px{{$}}
> +; RES: 1.o,r11,px{{$}}
> +; RES: 1.o,r12,px{{$}}
> +; RES: 1.o,a11,px{{$}}
> +; RES: 1.o,a12,px{{$}}
> +; RES: 1.o,a13,px{{$}}
> +; RES: 1.o,a14,px{{$}}
> +; RES: 1.o,a15,px{{$}}
> +
> +; RES: 2.o,f1,l{{$}}
> +; RES: 2.o,will_be_undefined,{{$}}
> +; RES: 2.o,v1,{{$}}
> +; RES: 2.o,r21,px{{$}}
> +; RES: 2.o,r22,px{{$}}
> +; RES: 2.o,a21,px{{$}}
> +; RES: 2.o,a22,px{{$}}
> +; RES: 2.o,a23,px{{$}}
> +; RES: 2.o,a24,px{{$}}
> +; RES: 2.o,a25,px{{$}}
> +
> +; f1's protected visibility should be reflected in the DSO.
> +
> +; OBJ: Name: f1 (
> +; OBJ-NEXT: Value:
> +; OBJ-NEXT: Size:
> +; OBJ-NEXT: Binding:
> +; OBJ-NEXT: Type:
> +; OBJ-NEXT: Other [
> +; OBJ-NEXT: STV_PROTECTED
> +; OBJ-NEXT: ]
>
> Modified: llvm/trunk/test/tools/gold/X86/common.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/common.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/common.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/common.ll Thu Aug 11 09:58:12 2016
> @@ -11,7 +11,7 @@
> ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=A
>
> ; Shared library case, we merge @a as common and keep it for the symbol table.
> -; A: @a = common global i32 0, align 8
> +; A: @a = common global [4 x i8] zeroinitializer, align 8
>
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=emit-llvm \
> @@ -19,7 +19,7 @@
> ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=B
>
> ; (i16 align 8) + (i8 align 16) = i16 align 16
> -; B: @a = common global i16 0, align 16
> +; B: @a = common global [2 x i8] zeroinitializer, align 16
>
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=emit-llvm \
> @@ -27,7 +27,7 @@
> ; RUN: llvm-dis %t3.o -o - | FileCheck %s --check-prefix=C
>
> ; (i16 align 8) + (i8 align 1) = i16 align 8.
> -; C: @a = common global i16 0, align 8
> +; C: @a = common global [2 x i8] zeroinitializer, align 8
>
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=emit-llvm \
> @@ -35,7 +35,7 @@
> ; RUN: llvm-dis %t3.o -o - | FileCheck --check-prefix=EXEC %s
>
> ; All IR case, we internalize a after merging.
> -; EXEC: @a = internal global i32 0, align 8
> +; EXEC: @a = internal global [4 x i8] zeroinitializer, align 8
>
> ; RUN: llc %p/Inputs/common.ll -o %t2native.o -filetype=obj
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> @@ -44,4 +44,4 @@
> ; RUN: llvm-dis %t3.o -o - | FileCheck --check-prefix=MIXED %s
>
> ; Mixed ELF and IR. We keep ours as common so the linker will finish the merge.
> -; MIXED: @a = common global i16 0, align 8
> +; MIXED: @a = common global [2 x i8] zeroinitializer, align 8
>
> Modified: llvm/trunk/test/tools/gold/X86/emit-llvm.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/emit-llvm.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/emit-llvm.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/emit-llvm.ll Thu Aug 11 09:58:12 2016
> @@ -2,17 +2,16 @@
>
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=emit-llvm \
> -; RUN: --plugin-opt=generate-api-file \
> ; RUN: -shared %t.o -o %t2.o
> ; RUN: llvm-dis %t2.o -o - | FileCheck %s
> -; RUN: FileCheck --check-prefix=API %s < %T/../apifile.txt
>
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: -m elf_x86_64 --plugin-opt=save-temps \
> ; RUN: -shared %t.o -o %t3.o
> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
> -; RUN: llvm-dis %t3.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
> -; RUN: llvm-dis %t3.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
> +; RUN: FileCheck --check-prefix=RES %s < %t3.o.resolution.txt
> +; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
> +; RUN: llvm-dis %t3.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
> +; RUN: llvm-dis %t3.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
> ; RUN: llvm-nm %t3.o.o | FileCheck --check-prefix=NM %s
>
> ; RUN: rm -f %t4.o
> @@ -25,19 +24,19 @@
>
> target triple = "x86_64-unknown-linux-gnu"
>
> -; CHECK-DAG: @g1 = linkonce_odr constant i32 32
> +; CHECK-DAG: @g1 = weak_odr constant i32 32
> @g1 = linkonce_odr constant i32 32
>
> -; CHECK-DAG: @g2 = internal local_unnamed_addr constant i32 32
> +; CHECK-DAG: @g2 = internal constant i32 32
> @g2 = linkonce_odr local_unnamed_addr constant i32 32
>
> ; CHECK-DAG: @g3 = internal unnamed_addr constant i32 32
> @g3 = linkonce_odr unnamed_addr constant i32 32
>
> -; CHECK-DAG: @g4 = linkonce_odr global i32 32
> +; CHECK-DAG: @g4 = weak_odr global i32 32
> @g4 = linkonce_odr global i32 32
>
> -; CHECK-DAG: @g5 = linkonce_odr local_unnamed_addr global i32 32
> +; CHECK-DAG: @g5 = weak_odr global i32 32
> @g5 = linkonce_odr local_unnamed_addr global i32 32
>
> ; CHECK-DAG: @g6 = internal unnamed_addr global i32 32
> @@ -75,8 +74,8 @@ define linkonce_odr void @f4() local_unn
> ret void
> }
>
> -; CHECK-DAG: define linkonce_odr void @f5()
> -; OPT-DAG: define linkonce_odr void @f5()
> +; CHECK-DAG: define weak_odr void @f5()
> +; OPT-DAG: define weak_odr void @f5()
> define linkonce_odr void @f5() {
> ret void
> }
> @@ -97,15 +96,21 @@ define i32* @f8() {
> ret i32* @g8
> }
>
> -; API: f1 PREVAILING_DEF_IRONLY
> -; API: f2 PREVAILING_DEF_IRONLY
> -; API: f3 PREVAILING_DEF_IRONLY_EXP
> -; API: f4 PREVAILING_DEF_IRONLY_EXP
> -; API: f5 PREVAILING_DEF_IRONLY_EXP
> -; API: f6 PREVAILING_DEF_IRONLY_EXP
> -; API: f7 PREVAILING_DEF_IRONLY_EXP
> -; API: f8 PREVAILING_DEF_IRONLY_EXP
> -; API: g7 UNDEF
> -; API: g8 UNDEF
> -; API: g9 PREVAILING_DEF_IRONLY_EXP
> -; API: g10 PREVAILING_DEF_IRONLY_EXP
> +; RES: .o,f1,pl{{$}}
> +; RES: .o,f2,pl{{$}}
> +; RES: .o,f3,px{{$}}
> +; RES: .o,f4,p{{$}}
> +; RES: .o,f5,px{{$}}
> +; RES: .o,f6,p{{$}}
> +; RES: .o,f7,px{{$}}
> +; RES: .o,f8,px{{$}}
> +; RES: .o,g1,px{{$}}
> +; RES: .o,g2,p{{$}}
> +; RES: .o,g3,p{{$}}
> +; RES: .o,g4,px{{$}}
> +; RES: .o,g5,px{{$}}
> +; RES: .o,g6,p{{$}}
> +; RES: .o,g7,{{$}}
> +; RES: .o,g8,{{$}}
> +; RES: .o,g9,px{{$}}
> +; RES: .o,g10,px{{$}}
>
> Modified: llvm/trunk/test/tools/gold/X86/opt-level.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/opt-level.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/opt-level.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/opt-level.ll Thu Aug 11 09:58:12 2016
> @@ -1,13 +1,13 @@
> ; RUN: llvm-as -o %t.bc %s
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
> ; RUN: -plugin-opt=O0 -r -o %t.o %t.bc
> -; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O0 %s
> +; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O0 %s
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
> ; RUN: -plugin-opt=O1 -r -o %t.o %t.bc
> -; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O1 %s
> +; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O1 %s
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so -plugin-opt=save-temps \
> ; RUN: -plugin-opt=O2 -r -o %t.o %t.bc
> -; RUN: llvm-dis < %t.o.opt.bc -o - | FileCheck --check-prefix=CHECK-O2 %s
> +; RUN: llvm-dis < %t.o.4.opt.bc -o - | FileCheck --check-prefix=CHECK-O2 %s
>
> ; CHECK-O0: define internal void @foo(
> ; CHECK-O1: define internal void @foo(
>
> Modified: llvm/trunk/test/tools/gold/X86/parallel.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/parallel.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/parallel.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/parallel.ll Thu Aug 11 09:58:12 2016
> @@ -1,7 +1,8 @@
> ; RUN: llvm-as -o %t.bc %s
> +; RUN: rm -f %t.opt.bc0 %t.opt.bc1 %t.o0 %t.o1
> ; RUN: env LD_PRELOAD=%llvmshlibdir/LLVMgold.so %gold -plugin %llvmshlibdir/LLVMgold.so -u foo -u bar -plugin-opt jobs=2 -plugin-opt save-temps -m elf_x86_64 -o %t %t.bc
> -; RUN: llvm-dis %t.opt.bc0 -o - | FileCheck --check-prefix=CHECK-BC0 %s
> -; RUN: llvm-dis %t.opt.bc1 -o - | FileCheck --check-prefix=CHECK-BC1 %s
> +; RUN: llvm-dis %t.5.precodegen.bc -o - | FileCheck --check-prefix=CHECK-BC0 %s
> +; RUN: llvm-dis %t.1.5.precodegen.bc -o - | FileCheck --check-prefix=CHECK-BC1 %s
> ; RUN: llvm-nm %t.o0 | FileCheck --check-prefix=CHECK0 %s
> ; RUN: llvm-nm %t.o1 | FileCheck --check-prefix=CHECK1 %s
>
>
> Modified: llvm/trunk/test/tools/gold/X86/slp-vectorize.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/slp-vectorize.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/slp-vectorize.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/slp-vectorize.ll Thu Aug 11 09:58:12 2016
> @@ -3,7 +3,7 @@
> ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=save-temps \
> ; RUN: -shared %t.o -o %t2.o
> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck %s
> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck %s
>
> ; test that the vectorizer is run.
> ; CHECK: fadd <4 x float>
>
> Modified: llvm/trunk/test/tools/gold/X86/start-lib-common.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/start-lib-common.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/start-lib-common.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/start-lib-common.ll Thu Aug 11 09:58:12 2016
> @@ -19,4 +19,4 @@
>
> ; Check that the common symbol is not dropped completely, which was a regression
> ; in r262676.
> -; CHECK: @x = common global i32 0
> +; CHECK: @x = common global [4 x i8] zeroinitializer
>
> Modified: llvm/trunk/test/tools/gold/X86/strip_names.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/strip_names.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/strip_names.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/strip_names.ll Thu Aug 11 09:58:12 2016
> @@ -3,7 +3,7 @@
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=save-temps \
> ; RUN: -shared %t.o -o %t2.o
> -; RUN: llvm-dis %t2.o.bc -o - | FileCheck %s
> +; RUN: llvm-dis %t2.o.2.internalize.bc -o - | FileCheck %s
>
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=emit-llvm \
>
> Modified: llvm/trunk/test/tools/gold/X86/thinlto.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/thinlto.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/thinlto.ll Thu Aug 11 09:58:12 2016
> @@ -25,21 +25,23 @@
> ; RUN: llvm-bcanalyzer -dump %t2.o.thinlto.bc | FileCheck %s --check-prefix=BACKEND2
> ; RUN: not test -e %t3
>
> -; Ensure gold generates an index as well as a binary by default in ThinLTO mode.
> +; Ensure gold generates an index as well as a binary with save-temps in ThinLTO mode.
> ; First force single-threaded mode
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> +; RUN: --plugin-opt=save-temps \
> ; RUN: --plugin-opt=thinlto \
> ; RUN: --plugin-opt=jobs=1 \
> ; RUN: -shared %t.o %t2.o -o %t4
> -; RUN: llvm-bcanalyzer -dump %t4.thinlto.bc | FileCheck %s --check-prefix=COMBINED
> +; RUN: llvm-bcanalyzer -dump %t4.index.bc | FileCheck %s --check-prefix=COMBINED
> ; RUN: llvm-nm %t4 | FileCheck %s --check-prefix=NM
>
> ; Next force multi-threaded mode
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> +; RUN: --plugin-opt=save-temps \
> ; RUN: --plugin-opt=thinlto \
> ; RUN: --plugin-opt=jobs=2 \
> ; RUN: -shared %t.o %t2.o -o %t4
> -; RUN: llvm-bcanalyzer -dump %t4.thinlto.bc | FileCheck %s --check-prefix=COMBINED
> +; RUN: llvm-bcanalyzer -dump %t4.index.bc | FileCheck %s --check-prefix=COMBINED
> ; RUN: llvm-nm %t4 | FileCheck %s --check-prefix=NM
>
> ; Test --plugin-opt=obj-path to ensure unique object files generated.
> @@ -48,8 +50,8 @@
> ; RUN: --plugin-opt=jobs=2 \
> ; RUN: --plugin-opt=obj-path=%t5.o \
> ; RUN: -shared %t.o %t2.o -o %t4
> -; RUN: llvm-nm %t5.o0 | FileCheck %s --check-prefix=NM2
> ; RUN: llvm-nm %t5.o1 | FileCheck %s --check-prefix=NM2
> +; RUN: llvm-nm %t5.o2 | FileCheck %s --check-prefix=NM2
>
> ; NM: T f
> ; NM2: T {{f|g}}
>
> Modified: llvm/trunk/test/tools/gold/X86/thinlto_alias.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_alias.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/thinlto_alias.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/thinlto_alias.ll Thu Aug 11 09:58:12 2016
> @@ -14,8 +14,14 @@
> ; RUN: --plugin-opt=save-temps \
> ; RUN: -o %t3.o %t2.o %t.o
> ; RUN: llvm-nm %t3.o | FileCheck %s
> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
> +
> +; This does not currently pass because the gold plugin now uses the
> +; combined summary rather than the IRMover to change the module's linkage
> +; during the ThinLTO backend. The internalization step implemented by IRMover
> +; for preempted symbols has not yet been implemented for the combined summary.
> +; XFAIL: *
>
> ; CHECK-NOT: U f
> ; OPT: define hidden void @weakfunc.llvm.0()
>
> Modified: llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/thinlto_internalize.ll Thu Aug 11 09:58:12 2016
> @@ -6,7 +6,7 @@
> ; RUN: --plugin-opt=-import-instr-limit=0 \
> ; RUN: --plugin-opt=save-temps \
> ; RUN: -o %t3.o %t2.o %t.o
> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck %s
> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck %s
>
> ; f() should be internalized and eliminated after inlining
> ; CHECK-NOT: @f()
>
> Modified: llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/thinlto_linkonceresolution.ll Thu Aug 11 09:58:12 2016
> @@ -14,13 +14,13 @@
> ; RUN: -shared \
> ; RUN: -o %t3.o %t2.o %t.o
> ; RUN: llvm-nm %t3.o | FileCheck %s
> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
>
> ; Ensure that f() is defined in resulting object file, and also
> ; confirm the weak linkage directly in the saved opt bitcode files.
> ; CHECK-NOT: U f
> -; OPT: declare hidden void @f()
> +; OPT-NOT: @f()
> ; OPT2: define weak_odr hidden void @f()
>
> target triple = "x86_64-unknown-linux-gnu"
>
> Modified: llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/thinlto_weak_resolution.ll Thu Aug 11 09:58:12 2016
> @@ -13,12 +13,17 @@
> ; RUN: llvm-nm %t3.o | FileCheck %s
> ; CHECK: weakfunc
>
> -; All of the preempted functions should have been eliminated (the plugin will
> -; not link them in).
> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
> +; Most of the preempted functions should have been eliminated (the plugin will
> +; set linkage of odr functions to available_externally and linkonce functions
> +; are removed by globaldce). FIXME: Need to introduce combined index linkage
> +; that means "drop this function" so we can avoid importing linkonce functions
> +; and drop weak functions.
> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck --check-prefix=OPT2 %s
> +; OPT2-NOT: @
> +; OPT2: @weakfunc
> ; OPT2-NOT: @
>
> -; RUN: llvm-dis %t.o.opt.bc -o - | FileCheck --check-prefix=OPT %s
> +; RUN: llvm-dis %t.o.4.opt.bc -o - | FileCheck --check-prefix=OPT %s
>
> target triple = "x86_64-unknown-linux-gnu"
>
>
> Modified: llvm/trunk/test/tools/gold/X86/type-merge2.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/type-merge2.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/type-merge2.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/type-merge2.ll Thu Aug 11 09:58:12 2016
> @@ -3,7 +3,7 @@
> ; RUN: %gold -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=save-temps \
> ; RUN: -shared %t.o %t2.o -o %t3.o
> -; RUN: llvm-dis %t3.o.bc -o - | FileCheck %s
> +; RUN: llvm-dis %t3.o.2.internalize.bc -o - | FileCheck %s
>
> %zed = type { i8 }
> define void @foo() {
>
> Modified: llvm/trunk/test/tools/gold/X86/vectorize.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/vectorize.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/vectorize.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/vectorize.ll Thu Aug 11 09:58:12 2016
> @@ -3,7 +3,7 @@
> ; RUN: %gold -m elf_x86_64 -plugin %llvmshlibdir/LLVMgold.so \
> ; RUN: --plugin-opt=save-temps \
> ; RUN: -shared %t.o -o %t2.o
> -; RUN: llvm-dis %t2.o.opt.bc -o - | FileCheck %s
> +; RUN: llvm-dis %t2.o.4.opt.bc -o - | FileCheck %s
>
> ; test that the vectorizer is run.
> ; CHECK: fadd <4 x float>
>
> Modified: llvm/trunk/test/tools/gold/X86/visibility.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/gold/X86/visibility.ll?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/test/tools/gold/X86/visibility.ll (original)
> +++ llvm/trunk/test/tools/gold/X86/visibility.ll Thu Aug 11 09:58:12 2016
> @@ -5,7 +5,7 @@
> ; RUN: --plugin-opt=save-temps \
> ; RUN: -shared %t.o %t2.o -o %t.so
> ; RUN: llvm-readobj -t %t.so | FileCheck %s
> -; RUN: llvm-dis %t.so.bc -o - | FileCheck --check-prefix=IR %s
> +; RUN: llvm-dis %t.so.2.internalize.bc -o - | FileCheck --check-prefix=IR %s
>
> ; CHECK: Name: foo
> ; CHECK-NEXT: Value:
> @@ -16,7 +16,7 @@
> ; CHECK-NEXT: STV_PROTECTED
> ; CHECK-NEXT: ]
>
> -; IR: define protected void @foo
> +; IR: define void @foo
>
> define weak protected void @foo() {
> ret void
>
> Added: llvm/trunk/test/tools/llvm-lto2/errors.ll
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/tools/llvm-lto2/errors.ll?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/test/tools/llvm-lto2/errors.ll (added)
> +++ llvm/trunk/test/tools/llvm-lto2/errors.ll Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,11 @@
> +; RUN: llvm-as %s -o %t.bc
> +; RUN: not llvm-lto2 -o %t2.o %t.bc 2>&1 | FileCheck --check-prefix=ERR1 %s
> +; RUN: not llvm-lto2 -o %t2.o -r %t.bc,foo,p -r %t.bc,bar,p %t.bc 2>&1 | FileCheck --check-prefix=ERR2 %s
> +; RUN: not llvm-lto2 -o %t2.o -r %t.bc,foo,q %t.bc 2>&1 | FileCheck --check-prefix=ERR3 %s
> +; RUN: not llvm-lto2 -o %t2.o -r foo %t.bc 2>&1 | FileCheck --check-prefix=ERR4 %s
> +
> +; ERR1: missing symbol resolution for {{.*}}.bc,foo
> +; ERR2: unused symbol resolution for {{.*}}.bc,bar
> +; ERR3: invalid character q in resolution: {{.*}}.bc,foo
> +; ERR4: invalid resolution: foo
> + at foo = global i32 0
>
> Modified: llvm/trunk/tools/gold/gold-plugin.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/gold/gold-plugin.cpp?rev=278338&r1=278337&r2=278338&view=diff
> ==============================================================================
> --- llvm/trunk/tools/gold/gold-plugin.cpp (original)
> +++ llvm/trunk/tools/gold/gold-plugin.cpp Thu Aug 11 09:58:12 2016
> @@ -12,44 +12,22 @@
> //
> //===----------------------------------------------------------------------===//
>
> -#include "llvm/ADT/StringSet.h"
> -#include "llvm/Analysis/TargetLibraryInfo.h"
> -#include "llvm/Analysis/TargetTransformInfo.h"
> #include "llvm/Bitcode/ReaderWriter.h"
> -#include "llvm/CodeGen/Analysis.h"
> #include "llvm/CodeGen/CommandFlags.h"
> -#include "llvm/CodeGen/ParallelCG.h"
> #include "llvm/Config/config.h" // plugin-api.h requires HAVE_STDINT_H
> -#include "llvm/IR/AutoUpgrade.h"
> #include "llvm/IR/Constants.h"
> -#include "llvm/IR/DiagnosticInfo.h"
> #include "llvm/IR/DiagnosticPrinter.h"
> -#include "llvm/IR/LLVMContext.h"
> -#include "llvm/IR/LegacyPassManager.h"
> -#include "llvm/IR/Module.h"
> -#include "llvm/IR/Verifier.h"
> #include "llvm/LTO/LTO.h"
> -#include "llvm/Linker/IRMover.h"
> -#include "llvm/MC/SubtargetFeature.h"
> -#include "llvm/Object/IRObjectFile.h"
> -#include "llvm/Object/ModuleSummaryIndexObjectFile.h"
> -#include "llvm/Support/Host.h"
> +#include "llvm/Support/CommandLine.h"
> #include "llvm/Support/ManagedStatic.h"
> #include "llvm/Support/MemoryBuffer.h"
> #include "llvm/Support/Path.h"
> -#include "llvm/Support/TargetRegistry.h"
> #include "llvm/Support/TargetSelect.h"
> -#include "llvm/Support/ThreadPool.h"
> #include "llvm/Support/raw_ostream.h"
> -#include "llvm/Support/thread.h"
> -#include "llvm/Transforms/IPO.h"
> -#include "llvm/Transforms/IPO/FunctionImport.h"
> -#include "llvm/Transforms/IPO/PassManagerBuilder.h"
> -#include "llvm/Transforms/Utils/FunctionImportUtils.h"
> -#include "llvm/Transforms/Utils/GlobalStatus.h"
> -#include "llvm/Transforms/Utils/ValueMapper.h"
> #include <list>
> +#include <map>
> #include <plugin-api.h>
> +#include <string>
> #include <system_error>
> #include <utility>
> #include <vector>
> @@ -61,6 +39,7 @@
> #define LDPT_GET_SYMBOLS_V3 28
>
> using namespace llvm;
> +using namespace lto;
>
> static ld_plugin_status discard_message(int level, const char *format, ...) {
> // Die loudly. Recent versions of Gold pass ld_plugin_message as the first
> @@ -106,38 +85,17 @@ struct PluginInputFile {
> };
>
> struct ResolutionInfo {
> - uint64_t CommonSize = 0;
> - unsigned CommonAlign = 0;
> - bool IsLinkonceOdr = true;
> - GlobalValue::UnnamedAddr UnnamedAddr = GlobalValue::UnnamedAddr::Global;
> - GlobalValue::VisibilityTypes Visibility = GlobalValue::DefaultVisibility;
> - bool CommonInternal = false;
> - bool UseCommon = false;
> + bool CanOmitFromDynSym = true;
> + bool DefaultVisibility = true;
> };
>
> -/// Class to own information used by a task or during its cleanup for a
> -/// ThinLTO backend instantiation.
> -class ThinLTOTaskInfo {
> - /// The output stream the task will codegen into.
> - std::unique_ptr<raw_fd_ostream> OS;
> -
> - /// The file name corresponding to the output stream, used during cleanup.
> - std::string Filename;
> -
> - /// Flag indicating whether the output file is a temp file that must be
> - /// added to the cleanup list during cleanup.
> - bool TempOutFile;
> -
> -public:
> - ThinLTOTaskInfo(std::unique_ptr<raw_fd_ostream> OS, std::string Filename,
> - bool TempOutFile)
> - : OS(std::move(OS)), Filename(std::move(Filename)),
> - TempOutFile(TempOutFile) {}
> -
> - /// Performs task related cleanup activities that must be done
> - /// single-threaded (i.e. call backs to gold).
> - void cleanup();
> +struct CommonResolution {
> + bool Prevailing = false;
> + bool VisibleToRegularObj = false;
> + uint64_t Size = 0;
> + unsigned Align = 0;
> };
> +
> }
>
> static ld_plugin_add_symbols add_symbols = nullptr;
> @@ -145,14 +103,16 @@ static ld_plugin_get_symbols get_symbols
> static ld_plugin_add_input_file add_input_file = nullptr;
> static ld_plugin_set_extra_library_path set_extra_library_path = nullptr;
> static ld_plugin_get_view get_view = nullptr;
> +static bool IsExecutable = false;
> static Optional<Reloc::Model> RelocationModel;
> static std::string output_name = "";
> static std::list<claimed_file> Modules;
> static DenseMap<int, void *> FDToLeaderHandle;
> static StringMap<ResolutionInfo> ResInfo;
> +static std::map<std::string, CommonResolution> Commons;
> static std::vector<std::string> Cleanup;
> static llvm::TargetOptions TargetOpts;
> -static std::string DefaultTriple = sys::getDefaultTargetTriple();
> +static size_t MaxTasks;
>
> namespace options {
> enum OutputType {
> @@ -161,7 +121,6 @@ namespace options {
> OT_BC_ONLY,
> OT_SAVE_TEMPS
> };
> - static bool generate_api_file = false;
> static OutputType TheOutputType = OT_NORMAL;
> static unsigned OptLevel = 2;
> // Default parallelism of 0 used to indicate that user did not specify.
> @@ -213,8 +172,6 @@ namespace options {
> // Additional options to pass into the code generator.
> // Note: This array will contain all plugin options which are not claimed
> // as plugin exclusive to pass to the code generator.
> - // For example, "generate-api-file" and "as"options are for the plugin
> - // use only and will not be passed.
> static std::vector<const char *> extra;
>
> static void process_plugin_option(const char *opt_)
> @@ -223,9 +180,7 @@ namespace options {
> return;
> llvm::StringRef opt = opt_;
>
> - if (opt == "generate-api-file") {
> - generate_api_file = true;
> - } else if (opt.startswith("mcpu=")) {
> + if (opt.startswith("mcpu=")) {
> mcpu = opt.substr(strlen("mcpu="));
> } else if (opt.startswith("extra-library-path=")) {
> extra_library_path = opt.substr(strlen("extra_library_path="));
> @@ -307,10 +262,15 @@ ld_plugin_status onload(ld_plugin_tv *tv
> switch (tv->tv_u.tv_val) {
> case LDPO_REL: // .o
> case LDPO_DYN: // .so
> + IsExecutable = false;
> + RelocationModel = Reloc::PIC_;
> + break;
> case LDPO_PIE: // position independent executable
> + IsExecutable = true;
> RelocationModel = Reloc::PIC_;
> break;
> case LDPO_EXEC: // .exe
> + IsExecutable = true;
> RelocationModel = Reloc::Static;
> break;
> default:
> @@ -404,20 +364,6 @@ ld_plugin_status onload(ld_plugin_tv *tv
> return LDPS_OK;
> }
>
> -static const GlobalObject *getBaseObject(const GlobalValue &GV) {
> - if (auto *GA = dyn_cast<GlobalAlias>(&GV))
> - return GA->getBaseObject();
> - return cast<GlobalObject>(&GV);
> -}
> -
> -static bool shouldSkip(uint32_t Symflags) {
> - if (!(Symflags & object::BasicSymbolRef::SF_Global))
> - return true;
> - if (Symflags & object::BasicSymbolRef::SF_FormatSpecific)
> - return true;
> - return false;
> -}
> -
> static void diagnosticHandler(const DiagnosticInfo &DI) {
> if (const auto *BDI = dyn_cast<BitcodeDiagnosticInfo>(&DI)) {
> std::error_code EC = BDI->getError();
> @@ -447,21 +393,18 @@ static void diagnosticHandler(const Diag
> message(Level, "LLVM gold plugin: %s", ErrStorage.c_str());
> }
>
> -static void diagnosticHandlerForContext(const DiagnosticInfo &DI,
> - void *Context) {
> - diagnosticHandler(DI);
> +static void check(Error E, std::string Msg = "LLVM gold plugin") {
> + handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
> + message(LDPL_FATAL, "%s: %s", Msg.c_str(), EIB.message().c_str());
> + return Error::success();
> + });
> }
>
> -static GlobalValue::VisibilityTypes
> -getMinVisibility(GlobalValue::VisibilityTypes A,
> - GlobalValue::VisibilityTypes B) {
> - if (A == GlobalValue::HiddenVisibility)
> - return A;
> - if (B == GlobalValue::HiddenVisibility)
> - return B;
> - if (A == GlobalValue::ProtectedVisibility)
> - return A;
> - return B;
> +template <typename T> static T check(Expected<T> E) {
> + if (E)
> + return std::move(*E);
> + check(E.takeError());
> + return T();
> }
>
> /// Called by gold to see whether this file is one that our plugin can handle.
> @@ -469,7 +412,6 @@ getMinVisibility(GlobalValue::Visibility
> /// possible.
> static ld_plugin_status claim_file_hook(const ld_plugin_input_file *file,
> int *claimed) {
> - LLVMContext Context;
> MemoryBufferRef BufferRef;
> std::unique_ptr<MemoryBuffer> Buffer;
> if (get_view) {
> @@ -498,22 +440,25 @@ static ld_plugin_status claim_file_hook(
> BufferRef = Buffer->getMemBufferRef();
> }
>
> - Context.setDiagnosticHandler(diagnosticHandlerForContext);
> - ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
> - object::IRObjectFile::create(BufferRef, Context);
> - std::error_code EC = ObjOrErr.getError();
> - if (EC == object::object_error::invalid_file_type ||
> - EC == object::object_error::bitcode_section_not_found)
> - return LDPS_OK;
> -
> *claimed = 1;
>
> - if (EC) {
> - message(LDPL_ERROR, "LLVM gold plugin has failed to create LTO module: %s",
> - EC.message().c_str());
> - return LDPS_ERR;
> + Expected<std::unique_ptr<InputFile>> ObjOrErr = InputFile::create(BufferRef);
> + if (!ObjOrErr) {
> + handleAllErrors(ObjOrErr.takeError(), [&](const ErrorInfoBase &EI) {
> + std::error_code EC = EI.convertToErrorCode();
> + if (EC == object::object_error::invalid_file_type ||
> + EC == object::object_error::bitcode_section_not_found)
> + *claimed = 0;
> + else
> + message(LDPL_ERROR,
> + "LLVM gold plugin has failed to create LTO module: %s",
> + EI.message().c_str());
> + });
> +
> + return *claimed ? LDPS_ERR : LDPS_OK;
> }
> - std::unique_ptr<object::IRObjectFile> Obj = std::move(*ObjOrErr);
> +
> + std::unique_ptr<InputFile> Obj = std::move(*ObjOrErr);
>
> Modules.resize(Modules.size() + 1);
> claimed_file &cf = Modules.back();
> @@ -535,72 +480,52 @@ static ld_plugin_status claim_file_hook(
> cf.name = file->name;
> if (file->offset)
> cf.name += ".llvm." + std::to_string(file->offset) + "." +
> - sys::path::filename(Obj->getModule().getSourceFileName()).str();
> + sys::path::filename(Obj->getSourceFileName()).str();
>
> for (auto &Sym : Obj->symbols()) {
> uint32_t Symflags = Sym.getFlags();
> - if (shouldSkip(Symflags))
> - continue;
>
> cf.syms.push_back(ld_plugin_symbol());
> ld_plugin_symbol &sym = cf.syms.back();
> sym.version = nullptr;
> + StringRef Name = Sym.getName();
> + sym.name = strdup(Name.str().c_str());
>
> - SmallString<64> Name;
> - {
> - raw_svector_ostream OS(Name);
> - Sym.printName(OS);
> - }
> - sym.name = strdup(Name.c_str());
> -
> - const GlobalValue *GV = Obj->getSymbolGV(Sym.getRawDataRefImpl());
> + ResolutionInfo &Res = ResInfo[Name];
>
> - ResolutionInfo &Res = ResInfo[sym.name];
> + Res.CanOmitFromDynSym &= Sym.canBeOmittedFromSymbolTable();
>
> sym.visibility = LDPV_DEFAULT;
> - if (GV) {
> - Res.UnnamedAddr =
> - GlobalValue::getMinUnnamedAddr(Res.UnnamedAddr, GV->getUnnamedAddr());
> - Res.IsLinkonceOdr &= GV->hasLinkOnceLinkage();
> - Res.Visibility = getMinVisibility(Res.Visibility, GV->getVisibility());
> - switch (GV->getVisibility()) {
> - case GlobalValue::DefaultVisibility:
> - break;
> - case GlobalValue::HiddenVisibility:
> - sym.visibility = LDPV_HIDDEN;
> - break;
> - case GlobalValue::ProtectedVisibility:
> - sym.visibility = LDPV_PROTECTED;
> - break;
> - }
> + GlobalValue::VisibilityTypes Vis = Sym.getVisibility();
> + if (Vis != GlobalValue::DefaultVisibility)
> + Res.DefaultVisibility = false;
> + switch (Vis) {
> + case GlobalValue::DefaultVisibility:
> + break;
> + case GlobalValue::HiddenVisibility:
> + sym.visibility = LDPV_HIDDEN;
> + break;
> + case GlobalValue::ProtectedVisibility:
> + sym.visibility = LDPV_PROTECTED;
> + break;
> }
>
> if (Symflags & object::BasicSymbolRef::SF_Undefined) {
> sym.def = LDPK_UNDEF;
> - if (GV && GV->hasExternalWeakLinkage())
> + if (Symflags & object::BasicSymbolRef::SF_Weak)
> sym.def = LDPK_WEAKUNDEF;
> - } else {
> + } else if (Symflags & object::BasicSymbolRef::SF_Common)
> + sym.def = LDPK_COMMON;
> + else if (Symflags & object::BasicSymbolRef::SF_Weak)
> + sym.def = LDPK_WEAKDEF;
> + else
> sym.def = LDPK_DEF;
> - if (GV) {
> - assert(!GV->hasExternalWeakLinkage() &&
> - !GV->hasAvailableExternallyLinkage() && "Not a declaration!");
> - if (GV->hasCommonLinkage())
> - sym.def = LDPK_COMMON;
> - else if (GV->isWeakForLinker())
> - sym.def = LDPK_WEAKDEF;
> - }
> - }
>
> sym.size = 0;
> sym.comdat_key = nullptr;
> - if (GV) {
> - const GlobalObject *Base = getBaseObject(*GV);
> - if (!Base)
> - message(LDPL_FATAL, "Unable to determine comdat of alias!");
> - const Comdat *C = Base->getComdat();
> - if (C)
> - sym.comdat_key = strdup(C->getName().str().c_str());
> - }
> + const Comdat *C = check(Sym.getComdat());
> + if (C)
> + sym.comdat_key = strdup(C->getName().str().c_str());
>
> sym.resolution = LDPR_UNKNOWN;
> }
> @@ -615,39 +540,6 @@ static ld_plugin_status claim_file_hook(
> return LDPS_OK;
> }
>
> -static void internalize(GlobalValue &GV) {
> - if (GV.isDeclarationForLinker())
> - return; // We get here if there is a matching asm definition.
> - if (!GV.hasLocalLinkage())
> - GV.setLinkage(GlobalValue::InternalLinkage);
> -}
> -
> -static const char *getResolutionName(ld_plugin_symbol_resolution R) {
> - switch (R) {
> - case LDPR_UNKNOWN:
> - return "UNKNOWN";
> - case LDPR_UNDEF:
> - return "UNDEF";
> - case LDPR_PREVAILING_DEF:
> - return "PREVAILING_DEF";
> - case LDPR_PREVAILING_DEF_IRONLY:
> - return "PREVAILING_DEF_IRONLY";
> - case LDPR_PREEMPTED_REG:
> - return "PREEMPTED_REG";
> - case LDPR_PREEMPTED_IR:
> - return "PREEMPTED_IR";
> - case LDPR_RESOLVED_IR:
> - return "RESOLVED_IR";
> - case LDPR_RESOLVED_EXEC:
> - return "RESOLVED_EXEC";
> - case LDPR_RESOLVED_DYN:
> - return "RESOLVED_DYN";
> - case LDPR_PREVAILING_DEF_IRONLY_EXP:
> - return "PREVAILING_DEF_IRONLY_EXP";
> - }
> - llvm_unreachable("Unknown resolution");
> -}
> -
> static void freeSymName(ld_plugin_symbol &Sym) {
> free(Sym.name);
> free(Sym.comdat_key);
> @@ -671,120 +563,27 @@ static const void *getSymbolsAndView(cla
> return View;
> }
>
> -static std::unique_ptr<ModuleSummaryIndex>
> -getModuleSummaryIndexForFile(claimed_file &F) {
> - const void *View = getSymbolsAndView(F);
> - if (!View)
> - return nullptr;
> -
> +static void addModule(LTO &Lto, claimed_file &F, const void *View) {
> MemoryBufferRef BufferRef(StringRef((const char *)View, F.filesize), F.name);
> + Expected<std::unique_ptr<InputFile>> ObjOrErr = InputFile::create(BufferRef);
>
> - // Don't bother trying to build an index if there is no summary information
> - // in this bitcode file.
> - if (!object::ModuleSummaryIndexObjectFile::hasGlobalValueSummaryInMemBuffer(
> - BufferRef, diagnosticHandler))
> - return std::unique_ptr<ModuleSummaryIndex>(nullptr);
> -
> - ErrorOr<std::unique_ptr<object::ModuleSummaryIndexObjectFile>> ObjOrErr =
> - object::ModuleSummaryIndexObjectFile::create(BufferRef,
> - diagnosticHandler);
> -
> - if (std::error_code EC = ObjOrErr.getError())
> - message(LDPL_FATAL,
> - "Could not read module summary index bitcode from file : %s",
> - EC.message().c_str());
> -
> - object::ModuleSummaryIndexObjectFile &Obj = **ObjOrErr;
> -
> - return Obj.takeIndex();
> -}
> -
> -static std::unique_ptr<Module>
> -getModuleForFile(LLVMContext &Context, claimed_file &F, const void *View,
> - StringRef Name, raw_fd_ostream *ApiFile,
> - StringSet<> &Internalize, std::vector<GlobalValue *> &Keep,
> - StringMap<unsigned> &Realign) {
> - MemoryBufferRef BufferRef(StringRef((const char *)View, F.filesize), Name);
> - ErrorOr<std::unique_ptr<object::IRObjectFile>> ObjOrErr =
> - object::IRObjectFile::create(BufferRef, Context);
> -
> - if (std::error_code EC = ObjOrErr.getError())
> + if (!ObjOrErr)
> message(LDPL_FATAL, "Could not read bitcode from file : %s",
> - EC.message().c_str());
> -
> - object::IRObjectFile &Obj = **ObjOrErr;
> + toString(ObjOrErr.takeError()).c_str());
>
> - Module &M = Obj.getModule();
> -
> - M.materializeMetadata();
> - UpgradeDebugInfo(M);
> -
> - SmallPtrSet<GlobalValue *, 8> Used;
> - collectUsedGlobalVariables(M, Used, /*CompilerUsed*/ false);
> + InputFile &Obj = **ObjOrErr;
>
> unsigned SymNum = 0;
> + std::vector<SymbolResolution> Resols(F.syms.size());
> for (auto &ObjSym : Obj.symbols()) {
> - GlobalValue *GV = Obj.getSymbolGV(ObjSym.getRawDataRefImpl());
> - if (GV && GV->hasAppendingLinkage())
> - Keep.push_back(GV);
> -
> - if (shouldSkip(ObjSym.getFlags()))
> - continue;
> ld_plugin_symbol &Sym = F.syms[SymNum];
> + SymbolResolution &R = Resols[SymNum];
> ++SymNum;
>
> ld_plugin_symbol_resolution Resolution =
> (ld_plugin_symbol_resolution)Sym.resolution;
>
> - if (options::generate_api_file)
> - *ApiFile << Sym.name << ' ' << getResolutionName(Resolution) << '\n';
> -
> - if (!GV) {
> - freeSymName(Sym);
> - continue; // Asm symbol.
> - }
> -
> ResolutionInfo &Res = ResInfo[Sym.name];
> - if (Resolution == LDPR_PREVAILING_DEF_IRONLY_EXP && !Res.IsLinkonceOdr)
> - Resolution = LDPR_PREVAILING_DEF;
> -
> - GV->setUnnamedAddr(Res.UnnamedAddr);
> - GV->setVisibility(Res.Visibility);
> -
> - // Override gold's resolution for common symbols. We want the largest
> - // one to win.
> - if (GV->hasCommonLinkage()) {
> - if (Resolution == LDPR_PREVAILING_DEF_IRONLY)
> - Res.CommonInternal = true;
> -
> - if (Resolution == LDPR_PREVAILING_DEF_IRONLY ||
> - Resolution == LDPR_PREVAILING_DEF)
> - Res.UseCommon = true;
> -
> - const DataLayout &DL = GV->getParent()->getDataLayout();
> - uint64_t Size = DL.getTypeAllocSize(GV->getType()->getElementType());
> - unsigned Align = GV->getAlignment();
> -
> - if (Res.UseCommon && Size >= Res.CommonSize) {
> - // Take GV.
> - if (Res.CommonInternal)
> - Resolution = LDPR_PREVAILING_DEF_IRONLY;
> - else
> - Resolution = LDPR_PREVAILING_DEF;
> - cast<GlobalVariable>(GV)->setAlignment(
> - std::max(Res.CommonAlign, Align));
> - } else {
> - // Do not take GV, it's smaller than what we already have in the
> - // combined module.
> - Resolution = LDPR_PREEMPTED_IR;
> - if (Align > Res.CommonAlign)
> - // Need to raise the alignment though.
> - Realign[Sym.name] = Align;
> - }
> -
> - Res.CommonSize = std::max(Res.CommonSize, Size);
> - Res.CommonAlign = std::max(Res.CommonAlign, Align);
> - }
>
> switch (Resolution) {
> case LDPR_UNKNOWN:
> @@ -795,57 +594,48 @@ getModuleForFile(LLVMContext &Context, c
> case LDPR_RESOLVED_DYN:
> case LDPR_PREEMPTED_IR:
> case LDPR_PREEMPTED_REG:
> - break;
> -
> case LDPR_UNDEF:
> - if (!GV->isDeclarationForLinker())
> - assert(GV->hasComdat());
> break;
>
> - case LDPR_PREVAILING_DEF_IRONLY: {
> - Keep.push_back(GV);
> - // The IR linker has to be able to map this value to a declaration,
> - // so we can only internalize after linking.
> - if (!Used.count(GV))
> - Internalize.insert(GV->getName());
> + case LDPR_PREVAILING_DEF_IRONLY:
> + R.Prevailing = true;
> break;
> - }
>
> case LDPR_PREVAILING_DEF:
> - Keep.push_back(GV);
> - // There is a non IR use, so we have to force optimizations to keep this.
> - switch (GV->getLinkage()) {
> - default:
> - break;
> - case GlobalValue::LinkOnceAnyLinkage:
> - GV->setLinkage(GlobalValue::WeakAnyLinkage);
> - break;
> - case GlobalValue::LinkOnceODRLinkage:
> - GV->setLinkage(GlobalValue::WeakODRLinkage);
> - break;
> - }
> + R.Prevailing = true;
> + R.VisibleToRegularObj = true;
> break;
>
> - case LDPR_PREVAILING_DEF_IRONLY_EXP: {
> - Keep.push_back(GV);
> - if (canBeOmittedFromSymbolTable(GV))
> - Internalize.insert(GV->getName());
> + case LDPR_PREVAILING_DEF_IRONLY_EXP:
> + R.Prevailing = true;
> + if (!Res.CanOmitFromDynSym)
> + R.VisibleToRegularObj = true;
> break;
> }
> +
> + if (Resolution != LDPR_RESOLVED_DYN && Resolution != LDPR_UNDEF &&
> + (IsExecutable || !Res.DefaultVisibility))
> + R.FinalDefinitionInLinkageUnit = true;
> +
> + if (ObjSym.getFlags() & object::BasicSymbolRef::SF_Common) {
> + // We ignore gold's resolution for common symbols. A common symbol with
> + // the correct size and alignment is added to the module by the pre-opt
> + // module hook if any common symbol prevailed.
> + CommonResolution &CommonRes = Commons[ObjSym.getIRName()];
> + if (R.Prevailing) {
> + CommonRes.Prevailing = true;
> + CommonRes.VisibleToRegularObj = R.VisibleToRegularObj;
> + }
> + CommonRes.Size = std::max(CommonRes.Size, ObjSym.getCommonSize());
> + CommonRes.Align = std::max(CommonRes.Align, ObjSym.getCommonAlignment());
> + R.Prevailing = false;
> }
>
> freeSymName(Sym);
> }
>
> - return Obj.takeModule();
> -}
> -
> -static void saveBCFile(StringRef Path, Module &M) {
> - std::error_code EC;
> - raw_fd_ostream OS(Path, EC, sys::fs::OpenFlags::F_None);
> - if (EC)
> - message(LDPL_FATAL, "Failed to write the output file.");
> - WriteBitcodeToFile(&M, OS, /* ShouldPreserveUseListOrder */ false);
> + check(Lto.add(std::move(*ObjOrErr), Resols),
> + std::string("Failed to link module ") + F.name);
> }
>
> static void recordFile(std::string Filename, bool TempOutFile) {
> @@ -857,190 +647,6 @@ static void recordFile(std::string Filen
> Cleanup.push_back(Filename.c_str());
> }
>
> -void ThinLTOTaskInfo::cleanup() {
> - // Close the output file descriptor before we pass it to gold.
> - OS->close();
> -
> - recordFile(Filename, TempOutFile);
> -}
> -
> -namespace {
> -/// Class to manage optimization and code generation for a module, possibly
> -/// in a thread (ThinLTO).
> -class CodeGen {
> - /// The module for which this will generate code.
> - std::unique_ptr<llvm::Module> M;
> -
> - /// The output stream to generate code into.
> - raw_fd_ostream *OS;
> -
> - /// The task ID when this was invoked in a thread (ThinLTO).
> - int TaskID;
> -
> - /// The module summary index for ThinLTO tasks.
> - const ModuleSummaryIndex *CombinedIndex;
> -
> - /// The target machine for generating code for this module.
> - std::unique_ptr<TargetMachine> TM;
> -
> - /// Filename to use as base when save-temps is enabled, used to get
> - /// a unique and identifiable save-temps output file for each ThinLTO backend.
> - std::string SaveTempsFilename;
> -
> - /// Map from a module name to the corresponding buffer holding a view of the
> - /// bitcode provided via the get_view gold callback.
> - StringMap<MemoryBufferRef> *ModuleMap;
> -
> - // Functions to import into this module.
> - FunctionImporter::ImportMapTy *ImportList;
> -
> - // Map of globals defined in this module to their summary.
> - std::map<GlobalValue::GUID, GlobalValueSummary *> *DefinedGlobals;
> -
> -public:
> - /// Constructor used by full LTO.
> - CodeGen(std::unique_ptr<llvm::Module> M)
> - : M(std::move(M)), OS(nullptr), TaskID(-1), CombinedIndex(nullptr),
> - ModuleMap(nullptr) {
> - initTargetMachine();
> - }
> - /// Constructor used by ThinLTO.
> - CodeGen(std::unique_ptr<llvm::Module> M, raw_fd_ostream *OS, int TaskID,
> - const ModuleSummaryIndex *CombinedIndex, std::string Filename,
> - StringMap<MemoryBufferRef> *ModuleMap,
> - FunctionImporter::ImportMapTy *ImportList,
> - std::map<GlobalValue::GUID, GlobalValueSummary *> *DefinedGlobals)
> - : M(std::move(M)), OS(OS), TaskID(TaskID), CombinedIndex(CombinedIndex),
> - SaveTempsFilename(std::move(Filename)), ModuleMap(ModuleMap),
> - ImportList(ImportList), DefinedGlobals(DefinedGlobals) {
> - assert(options::thinlto == !!CombinedIndex &&
> - "Expected module summary index iff performing ThinLTO");
> - initTargetMachine();
> - }
> -
> - /// Invoke LTO passes and the code generator for the module.
> - void runAll();
> -
> - /// Invoke the actual code generation to emit Module's object to file.
> - void runCodegenPasses();
> -
> -private:
> - const Target *TheTarget;
> - std::string TripleStr;
> - std::string FeaturesString;
> - TargetOptions Options;
> -
> - /// Create a target machine for the module. Must be unique for each
> - /// module/task.
> - void initTargetMachine();
> -
> - std::unique_ptr<TargetMachine> createTargetMachine();
> -
> - /// Run all LTO passes on the module.
> - void runLTOPasses();
> -
> - /// Sets up output files necessary to perform optional multi-threaded
> - /// split code generation, and invokes the code generation implementation.
> - /// If BCFileName is not empty, saves bitcode for module partitions into
> - /// {BCFileName}0 .. {BCFileName}N.
> - void runSplitCodeGen(const SmallString<128> &BCFilename);
> -};
> -}
> -
> -static SubtargetFeatures getFeatures(Triple &TheTriple) {
> - SubtargetFeatures Features;
> - Features.getDefaultSubtargetFeatures(TheTriple);
> - for (const std::string &A : MAttrs)
> - Features.AddFeature(A);
> - return Features;
> -}
> -
> -static CodeGenOpt::Level getCGOptLevel() {
> - switch (options::OptLevel) {
> - case 0:
> - return CodeGenOpt::None;
> - case 1:
> - return CodeGenOpt::Less;
> - case 2:
> - return CodeGenOpt::Default;
> - case 3:
> - return CodeGenOpt::Aggressive;
> - }
> - llvm_unreachable("Invalid optimization level");
> -}
> -
> -void CodeGen::initTargetMachine() {
> - TripleStr = M->getTargetTriple();
> - Triple TheTriple(TripleStr);
> -
> - std::string ErrMsg;
> - TheTarget = TargetRegistry::lookupTarget(TripleStr, ErrMsg);
> - if (!TheTarget)
> - message(LDPL_FATAL, "Target not found: %s", ErrMsg.c_str());
> -
> - SubtargetFeatures Features = getFeatures(TheTriple);
> - FeaturesString = Features.getString();
> - Options = InitTargetOptionsFromCodeGenFlags();
> -
> - // Disable the new X86 relax relocations since gold might not support them.
> - // FIXME: Check the gold version or add a new option to enable them.
> - Options.RelaxELFRelocations = false;
> -
> - TM = createTargetMachine();
> -}
> -
> -std::unique_ptr<TargetMachine> CodeGen::createTargetMachine() {
> - CodeGenOpt::Level CGOptLevel = getCGOptLevel();
> -
> - return std::unique_ptr<TargetMachine>(TheTarget->createTargetMachine(
> - TripleStr, options::mcpu, FeaturesString, Options, RelocationModel,
> - CodeModel::Default, CGOptLevel));
> -}
> -
> -void CodeGen::runLTOPasses() {
> - M->setDataLayout(TM->createDataLayout());
> -
> - if (CombinedIndex) {
> - // Apply summary-based LinkOnce/Weak resolution decisions.
> - thinLTOResolveWeakForLinkerModule(*M, *DefinedGlobals);
> -
> - // Apply summary-based internalization decisions. Skip if there are no
> - // defined globals from the summary since not only is it unnecessary, but
> - // if this module did not have a summary section the internalizer will
> - // assert if it finds any definitions in this module that aren't in the
> - // DefinedGlobals set.
> - if (!DefinedGlobals->empty())
> - thinLTOInternalizeModule(*M, *DefinedGlobals);
> -
> - // Create a loader that will parse the bitcode from the buffers
> - // in the ModuleMap.
> - ModuleLoader Loader(M->getContext(), *ModuleMap);
> -
> - // Perform function importing.
> - FunctionImporter Importer(*CombinedIndex, Loader);
> - Importer.importFunctions(*M, *ImportList);
> - }
> -
> - legacy::PassManager passes;
> - passes.add(createTargetTransformInfoWrapperPass(TM->getTargetIRAnalysis()));
> -
> - PassManagerBuilder PMB;
> - PMB.LibraryInfo = new TargetLibraryInfoImpl(Triple(TM->getTargetTriple()));
> - PMB.Inliner = createFunctionInliningPass();
> - // Unconditionally verify input since it is not verified before this
> - // point and has unknown origin.
> - PMB.VerifyInput = true;
> - PMB.VerifyOutput = !options::DisableVerify;
> - PMB.LoopVectorize = true;
> - PMB.SLPVectorize = true;
> - PMB.OptLevel = options::OptLevel;
> - if (options::thinlto)
> - PMB.populateThinLTOPassManager(passes);
> - else
> - PMB.populateLTOPassManager(passes);
> - passes.run(*M);
> -}
> -
> /// Open a file and return the new file descriptor given a base input
> /// file name, a flag indicating whether a temp file should be generated,
> /// and an optional task id. The new filename generated is
> @@ -1066,220 +672,44 @@ static int openOutputFile(SmallString<12
> return FD;
> }
>
> -void CodeGen::runCodegenPasses() {
> - assert(OS && "Output stream must be set before emitting to file");
> - legacy::PassManager CodeGenPasses;
> - if (TM->addPassesToEmitFile(CodeGenPasses, *OS,
> - TargetMachine::CGFT_ObjectFile))
> - report_fatal_error("Failed to setup codegen");
> - CodeGenPasses.run(*M);
> -}
> -
> -void CodeGen::runSplitCodeGen(const SmallString<128> &BCFilename) {
> - SmallString<128> Filename;
> - // Note that openOutputFile will append a unique ID for each task
> - if (!options::obj_path.empty())
> - Filename = options::obj_path;
> - else if (options::TheOutputType == options::OT_SAVE_TEMPS)
> - Filename = output_name + ".o";
> -
> - // Note that the default parallelism is 1 instead of the
> - // hardware_concurrency, as there are behavioral differences between
> - // parallelism levels (e.g. symbol ordering will be different, and some uses
> - // of inline asm currently have issues with parallelism >1).
> - unsigned int MaxThreads = options::Parallelism ? options::Parallelism : 1;
> -
> - std::vector<SmallString<128>> Filenames(MaxThreads);
> - std::vector<SmallString<128>> BCFilenames(MaxThreads);
> - bool TempOutFile = Filename.empty();
> - {
> - // Open a file descriptor for each backend task. This is done in a block
> - // so that the output file descriptors are closed before gold opens them.
> - std::list<llvm::raw_fd_ostream> OSs;
> - std::vector<llvm::raw_pwrite_stream *> OSPtrs(MaxThreads);
> - for (unsigned I = 0; I != MaxThreads; ++I) {
> - int FD = openOutputFile(Filename, TempOutFile, Filenames[I],
> - // Only append ID if there are multiple tasks.
> - MaxThreads > 1 ? I : -1);
> - OSs.emplace_back(FD, true);
> - OSPtrs[I] = &OSs.back();
> - }
> -
> - std::list<llvm::raw_fd_ostream> BCOSs;
> - std::vector<llvm::raw_pwrite_stream *> BCOSPtrs;
> - if (!BCFilename.empty() && MaxThreads > 1) {
> - for (unsigned I = 0; I != MaxThreads; ++I) {
> - int FD = openOutputFile(BCFilename, false, BCFilenames[I], I);
> - BCOSs.emplace_back(FD, true);
> - BCOSPtrs.push_back(&BCOSs.back());
> - }
> - }
> -
> - // Run backend tasks.
> - splitCodeGen(std::move(M), OSPtrs, BCOSPtrs,
> - [&]() { return createTargetMachine(); });
> - }
> -
> - for (auto &Filename : Filenames)
> - recordFile(Filename.c_str(), TempOutFile);
> -}
> -
> -void CodeGen::runAll() {
> - runLTOPasses();
> -
> - SmallString<128> OptFilename;
> - if (options::TheOutputType == options::OT_SAVE_TEMPS) {
> - OptFilename = output_name;
> - // If the CodeGen client provided a filename, use it. Always expect
> - // a provided filename if we are in a task (i.e. ThinLTO backend).
> - assert(!SaveTempsFilename.empty() || TaskID == -1);
> - if (!SaveTempsFilename.empty())
> - OptFilename = SaveTempsFilename;
> - OptFilename += ".opt.bc";
> - saveBCFile(OptFilename, *M);
> - }
> -
> - // If we are already in a thread (i.e. ThinLTO), just perform
> - // codegen passes directly.
> - if (TaskID >= 0)
> - runCodegenPasses();
> - // Otherwise attempt split code gen.
> - else
> - runSplitCodeGen(OptFilename);
> -}
> -
> -/// Links the module in \p View from file \p F into the combined module
> -/// saved in the IRMover \p L.
> -static void linkInModule(LLVMContext &Context, IRMover &L, claimed_file &F,
> - const void *View, StringRef Name,
> - raw_fd_ostream *ApiFile, StringSet<> &Internalize,
> - bool SetName = false) {
> - std::vector<GlobalValue *> Keep;
> - StringMap<unsigned> Realign;
> - std::unique_ptr<Module> M = getModuleForFile(Context, F, View, Name, ApiFile,
> - Internalize, Keep, Realign);
> - if (!M.get())
> - return;
> - if (!options::triple.empty())
> - M->setTargetTriple(options::triple.c_str());
> - else if (M->getTargetTriple().empty()) {
> - M->setTargetTriple(DefaultTriple);
> - }
> -
> - // For ThinLTO we want to propagate the source file name to ensure
> - // we can create the correct global identifiers matching those in the
> - // original module.
> - if (SetName)
> - L.getModule().setSourceFileName(M->getSourceFileName());
> -
> - if (Error E = L.move(std::move(M), Keep,
> - [](GlobalValue &, IRMover::ValueAdder) {})) {
> - handleAllErrors(std::move(E), [&](const llvm::ErrorInfoBase &EIB) {
> - message(LDPL_FATAL, "Failed to link module %s: %s", Name.str().c_str(),
> - EIB.message().c_str());
> - });
> - }
> -
> - for (const auto &I : Realign) {
> - GlobalValue *Dst = L.getModule().getNamedValue(I.first());
> - if (!Dst)
> +/// Add all required common symbols to M, which is expected to be the first
> +/// combined module.
> +static void addCommons(Module &M) {
> + for (auto &I : Commons) {
> + if (!I.second.Prevailing)
> continue;
> - cast<GlobalVariable>(Dst)->setAlignment(I.second);
> + ArrayType *Ty =
> + ArrayType::get(Type::getInt8Ty(M.getContext()), I.second.Size);
> + GlobalVariable *OldGV = M.getNamedGlobal(I.first);
> + auto *GV = new GlobalVariable(M, Ty, false, GlobalValue::CommonLinkage,
> + ConstantAggregateZero::get(Ty), "");
> + GV->setAlignment(I.second.Align);
> + if (OldGV) {
> + OldGV->replaceAllUsesWith(ConstantExpr::getBitCast(GV, OldGV->getType()));
> + GV->takeName(OldGV);
> + OldGV->eraseFromParent();
> + } else {
> + GV->setName(I.first);
> + }
> + // We may only internalize commons if there is a single LTO task because
> + // other native object files may require the common.
> + if (MaxTasks == 1 && !I.second.VisibleToRegularObj)
> + GV->setLinkage(GlobalValue::InternalLinkage);
> }
> }
>
> -/// Perform the ThinLTO backend on a single module, invoking the LTO and codegen
> -/// pipelines.
> -static void thinLTOBackendTask(claimed_file &F, const void *View,
> - StringRef Name, raw_fd_ostream *ApiFile,
> - const ModuleSummaryIndex &CombinedIndex,
> - raw_fd_ostream *OS, unsigned TaskID,
> - StringMap<MemoryBufferRef> &ModuleMap,
> - FunctionImporter::ImportMapTy &ImportList,
> - std::map<GlobalValue::GUID, GlobalValueSummary *> &DefinedGlobals) {
> - // Need to use a separate context for each task
> - LLVMContext Context;
> - Context.setDiscardValueNames(options::TheOutputType !=
> - options::OT_SAVE_TEMPS);
> - Context.enableDebugTypeODRUniquing(); // Merge debug info types.
> - Context.setDiagnosticHandler(diagnosticHandlerForContext, nullptr, true);
> -
> - std::unique_ptr<llvm::Module> NewModule(new llvm::Module(Name, Context));
> - IRMover L(*NewModule.get());
> -
> - StringSet<> Dummy;
> - linkInModule(Context, L, F, View, Name, ApiFile, Dummy, true);
> - if (renameModuleForThinLTO(*NewModule, CombinedIndex))
> - message(LDPL_FATAL, "Failed to rename module for ThinLTO");
> -
> - CodeGen codeGen(std::move(NewModule), OS, TaskID, &CombinedIndex, Name,
> - &ModuleMap, &ImportList, &DefinedGlobals);
> - codeGen.runAll();
> -}
> -
> -/// Launch each module's backend pipeline in a separate task in a thread pool.
> -static void
> -thinLTOBackends(raw_fd_ostream *ApiFile,
> - const ModuleSummaryIndex &CombinedIndex,
> - StringMap<MemoryBufferRef> &ModuleMap,
> - StringMap<FunctionImporter::ImportMapTy> &ImportLists,
> - StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
> - &ModuleToDefinedGVSummaries) {
> - unsigned TaskCount = 0;
> - std::vector<ThinLTOTaskInfo> Tasks;
> - Tasks.reserve(Modules.size());
> - unsigned int MaxThreads = options::Parallelism
> - ? options::Parallelism
> - : thread::hardware_concurrency();
> -
> - // Create ThreadPool in nested scope so that threads will be joined
> - // on destruction.
> - {
> - ThreadPool ThinLTOThreadPool(MaxThreads);
> - for (claimed_file &F : Modules) {
> - // Do all the gold callbacks in the main thread, since gold is not thread
> - // safe by default.
> - const void *View = getSymbolsAndView(F);
> - if (!View)
> - continue;
> -
> - SmallString<128> Filename;
> - if (!options::obj_path.empty())
> - // Note that openOutputFile will append a unique ID for each task
> - Filename = options::obj_path;
> - else if (options::TheOutputType == options::OT_SAVE_TEMPS) {
> - // Use the input file name so that we get a unique and identifiable
> - // output file for each ThinLTO backend task.
> - Filename = F.name;
> - Filename += ".thinlto.o";
> - }
> - bool TempOutFile = Filename.empty();
> -
> - SmallString<128> NewFilename;
> - int FD = openOutputFile(Filename, TempOutFile, NewFilename,
> - // Only append the TaskID if we will use the
> - // non-unique obj_path.
> - !options::obj_path.empty() ? TaskCount : -1);
> - TaskCount++;
> - std::unique_ptr<raw_fd_ostream> OS =
> - llvm::make_unique<raw_fd_ostream>(FD, true);
> -
> - // Enqueue the task
> - ThinLTOThreadPool.async(thinLTOBackendTask, std::ref(F), View, F.name,
> - ApiFile, std::ref(CombinedIndex), OS.get(),
> - TaskCount, std::ref(ModuleMap),
> - std::ref(ImportLists[F.name]),
> - std::ref(ModuleToDefinedGVSummaries[F.name]));
> -
> - // Record the information needed by the task or during its cleanup
> - // to a ThinLTOTaskInfo instance. For information needed by the task
> - // the unique_ptr ownership is transferred to the ThinLTOTaskInfo.
> - Tasks.emplace_back(std::move(OS), NewFilename.c_str(), TempOutFile);
> - }
> +static CodeGenOpt::Level getCGOptLevel() {
> + switch (options::OptLevel) {
> + case 0:
> + return CodeGenOpt::None;
> + case 1:
> + return CodeGenOpt::Less;
> + case 2:
> + return CodeGenOpt::Default;
> + case 3:
> + return CodeGenOpt::Aggressive;
> }
> -
> - for (auto &Task : Tasks)
> - Task.cleanup();
> + llvm_unreachable("Invalid optimization level");
> }
>
> /// Parse the thinlto_prefix_replace option into the \p OldPrefix and
> @@ -1293,266 +723,130 @@ static void getThinLTOOldAndNewPrefix(st
> NewPrefix = Split.second.str();
> }
>
> -/// Given the original \p Path to an output file, replace any path
> -/// prefix matching \p OldPrefix with \p NewPrefix. Also, create the
> -/// resulting directory if it does not yet exist.
> -static std::string getThinLTOOutputFile(const std::string &Path,
> - const std::string &OldPrefix,
> - const std::string &NewPrefix) {
> - if (OldPrefix.empty() && NewPrefix.empty())
> - return Path;
> - SmallString<128> NewPath(Path);
> - llvm::sys::path::replace_path_prefix(NewPath, OldPrefix, NewPrefix);
> - StringRef ParentPath = llvm::sys::path::parent_path(NewPath.str());
> - if (!ParentPath.empty()) {
> - // Make sure the new directory exists, creating it if necessary.
> - if (std::error_code EC = llvm::sys::fs::create_directories(ParentPath))
> - llvm::errs() << "warning: could not create directory '" << ParentPath
> - << "': " << EC.message() << '\n';
> - }
> - return NewPath.str();
> -}
> -
> -/// Perform ThinLTO link, which creates the combined index file.
> -/// Also, either launch backend threads or (under thinlto-index-only)
> -/// emit individual index files for distributed backends and exit.
> -static ld_plugin_status thinLTOLink(raw_fd_ostream *ApiFile) {
> - // Map from a module name to the corresponding buffer holding a view of the
> - // bitcode provided via the get_view gold callback.
> - StringMap<MemoryBufferRef> ModuleMap;
> - // Map to own RAII objects that manage the file opening and releasing
> - // interfaces with gold.
> - DenseMap<void *, std::unique_ptr<PluginInputFile>> HandleToInputFile;
> -
> - // Keep track of symbols that must not be internalized because they
> - // are referenced outside of a single IR module.
> - DenseSet<GlobalValue::GUID> Preserve;
> -
> - // Keep track of the prevailing copy for each GUID, for use in resolving
> - // weak linkages.
> - DenseMap<GlobalValue::GUID, const GlobalValueSummary *> PrevailingCopy;
> -
> - ModuleSummaryIndex CombinedIndex;
> - uint64_t NextModuleId = 0;
> - for (claimed_file &F : Modules) {
> - if (!HandleToInputFile.count(F.leader_handle))
> - HandleToInputFile.insert(std::make_pair(
> - F.leader_handle, llvm::make_unique<PluginInputFile>(F.handle)));
> - // Pass this into getModuleSummaryIndexForFile
> - const void *View = getSymbolsAndView(F);
> - if (!View)
> - continue;
> -
> - MemoryBufferRef ModuleBuffer(StringRef((const char *)View, F.filesize),
> - F.name);
> - assert(ModuleMap.find(ModuleBuffer.getBufferIdentifier()) ==
> - ModuleMap.end() &&
> - "Expect unique Buffer Identifier");
> - ModuleMap[ModuleBuffer.getBufferIdentifier()] = ModuleBuffer;
> -
> - std::unique_ptr<ModuleSummaryIndex> Index = getModuleSummaryIndexForFile(F);
> -
> - // Use gold's symbol resolution information to identify symbols referenced
> - // by more than a single IR module (i.e. referenced by multiple IR modules
> - // or by a non-IR module). Cross references introduced by importing are
> - // checked separately via the export lists. Also track the prevailing copy
> - // for later symbol resolution.
> - for (auto &Sym : F.syms) {
> - ld_plugin_symbol_resolution Resolution =
> - (ld_plugin_symbol_resolution)Sym.resolution;
> - GlobalValue::GUID SymGUID = GlobalValue::getGUID(Sym.name);
> - if (Resolution != LDPR_PREVAILING_DEF_IRONLY)
> - Preserve.insert(SymGUID);
> -
> - if (Index && (Resolution == LDPR_PREVAILING_DEF ||
> - Resolution == LDPR_PREVAILING_DEF_IRONLY ||
> - Resolution == LDPR_PREVAILING_DEF_IRONLY_EXP))
> - PrevailingCopy[SymGUID] = Index->getGlobalValueSummary(SymGUID);
> - }
> -
> - // Skip files without a module summary.
> - if (Index)
> - CombinedIndex.mergeFrom(std::move(Index), ++NextModuleId);
> - }
> -
> - // Collect for each module the list of function it defines (GUID ->
> - // Summary).
> - StringMap<std::map<GlobalValue::GUID, GlobalValueSummary *>>
> - ModuleToDefinedGVSummaries(NextModuleId);
> - CombinedIndex.collectDefinedGVSummariesPerModule(ModuleToDefinedGVSummaries);
> -
> - StringMap<FunctionImporter::ImportMapTy> ImportLists(NextModuleId);
> - StringMap<FunctionImporter::ExportSetTy> ExportLists(NextModuleId);
> - ComputeCrossModuleImport(CombinedIndex, ModuleToDefinedGVSummaries,
> - ImportLists, ExportLists);
> -
> - auto isPrevailing = [&](GlobalValue::GUID GUID, const GlobalValueSummary *S) {
> - const auto &Prevailing = PrevailingCopy.find(GUID);
> - assert(Prevailing != PrevailingCopy.end());
> - return Prevailing->second == S;
> - };
> +static std::unique_ptr<LTO> createLTO() {
> + Config Conf;
> + ThinBackend Backend;
> + unsigned ParallelCodeGenParallelismLevel = 1;
>
> - // Callback for internalization, to prevent internalization of symbols
> - // that were not candidates initially, and those that are being imported
> - // (which introduces new cross references).
> - auto isExported = [&](StringRef ModuleIdentifier, GlobalValue::GUID GUID) {
> - const auto &ExportList = ExportLists.find(ModuleIdentifier);
> - return (ExportList != ExportLists.end() &&
> - ExportList->second.count(GUID)) ||
> - Preserve.count(GUID);
> - };
> + Conf.CPU = options::mcpu;
> + Conf.Options = InitTargetOptionsFromCodeGenFlags();
>
> - thinLTOResolveWeakForLinkerInIndex(
> - CombinedIndex, isPrevailing,
> - [](StringRef ModuleIdentifier, GlobalValue::GUID GUID,
> - GlobalValue::LinkageTypes NewLinkage) {});
> -
> - // Use global summary-based analysis to identify symbols that can be
> - // internalized (because they aren't exported or preserved as per callback).
> - // Changes are made in the index, consumed in the ThinLTO backends.
> - thinLTOInternalizeAndPromoteInIndex(CombinedIndex, isExported);
> -
> - if (options::thinlto_emit_imports_files && !options::thinlto_index_only)
> - message(LDPL_WARNING,
> - "thinlto-emit-imports-files ignored unless thinlto-index-only");
> + // Disable the new X86 relax relocations since gold might not support them.
> + // FIXME: Check the gold version or add a new option to enable them.
> + Conf.Options.RelaxELFRelocations = false;
>
> + Conf.MAttrs = MAttrs;
> + Conf.RelocModel = *RelocationModel;
> + Conf.CGOptLevel = getCGOptLevel();
> + Conf.DisableVerify = options::DisableVerify;
> + Conf.OptLevel = options::OptLevel;
> + if (options::Parallelism) {
> + if (options::thinlto)
> + Backend = createInProcessThinBackend(options::Parallelism);
> + else
> + ParallelCodeGenParallelismLevel = options::Parallelism;
> + }
> if (options::thinlto_index_only) {
> - // If the thinlto-prefix-replace option was specified, parse it and
> - // extract the old and new prefixes.
> std::string OldPrefix, NewPrefix;
> getThinLTOOldAndNewPrefix(OldPrefix, NewPrefix);
> + Backend = createWriteIndexesThinBackend(
> + OldPrefix, NewPrefix, options::thinlto_emit_imports_files,
> + options::thinlto_linked_objects_file);
> + }
>
> - // If the user requested a list of objects gold included in the link,
> - // create and open the requested file.
> - raw_fd_ostream *ObjFileOS = nullptr;
> - if (!options::thinlto_linked_objects_file.empty()) {
> - std::error_code EC;
> - ObjFileOS = new raw_fd_ostream(options::thinlto_linked_objects_file, EC,
> - sys::fs::OpenFlags::F_None);
> - if (EC)
> - message(LDPL_FATAL, "Unable to open %s for writing: %s",
> - options::thinlto_linked_objects_file.c_str(),
> - EC.message().c_str());
> - }
> - // For each input bitcode file, generate an individual index that
> - // contains summaries only for its own global values, and for any that
> - // should be imported.
> - for (claimed_file &F : Modules) {
> - std::error_code EC;
> + Conf.OverrideTriple = options::triple;
> + Conf.DefaultTriple = sys::getDefaultTargetTriple();
>
> - std::string NewModulePath =
> - getThinLTOOutputFile(F.name, OldPrefix, NewPrefix);
> + Conf.DiagHandler = diagnosticHandler;
>
> - if (!options::thinlto_linked_objects_file.empty()) {
> - // If gold included any symbols from ths file in the link, emit path
> - // to the final object file, which should be included in the final
> - // native link.
> - if (get_symbols(F.handle, F.syms.size(), F.syms.data()) !=
> - LDPS_NO_SYMS) {
> - assert(ObjFileOS);
> - *ObjFileOS << NewModulePath << "\n";
> - }
> - }
> + Conf.PreOptModuleHook = [](size_t Task, Module &M) {
> + if (Task == 0)
> + addCommons(M);
> + return true;
> + };
>
> - raw_fd_ostream OS((Twine(NewModulePath) + ".thinlto.bc").str(), EC,
> - sys::fs::OpenFlags::F_None);
> - if (EC)
> - message(LDPL_FATAL, "Unable to open %s.thinlto.bc for writing: %s",
> - NewModulePath.c_str(), EC.message().c_str());
> - // Build a map of module to the GUIDs and summary objects that should
> - // be written to its index.
> - std::map<std::string, GVSummaryMapTy> ModuleToSummariesForIndex;
> - gatherImportedSummariesForModule(F.name, ModuleToDefinedGVSummaries,
> - ImportLists, ModuleToSummariesForIndex);
> - WriteIndexToFile(CombinedIndex, OS, &ModuleToSummariesForIndex);
> -
> - if (options::thinlto_emit_imports_files) {
> - if ((EC = EmitImportsFiles(F.name,
> - (Twine(NewModulePath) + ".imports").str(),
> - ImportLists)))
> - message(LDPL_FATAL, "Unable to open %s.imports",
> - NewModulePath.c_str(), EC.message().c_str());
> - }
> - }
> + switch (options::TheOutputType) {
> + case options::OT_NORMAL:
> + break;
>
> - if (ObjFileOS)
> - ObjFileOS->close();
> + case options::OT_DISABLE:
> + Conf.PreOptModuleHook = [](size_t Task, Module &M) { return false; };
> + break;
>
> - cleanup_hook();
> - exit(0);
> - }
> + case options::OT_BC_ONLY:
> + Conf.PostInternalizeModuleHook = [](size_t Task, Module &M) {
> + std::error_code EC;
> + raw_fd_ostream OS(output_name, EC, sys::fs::OpenFlags::F_None);
> + if (EC)
> + message(LDPL_FATAL, "Failed to write the output file.");
> + WriteBitcodeToFile(&M, OS, /* ShouldPreserveUseListOrder */ false);
> + return false;
> + };
> + break;
>
> - // Create OS in nested scope so that it will be closed on destruction.
> - {
> - std::error_code EC;
> - raw_fd_ostream OS(output_name + ".thinlto.bc", EC,
> - sys::fs::OpenFlags::F_None);
> - if (EC)
> - message(LDPL_FATAL, "Unable to open %s.thinlto.bc for writing: %s",
> - output_name.data(), EC.message().c_str());
> - WriteIndexToFile(CombinedIndex, OS);
> + case options::OT_SAVE_TEMPS:
> + check(Conf.addSaveTemps(output_name, /* UseInputModulePath */ true));
> + break;
> }
>
> - thinLTOBackends(ApiFile, CombinedIndex, ModuleMap, ImportLists,
> - ModuleToDefinedGVSummaries);
> - return LDPS_OK;
> + return llvm::make_unique<LTO>(std::move(Conf), Backend,
> + ParallelCodeGenParallelismLevel);
> }
>
> /// gold informs us that all symbols have been read. At this point, we use
> /// get_symbols to see if any of our definitions have been overridden by a
> /// native object file. Then, perform optimization and codegen.
> -static ld_plugin_status allSymbolsReadHook(raw_fd_ostream *ApiFile) {
> +static ld_plugin_status allSymbolsReadHook() {
> if (Modules.empty())
> return LDPS_OK;
>
> if (unsigned NumOpts = options::extra.size())
> cl::ParseCommandLineOptions(NumOpts, &options::extra[0]);
>
> - if (options::thinlto)
> - return thinLTOLink(ApiFile);
> + std::unique_ptr<LTO> Lto = createLTO();
>
> - LLVMContext Context;
> - Context.setDiscardValueNames(options::TheOutputType !=
> - options::OT_SAVE_TEMPS);
> - Context.enableDebugTypeODRUniquing(); // Merge debug info types.
> - Context.setDiagnosticHandler(diagnosticHandlerForContext, nullptr, true);
> -
> - std::unique_ptr<Module> Combined(new Module("ld-temp.o", Context));
> - IRMover L(*Combined);
> -
> - StringSet<> Internalize;
> for (claimed_file &F : Modules) {
> - // RAII object to manage the file opening and releasing interfaces with
> - // gold.
> PluginInputFile InputFile(F.handle);
> const void *View = getSymbolsAndView(F);
> if (!View)
> continue;
> - linkInModule(Context, L, F, View, F.name, ApiFile, Internalize);
> + addModule(*Lto, F, View);
> }
>
> - for (const auto &Name : Internalize) {
> - GlobalValue *GV = Combined->getNamedValue(Name.first());
> - if (GV)
> - internalize(*GV);
> - }
> + SmallString<128> Filename;
> + // Note that openOutputFile will append a unique ID for each task
> + if (!options::obj_path.empty())
> + Filename = options::obj_path;
> + else if (options::TheOutputType == options::OT_SAVE_TEMPS)
> + Filename = output_name + ".o";
> + bool SaveTemps = !Filename.empty();
>
> - if (options::TheOutputType == options::OT_DISABLE)
> + MaxTasks = Lto->getMaxTasks();
> + std::vector<uintptr_t> IsTemporary(MaxTasks);
> + std::vector<SmallString<128>> Filenames(MaxTasks);
> +
> + auto AddStream = [&](size_t Task) {
> + int FD = openOutputFile(Filename, /*TempOutFile=*/!SaveTemps,
> + Filenames[Task], MaxTasks > 1 ? Task : -1);
> + IsTemporary[Task] = !SaveTemps;
> +
> + return llvm::make_unique<llvm::raw_fd_ostream>(FD, true);
> + };
> +
> + check(Lto->run(AddStream));
> +
> + if (options::TheOutputType == options::OT_DISABLE ||
> + options::TheOutputType == options::OT_BC_ONLY)
> return LDPS_OK;
>
> - if (options::TheOutputType != options::OT_NORMAL) {
> - std::string path;
> - if (options::TheOutputType == options::OT_BC_ONLY)
> - path = output_name;
> - else
> - path = output_name + ".bc";
> - saveBCFile(path, *Combined);
> - if (options::TheOutputType == options::OT_BC_ONLY)
> - return LDPS_OK;
> + if (options::thinlto_index_only) {
> + cleanup_hook();
> + exit(0);
> }
>
> - CodeGen codeGen(std::move(Combined));
> - codeGen.runAll();
> + for (unsigned I = 0; I != MaxTasks; ++I)
> + if (!Filenames[I].empty())
> + recordFile(Filenames[I].str(), IsTemporary[I]);
>
> if (!options::extra_library_path.empty() &&
> set_extra_library_path(options::extra_library_path.c_str()) != LDPS_OK)
> @@ -1562,18 +856,7 @@ static ld_plugin_status allSymbolsReadHo
> }
>
> static ld_plugin_status all_symbols_read_hook(void) {
> - ld_plugin_status Ret;
> - if (!options::generate_api_file) {
> - Ret = allSymbolsReadHook(nullptr);
> - } else {
> - std::error_code EC;
> - raw_fd_ostream ApiFile("apifile.txt", EC, sys::fs::F_None);
> - if (EC)
> - message(LDPL_FATAL, "Unable to open apifile.txt for writing: %s",
> - EC.message().c_str());
> - Ret = allSymbolsReadHook(&ApiFile);
> - }
> -
> + ld_plugin_status Ret = allSymbolsReadHook();
> llvm_shutdown();
>
> if (options::TheOutputType == options::OT_BC_ONLY ||
>
> Added: llvm/trunk/tools/llvm-lto2/CMakeLists.txt
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-lto2/CMakeLists.txt?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-lto2/CMakeLists.txt (added)
> +++ llvm/trunk/tools/llvm-lto2/CMakeLists.txt Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,10 @@
> +set(LLVM_LINK_COMPONENTS
> + ${LLVM_TARGETS_TO_BUILD}
> + LTO
> + Object
> + Support
> + )
> +
> +add_llvm_tool(llvm-lto2
> + llvm-lto2.cpp
> + )
>
> Copied: llvm/trunk/tools/llvm-lto2/LLVMBuild.txt (from r278331, llvm/trunk/lib/LTO/LLVMBuild.txt)
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-lto2/LLVMBuild.txt?p2=llvm/trunk/tools/llvm-lto2/LLVMBuild.txt&p1=llvm/trunk/lib/LTO/LLVMBuild.txt&r1=278331&r2=278338&rev=278338&view=diff
> ==============================================================================
> --- llvm/trunk/lib/LTO/LLVMBuild.txt (original)
> +++ llvm/trunk/tools/llvm-lto2/LLVMBuild.txt Thu Aug 11 09:58:12 2016
> @@ -1,4 +1,4 @@
> -;===- ./lib/LTO/LLVMBuild.txt ----------------------------------*- Conf -*--===;
> +;===- ./tools/llvm-lto2/LLVMBuild.txt --------------------------*- Conf -*--===;
> ;
> ; The LLVM Compiler Infrastructure
> ;
> @@ -16,22 +16,7 @@
> ;===------------------------------------------------------------------------===;
>
> [component_0]
> -type = Library
> -name = LTO
> -parent = Libraries
> -required_libraries =
> - Analysis
> - BitReader
> - BitWriter
> - CodeGen
> - Core
> - IPO
> - InstCombine
> - Linker
> - MC
> - ObjCARC
> - Object
> - Scalar
> - Support
> - Target
> - TransformUtils
> \ No newline at end of file
> +type = Tool
> +name = llvm-lto2
> +parent = Tools
> +required_libraries = LTO Object all-targets
>
> Added: llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp
> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp?rev=278338&view=auto
> ==============================================================================
> --- llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp (added)
> +++ llvm/trunk/tools/llvm-lto2/llvm-lto2.cpp Thu Aug 11 09:58:12 2016
> @@ -0,0 +1,168 @@
> +//===-- llvm-lto2: test harness for the resolution-based LTO interface ----===//
> +//
> +// The LLVM Compiler Infrastructure
> +//
> +// This file is distributed under the University of Illinois Open Source
> +// License. See LICENSE.TXT for details.
> +//
> +//===----------------------------------------------------------------------===//
> +//
> +// This program takes in a list of bitcode files, links them and performs
> +// link-time optimization according to the provided symbol resolutions using the
> +// resolution-based LTO interface, and outputs one or more object files.
> +//
> +// This program is intended to eventually replace llvm-lto which uses the legacy
> +// LTO interface.
> +//
> +//===----------------------------------------------------------------------===//
> +
> +#include "llvm/LTO/LTO.h"
> +#include "llvm/Support/CommandLine.h"
> +#include "llvm/Support/TargetSelect.h"
> +
> +using namespace llvm;
> +using namespace lto;
> +using namespace object;
> +
> +static cl::list<std::string> InputFilenames(cl::Positional, cl::OneOrMore,
> + cl::desc("<input bitcode files>"));
> +
> +static cl::opt<std::string> OutputFilename("o", cl::Required,
> + cl::desc("Output filename"),
> + cl::value_desc("filename"));
> +
> +static cl::opt<bool> SaveTemps("save-temps", cl::desc("Save temporary files"));
> +
> +static cl::list<std::string> SymbolResolutions(
> + "r",
> + cl::desc("Specify a symbol resolution: filename,symbolname,resolution\n"
> + "where \"resolution\" is a sequence (which may be empty) of the\n"
> + "following characters:\n"
> + " p - prevailing: the linker has chosen this definition of the\n"
> + " symbol\n"
> + " l - local: the definition of this symbol is unpreemptable at\n"
> + " runtime and is known to be in this linkage unit\n"
> + " x - externally visible: the definition of this symbol is\n"
> + " visible outside of the LTO unit\n"
> + "A resolution for each symbol must be specified."),
> + cl::ZeroOrMore);
> +
> +static void check(Error E, std::string Msg) {
> + if (!E)
> + return;
> + handleAllErrors(std::move(E), [&](ErrorInfoBase &EIB) {
> + errs() << "llvm-lto: " << Msg << ": " << EIB.message().c_str() << '\n';
> + });
> + exit(1);
> +}
> +
> +template <typename T> static T check(Expected<T> E, std::string Msg) {
> + if (E)
> + return std::move(*E);
> + check(E.takeError(), Msg);
> + return T();
> +}
> +
> +static void check(std::error_code EC, std::string Msg) {
> + check(errorCodeToError(EC), Msg);
> +}
> +
> +template <typename T> static T check(ErrorOr<T> E, std::string Msg) {
> + if (E)
> + return std::move(*E);
> + check(E.getError(), Msg);
> + return T();
> +}
> +
> +int main(int argc, char **argv) {
> + InitializeAllTargets();
> + InitializeAllTargetMCs();
> + InitializeAllAsmPrinters();
> + InitializeAllAsmParsers();
> +
> + cl::ParseCommandLineOptions(argc, argv, "Resolution-based LTO test harness");
> +
> + std::map<std::pair<std::string, std::string>, SymbolResolution>
> + CommandLineResolutions;
> + for (std::string R : SymbolResolutions) {
> + StringRef Rest = R;
> + StringRef FileName, SymbolName;
> + std::tie(FileName, Rest) = Rest.split(',');
> + if (Rest.empty()) {
> + llvm::errs() << "invalid resolution: " << R << '\n';
> + return 1;
> + }
> + std::tie(SymbolName, Rest) = Rest.split(',');
> + SymbolResolution Res;
> + for (char C : Rest) {
> + if (C == 'p')
> + Res.Prevailing = true;
> + else if (C == 'l')
> + Res.FinalDefinitionInLinkageUnit = true;
> + else if (C == 'x')
> + Res.VisibleToRegularObj = true;
> + else
> + llvm::errs() << "invalid character " << C << " in resolution: " << R
> + << '\n';
> + }
> + CommandLineResolutions[{FileName, SymbolName}] = Res;
> + }
> +
> + std::vector<std::unique_ptr<MemoryBuffer>> MBs;
> +
> + Config Conf;
> + Conf.DiagHandler = [](const DiagnosticInfo &) {
> + exit(1);
> + };
> +
> + if (SaveTemps)
> + check(Conf.addSaveTemps(OutputFilename), "Config::addSaveTemps failed");
> +
> + LTO Lto(std::move(Conf));
> +
> + bool HasErrors = false;
> + for (std::string F : InputFilenames) {
> + std::unique_ptr<MemoryBuffer> MB = check(MemoryBuffer::getFile(F), F);
> + std::unique_ptr<InputFile> Input =
> + check(InputFile::create(MB->getMemBufferRef()), F);
> +
> + std::vector<SymbolResolution> Res;
> + for (const InputFile::Symbol &Sym : Input->symbols()) {
> + auto I = CommandLineResolutions.find({F, Sym.getName()});
> + if (I == CommandLineResolutions.end()) {
> + llvm::errs() << argv[0] << ": missing symbol resolution for " << F
> + << ',' << Sym.getName() << '\n';
> + HasErrors = true;
> + } else {
> + Res.push_back(I->second);
> + CommandLineResolutions.erase(I);
> + }
> + }
> +
> + if (HasErrors)
> + continue;
> +
> + MBs.push_back(std::move(MB));
> + check(Lto.add(std::move(Input), Res), F);
> + }
> +
> + if (!CommandLineResolutions.empty()) {
> + HasErrors = true;
> + for (auto UnusedRes : CommandLineResolutions)
> + llvm::errs() << argv[0] << ": unused symbol resolution for "
> + << UnusedRes.first.first << ',' << UnusedRes.first.second
> + << '\n';
> + }
> + if (HasErrors)
> + return 1;
> +
> + auto AddStream = [&](size_t Task) {
> + std::string Path = OutputFilename + "." + utostr(Task);
> + std::error_code EC;
> + auto S = llvm::make_unique<raw_fd_ostream>(Path, EC, sys::fs::F_None);
> + check(EC, Path);
> + return S;
> + };
> +
> + check(Lto.run(AddStream), "LTO::run failed");
> +}
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
More information about the llvm-commits
mailing list