[llvm] r255909 - [ThinLTO] Metadata linking for imported functions
Teresa Johnson via llvm-commits
llvm-commits at lists.llvm.org
Wed Dec 30 09:50:05 PST 2015
On Tue, Dec 29, 2015 at 6:00 PM, Teresa Johnson <tejohnson at google.com> wrote:
> On Wed, Dec 23, 2015 at 8:34 AM, Duncan P. N. Exon Smith
> <dexonsmith at apple.com> wrote:
>>
>>> On 2015-Dec-17, at 12:14, Teresa Johnson via llvm-commits <llvm-commits at lists.llvm.org> wrote:
>>>
>>> Author: tejohnson
>>> Date: Thu Dec 17 11:14:09 2015
>>> New Revision: 255909
>>>
>>> URL: http://llvm.org/viewvc/llvm-project?rev=255909&view=rev
>>> Log:
>>> [ThinLTO] Metadata linking for imported functions
>>>
>>> Summary:
>>> Second patch split out from http://reviews.llvm.org/D14752.
>>>
>>> Maps metadata as a post-pass from each module when importing complete,
>>> suturing up final metadata to the temporary metadata left on the
>>> imported instructions.
>>>
>>> This entails saving the mapping from bitcode value id to temporary
>>> metadata in the importing pass, and from bitcode value id to final
>>> metadata during the metadata linking postpass.
>>>
>>> Depends on D14825.
>>>
>>> Reviewers: dexonsmith, joker.eph
>>>
>>> Subscribers: davidxl, llvm-commits, joker.eph
>>>
>>> Differential Revision: http://reviews.llvm.org/D14838
>>>
>>> Added:
>>> llvm/trunk/test/Linker/Inputs/thinlto_funcimport_debug.ll
>>> llvm/trunk/test/Linker/thinlto_funcimport_debug.ll
>>> llvm/trunk/test/Transforms/FunctionImport/Inputs/funcimport_debug.ll
>>> llvm/trunk/test/Transforms/FunctionImport/funcimport_debug.ll
>>> Modified:
>>> llvm/trunk/include/llvm/IR/GVMaterializer.h
>>> llvm/trunk/include/llvm/IR/Metadata.h
>>> llvm/trunk/include/llvm/IRReader/IRReader.h
>>> llvm/trunk/include/llvm/Linker/IRMover.h
>>> llvm/trunk/include/llvm/Linker/Linker.h
>>> llvm/trunk/include/llvm/Transforms/Utils/ValueMapper.h
>>> llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp
>>> llvm/trunk/lib/IR/Metadata.cpp
>>> llvm/trunk/lib/IRReader/IRReader.cpp
>>> llvm/trunk/lib/Linker/IRMover.cpp
>>> llvm/trunk/lib/Linker/LinkModules.cpp
>>> llvm/trunk/lib/Transforms/IPO/FunctionImport.cpp
>>> llvm/trunk/lib/Transforms/Utils/ValueMapper.cpp
>>> llvm/trunk/tools/llvm-link/llvm-link.cpp
>>>
>>> Modified: llvm/trunk/include/llvm/IR/GVMaterializer.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/GVMaterializer.h?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/IR/GVMaterializer.h (original)
>>> +++ llvm/trunk/include/llvm/IR/GVMaterializer.h Thu Dec 17 11:14:09 2015
>>> @@ -18,12 +18,14 @@
>>> #ifndef LLVM_IR_GVMATERIALIZER_H
>>> #define LLVM_IR_GVMATERIALIZER_H
>>>
>>> +#include "llvm/ADT/DenseMap.h"
>>> #include <system_error>
>>> #include <vector>
>>>
>>> namespace llvm {
>>> class Function;
>>> class GlobalValue;
>>> +class Metadata;
>>> class Module;
>>> class StructType;
>>>
>>> @@ -56,6 +58,14 @@ public:
>>> virtual std::error_code materializeMetadata() = 0;
>>> virtual void setStripDebugInfo() = 0;
>>>
>>> + /// Client should define this interface if the mapping between metadata
>>> + /// values and value ids needs to be preserved, e.g. across materializer
>>> + /// instantiations. If OnlyTempMD is true, only those that have remained
>>> + /// temporary metadata are recorded in the map.
>>> + virtual void
>>> + saveMDValueList(DenseMap<const Metadata *, unsigned> &MDValueToValIDMap,
>>> + bool OnlyTempMD) {}
>>
>> Please update callers to say `/* OnlyTempMD = */ true/false`. I found
>> it hard to know which was which.
>
> Ok
>
>>
>>> +
>>> virtual std::vector<StructType *> getIdentifiedStructTypes() const = 0;
>>> };
>>>
>>>
>>> Modified: llvm/trunk/include/llvm/IR/Metadata.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IR/Metadata.h?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/IR/Metadata.h (original)
>>> +++ llvm/trunk/include/llvm/IR/Metadata.h Thu Dec 17 11:14:09 2015
>>> @@ -832,10 +832,11 @@ public:
>>> /// \brief Resolve cycles.
>>> ///
>>> /// Once all forward declarations have been resolved, force cycles to be
>>> - /// resolved.
>>> + /// resolved. If \p MDMaterialized is true, then any temporary metadata
>>> + /// is ignored, otherwise it asserts when encountering temporary metadata.
>>
>> Please expand a little on the semantic difference between `true` and
>> `false` here. In the `false` case, you're blocking identical `MDNode`s
>> from getting uniqued together.
>
> Looks like I got the comment backwards. We ignore temp metadata when
> MDMaterialized==false, otherwise we assert when seeing temp metadata
> here (the default behavior). I don't think there is any other semantic
> difference - the flag only affects temporary MD, which we shouldn't
> normally see (which is why the default is the old behavior of
> asserting on temp metadata).
>
>>
>>> ///
>>> /// \pre No operands (or operands' operands, etc.) have \a isTemporary().
>>> - void resolveCycles();
>>> + void resolveCycles(bool MDMaterialized = true);
>>
>> I think this variable would be better named: "AllowTemps" (same comment
>> for the definition as well).
>
> Ok, I think that is clearer.
>
>>
>>>
>>> /// \brief Replace a temporary node with a permanent one.
>>> ///
>>>
>>> Modified: llvm/trunk/include/llvm/IRReader/IRReader.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/IRReader/IRReader.h?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/IRReader/IRReader.h (original)
>>> +++ llvm/trunk/include/llvm/IRReader/IRReader.h Thu Dec 17 11:14:09 2015
>>> @@ -27,10 +27,11 @@ class LLVMContext;
>>> /// If the given file holds a bitcode image, return a Module
>>> /// for it which does lazy deserialization of function bodies. Otherwise,
>>> /// attempt to parse it as LLVM Assembly and return a fully populated
>>> -/// Module.
>>> -std::unique_ptr<Module> getLazyIRFileModule(StringRef Filename,
>>> - SMDiagnostic &Err,
>>> - LLVMContext &Context);
>>> +/// Module. The ShouldLazyLoadMetadata flag is passed down to the bitcode
>>> +/// reader to optionally enable lazy metadata loading.
>>> +std::unique_ptr<Module>
>>> +getLazyIRFileModule(StringRef Filename, SMDiagnostic &Err, LLVMContext &Context,
>>> + bool ShouldLazyLoadMetadata = false);
>>>
>>> /// If the given MemoryBuffer holds a bitcode image, return a Module
>>> /// for it. Otherwise, attempt to parse it as LLVM Assembly and return
>>>
>>> Modified: llvm/trunk/include/llvm/Linker/IRMover.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Linker/IRMover.h?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/Linker/IRMover.h (original)
>>> +++ llvm/trunk/include/llvm/Linker/IRMover.h Thu Dec 17 11:14:09 2015
>>> @@ -16,6 +16,7 @@
>>>
>>> namespace llvm {
>>> class GlobalValue;
>>> +class MDNode;
>>> class Module;
>>> class StructType;
>>> class Type;
>>> @@ -60,7 +61,9 @@ public:
>>> /// Move in the provide values. The source is destroyed.
>>> /// Returns true on error.
>>> bool move(Module &Src, ArrayRef<GlobalValue *> ValuesToLink,
>>> - std::function<void(GlobalValue &GV, ValueAdder Add)> AddLazyFor);
>>> + std::function<void(GlobalValue &GV, ValueAdder Add)> AddLazyFor,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap = nullptr,
>>> + bool IsMetadataLinkingPostpass = false);
>>> Module &getModule() { return Composite; }
>>>
>>> private:
>>>
>>> Modified: llvm/trunk/include/llvm/Linker/Linker.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Linker/Linker.h?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/Linker/Linker.h (original)
>>> +++ llvm/trunk/include/llvm/Linker/Linker.h Thu Dec 17 11:14:09 2015
>>> @@ -42,11 +42,14 @@ public:
>>> /// For ThinLTO function importing/exporting the \p FunctionInfoIndex
>>> /// is passed. If \p FunctionsToImport is provided, only the functions that
>>> /// are part of the set will be imported from the source module.
>>> + /// The \p ValIDToTempMDMap is populated by the linker when function
>>> + /// importing is performed.
>>> ///
>>> /// Returns true on error.
>>> bool linkInModule(std::unique_ptr<Module> Src, unsigned Flags = Flags::None,
>>> const FunctionInfoIndex *Index = nullptr,
>>> - DenseSet<const GlobalValue *> *FunctionsToImport = nullptr);
>>> + DenseSet<const GlobalValue *> *FunctionsToImport = nullptr,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap = nullptr);
>>>
>>> /// This exists to implement the deprecated LLVMLinkModules C api. Don't use
>>> /// for anything else.
>>> @@ -54,6 +57,14 @@ public:
>>>
>>> static bool linkModules(Module &Dest, std::unique_ptr<Module> Src,
>>> unsigned Flags = Flags::None);
>>> +
>>> + /// \brief Link metadata from \p Src into the composite. The source is
>>> + /// destroyed.
>>> + ///
>>> + /// The \p ValIDToTempMDMap sound have been populated earlier during function
>>> + /// importing from \p Src.
>>> + bool linkInMetadata(Module &Src,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap);
>>> };
>>>
>>> /// Create a new module with exported local functions renamed and promoted
>>>
>>> Modified: llvm/trunk/include/llvm/Transforms/Utils/ValueMapper.h
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/include/llvm/Transforms/Utils/ValueMapper.h?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/include/llvm/Transforms/Utils/ValueMapper.h (original)
>>> +++ llvm/trunk/include/llvm/Transforms/Utils/ValueMapper.h Thu Dec 17 11:14:09 2015
>>> @@ -55,6 +55,12 @@ namespace llvm {
>>> /// It is called after the mapping is recorded, so it doesn't need to worry
>>> /// about recursion.
>>> virtual void materializeInitFor(GlobalValue *New, GlobalValue *Old);
>>> +
>>> + /// If the client needs to handle temporary metadata it must implement
>>> + /// these methods.
>>> + virtual Metadata *mapTemporaryMetadata(Metadata *MD) { return nullptr; }
>>> + virtual void replaceTemporaryMetadata(const Metadata *OrigMD,
>>> + Metadata *NewMD) {}
>>> };
>>>
>>> /// RemapFlags - These are flags that the value mapping APIs allow.
>>> @@ -78,6 +84,11 @@ namespace llvm {
>>> /// Any global values not in value map are mapped to null instead of
>>> /// mapping to self. Illegal if RF_IgnoreMissingEntries is also set.
>>> RF_NullMapMissingGlobalValues = 8,
>>> +
>>> + /// Set when there is still temporary metadata that must be handled,
>>> + /// such as when we are doing function importing and will materialize
>>> + /// and link metadata as a postpass.
>>> + RF_HaveUnmaterializedMetadata = 16,
>>> };
>>>
>>> static inline RemapFlags operator|(RemapFlags LHS, RemapFlags RHS) {
>>>
>>> Modified: llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp (original)
>>> +++ llvm/trunk/lib/Bitcode/Reader/BitcodeReader.cpp Thu Dec 17 11:14:09 2015
>>> @@ -97,6 +97,7 @@ public:
>>> class BitcodeReaderMDValueList {
>>> unsigned NumFwdRefs;
>>> bool AnyFwdRefs;
>>> + bool SavedFwdRefs;
>>
>> Should have a Boolean verb, something like "HasSavedFwdRefs".
>
> Ok
>
>>
>>> unsigned MinFwdRef;
>>> unsigned MaxFwdRef;
>>> std::vector<TrackingMDRef> MDValuePtrs;
>>> @@ -104,7 +105,12 @@ class BitcodeReaderMDValueList {
>>> LLVMContext &Context;
>>> public:
>>> BitcodeReaderMDValueList(LLVMContext &C)
>>> - : NumFwdRefs(0), AnyFwdRefs(false), Context(C) {}
>>> + : NumFwdRefs(0), AnyFwdRefs(false), SavedFwdRefs(false), Context(C) {}
>>> + ~BitcodeReaderMDValueList() {
>>> + // Assert that we either replaced all forward references, or saved
>>> + // them for later replacement.
>>> + assert(!NumFwdRefs || SavedFwdRefs);
>>> + }
>>>
>>> // vector compatibility methods
>>> unsigned size() const { return MDValuePtrs.size(); }
>>> @@ -115,6 +121,8 @@ public:
>>> void pop_back() { MDValuePtrs.pop_back(); }
>>> bool empty() const { return MDValuePtrs.empty(); }
>>>
>>> + void savedFwdRefs() { SavedFwdRefs = true; }
>>
>> Can you rename this something like "setHasSavedFwdRefs()"? The first
>> few times I read this at the call site, I thought it was doing something,
>> and I couldn't figure out what.
>
> Ok
>
>>
>>> +
>>> Metadata *operator[](unsigned i) const {
>>> assert(i < MDValuePtrs.size());
>>> return MDValuePtrs[i];
>>> @@ -274,6 +282,14 @@ public:
>>>
>>> void setStripDebugInfo() override;
>>>
>>> + /// Save the mapping between the metadata values and the corresponding
>>> + /// value id that were recorded in the MDValueList during parsing. If
>>> + /// OnlyTempMD is true, then only record those entries that are still
>>> + /// temporary metadata. This interface is used when metadata linking is
>>> + /// performed as a postpass, such as during function importing.
>>> + void saveMDValueList(DenseMap<const Metadata *, unsigned> &MDValueToValIDMap,
>>> + bool OnlyTempMD) override;
>>> +
>>> private:
>>> /// Parse the "IDENTIFICATION_BLOCK_ID" block, populate the
>>> // ProducerIdentification data member, and do some basic enforcement on the
>>> @@ -1069,6 +1085,9 @@ Metadata *BitcodeReaderMDValueList::getV
>>> MinFwdRef = MaxFwdRef = Idx;
>>> }
>>> ++NumFwdRefs;
>>> + // Reset flag to ensure that we save this forward reference if we
>>> + // are delaying metadata mapping (e.g. for function importing).
>>> + SavedFwdRefs = false;
>>>
>>> // Create and return a placeholder, which will later be RAUW'd.
>>> Metadata *MD = MDNode::getTemporary(Context, None).release();
>>> @@ -3062,6 +3081,30 @@ std::error_code BitcodeReader::materiali
>>>
>>> void BitcodeReader::setStripDebugInfo() { StripDebugInfo = true; }
>>>
>>> +void BitcodeReader::saveMDValueList(
>>> + DenseMap<const Metadata *, unsigned> &MDValueToValIDMap, bool OnlyTempMD) {
>>
>> "MDValue" is a horrible term that I wish I'd never come up with.
>> Please don't propagate it. If you get a chance, please clean up
>> instances of it (even the ones you didn't add).
>>
>> I think this should be called `saveMetadataList()`, and the map
>> should be `MetadataToIDs`.
>
> I did both of these just now in r256593.
>
>>
>>> + for (unsigned ValID = 0; ValID < MDValueList.size(); ++ValID) {
>>
>> I suggest just using `ID` here instead of `ValID`.
>
> Ditto
>
>>
>>> + Metadata *MD = MDValueList[ValID];
>>> + auto *N = dyn_cast_or_null<MDNode>(MD);
>>> + // Save all values if !OnlyTempMD, otherwise just the temporary metadata.
>>> + if (!OnlyTempMD || (N && N->isTemporary())) {
>>
>> Why do you need to record metadata that isn't an MDNode here (i.e., why
>> not `continue` if `!N`)?
>
> I don't remember, but will take a look if I can change that. (I just
> did and am getting an assert in one of the ThinLTO regression tests,
> need to see if I just need to modify a use or something).
Can't do this as there are ValueAsMetadata in the module level
metadata bitcode that are referenced from instructions (the example I
found was from an llvm.dbg.value instruction).Therefore, the
instruction gets a temp MD node during function importing, and we need
to record the ValueAsMetadata here as well so that the temp MD is
replaced with it during metadata linking. Will add a comment.
>
>>
>> Also, it would be great to be able to assert:
>> --
>> assert(N->isResolved() || N->isTemporary());
>> --
>> This confirms that you've called `resolveCycles()`. If you haven't
>> called `resolveCycles()` by now, then I don't think it's safe to put `N`
>> in the map.
>
> Ok, will try that.
Added the assert, although modified to handle the !N case, and that
works fine as expected.
>
>>
>> Finally, there's an important missing assertion that I'm not sure how
>> to provide. But it's important.
>>
>> IIUC, it's vital that this *doesn't* happen:
>>
>> 1. Temporary metadata T is put into MetadataToIDs.
>> 2. T.replaceAllUsesWith(MD) gets called.
>> 3. MetadataToIDs.count/find/etc.(T) or (MD).
>>
>> First case (T): since `T` has been freed, some new temporary (or normal)
>> metadata might get allocated in the same spot. Someone could be
>> checking for this new `N`, and get the ID for the old `T`.
>>
>> Second case (MD): the map contains the ID for `MD`, but it's indexed
>> under the old pointer `T`. Checking for `MD` will give the wrong answer.
>>
>> As I mentioned, I don't know how to add an assertion or verification for
>> this, but I think it's really important. Please look into how to
>> verify that these cases can't occur.
>
> In the case where we are saving temporary metadata in the
> MetadataToIDs map, we should never replace it once we save it to that
> map, until after we delete that instance of the MetadataToIDs map when
> the IRLinker is destructed.
>
> So what I could do is when we save temp metadata to the map, set a
> flag on the temp MD's ReplaceableMetadataImpl, and assert in
> ReplaceableMetadataImpl::replaceAllUsesWith if the flag is set, then
> clear the flag when we destruct the IRLinker and its associated
> MetadataToIDs map. Does that seem reasonable? I'll give it a shot.
This works. I also confirmed that I get an assert during metadata
linking if I skip resetting the flag during IRLinker destruction, so
it does catch a RAUW on temporary data with the flag set.
Will commit these new asserts and fix the other issues you pointed out
during your review.
>
>>
>>> + // Will call this after materializing each function, in order to
>>> + // handle remapping of the function's instructions/metadata.
>>> + // See if we already have an entry in that case.
>>> + if (OnlyTempMD && MDValueToValIDMap.count(MD)) {
>>> + assert(MDValueToValIDMap[MD] == ValID &&
>>> + "Inconsistent metadata value id");
>>> + continue;
>>> + }
>>> + MDValueToValIDMap[MD] = ValID;
>>> + // Flag that we saved the forward refs (temporary metadata) for error
>>> + // checking during MDValueList destruction.
>>> + if (OnlyTempMD)
>>> + MDValueList.savedFwdRefs();
>>> + }
>>> + }
>>> +}
>>> +
>>> /// When we see the block for a function body, remember where it is and then
>>> /// skip it. This lets us lazily deserialize the functions.
>>> std::error_code BitcodeReader::rememberAndSkipFunctionBody() {
>>>
>>> Modified: llvm/trunk/lib/IR/Metadata.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IR/Metadata.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/IR/Metadata.cpp (original)
>>> +++ llvm/trunk/lib/IR/Metadata.cpp Thu Dec 17 11:14:09 2015
>>> @@ -517,7 +517,7 @@ void MDNode::decrementUnresolvedOperandC
>>> resolve();
>>> }
>>>
>>> -void MDNode::resolveCycles() {
>>> +void MDNode::resolveCycles(bool MDMaterialized) {
>>> if (isResolved())
>>> return;
>>>
>>> @@ -530,6 +530,8 @@ void MDNode::resolveCycles() {
>>> if (!N)
>>> continue;
>>>
>>> + if (N->isTemporary() && !MDMaterialized)
>>> + continue;
>>> assert(!N->isTemporary() &&
>>> "Expected all forward declarations to be resolved");
>>> if (!N->isResolved())
>>>
>>> Modified: llvm/trunk/lib/IRReader/IRReader.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/IRReader/IRReader.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/IRReader/IRReader.cpp (original)
>>> +++ llvm/trunk/lib/IRReader/IRReader.cpp Thu Dec 17 11:14:09 2015
>>> @@ -31,11 +31,11 @@ static const char *const TimeIRParsingNa
>>>
>>> static std::unique_ptr<Module>
>>> getLazyIRModule(std::unique_ptr<MemoryBuffer> Buffer, SMDiagnostic &Err,
>>> - LLVMContext &Context) {
>>> + LLVMContext &Context, bool ShouldLazyLoadMetadata) {
>>> if (isBitcode((const unsigned char *)Buffer->getBufferStart(),
>>> (const unsigned char *)Buffer->getBufferEnd())) {
>>> - ErrorOr<std::unique_ptr<Module>> ModuleOrErr =
>>> - getLazyBitcodeModule(std::move(Buffer), Context);
>>> + ErrorOr<std::unique_ptr<Module>> ModuleOrErr = getLazyBitcodeModule(
>>> + std::move(Buffer), Context, ShouldLazyLoadMetadata);
>>> if (std::error_code EC = ModuleOrErr.getError()) {
>>> Err = SMDiagnostic(Buffer->getBufferIdentifier(), SourceMgr::DK_Error,
>>> EC.message());
>>> @@ -49,7 +49,8 @@ getLazyIRModule(std::unique_ptr<MemoryBu
>>>
>>> std::unique_ptr<Module> llvm::getLazyIRFileModule(StringRef Filename,
>>> SMDiagnostic &Err,
>>> - LLVMContext &Context) {
>>> + LLVMContext &Context,
>>> + bool ShouldLazyLoadMetadata) {
>>> ErrorOr<std::unique_ptr<MemoryBuffer>> FileOrErr =
>>> MemoryBuffer::getFileOrSTDIN(Filename);
>>> if (std::error_code EC = FileOrErr.getError()) {
>>> @@ -58,7 +59,8 @@ std::unique_ptr<Module> llvm::getLazyIRF
>>> return nullptr;
>>> }
>>>
>>> - return getLazyIRModule(std::move(FileOrErr.get()), Err, Context);
>>> + return getLazyIRModule(std::move(FileOrErr.get()), Err, Context,
>>> + ShouldLazyLoadMetadata);
>>> }
>>>
>>> std::unique_ptr<Module> llvm::parseIR(MemoryBufferRef Buffer, SMDiagnostic &Err,
>>>
>>> Modified: llvm/trunk/lib/Linker/IRMover.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Linker/IRMover.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/Linker/IRMover.cpp (original)
>>> +++ llvm/trunk/lib/Linker/IRMover.cpp Thu Dec 17 11:14:09 2015
>>> @@ -14,6 +14,7 @@
>>> #include "llvm/ADT/Triple.h"
>>> #include "llvm/IR/Constants.h"
>>> #include "llvm/IR/DiagnosticPrinter.h"
>>> +#include "llvm/IR/GVMaterializer.h"
>>> #include "llvm/IR/TypeFinder.h"
>>> #include "llvm/Transforms/Utils/Cloning.h"
>>> using namespace llvm;
>>> @@ -349,6 +350,9 @@ public:
>>> GlobalValueMaterializer(IRLinker *ModLinker) : ModLinker(ModLinker) {}
>>> Value *materializeDeclFor(Value *V) override;
>>> void materializeInitFor(GlobalValue *New, GlobalValue *Old) override;
>>> + Metadata *mapTemporaryMetadata(Metadata *MD) override;
>>> + void replaceTemporaryMetadata(const Metadata *OrigMD,
>>> + Metadata *NewMD) override;
>>> };
>>>
>>> class LocalValueMaterializer final : public ValueMaterializer {
>>> @@ -358,6 +362,9 @@ public:
>>> LocalValueMaterializer(IRLinker *ModLinker) : ModLinker(ModLinker) {}
>>> Value *materializeDeclFor(Value *V) override;
>>> void materializeInitFor(GlobalValue *New, GlobalValue *Old) override;
>>> + Metadata *mapTemporaryMetadata(Metadata *MD) override;
>>> + void replaceTemporaryMetadata(const Metadata *OrigMD,
>>> + Metadata *NewMD) override;
>>> };
>>>
>>> /// This is responsible for keeping track of the state used for moving data
>>> @@ -394,6 +401,24 @@ class IRLinker {
>>>
>>> bool HasError = false;
>>>
>>> + /// Flag indicating that we are just linking metadata (after function
>>> + /// importing).
>>> + bool IsMetadataLinkingPostpass;
>>> +
>>> + /// Flags to pass to value mapper invocations.
>>> + RemapFlags ValueMapperFlags = RF_MoveDistinctMDs;
>>> +
>>> + /// Association between metadata values created during bitcode parsing and
>>> + /// the value id. Used to correlate temporary metadata created during
>>> + /// function importing with the final metadata parsed during the subsequent
>>> + /// metadata linking postpass.
>>> + DenseMap<const Metadata *, unsigned> MDValueToValIDMap;
>>> +
>>> + /// Association between metadata value id and temporary metadata that
>>> + /// remains unmapped after function importing. Saved during function
>>> + /// importing and consumed during the metadata linking postpass.
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap;
>>> +
>>> /// Handles cloning of a global values from the source module into
>>> /// the destination module, including setting the attributes and visibility.
>>> GlobalValue *copyGlobalValueProto(const GlobalValue *SGV, bool ForDefinition);
>>> @@ -409,6 +434,14 @@ class IRLinker {
>>> SrcM.getContext().diagnose(LinkDiagnosticInfo(DS_Warning, Message));
>>> }
>>>
>>> + /// Check whether we should be linking metadata from the source module.
>>> + bool shouldLinkMetadata() {
>>> + // ValIDToTempMDMap will be non-null when we are importing or otherwise want
>>> + // to link metadata lazily, and then when linking the metadata.
>>> + // We only want to return true for the former case.
>>> + return ValIDToTempMDMap == nullptr || IsMetadataLinkingPostpass;
>>> + }
>>> +
>>> /// Given a global in the source module, return the global in the
>>> /// destination module that is being linked to, if any.
>>> GlobalValue *getLinkedToGlobal(const GlobalValue *SrcGV) {
>>> @@ -457,16 +490,35 @@ class IRLinker {
>>> public:
>>> IRLinker(Module &DstM, IRMover::IdentifiedStructTypeSet &Set, Module &SrcM,
>>> ArrayRef<GlobalValue *> ValuesToLink,
>>> - std::function<void(GlobalValue &, IRMover::ValueAdder)> AddLazyFor)
>>> + std::function<void(GlobalValue &, IRMover::ValueAdder)> AddLazyFor,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap = nullptr,
>>> + bool IsMetadataLinkingPostpass = false)
>>> : DstM(DstM), SrcM(SrcM), AddLazyFor(AddLazyFor), TypeMap(Set),
>>> - GValMaterializer(this), LValMaterializer(this) {
>>> + GValMaterializer(this), LValMaterializer(this),
>>> + IsMetadataLinkingPostpass(IsMetadataLinkingPostpass),
>>> + ValIDToTempMDMap(ValIDToTempMDMap) {
>>> for (GlobalValue *GV : ValuesToLink)
>>> maybeAdd(GV);
>>> +
>>> + // If appropriate, tell the value mapper that it can expect to see
>>> + // temporary metadata.
>>> + if (!shouldLinkMetadata())
>>> + ValueMapperFlags = ValueMapperFlags | RF_HaveUnmaterializedMetadata;
>>> }
>>>
>>> bool run();
>>> Value *materializeDeclFor(Value *V, bool ForAlias);
>>> void materializeInitFor(GlobalValue *New, GlobalValue *Old, bool ForAlias);
>>> +
>>> + /// Save the mapping between the given temporary metadata and its metadata
>>> + /// value id. Used to support metadata linking as a postpass for function
>>> + /// importing.
>>> + Metadata *mapTemporaryMetadata(Metadata *MD);
>>> +
>>> + /// Replace any temporary metadata saved for the source metadata's id with
>>> + /// the new non-temporary metadata. Used when metadata linking as a postpass
>>> + /// for function importing.
>>> + void replaceTemporaryMetadata(const Metadata *OrigMD, Metadata *NewMD);
>>> };
>>> }
>>>
>>> @@ -500,6 +552,15 @@ void GlobalValueMaterializer::materializ
>>> ModLinker->materializeInitFor(New, Old, false);
>>> }
>>>
>>> +Metadata *GlobalValueMaterializer::mapTemporaryMetadata(Metadata *MD) {
>>> + return ModLinker->mapTemporaryMetadata(MD);
>>> +}
>>> +
>>> +void GlobalValueMaterializer::replaceTemporaryMetadata(const Metadata *OrigMD,
>>> + Metadata *NewMD) {
>>> + ModLinker->replaceTemporaryMetadata(OrigMD, NewMD);
>>> +}
>>> +
>>> Value *LocalValueMaterializer::materializeDeclFor(Value *V) {
>>> return ModLinker->materializeDeclFor(V, true);
>>> }
>>> @@ -509,6 +570,15 @@ void LocalValueMaterializer::materialize
>>> ModLinker->materializeInitFor(New, Old, true);
>>> }
>>>
>>> +Metadata *LocalValueMaterializer::mapTemporaryMetadata(Metadata *MD) {
>>> + return ModLinker->mapTemporaryMetadata(MD);
>>> +}
>>> +
>>> +void LocalValueMaterializer::replaceTemporaryMetadata(const Metadata *OrigMD,
>>> + Metadata *NewMD) {
>>> + ModLinker->replaceTemporaryMetadata(OrigMD, NewMD);
>>> +}
>>> +
>>> Value *IRLinker::materializeDeclFor(Value *V, bool ForAlias) {
>>> auto *SGV = dyn_cast<GlobalValue>(V);
>>> if (!SGV)
>>> @@ -536,6 +606,51 @@ void IRLinker::materializeInitFor(Global
>>> linkGlobalValueBody(*New, *Old);
>>> }
>>>
>>> +Metadata *IRLinker::mapTemporaryMetadata(Metadata *MD) {
>>> + if (!ValIDToTempMDMap)
>>> + return nullptr;
>>> + // If this temporary metadata has a value id recorded during function
>>> + // parsing, record that in the ValIDToTempMDMap if one was provided.
>>> + if (MDValueToValIDMap.count(MD)) {
>>> + unsigned Idx = MDValueToValIDMap[MD];
>>> + // Check if we created a temp MD when importing a different function from
>>> + // this module. If so, reuse it the same temporary metadata, otherwise
>>> + // add this temporary metadata to the map.
>>> + if (!ValIDToTempMDMap->count(Idx)) {
>>> + MDNode *Node = cast<MDNode>(MD);
>>> + assert(Node->isTemporary());
>>> + (*ValIDToTempMDMap)[Idx] = Node;
>>> + }
>>> + return (*ValIDToTempMDMap)[Idx];
>>> + }
>>> + return nullptr;
>>> +}
>>> +
>>> +void IRLinker::replaceTemporaryMetadata(const Metadata *OrigMD,
>>> + Metadata *NewMD) {
>>> + if (!ValIDToTempMDMap)
>>> + return;
>>> +#ifndef NDEBUG
>>> + auto *N = dyn_cast_or_null<MDNode>(NewMD);
>>> + assert(!N || !N->isTemporary());
>>> +#endif
>>> + // If a mapping between metadata value ids and temporary metadata
>>> + // created during function importing was provided, and the source
>>> + // metadata has a value id recorded during metadata parsing, replace
>>> + // the temporary metadata with the final mapped metadata now.
>>> + if (MDValueToValIDMap.count(OrigMD)) {
>>> + unsigned Idx = MDValueToValIDMap[OrigMD];
>>> + // Nothing to do if we didn't need to create a temporary metadata during
>>> + // function importing.
>>> + if (!ValIDToTempMDMap->count(Idx))
>>> + return;
>>> + MDNode *TempMD = (*ValIDToTempMDMap)[Idx];
>>> + TempMD->replaceAllUsesWith(NewMD);
>>> + MDNode::deleteTemporary(TempMD);
>>> + ValIDToTempMDMap->erase(Idx);
>>> + }
>>> +}
>>> +
>>> /// Loop through the global variables in the src module and merge them into the
>>> /// dest module.
>>> GlobalVariable *IRLinker::copyGlobalVariableProto(const GlobalVariable *SGVar) {
>>> @@ -793,16 +908,16 @@ Constant *IRLinker::linkAppendingVarProt
>>> Constant *NewV;
>>> if (IsOldStructor) {
>>> auto *S = cast<ConstantStruct>(V);
>>> - auto *E1 = MapValue(S->getOperand(0), ValueMap, RF_MoveDistinctMDs,
>>> + auto *E1 = MapValue(S->getOperand(0), ValueMap, ValueMapperFlags,
>>> &TypeMap, &GValMaterializer);
>>> - auto *E2 = MapValue(S->getOperand(1), ValueMap, RF_MoveDistinctMDs,
>>> + auto *E2 = MapValue(S->getOperand(1), ValueMap, ValueMapperFlags,
>>> &TypeMap, &GValMaterializer);
>>> Value *Null = Constant::getNullValue(VoidPtrTy);
>>> NewV =
>>> ConstantStruct::get(cast<StructType>(EltTy), E1, E2, Null, nullptr);
>>> } else {
>>> - NewV = MapValue(V, ValueMap, RF_MoveDistinctMDs, &TypeMap,
>>> - &GValMaterializer);
>>> + NewV =
>>> + MapValue(V, ValueMap, ValueMapperFlags, &TypeMap, &GValMaterializer);
>>> }
>>> DstElements.push_back(NewV);
>>> }
>>> @@ -837,6 +952,14 @@ static bool useExistingDest(GlobalValue
>>> }
>>>
>>> bool IRLinker::shouldLink(GlobalValue *DGV, GlobalValue &SGV) {
>>> + // Already imported all the values. Just map to the Dest value
>>> + // in case it is referenced in the metadata.
>>> + if (IsMetadataLinkingPostpass) {
>>> + assert(!ValuesToLink.count(&SGV) &&
>>> + "Source value unexpectedly requested for link during metadata link");
>>> + return false;
>>> + }
>>> +
>>> if (ValuesToLink.count(&SGV))
>>> return true;
>>>
>>> @@ -925,8 +1048,8 @@ Constant *IRLinker::linkGlobalValueProto
>>> /// referenced are in Dest.
>>> void IRLinker::linkGlobalInit(GlobalVariable &Dst, GlobalVariable &Src) {
>>> // Figure out what the initializer looks like in the dest module.
>>> - Dst.setInitializer(MapValue(Src.getInitializer(), ValueMap,
>>> - RF_MoveDistinctMDs, &TypeMap, &GValMaterializer));
>>> + Dst.setInitializer(MapValue(Src.getInitializer(), ValueMap, ValueMapperFlags,
>>> + &TypeMap, &GValMaterializer));
>>> }
>>>
>>> /// Copy the source function over into the dest function and fix up references
>>> @@ -939,22 +1062,28 @@ bool IRLinker::linkFunctionBody(Function
>>> if (std::error_code EC = Src.materialize())
>>> return emitError(EC.message());
>>>
>>> + if (!shouldLinkMetadata())
>>> + // This is only supported for lazy links. Do after materialization of
>>> + // a function and before remapping metadata on instructions below
>>> + // in RemapInstruction, as the saved mapping is used to handle
>>> + // the temporary metadata hanging off instructions.
>>> + SrcM.getMaterializer()->saveMDValueList(MDValueToValIDMap, true);
>>> +
>>> // Link in the prefix data.
>>> if (Src.hasPrefixData())
>>> - Dst.setPrefixData(MapValue(Src.getPrefixData(), ValueMap,
>>> - RF_MoveDistinctMDs, &TypeMap,
>>> - &GValMaterializer));
>>> + Dst.setPrefixData(MapValue(Src.getPrefixData(), ValueMap, ValueMapperFlags,
>>> + &TypeMap, &GValMaterializer));
>>>
>>> // Link in the prologue data.
>>> if (Src.hasPrologueData())
>>> Dst.setPrologueData(MapValue(Src.getPrologueData(), ValueMap,
>>> - RF_MoveDistinctMDs, &TypeMap,
>>> + ValueMapperFlags, &TypeMap,
>>> &GValMaterializer));
>>>
>>> // Link in the personality function.
>>> if (Src.hasPersonalityFn())
>>> Dst.setPersonalityFn(MapValue(Src.getPersonalityFn(), ValueMap,
>>> - RF_MoveDistinctMDs, &TypeMap,
>>> + ValueMapperFlags, &TypeMap,
>>> &GValMaterializer));
>>>
>>> // Go through and convert function arguments over, remembering the mapping.
>>> @@ -971,7 +1100,7 @@ bool IRLinker::linkFunctionBody(Function
>>> SmallVector<std::pair<unsigned, MDNode *>, 8> MDs;
>>> Src.getAllMetadata(MDs);
>>> for (const auto &I : MDs)
>>> - Dst.setMetadata(I.first, MapMetadata(I.second, ValueMap, RF_MoveDistinctMDs,
>>> + Dst.setMetadata(I.first, MapMetadata(I.second, ValueMap, ValueMapperFlags,
>>> &TypeMap, &GValMaterializer));
>>>
>>> // Splice the body of the source function into the dest function.
>>> @@ -983,9 +1112,8 @@ bool IRLinker::linkFunctionBody(Function
>>> // functions and patch them up to point to the local versions.
>>> for (BasicBlock &BB : Dst)
>>> for (Instruction &I : BB)
>>> - RemapInstruction(&I, ValueMap,
>>> - RF_IgnoreMissingEntries | RF_MoveDistinctMDs, &TypeMap,
>>> - &GValMaterializer);
>>> + RemapInstruction(&I, ValueMap, RF_IgnoreMissingEntries | ValueMapperFlags,
>>> + &TypeMap, &GValMaterializer);
>>>
>>> // There is no need to map the arguments anymore.
>>> for (Argument &Arg : Src.args())
>>> @@ -997,7 +1125,7 @@ bool IRLinker::linkFunctionBody(Function
>>>
>>> void IRLinker::linkAliasBody(GlobalAlias &Dst, GlobalAlias &Src) {
>>> Constant *Aliasee = Src.getAliasee();
>>> - Constant *Val = MapValue(Aliasee, AliasValueMap, RF_MoveDistinctMDs, &TypeMap,
>>> + Constant *Val = MapValue(Aliasee, AliasValueMap, ValueMapperFlags, &TypeMap,
>>> &LValMaterializer);
>>> Dst.setAliasee(Val);
>>> }
>>> @@ -1024,7 +1152,7 @@ void IRLinker::linkNamedMDNodes() {
>>> // Add Src elements into Dest node.
>>> for (const MDNode *op : NMD.operands())
>>> DestNMD->addOperand(MapMetadata(
>>> - op, ValueMap, RF_MoveDistinctMDs | RF_NullMapMissingGlobalValues,
>>> + op, ValueMap, ValueMapperFlags | RF_NullMapMissingGlobalValues,
>>> &TypeMap, &GValMaterializer));
>>> }
>>> }
>>> @@ -1260,7 +1388,7 @@ bool IRLinker::run() {
>>> continue;
>>>
>>> assert(!GV->isDeclaration());
>>> - MapValue(GV, ValueMap, RF_MoveDistinctMDs, &TypeMap, &GValMaterializer);
>>> + MapValue(GV, ValueMap, ValueMapperFlags, &TypeMap, &GValMaterializer);
>>> if (HasError)
>>> return true;
>>> }
>>> @@ -1272,11 +1400,39 @@ bool IRLinker::run() {
>>> // Remap all of the named MDNodes in Src into the DstM module. We do this
>>> // after linking GlobalValues so that MDNodes that reference GlobalValues
>>> // are properly remapped.
>>> - linkNamedMDNodes();
>>> + if (shouldLinkMetadata()) {
>>> + // Even if just linking metadata we should link decls above in case
>>> + // any are referenced by metadata. IRLinker::shouldLink ensures that
>>> + // we don't actually link anything from source.
>>> + if (IsMetadataLinkingPostpass) {
>>> + // Ensure metadata materialized
>>> + if (SrcM.getMaterializer()->materializeMetadata())
>>> + return true;
>>> + SrcM.getMaterializer()->saveMDValueList(MDValueToValIDMap, false);
>>> + }
>>> +
>>> + linkNamedMDNodes();
>>> +
>>> + if (IsMetadataLinkingPostpass) {
>>> + // Handle anything left in the ValIDToTempMDMap, such as metadata nodes
>>> + // not reached by the dbg.cu NamedMD (i.e. only reached from
>>> + // instructions).
>>> + // Walk the MDValueToValIDMap once to find the set of new (imported) MD
>>> + // that still has corresponding temporary metadata, and invoke metadata
>>> + // mapping on each one.
>>> + for (auto MDI : MDValueToValIDMap) {
>>> + if (!ValIDToTempMDMap->count(MDI.second))
>>> + continue;
>>> + MapMetadata(MDI.first, ValueMap, ValueMapperFlags, &TypeMap,
>>> + &GValMaterializer);
>>> + }
>>> + assert(ValIDToTempMDMap->empty());
>>> + }
>>>
>>> - // Merge the module flags into the DstM module.
>>> - if (linkModuleFlagsMetadata())
>>> - return true;
>>> + // Merge the module flags into the DstM module.
>>> + if (linkModuleFlagsMetadata())
>>> + return true;
>>> + }
>>>
>>> return false;
>>> }
>>> @@ -1384,9 +1540,11 @@ IRMover::IRMover(Module &M) : Composite(
>>>
>>> bool IRMover::move(
>>> Module &Src, ArrayRef<GlobalValue *> ValuesToLink,
>>> - std::function<void(GlobalValue &, ValueAdder Add)> AddLazyFor) {
>>> + std::function<void(GlobalValue &, ValueAdder Add)> AddLazyFor,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap,
>>> + bool IsMetadataLinkingPostpass) {
>>> IRLinker TheLinker(Composite, IdentifiedStructTypes, Src, ValuesToLink,
>>> - AddLazyFor);
>>> + AddLazyFor, ValIDToTempMDMap, IsMetadataLinkingPostpass);
>>> bool RetCode = TheLinker.run();
>>> Composite.dropTriviallyDeadConstantArrays();
>>> return RetCode;
>>>
>>> Modified: llvm/trunk/lib/Linker/LinkModules.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Linker/LinkModules.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/Linker/LinkModules.cpp (original)
>>> +++ llvm/trunk/lib/Linker/LinkModules.cpp Thu Dec 17 11:14:09 2015
>>> @@ -48,6 +48,11 @@ class ModuleLinker {
>>> /// as part of a different backend compilation process.
>>> bool HasExportedFunctions = false;
>>>
>>> + /// Association between metadata value id and temporary metadata that
>>> + /// remains unmapped after function importing. Saved during function
>>> + /// importing and consumed during the metadata linking postpass.
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap;
>>> +
>>> /// Used as the callback for lazy linking.
>>> /// The mover has just hit GV and we have to decide if it, and other members
>>> /// of the same comdat, should be linked. Every member to be linked is passed
>>> @@ -150,9 +155,10 @@ class ModuleLinker {
>>> public:
>>> ModuleLinker(IRMover &Mover, Module &SrcM, unsigned Flags,
>>> const FunctionInfoIndex *Index = nullptr,
>>> - DenseSet<const GlobalValue *> *FunctionsToImport = nullptr)
>>> + DenseSet<const GlobalValue *> *FunctionsToImport = nullptr,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap = nullptr)
>>> : Mover(Mover), SrcM(SrcM), Flags(Flags), ImportIndex(Index),
>>> - ImportFunction(FunctionsToImport) {
>>> + ImportFunction(FunctionsToImport), ValIDToTempMDMap(ValIDToTempMDMap) {
>>> assert((ImportIndex || !ImportFunction) &&
>>> "Expect a FunctionInfoIndex when importing");
>>> // If we have a FunctionInfoIndex but no function to import,
>>> @@ -161,6 +167,8 @@ public:
>>> // may be exported to another backend compilation.
>>> if (ImportIndex && !ImportFunction)
>>> HasExportedFunctions = ImportIndex->hasExportedFunctions(SrcM);
>>> + assert((ValIDToTempMDMap || !ImportFunction) &&
>>> + "Function importing must provide a ValIDToTempMDMap");
>>> }
>>>
>>> bool run();
>>> @@ -502,6 +510,7 @@ bool ModuleLinker::getComdatResult(const
>>> bool ModuleLinker::shouldLinkFromSource(bool &LinkFromSrc,
>>> const GlobalValue &Dest,
>>> const GlobalValue &Src) {
>>> +
>>> // Should we unconditionally use the Src?
>>> if (shouldOverrideFromSrc()) {
>>> LinkFromSrc = true;
>>> @@ -776,7 +785,8 @@ bool ModuleLinker::run() {
>>> if (Mover.move(SrcM, ValuesToLink.getArrayRef(),
>>> [this](GlobalValue &GV, IRMover::ValueAdder Add) {
>>> addLazyFor(GV, Add);
>>> - }))
>>> + },
>>> + ValIDToTempMDMap, false))
>>> return true;
>>> Module &DstM = Mover.getModule();
>>> for (auto &P : Internalize) {
>>> @@ -791,8 +801,10 @@ Linker::Linker(Module &M) : Mover(M) {}
>>>
>>> bool Linker::linkInModule(std::unique_ptr<Module> Src, unsigned Flags,
>>> const FunctionInfoIndex *Index,
>>> - DenseSet<const GlobalValue *> *FunctionsToImport) {
>>> - ModuleLinker TheLinker(Mover, *Src, Flags, Index, FunctionsToImport);
>>> + DenseSet<const GlobalValue *> *FunctionsToImport,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap) {
>>> + ModuleLinker TheLinker(Mover, *Src, Flags, Index, FunctionsToImport,
>>> + ValIDToTempMDMap);
>>> return TheLinker.run();
>>> }
>>>
>>> @@ -801,6 +813,17 @@ bool Linker::linkInModuleForCAPI(Module
>>> return TheLinker.run();
>>> }
>>>
>>> +bool Linker::linkInMetadata(Module &Src,
>>> + DenseMap<unsigned, MDNode *> *ValIDToTempMDMap) {
>>> + SetVector<GlobalValue *> ValuesToLink;
>>> + if (Mover.move(
>>> + Src, ValuesToLink.getArrayRef(),
>>> + [this](GlobalValue &GV, IRMover::ValueAdder Add) { assert(false); },
>>> + ValIDToTempMDMap, true))
>>> + return true;
>>> + return false;
>>> +}
>>> +
>>> //===----------------------------------------------------------------------===//
>>> // LinkModules entrypoint.
>>> //===----------------------------------------------------------------------===//
>>>
>>> Modified: llvm/trunk/lib/Transforms/IPO/FunctionImport.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/IPO/FunctionImport.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/Transforms/IPO/FunctionImport.cpp (original)
>>> +++ llvm/trunk/lib/Transforms/IPO/FunctionImport.cpp Thu Dec 17 11:14:09 2015
>>> @@ -292,6 +292,9 @@ bool FunctionImporter::importFunctions(M
>>> ModuleToFunctionsToImportMap, Index, ModuleLoaderCache);
>>> assert(Worklist.empty() && "Worklist hasn't been flushed in GetImportList");
>>>
>>> + StringMap<std::unique_ptr<DenseMap<unsigned, MDNode *>>>
>>> + ModuleToTempMDValsMap;
>>> +
>>> // Do the actual import of functions now, one Module at a time
>>> for (auto &FunctionsToImportPerModule : ModuleToFunctionsToImportMap) {
>>> // Get the module for the import
>>> @@ -301,13 +304,32 @@ bool FunctionImporter::importFunctions(M
>>> assert(&DestModule.getContext() == &SrcModule->getContext() &&
>>> "Context mismatch");
>>>
>>> + // Save the mapping of value ids to temporary metadata created when
>>> + // importing this function. If we have already imported from this module,
>>> + // add new temporary metadata to the existing mapping.
>>> + auto &TempMDVals = ModuleToTempMDValsMap[SrcModule->getModuleIdentifier()];
>>> + if (!TempMDVals)
>>> + TempMDVals = llvm::make_unique<DenseMap<unsigned, MDNode *>>();
>>> +
>>> // Link in the specified functions.
>>> if (TheLinker.linkInModule(std::move(SrcModule), Linker::Flags::None,
>>> - &Index, &FunctionsToImport))
>>> + &Index, &FunctionsToImport, TempMDVals.get()))
>>> report_fatal_error("Function Import: link error");
>>>
>>> ImportedCount += FunctionsToImport.size();
>>> }
>>> +
>>> + // Now link in metadata for all modules from which we imported functions.
>>> + for (StringMapEntry<std::unique_ptr<DenseMap<unsigned, MDNode *>>> &SME :
>>> + ModuleToTempMDValsMap) {
>>> + // Load the specified source module.
>>> + auto &SrcModule = ModuleLoaderCache(SME.getKey());
>>> +
>>> + // Link in all necessary metadata from this module.
>>> + if (TheLinker.linkInMetadata(SrcModule, SME.getValue().get()))
>>> + return false;
>>> + }
>>> +
>>> DEBUG(dbgs() << "Imported " << ImportedCount << " functions for Module "
>>> << DestModule.getModuleIdentifier() << "\n");
>>> return ImportedCount;
>>>
>>> Modified: llvm/trunk/lib/Transforms/Utils/ValueMapper.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/lib/Transforms/Utils/ValueMapper.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/lib/Transforms/Utils/ValueMapper.cpp (original)
>>> +++ llvm/trunk/lib/Transforms/Utils/ValueMapper.cpp Thu Dec 17 11:14:09 2015
>>> @@ -167,13 +167,21 @@ Value *llvm::MapValue(const Value *V, Va
>>> }
>>>
>>> static Metadata *mapToMetadata(ValueToValueMapTy &VM, const Metadata *Key,
>>> - Metadata *Val) {
>>> + Metadata *Val, ValueMaterializer *Materializer,
>>> + RemapFlags Flags) {
>>> VM.MD()[Key].reset(Val);
>>> + if (Materializer && !(Flags & RF_HaveUnmaterializedMetadata)) {
>>> + auto *N = dyn_cast_or_null<MDNode>(Val);
>>> + // Need to invoke this once we have non-temporary MD.
>>> + if (!N || !N->isTemporary())
>>> + Materializer->replaceTemporaryMetadata(Key, Val);
>>> + }
>>> return Val;
>>> }
>>>
>>> -static Metadata *mapToSelf(ValueToValueMapTy &VM, const Metadata *MD) {
>>> - return mapToMetadata(VM, MD, const_cast<Metadata *>(MD));
>>> +static Metadata *mapToSelf(ValueToValueMapTy &VM, const Metadata *MD,
>>> + ValueMaterializer *Materializer, RemapFlags Flags) {
>>> + return mapToMetadata(VM, MD, const_cast<Metadata *>(MD), Materializer, Flags);
>>> }
>>>
>>> static Metadata *MapMetadataImpl(const Metadata *MD,
>>> @@ -206,10 +214,13 @@ static Metadata *mapMetadataOp(Metadata
>>> }
>>>
>>> /// Resolve uniquing cycles involving the given metadata.
>>> -static void resolveCycles(Metadata *MD) {
>>> - if (auto *N = dyn_cast_or_null<MDNode>(MD))
>>> +static void resolveCycles(Metadata *MD, bool MDMaterialized) {
>>> + if (auto *N = dyn_cast_or_null<MDNode>(MD)) {
>>> + if (!MDMaterialized && N->isTemporary())
>>> + return;
>>> if (!N->isResolved())
>>> - N->resolveCycles();
>>> + N->resolveCycles(MDMaterialized);
>>> + }
>>> }
>>>
>>> /// Remap the operands of an MDNode.
>>> @@ -238,7 +249,7 @@ static bool remapOperands(MDNode &Node,
>>> // Resolve uniquing cycles underneath distinct nodes on the fly so they
>>> // don't infect later operands.
>>> if (IsDistinct)
>>> - resolveCycles(New);
>>> + resolveCycles(New, !(Flags & RF_HaveUnmaterializedMetadata));
>>> }
>>> }
>>>
>>> @@ -266,7 +277,7 @@ static Metadata *mapDistinctNode(const M
>>>
>>> // Remap operands later.
>>> DistinctWorklist.push_back(NewMD);
>>> - return mapToMetadata(VM, Node, NewMD);
>>> + return mapToMetadata(VM, Node, NewMD, Materializer, Flags);
>>> }
>>>
>>> /// \brief Map a uniqued MDNode.
>>> @@ -277,22 +288,29 @@ static Metadata *mapUniquedNode(const MD
>>> ValueToValueMapTy &VM, RemapFlags Flags,
>>> ValueMapTypeRemapper *TypeMapper,
>>> ValueMaterializer *Materializer) {
>>> - assert(Node->isUniqued() && "Expected uniqued node");
>>> + assert(((Flags & RF_HaveUnmaterializedMetadata) || Node->isUniqued()) &&
>>> + "Expected uniqued node");
>>>
>>> // Create a temporary node and map it upfront in case we have a uniquing
>>> // cycle. If necessary, this mapping will get updated by RAUW logic before
>>> // returning.
>>> auto ClonedMD = Node->clone();
>>> - mapToMetadata(VM, Node, ClonedMD.get());
>>> + mapToMetadata(VM, Node, ClonedMD.get(), Materializer, Flags);
>>> if (!remapOperands(*ClonedMD, DistinctWorklist, VM, Flags, TypeMapper,
>>> Materializer)) {
>>> // No operands changed, so use the original.
>>> ClonedMD->replaceAllUsesWith(const_cast<MDNode *>(Node));
>>> - return const_cast<MDNode *>(Node);
>>> + // Even though replaceAllUsesWith would have replaced the value map
>>> + // entry, we need to explictly map with the final non-temporary node
>>> + // to replace any temporary metadata via the callback.
>>> + return mapToSelf(VM, Node, Materializer, Flags);
>>> }
>>>
>>> - // Uniquify the cloned node.
>>> - return MDNode::replaceWithUniqued(std::move(ClonedMD));
>>> + // Uniquify the cloned node. Explicitly map it with the final non-temporary
>>> + // node so that replacement of temporary metadata via the callback occurs.
>>> + return mapToMetadata(VM, Node,
>>> + MDNode::replaceWithUniqued(std::move(ClonedMD)),
>>> + Materializer, Flags);
>>> }
>>>
>>> static Metadata *MapMetadataImpl(const Metadata *MD,
>>> @@ -305,18 +323,18 @@ static Metadata *MapMetadataImpl(const M
>>> return NewMD;
>>>
>>> if (isa<MDString>(MD))
>>> - return mapToSelf(VM, MD);
>>> + return mapToSelf(VM, MD, Materializer, Flags);
>>>
>>> if (isa<ConstantAsMetadata>(MD))
>>> if ((Flags & RF_NoModuleLevelChanges))
>>> - return mapToSelf(VM, MD);
>>> + return mapToSelf(VM, MD, Materializer, Flags);
>>>
>>> if (const auto *VMD = dyn_cast<ValueAsMetadata>(MD)) {
>>> Value *MappedV =
>>> MapValue(VMD->getValue(), VM, Flags, TypeMapper, Materializer);
>>> if (VMD->getValue() == MappedV ||
>>> (!MappedV && (Flags & RF_IgnoreMissingEntries)))
>>> - return mapToSelf(VM, MD);
>>> + return mapToSelf(VM, MD, Materializer, Flags);
>>>
>>> // FIXME: This assert crashes during bootstrap, but I think it should be
>>> // correct. For now, just match behaviour from before the metadata/value
>>> @@ -325,7 +343,8 @@ static Metadata *MapMetadataImpl(const M
>>> // assert((MappedV || (Flags & RF_NullMapMissingGlobalValues)) &&
>>> // "Referenced metadata not in value map!");
>>> if (MappedV)
>>> - return mapToMetadata(VM, MD, ValueAsMetadata::get(MappedV));
>>> + return mapToMetadata(VM, MD, ValueAsMetadata::get(MappedV), Materializer,
>>> + Flags);
>>> return nullptr;
>>> }
>>>
>>> @@ -336,10 +355,25 @@ static Metadata *MapMetadataImpl(const M
>>> // If this is a module-level metadata and we know that nothing at the
>>> // module level is changing, then use an identity mapping.
>>> if (Flags & RF_NoModuleLevelChanges)
>>> - return mapToSelf(VM, MD);
>>> + return mapToSelf(VM, MD, Materializer, Flags);
>>>
>>> // Require resolved nodes whenever metadata might be remapped.
>>> - assert(Node->isResolved() && "Unexpected unresolved node");
>>> + assert(((Flags & RF_HaveUnmaterializedMetadata) || Node->isResolved()) &&
>>> + "Unexpected unresolved node");
>>> +
>>> + if (Materializer && Node->isTemporary()) {
>>> + assert(Flags & RF_HaveUnmaterializedMetadata);
>>> + Metadata *TempMD =
>>> + Materializer->mapTemporaryMetadata(const_cast<Metadata *>(MD));
>>> + // If the above callback returned an existing temporary node, use it
>>> + // instead of the current temporary node. This happens when earlier
>>> + // function importing passes already created and saved a temporary
>>> + // metadata node for the same value id.
>>> + if (TempMD) {
>>> + mapToMetadata(VM, MD, TempMD, Materializer, Flags);
>>> + return TempMD;
>>> + }
>>> + }
>>>
>>> if (Node->isDistinct())
>>> return mapDistinctNode(Node, DistinctWorklist, VM, Flags, TypeMapper,
>>> @@ -363,7 +397,7 @@ Metadata *llvm::MapMetadata(const Metada
>>> return NewMD;
>>>
>>> // Resolve cycles involving the entry metadata.
>>> - resolveCycles(NewMD);
>>> + resolveCycles(NewMD, !(Flags & RF_HaveUnmaterializedMetadata));
>>>
>>> // Remap the operands of distinct MDNodes.
>>> while (!DistinctWorklist.empty())
>>>
>>> Added: llvm/trunk/test/Linker/Inputs/thinlto_funcimport_debug.ll
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Linker/Inputs/thinlto_funcimport_debug.ll?rev=255909&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/Linker/Inputs/thinlto_funcimport_debug.ll (added)
>>> +++ llvm/trunk/test/Linker/Inputs/thinlto_funcimport_debug.ll Thu Dec 17 11:14:09 2015
>>> @@ -0,0 +1,38 @@
>>> +; ModuleID = 'dbg_main.o'
>>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>> +target triple = "x86_64-unknown-linux-gnu"
>>> +
>>> +; Function Attrs: nounwind uwtable
>>> +define i32 @main() #0 !dbg !4 {
>>> +entry:
>>> + %call = tail call i32 @func1(i32 10) #2, !dbg !11
>>> + %call1 = tail call i32 @func2(i32 10) #2, !dbg !12
>>> + ret i32 0, !dbg !13
>>> +}
>>> +
>>> +declare i32 @func1(i32) #1
>>> +
>>> +declare i32 @func2(i32) #1
>>> +
>>> +attributes #0 = { nounwind uwtable "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+fxsr,+mmx,+sse,+sse2" "unsafe-fp-math"="false" "use-soft-float"="false" }
>>> +attributes #1 = { "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+fxsr,+mmx,+sse,+sse2" "unsafe-fp-math"="false" "use-soft-float"="false" }
>>> +attributes #2 = { nounwind }
>>> +
>>> +!llvm.dbg.cu = !{!0}
>>> +!llvm.module.flags = !{!8, !9}
>>> +!llvm.ident = !{!10}
>>> +
>>> +!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.8.0 (trunk 251407) (llvm/trunk 251401)", isOptimized: true, runtimeVersion: 0, emissionKind: 1, enums: !2, subprograms: !3)
>>> +!1 = !DIFile(filename: "dbg_main.c", directory: ".")
>>> +!2 = !{}
>>> +!3 = !{!4}
>>> +!4 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line: 3, type: !5, isLocal: false, isDefinition: true, scopeLine: 3, isOptimized: true, variables: !2)
>>> +!5 = !DISubroutineType(types: !6)
>>> +!6 = !{!7}
>>> +!7 = !DIBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed)
>>> +!8 = !{i32 2, !"Dwarf Version", i32 4}
>>> +!9 = !{i32 2, !"Debug Info Version", i32 3}
>>> +!10 = !{!"clang version 3.8.0 (trunk 251407) (llvm/trunk 251401)"}
>>> +!11 = !DILocation(line: 4, column: 3, scope: !4)
>>> +!12 = !DILocation(line: 5, column: 3, scope: !4)
>>> +!13 = !DILocation(line: 6, column: 1, scope: !4)
>>>
>>> Added: llvm/trunk/test/Linker/thinlto_funcimport_debug.ll
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Linker/thinlto_funcimport_debug.ll?rev=255909&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/Linker/thinlto_funcimport_debug.ll (added)
>>> +++ llvm/trunk/test/Linker/thinlto_funcimport_debug.ll Thu Dec 17 11:14:09 2015
>>> @@ -0,0 +1,71 @@
>>> +; Do setup work for all below tests: generate bitcode and combined index
>>> +; RUN: llvm-as -function-summary %s -o %t.bc
>>> +; RUN: llvm-as -function-summary %p/Inputs/thinlto_funcimport_debug.ll -o %t2.bc
>>> +; RUN: llvm-lto -thinlto -o %t3 %t.bc %t2.bc
>>> +
>>> +; Confirm that we link the metadata for the imported module.
>>> +; RUN: llvm-link %t2.bc -functionindex=%t3.thinlto.bc -import=func1:%t.bc -S | FileCheck %s
>>> +
>>> +; CHECK: declare i32 @func2
>>> +; CHECK: define available_externally i32 @func1
>>> +; CHECK: distinct !DISubprogram(name: "func1"
>>> +; CHECK: distinct !DISubprogram(name: "func2"
>>> +
>>> +; ModuleID = 'dbg.o'
>>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>> +target triple = "x86_64-unknown-linux-gnu"
>>> +
>>> +; Function Attrs: nounwind readnone uwtable
>>> +define i32 @func1(i32 %n) #0 !dbg !4 {
>>> +entry:
>>> + tail call void @llvm.dbg.value(metadata i32 %n, i64 0, metadata !9, metadata !17), !dbg !18
>>> + tail call void @llvm.dbg.value(metadata i32 5, i64 0, metadata !10, metadata !17), !dbg !19
>>> + %cmp = icmp sgt i32 %n, 10, !dbg !20
>>> + %. = select i1 %cmp, i32 10, i32 5, !dbg !22
>>> + tail call void @llvm.dbg.value(metadata i32 %., i64 0, metadata !10, metadata !17), !dbg !19
>>> + ret i32 %., !dbg !23
>>> +}
>>> +
>>> +; Function Attrs: nounwind readnone uwtable
>>> +define i32 @func2(i32 %n) #0 !dbg !11 {
>>> +entry:
>>> + tail call void @llvm.dbg.value(metadata i32 %n, i64 0, metadata !13, metadata !17), !dbg !24
>>> + ret i32 %n, !dbg !25
>>> +}
>>> +
>>> +; Function Attrs: nounwind readnone
>>> +declare void @llvm.dbg.value(metadata, i64, metadata, metadata) #1
>>> +
>>> +attributes #0 = { nounwind readnone uwtable "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+fxsr,+mmx,+sse,+sse2" "unsafe-fp-math"="false" "use-soft-float"="false" }
>>> +attributes #1 = { nounwind readnone }
>>> +
>>> +!llvm.dbg.cu = !{!0}
>>> +!llvm.module.flags = !{!14, !15}
>>> +!llvm.ident = !{!16}
>>> +
>>> +!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.8.0 (trunk 251407) (llvm/trunk 251401)", isOptimized: true, runtimeVersion: 0, emissionKind: 1, enums: !2, subprograms: !3)
>>> +!1 = !DIFile(filename: "dbg.c", directory: ".")
>>> +!2 = !{}
>>> +!3 = !{!4, !11}
>>> +!4 = distinct !DISubprogram(name: "func1", scope: !1, file: !1, line: 1, type: !5, isLocal: false, isDefinition: true, scopeLine: 1, flags: DIFlagPrototyped, isOptimized: true, variables: !8)
>>> +!5 = !DISubroutineType(types: !6)
>>> +!6 = !{!7, !7}
>>> +!7 = !DIBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed)
>>> +!8 = !{!9, !10}
>>> +!9 = !DILocalVariable(name: "n", arg: 1, scope: !4, file: !1, line: 1, type: !7)
>>> +!10 = !DILocalVariable(name: "x", scope: !4, file: !1, line: 2, type: !7)
>>> +!11 = distinct !DISubprogram(name: "func2", scope: !1, file: !1, line: 8, type: !5, isLocal: false, isDefinition: true, scopeLine: 8, flags: DIFlagPrototyped, isOptimized: true, variables: !12)
>>> +!12 = !{!13}
>>> +!13 = !DILocalVariable(name: "n", arg: 1, scope: !11, file: !1, line: 8, type: !7)
>>> +!14 = !{i32 2, !"Dwarf Version", i32 4}
>>> +!15 = !{i32 2, !"Debug Info Version", i32 3}
>>> +!16 = !{!"clang version 3.8.0 (trunk 251407) (llvm/trunk 251401)"}
>>> +!17 = !DIExpression()
>>> +!18 = !DILocation(line: 1, column: 15, scope: !4)
>>> +!19 = !DILocation(line: 2, column: 7, scope: !4)
>>> +!20 = !DILocation(line: 3, column: 9, scope: !21)
>>> +!21 = distinct !DILexicalBlock(scope: !4, file: !1, line: 3, column: 7)
>>> +!22 = !DILocation(line: 3, column: 7, scope: !4)
>>> +!23 = !DILocation(line: 5, column: 3, scope: !4)
>>> +!24 = !DILocation(line: 8, column: 15, scope: !11)
>>> +!25 = !DILocation(line: 9, column: 3, scope: !11)
>>>
>>> Added: llvm/trunk/test/Transforms/FunctionImport/Inputs/funcimport_debug.ll
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/FunctionImport/Inputs/funcimport_debug.ll?rev=255909&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/Transforms/FunctionImport/Inputs/funcimport_debug.ll (added)
>>> +++ llvm/trunk/test/Transforms/FunctionImport/Inputs/funcimport_debug.ll Thu Dec 17 11:14:09 2015
>>> @@ -0,0 +1,27 @@
>>> +; ModuleID = 'funcimport_debug.o'
>>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>> +target triple = "x86_64-unknown-linux-gnu"
>>> +
>>> +; Function Attrs: nounwind uwtable
>>> +define void @func() #0 !dbg !4 {
>>> +entry:
>>> + ret void, !dbg !10
>>> +}
>>> +
>>> +attributes #0 = { nounwind uwtable "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+fxsr,+mmx,+sse,+sse2" "unsafe-fp-math"="false" "use-soft-float"="false" }
>>> +
>>> +!llvm.dbg.cu = !{!0}
>>> +!llvm.module.flags = !{!7, !8}
>>> +!llvm.ident = !{!9}
>>> +
>>> +!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.8.0 (trunk 255685) (llvm/trunk 255682)", isOptimized: false, runtimeVersion: 0, emissionKind: 1, enums: !2, subprograms: !3)
>>> +!1 = !DIFile(filename: "funcimport_debug.c", directory: ".")
>>> +!2 = !{}
>>> +!3 = !{!4}
>>> +!4 = distinct !DISubprogram(name: "func", scope: !1, file: !1, line: 1, type: !5, isLocal: false, isDefinition: true, scopeLine: 1, isOptimized: false, variables: !2)
>>> +!5 = !DISubroutineType(types: !6)
>>> +!6 = !{null}
>>> +!7 = !{i32 2, !"Dwarf Version", i32 4}
>>> +!8 = !{i32 2, !"Debug Info Version", i32 3}
>>> +!9 = !{!"clang version 3.8.0 (trunk 255685) (llvm/trunk 255682)"}
>>> +!10 = !DILocation(line: 2, column: 1, scope: !4)
>>>
>>> Added: llvm/trunk/test/Transforms/FunctionImport/funcimport_debug.ll
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/test/Transforms/FunctionImport/funcimport_debug.ll?rev=255909&view=auto
>>> ==============================================================================
>>> --- llvm/trunk/test/Transforms/FunctionImport/funcimport_debug.ll (added)
>>> +++ llvm/trunk/test/Transforms/FunctionImport/funcimport_debug.ll Thu Dec 17 11:14:09 2015
>>> @@ -0,0 +1,45 @@
>>> +; Do setup work for all below tests: generate bitcode and combined index
>>> +; RUN: llvm-as -function-summary %s -o %t.bc
>>> +; RUN: llvm-as -function-summary %p/Inputs/funcimport_debug.ll -o %t2.bc
>>> +; RUN: llvm-lto -thinlto -o %t3 %t.bc %t2.bc
>>> +
>>> +; Do the import now and confirm that metadata is linked for imported function.
>>> +; RUN: opt -function-import -summary-file %t3.thinlto.bc %s -S | FileCheck %s
>>> +
>>> +; CHECK: define available_externally void @func()
>>> +; CHECK: distinct !DISubprogram(name: "main"
>>> +; CHECK: distinct !DISubprogram(name: "func"
>>> +
>>> +; ModuleID = 'funcimport_debug.o'
>>> +target datalayout = "e-m:e-i64:64-f80:128-n8:16:32:64-S128"
>>> +target triple = "x86_64-unknown-linux-gnu"
>>> +
>>> +; Function Attrs: nounwind uwtable
>>> +define i32 @main() #0 !dbg !4 {
>>> +entry:
>>> + call void (...) @func(), !dbg !11
>>> + ret i32 0, !dbg !12
>>> +}
>>> +
>>> +declare void @func(...) #1
>>> +
>>> +attributes #0 = { nounwind uwtable "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+fxsr,+mmx,+sse,+sse2" "unsafe-fp-math"="false" "use-soft-float"="false" }
>>> +attributes #1 = { "disable-tail-calls"="false" "less-precise-fpmad"="false" "no-frame-pointer-elim"="true" "no-frame-pointer-elim-non-leaf" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+fxsr,+mmx,+sse,+sse2" "unsafe-fp-math"="false" "use-soft-float"="false" }
>>> +
>>> +!llvm.dbg.cu = !{!0}
>>> +!llvm.module.flags = !{!8, !9}
>>> +!llvm.ident = !{!10}
>>> +
>>> +!0 = distinct !DICompileUnit(language: DW_LANG_C99, file: !1, producer: "clang version 3.8.0 (trunk 255685) (llvm/trunk 255682)", isOptimized: false, runtimeVersion: 0, emissionKind: 1, enums: !2, subprograms: !3)
>>> +!1 = !DIFile(filename: "funcimport_debug.c", directory: ".")
>>> +!2 = !{}
>>> +!3 = !{!4}
>>> +!4 = distinct !DISubprogram(name: "main", scope: !1, file: !1, line: 2, type: !5, isLocal: false, isDefinition: true, scopeLine: 2, isOptimized: false, variables: !2)
>>> +!5 = !DISubroutineType(types: !6)
>>> +!6 = !{!7}
>>> +!7 = !DIBasicType(name: "int", size: 32, align: 32, encoding: DW_ATE_signed)
>>> +!8 = !{i32 2, !"Dwarf Version", i32 4}
>>> +!9 = !{i32 2, !"Debug Info Version", i32 3}
>>> +!10 = !{!"clang version 3.8.0 (trunk 255685) (llvm/trunk 255682)"}
>>> +!11 = !DILocation(line: 3, column: 3, scope: !4)
>>> +!12 = !DILocation(line: 4, column: 1, scope: !4)
>>>
>>> Modified: llvm/trunk/tools/llvm-link/llvm-link.cpp
>>> URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/tools/llvm-link/llvm-link.cpp?rev=255909&r1=255908&r2=255909&view=diff
>>> ==============================================================================
>>> --- llvm/trunk/tools/llvm-link/llvm-link.cpp (original)
>>> +++ llvm/trunk/tools/llvm-link/llvm-link.cpp Thu Dec 17 11:14:09 2015
>>> @@ -106,16 +106,21 @@ static cl::opt<bool> PreserveAssemblyUse
>>> // Read the specified bitcode file in and return it. This routine searches the
>>> // link path for the specified file to try to find it...
>>> //
>>> -static std::unique_ptr<Module>
>>> -loadFile(const char *argv0, const std::string &FN, LLVMContext &Context) {
>>> +static std::unique_ptr<Module> loadFile(const char *argv0,
>>> + const std::string &FN,
>>> + LLVMContext &Context,
>>> + bool MaterializeMetadata = true) {
>>> SMDiagnostic Err;
>>> if (Verbose) errs() << "Loading '" << FN << "'\n";
>>> - std::unique_ptr<Module> Result = getLazyIRFileModule(FN, Err, Context);
>>> + std::unique_ptr<Module> Result =
>>> + getLazyIRFileModule(FN, Err, Context, !MaterializeMetadata);
>>> if (!Result)
>>> Err.print(argv0, errs());
>>>
>>> - Result->materializeMetadata();
>>> - UpgradeDebugInfo(*Result);
>>> + if (MaterializeMetadata) {
>>> + Result->materializeMetadata();
>>> + UpgradeDebugInfo(*Result);
>>> + }
>>>
>>> return Result;
>>> }
>>> @@ -148,6 +153,8 @@ static void diagnosticHandlerWithContext
>>> /// Import any functions requested via the -import option.
>>> static bool importFunctions(const char *argv0, LLVMContext &Context,
>>> Linker &L) {
>>> + StringMap<std::unique_ptr<DenseMap<unsigned, MDNode *>>>
>>> + ModuleToTempMDValsMap;
>>> for (const auto &Import : Imports) {
>>> // Identify the requested function and its bitcode source file.
>>> size_t Idx = Import.find(':');
>>> @@ -159,7 +166,7 @@ static bool importFunctions(const char *
>>> std::string FileName = Import.substr(Idx + 1, std::string::npos);
>>>
>>> // Load the specified source module.
>>> - std::unique_ptr<Module> M = loadFile(argv0, FileName, Context);
>>> + std::unique_ptr<Module> M = loadFile(argv0, FileName, Context, false);
>>> if (!M.get()) {
>>> errs() << argv0 << ": error loading file '" << FileName << "'\n";
>>> return false;
>>> @@ -201,11 +208,39 @@ static bool importFunctions(const char *
>>> Index = std::move(IndexOrErr.get());
>>> }
>>>
>>> + // Save the mapping of value ids to temporary metadata created when
>>> + // importing this function. If we have already imported from this module,
>>> + // add new temporary metadata to the existing mapping.
>>> + auto &TempMDVals = ModuleToTempMDValsMap[FileName];
>>> + if (!TempMDVals)
>>> + TempMDVals = llvm::make_unique<DenseMap<unsigned, MDNode *>>();
>>> +
>>> // Link in the specified function.
>>> DenseSet<const GlobalValue *> FunctionsToImport;
>>> FunctionsToImport.insert(F);
>>> if (L.linkInModule(std::move(M), Linker::Flags::None, Index.get(),
>>> - &FunctionsToImport))
>>> + &FunctionsToImport, TempMDVals.get()))
>>> + return false;
>>> + }
>>> +
>>> + // Now link in metadata for all modules from which we imported functions.
>>> + for (StringMapEntry<std::unique_ptr<DenseMap<unsigned, MDNode *>>> &SME :
>>> + ModuleToTempMDValsMap) {
>>> + // Load the specified source module.
>>> + std::unique_ptr<Module> M = loadFile(argv0, SME.getKey(), Context, true);
>>> + if (!M.get()) {
>>> + errs() << argv0 << ": error loading file '" << SME.getKey() << "'\n";
>>> + return false;
>>> + }
>>> +
>>> + if (verifyModule(*M, &errs())) {
>>> + errs() << argv0 << ": " << SME.getKey()
>>> + << ": error: input module is broken!\n";
>>> + return false;
>>> + }
>>> +
>>> + // Link in all necessary metadata from this module.
>>> + if (L.linkInMetadata(*M, SME.getValue().get()))
>>> return false;
>>> }
>>> return true;
>>>
>>>
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>>
>
>
>
> --
> Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413
--
Teresa Johnson | Software Engineer | tejohnson at google.com | 408-460-2413
More information about the llvm-commits
mailing list