[llvm] a97f628 - [llvm][IR] Add dso_local_equivalent Constant
Leonard Chan via llvm-commits
llvm-commits at lists.llvm.org
Thu Nov 19 10:26:56 PST 2020
Author: Leonard Chan
Date: 2020-11-19T10:26:17-08:00
New Revision: a97f62837f59620a60b3ed9f29568440c7d0553a
URL: https://github.com/llvm/llvm-project/commit/a97f62837f59620a60b3ed9f29568440c7d0553a
DIFF: https://github.com/llvm/llvm-project/commit/a97f62837f59620a60b3ed9f29568440c7d0553a.diff
LOG: [llvm][IR] Add dso_local_equivalent Constant
The `dso_local_equivalent` constant is a wrapper for functions that represents a
value which is functionally equivalent to the global passed to this. That is, if
this accepts a function, calling this constant should have the same effects as
calling the function directly. This could be a direct reference to the function,
the `@plt` modifier on X86/AArch64, a thunk, or anything that's equivalent to the
resolved function as a call target.
When lowered, the returned address must have a constant offset at link time from
some other symbol defined within the same binary. The address of this value is
also insignificant. The name is leveraged from `dso_local` where use of a function
or variable is resolved to a symbol in the same linkage unit.
In this patch:
- Addition of `dso_local_equivalent` and handling it
- Update Constant::needsRelocation() to strip constant inbound GEPs and take
advantage of `dso_local_equivalent` for relative references
This is useful for the [Relative VTables C++ ABI](https://reviews.llvm.org/D72959)
which makes vtables readonly. This works by replacing the dynamic relocations for
function pointers in them with static relocations that represent the offset between
the vtable and virtual functions. If a function is externally defined,
`dso_local_equivalent` can be used as a generic wrapper for the function to still
allow for this static offset calculation to be done.
See [RFC](http://lists.llvm.org/pipermail/llvm-dev/2020-August/144469.html) for more details.
Differential Revision: https://reviews.llvm.org/D77248
Added:
llvm/test/CodeGen/X86/dso_local_equivalent.ll
Modified:
llvm/docs/LangRef.rst
llvm/include/llvm/Analysis/ConstantFolding.h
llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h
llvm/include/llvm/IR/Constants.h
llvm/include/llvm/IR/Value.def
llvm/include/llvm/Target/TargetLoweringObjectFile.h
llvm/lib/Analysis/ConstantFolding.cpp
llvm/lib/AsmParser/LLLexer.cpp
llvm/lib/AsmParser/LLParser.cpp
llvm/lib/AsmParser/LLToken.h
llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
llvm/lib/IR/AsmWriter.cpp
llvm/lib/IR/Constants.cpp
llvm/lib/IR/Core.cpp
llvm/lib/IR/LLVMContextImpl.h
llvm/test/CodeGen/X86/relptr-rodata.ll
Removed:
################################################################################
diff --git a/llvm/docs/LangRef.rst b/llvm/docs/LangRef.rst
index 0a71ba1abf74..e62d4d428997 100644
--- a/llvm/docs/LangRef.rst
+++ b/llvm/docs/LangRef.rst
@@ -3783,6 +3783,43 @@ long as the original value is reconstituted before the ``indirectbr`` or
Finally, some targets may provide defined semantics when using the value
as the operand to an inline assembly, but that is target specific.
+.. _dso_local_equivalent:
+
+DSO Local Equivalent
+--------------------
+
+``dso_local_equivalent @func``
+
+A '``dso_local_equivalent``' constant represents a function which is
+functionally equivalent to a given function, but is always defined in the
+current linkage unit. The resulting pointer has the same type as the underlying
+function. The resulting pointer is permitted, but not required, to be
diff erent
+from a pointer to the function, and it may have
diff erent values in
diff erent
+translation units.
+
+The target function may not have ``extern_weak`` linkage.
+
+``dso_local_equivalent`` can be implemented as such:
+
+- If the function has local linkage, hidden visibility, or is
+ ``dso_local``, ``dso_local_equivalent`` can be implemented as simply a pointer
+ to the function.
+- ``dso_local_equivalent`` can be implemented with a stub that tail-calls the
+ function. Many targets support relocations that resolve at link time to either
+ a function or a stub for it, depending on if the function is defined within the
+ linkage unit; LLVM will use this when available. (This is commonly called a
+ "PLT stub".) On other targets, the stub may need to be emitted explicitly.
+
+This can be used wherever a ``dso_local`` instance of a function is needed without
+needing to explicitly make the original function ``dso_local``. An instance where
+this can be used is for static offset calculations between a function and some other
+``dso_local`` symbol. This is especially useful for the Relative VTables C++ ABI,
+where dynamic relocations for function pointers in VTables can be replaced with
+static relocations for offsets between the VTable and virtual functions which
+may not be ``dso_local``.
+
+This is currently only supported for ELF binary formats.
+
.. _constantexprs:
Constant Expressions
diff --git a/llvm/include/llvm/Analysis/ConstantFolding.h b/llvm/include/llvm/Analysis/ConstantFolding.h
index 0ccc782ad6f5..ef6e66b2b88e 100644
--- a/llvm/include/llvm/Analysis/ConstantFolding.h
+++ b/llvm/include/llvm/Analysis/ConstantFolding.h
@@ -25,6 +25,7 @@ template <typename T> class ArrayRef;
class CallBase;
class Constant;
class ConstantExpr;
+class DSOLocalEquivalent;
class DataLayout;
class Function;
class GlobalValue;
@@ -34,8 +35,11 @@ class Type;
/// If this constant is a constant offset from a global, return the global and
/// the constant. Because of constantexprs, this function is recursive.
+/// If the global is part of a dso_local_equivalent constant, return it through
+/// `Equiv` if it is provided.
bool IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV, APInt &Offset,
- const DataLayout &DL);
+ const DataLayout &DL,
+ DSOLocalEquivalent **DSOEquiv = nullptr);
/// ConstantFoldInstruction - Try to constant fold the specified instruction.
/// If successful, the constant result is returned, if not, null is returned.
diff --git a/llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h b/llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h
index 50488cfdef08..bd4bb1b23680 100644
--- a/llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h
+++ b/llvm/include/llvm/CodeGen/TargetLoweringObjectFileImpl.h
@@ -38,7 +38,7 @@ class TargetLoweringObjectFileELF : public TargetLoweringObjectFile {
const TargetMachine *TM = nullptr;
public:
- TargetLoweringObjectFileELF() = default;
+ TargetLoweringObjectFileELF();
~TargetLoweringObjectFileELF() override = default;
void Initialize(MCContext &Ctx, const TargetMachine &TM) override;
@@ -97,6 +97,9 @@ class TargetLoweringObjectFileELF : public TargetLoweringObjectFile {
const GlobalValue *RHS,
const TargetMachine &TM) const override;
+ const MCExpr *lowerDSOLocalEquivalent(const DSOLocalEquivalent *Equiv,
+ const TargetMachine &TM) const override;
+
MCSection *getSectionForCommandLines() const override;
};
diff --git a/llvm/include/llvm/IR/Constants.h b/llvm/include/llvm/IR/Constants.h
index febcdb4d02c4..88684dc8fafe 100644
--- a/llvm/include/llvm/IR/Constants.h
+++ b/llvm/include/llvm/IR/Constants.h
@@ -888,6 +888,42 @@ struct OperandTraits<BlockAddress> :
DEFINE_TRANSPARENT_OPERAND_ACCESSORS(BlockAddress, Value)
+/// Wrapper for a function that represents a value that
+/// functionally represents the original function. This can be a function,
+/// global alias to a function, or an ifunc.
+class DSOLocalEquivalent final : public Constant {
+ friend class Constant;
+
+ DSOLocalEquivalent(GlobalValue *GV);
+
+ void *operator new(size_t s) { return User::operator new(s, 1); }
+
+ void destroyConstantImpl();
+ Value *handleOperandChangeImpl(Value *From, Value *To);
+
+public:
+ /// Return a DSOLocalEquivalent for the specified global value.
+ static DSOLocalEquivalent *get(GlobalValue *GV);
+
+ /// Transparently provide more efficient getOperand methods.
+ DECLARE_TRANSPARENT_OPERAND_ACCESSORS(Value);
+
+ GlobalValue *getGlobalValue() const {
+ return cast<GlobalValue>(Op<0>().get());
+ }
+
+ /// Methods for support type inquiry through isa, cast, and dyn_cast:
+ static bool classof(const Value *V) {
+ return V->getValueID() == DSOLocalEquivalentVal;
+ }
+};
+
+template <>
+struct OperandTraits<DSOLocalEquivalent>
+ : public FixedNumOperandTraits<DSOLocalEquivalent, 1> {};
+
+DEFINE_TRANSPARENT_OPERAND_ACCESSORS(DSOLocalEquivalent, Value)
+
//===----------------------------------------------------------------------===//
/// A constant value that is initialized with an expression using
/// other constant values.
diff --git a/llvm/include/llvm/IR/Value.def b/llvm/include/llvm/IR/Value.def
index aaf1651979a9..0a14b94a501d 100644
--- a/llvm/include/llvm/IR/Value.def
+++ b/llvm/include/llvm/IR/Value.def
@@ -23,6 +23,11 @@
#error "Missing macro definition of HANDLE_VALUE*"
#endif
+// If the LLVM_C_API macro is set, then values handled via HANDLE_*_EXCLUDE_LLVM_C_API will not be expanded in areas the HANDLE_* macro is used. If it is not set, then HANDLE_*_EXCLUDE_LLVM_C_API values are handled normally as their HANDLE_* counterparts.
+#ifndef LLVM_C_API
+#define LLVM_C_API 0
+#endif
+
#ifndef HANDLE_MEMORY_VALUE
#define HANDLE_MEMORY_VALUE(ValueName) HANDLE_VALUE(ValueName)
#endif
@@ -55,6 +60,15 @@
#define HANDLE_CONSTANT_MARKER(MarkerName, ValueName)
#endif
+#ifndef HANDLE_CONSTANT_EXCLUDE_LLVM_C_API
+#define HANDLE_CONSTANT_EXCLUDE_LLVM_C_API(ValueName) HANDLE_CONSTANT(ValueName)
+#endif
+
+#if LLVM_C_API
+#undef HANDLE_CONSTANT_EXCLUDE_LLVM_C_API
+#define HANDLE_CONSTANT_EXCLUDE_LLVM_C_API(ValueName)
+#endif
+
// Having constant first makes the range check for isa<Constant> faster
// and smaller by one operation.
@@ -65,6 +79,7 @@ HANDLE_GLOBAL_VALUE(GlobalIFunc)
HANDLE_GLOBAL_VALUE(GlobalVariable)
HANDLE_CONSTANT(BlockAddress)
HANDLE_CONSTANT(ConstantExpr)
+HANDLE_CONSTANT_EXCLUDE_LLVM_C_API(DSOLocalEquivalent)
// ConstantAggregate.
HANDLE_CONSTANT(ConstantArray)
@@ -114,3 +129,5 @@ HANDLE_INSTRUCTION(Instruction)
#undef HANDLE_INLINE_ASM_VALUE
#undef HANDLE_VALUE
#undef HANDLE_CONSTANT_MARKER
+#undef HANDLE_CONSTANT_EXCLUDE_LLVM_C_API
+#undef LLVM_C_API
diff --git a/llvm/include/llvm/Target/TargetLoweringObjectFile.h b/llvm/include/llvm/Target/TargetLoweringObjectFile.h
index eb6b683e4d68..edae4bb509ee 100644
--- a/llvm/include/llvm/Target/TargetLoweringObjectFile.h
+++ b/llvm/include/llvm/Target/TargetLoweringObjectFile.h
@@ -38,6 +38,7 @@ class Module;
class SectionKind;
class StringRef;
class TargetMachine;
+class DSOLocalEquivalent;
class TargetLoweringObjectFile : public MCObjectFileInfo {
/// Name-mangler for global names.
@@ -47,6 +48,7 @@ class TargetLoweringObjectFile : public MCObjectFileInfo {
bool SupportIndirectSymViaGOTPCRel = false;
bool SupportGOTPCRelWithOffset = true;
bool SupportDebugThreadLocalLocation = true;
+ bool SupportDSOLocalEquivalentLowering = false;
/// PersonalityEncoding, LSDAEncoding, TTypeEncoding - Some encoding values
/// for EH.
@@ -180,6 +182,17 @@ class TargetLoweringObjectFile : public MCObjectFileInfo {
return nullptr;
}
+ /// Target supports a native lowering of a dso_local_equivalent constant
+ /// without needing to replace it with equivalent IR.
+ bool supportDSOLocalEquivalentLowering() const {
+ return SupportDSOLocalEquivalentLowering;
+ }
+
+ virtual const MCExpr *lowerDSOLocalEquivalent(const DSOLocalEquivalent *Equiv,
+ const TargetMachine &TM) const {
+ return nullptr;
+ }
+
/// Target supports replacing a data "PC"-relative access to a symbol
/// through another symbol, by accessing the later via a GOT entry instead?
bool supportIndirectSymViaGOTPCRel() const {
diff --git a/llvm/lib/Analysis/ConstantFolding.cpp b/llvm/lib/Analysis/ConstantFolding.cpp
index bf445375962d..d68998dcc1e8 100644
--- a/llvm/lib/Analysis/ConstantFolding.cpp
+++ b/llvm/lib/Analysis/ConstantFolding.cpp
@@ -295,7 +295,11 @@ Constant *FoldBitCast(Constant *C, Type *DestTy, const DataLayout &DL) {
/// If this constant is a constant offset from a global, return the global and
/// the constant. Because of constantexprs, this function is recursive.
bool llvm::IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV,
- APInt &Offset, const DataLayout &DL) {
+ APInt &Offset, const DataLayout &DL,
+ DSOLocalEquivalent **DSOEquiv) {
+ if (DSOEquiv)
+ *DSOEquiv = nullptr;
+
// Trivial case, constant is the global.
if ((GV = dyn_cast<GlobalValue>(C))) {
unsigned BitWidth = DL.getIndexTypeSizeInBits(GV->getType());
@@ -303,6 +307,15 @@ bool llvm::IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV,
return true;
}
+ if (auto *FoundDSOEquiv = dyn_cast<DSOLocalEquivalent>(C)) {
+ if (DSOEquiv)
+ *DSOEquiv = FoundDSOEquiv;
+ GV = FoundDSOEquiv->getGlobalValue();
+ unsigned BitWidth = DL.getIndexTypeSizeInBits(GV->getType());
+ Offset = APInt(BitWidth, 0);
+ return true;
+ }
+
// Otherwise, if this isn't a constant expr, bail out.
auto *CE = dyn_cast<ConstantExpr>(C);
if (!CE) return false;
@@ -310,7 +323,8 @@ bool llvm::IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV,
// Look through ptr->int and ptr->ptr casts.
if (CE->getOpcode() == Instruction::PtrToInt ||
CE->getOpcode() == Instruction::BitCast)
- return IsConstantOffsetFromGlobal(CE->getOperand(0), GV, Offset, DL);
+ return IsConstantOffsetFromGlobal(CE->getOperand(0), GV, Offset, DL,
+ DSOEquiv);
// i32* getelementptr ([5 x i32]* @a, i32 0, i32 5)
auto *GEP = dyn_cast<GEPOperator>(CE);
@@ -321,7 +335,8 @@ bool llvm::IsConstantOffsetFromGlobal(Constant *C, GlobalValue *&GV,
APInt TmpOffset(BitWidth, 0);
// If the base isn't a global+constant, we aren't either.
- if (!IsConstantOffsetFromGlobal(CE->getOperand(0), GV, TmpOffset, DL))
+ if (!IsConstantOffsetFromGlobal(CE->getOperand(0), GV, TmpOffset, DL,
+ DSOEquiv))
return false;
// Otherwise, add any offset that our operands provide.
diff --git a/llvm/lib/AsmParser/LLLexer.cpp b/llvm/lib/AsmParser/LLLexer.cpp
index 4dbc3f25f150..b7e3a12cebf6 100644
--- a/llvm/lib/AsmParser/LLLexer.cpp
+++ b/llvm/lib/AsmParser/LLLexer.cpp
@@ -724,6 +724,7 @@ lltok::Kind LLLexer::LexIdentifier() {
KEYWORD(vscale);
KEYWORD(x);
KEYWORD(blockaddress);
+ KEYWORD(dso_local_equivalent);
// Metadata types.
KEYWORD(distinct);
diff --git a/llvm/lib/AsmParser/LLParser.cpp b/llvm/lib/AsmParser/LLParser.cpp
index f3c97c10469b..4f343dc9595d 100644
--- a/llvm/lib/AsmParser/LLParser.cpp
+++ b/llvm/lib/AsmParser/LLParser.cpp
@@ -3491,6 +3491,39 @@ bool LLParser::parseValID(ValID &ID, PerFunctionState *PFS) {
return false;
}
+ case lltok::kw_dso_local_equivalent: {
+ // ValID ::= 'dso_local_equivalent' @foo
+ Lex.Lex();
+
+ ValID Fn;
+
+ if (parseValID(Fn))
+ return true;
+
+ if (Fn.Kind != ValID::t_GlobalID && Fn.Kind != ValID::t_GlobalName)
+ return error(Fn.Loc,
+ "expected global value name in dso_local_equivalent");
+
+ // Try to find the function (but skip it if it's forward-referenced).
+ GlobalValue *GV = nullptr;
+ if (Fn.Kind == ValID::t_GlobalID) {
+ if (Fn.UIntVal < NumberedVals.size())
+ GV = NumberedVals[Fn.UIntVal];
+ } else if (!ForwardRefVals.count(Fn.StrVal)) {
+ GV = M->getNamedValue(Fn.StrVal);
+ }
+
+ assert(GV && "Could not find a corresponding global variable");
+
+ if (!GV->getValueType()->isFunctionTy())
+ return error(Fn.Loc, "expected a function, alias to function, or ifunc "
+ "in dso_local_equivalent");
+
+ ID.ConstantVal = DSOLocalEquivalent::get(GV);
+ ID.Kind = ValID::t_Constant;
+ return false;
+ }
+
case lltok::kw_trunc:
case lltok::kw_zext:
case lltok::kw_sext:
diff --git a/llvm/lib/AsmParser/LLToken.h b/llvm/lib/AsmParser/LLToken.h
index 02df4d0ad8b0..c2b33355024d 100644
--- a/llvm/lib/AsmParser/LLToken.h
+++ b/llvm/lib/AsmParser/LLToken.h
@@ -360,6 +360,7 @@ enum Kind {
kw_extractvalue,
kw_insertvalue,
kw_blockaddress,
+ kw_dso_local_equivalent,
kw_freeze,
diff --git a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
index e76d4044b572..8287a45e6722 100644
--- a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
@@ -2341,6 +2341,9 @@ const MCExpr *AsmPrinter::lowerConstant(const Constant *CV) {
if (const BlockAddress *BA = dyn_cast<BlockAddress>(CV))
return MCSymbolRefExpr::create(GetBlockAddressSymbol(BA), Ctx);
+ if (const auto *Equiv = dyn_cast<DSOLocalEquivalent>(CV))
+ return getObjFileLowering().lowerDSOLocalEquivalent(Equiv, TM);
+
const ConstantExpr *CE = dyn_cast<ConstantExpr>(CV);
if (!CE) {
llvm_unreachable("Unknown constant value to lower!");
@@ -2437,18 +2440,25 @@ const MCExpr *AsmPrinter::lowerConstant(const Constant *CV) {
case Instruction::Sub: {
GlobalValue *LHSGV;
APInt LHSOffset;
+ DSOLocalEquivalent *DSOEquiv;
if (IsConstantOffsetFromGlobal(CE->getOperand(0), LHSGV, LHSOffset,
- getDataLayout())) {
+ getDataLayout(), &DSOEquiv)) {
GlobalValue *RHSGV;
APInt RHSOffset;
if (IsConstantOffsetFromGlobal(CE->getOperand(1), RHSGV, RHSOffset,
getDataLayout())) {
const MCExpr *RelocExpr =
getObjFileLowering().lowerRelativeReference(LHSGV, RHSGV, TM);
- if (!RelocExpr)
+ if (!RelocExpr) {
+ const MCExpr *LHSExpr =
+ MCSymbolRefExpr::create(getSymbol(LHSGV), Ctx);
+ if (DSOEquiv &&
+ getObjFileLowering().supportDSOLocalEquivalentLowering())
+ LHSExpr =
+ getObjFileLowering().lowerDSOLocalEquivalent(DSOEquiv, TM);
RelocExpr = MCBinaryExpr::createSub(
- MCSymbolRefExpr::create(getSymbol(LHSGV), Ctx),
- MCSymbolRefExpr::create(getSymbol(RHSGV), Ctx), Ctx);
+ LHSExpr, MCSymbolRefExpr::create(getSymbol(RHSGV), Ctx), Ctx);
+ }
int64_t Addend = (LHSOffset - RHSOffset).getSExtValue();
if (Addend != 0)
RelocExpr = MCBinaryExpr::createAdd(
diff --git a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
index 24975e23f3b4..375c3e9d5304 100644
--- a/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/SelectionDAGBuilder.cpp
@@ -1514,6 +1514,9 @@ SDValue SelectionDAGBuilder::getValueImpl(const Value *V) {
if (const BlockAddress *BA = dyn_cast<BlockAddress>(C))
return DAG.getBlockAddress(BA, VT);
+ if (const auto *Equiv = dyn_cast<DSOLocalEquivalent>(C))
+ return getValue(Equiv->getGlobalValue());
+
VectorType *VecTy = cast<VectorType>(V->getType());
// Now that we know the number and type of the elements, get that number of
diff --git a/llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp b/llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
index 6f9268fbb4ec..a6c5a26bcd6b 100644
--- a/llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
+++ b/llvm/lib/CodeGen/TargetLoweringObjectFileImpl.cpp
@@ -105,6 +105,11 @@ static void GetObjCImageInfo(Module &M, unsigned &Version, unsigned &Flags,
// ELF
//===----------------------------------------------------------------------===//
+TargetLoweringObjectFileELF::TargetLoweringObjectFileELF()
+ : TargetLoweringObjectFile() {
+ SupportDSOLocalEquivalentLowering = true;
+}
+
void TargetLoweringObjectFileELF::Initialize(MCContext &Ctx,
const TargetMachine &TgtM) {
TargetLoweringObjectFile::Initialize(Ctx, TgtM);
@@ -1007,6 +1012,20 @@ const MCExpr *TargetLoweringObjectFileELF::lowerRelativeReference(
MCSymbolRefExpr::create(TM.getSymbol(RHS), getContext()), getContext());
}
+const MCExpr *TargetLoweringObjectFileELF::lowerDSOLocalEquivalent(
+ const DSOLocalEquivalent *Equiv, const TargetMachine &TM) const {
+ assert(supportDSOLocalEquivalentLowering());
+
+ const auto *GV = Equiv->getGlobalValue();
+
+ // A PLT entry is not needed for dso_local globals.
+ if (GV->isDSOLocal() || GV->isImplicitDSOLocal())
+ return MCSymbolRefExpr::create(TM.getSymbol(GV), getContext());
+
+ return MCSymbolRefExpr::create(TM.getSymbol(GV), PLTRelativeVariantKind,
+ getContext());
+}
+
MCSection *TargetLoweringObjectFileELF::getSectionForCommandLines() const {
// Use ".GCC.command.line" since this feature is to support clang's
// -frecord-gcc-switches which in turn attempts to mimic GCC's switch of the
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index 83443fefcd49..75c25cde623c 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -1455,6 +1455,13 @@ static void WriteConstantInternal(raw_ostream &Out, const Constant *CV,
return;
}
+ if (const auto *Equiv = dyn_cast<DSOLocalEquivalent>(CV)) {
+ Out << "dso_local_equivalent ";
+ WriteAsOperandInternal(Out, Equiv->getGlobalValue(), &TypePrinter, Machine,
+ Context);
+ return;
+ }
+
if (const ConstantArray *CA = dyn_cast<ConstantArray>(CV)) {
Type *ETy = CA->getType()->getElementType();
Out << '[';
diff --git a/llvm/lib/IR/Constants.cpp b/llvm/lib/IR/Constants.cpp
index 3d8555bb44f2..732d6e69ba92 100644
--- a/llvm/lib/IR/Constants.cpp
+++ b/llvm/lib/IR/Constants.cpp
@@ -509,6 +509,9 @@ void llvm::deleteConstant(Constant *C) {
case Constant::BlockAddressVal:
delete static_cast<BlockAddress *>(C);
break;
+ case Constant::DSOLocalEquivalentVal:
+ delete static_cast<DSOLocalEquivalent *>(C);
+ break;
case Constant::UndefValueVal:
delete static_cast<UndefValue *>(C);
break;
@@ -654,10 +657,17 @@ bool Constant::needsRelocation() const {
return false;
// Relative pointers do not need to be dynamically relocated.
- if (auto *LHSGV = dyn_cast<GlobalValue>(LHSOp0->stripPointerCasts()))
- if (auto *RHSGV = dyn_cast<GlobalValue>(RHSOp0->stripPointerCasts()))
+ if (auto *RHSGV =
+ dyn_cast<GlobalValue>(RHSOp0->stripInBoundsConstantOffsets())) {
+ auto *LHS = LHSOp0->stripInBoundsConstantOffsets();
+ if (auto *LHSGV = dyn_cast<GlobalValue>(LHS)) {
if (LHSGV->isDSOLocal() && RHSGV->isDSOLocal())
return false;
+ } else if (isa<DSOLocalEquivalent>(LHS)) {
+ if (RHSGV->isDSOLocal())
+ return false;
+ }
+ }
}
}
}
@@ -1763,6 +1773,54 @@ Value *BlockAddress::handleOperandChangeImpl(Value *From, Value *To) {
return nullptr;
}
+DSOLocalEquivalent *DSOLocalEquivalent::get(GlobalValue *GV) {
+ DSOLocalEquivalent *&Equiv = GV->getContext().pImpl->DSOLocalEquivalents[GV];
+ if (!Equiv)
+ Equiv = new DSOLocalEquivalent(GV);
+
+ assert(Equiv->getGlobalValue() == GV &&
+ "DSOLocalFunction does not match the expected global value");
+ return Equiv;
+}
+
+DSOLocalEquivalent::DSOLocalEquivalent(GlobalValue *GV)
+ : Constant(GV->getType(), Value::DSOLocalEquivalentVal, &Op<0>(), 1) {
+ setOperand(0, GV);
+}
+
+/// Remove the constant from the constant table.
+void DSOLocalEquivalent::destroyConstantImpl() {
+ const GlobalValue *GV = getGlobalValue();
+ GV->getContext().pImpl->DSOLocalEquivalents.erase(GV);
+}
+
+Value *DSOLocalEquivalent::handleOperandChangeImpl(Value *From, Value *To) {
+ assert(From == getGlobalValue() && "Changing value does not match operand.");
+ assert(To->getType() == getType() && "Mismatched types");
+ assert(isa<Constant>(To) && "Can only replace the operands with a constant");
+
+ // The replacement is with another global value.
+ if (const auto *ToObj = dyn_cast<GlobalValue>(To)) {
+ DSOLocalEquivalent *&NewEquiv =
+ getContext().pImpl->DSOLocalEquivalents[ToObj];
+ if (NewEquiv)
+ return NewEquiv;
+ }
+
+ // The replacement could be a bitcast or an alias to another function. We can
+ // replace it with a bitcast to the dso_local_equivalent of that function.
+ auto *Func = cast<Function>(To->stripPointerCastsAndAliases());
+ DSOLocalEquivalent *&NewEquiv = getContext().pImpl->DSOLocalEquivalents[Func];
+ if (NewEquiv)
+ return llvm::ConstantExpr::getBitCast(NewEquiv, getType());
+
+ // Replace this with the new one.
+ getContext().pImpl->DSOLocalEquivalents.erase(getGlobalValue());
+ NewEquiv = this;
+ setOperand(0, Func);
+ return nullptr;
+}
+
//---- ConstantExpr::get() implementations.
//
diff --git a/llvm/lib/IR/Core.cpp b/llvm/lib/IR/Core.cpp
index 256317cf354d..28ada52ab911 100644
--- a/llvm/lib/IR/Core.cpp
+++ b/llvm/lib/IR/Core.cpp
@@ -826,6 +826,7 @@ LLVMTypeRef LLVMTypeOf(LLVMValueRef Val) {
LLVMValueKind LLVMGetValueKind(LLVMValueRef Val) {
switch(unwrap(Val)->getValueID()) {
+#define LLVM_C_API 1
#define HANDLE_VALUE(Name) \
case Value::Name##Val: \
return LLVM##Name##ValueKind;
diff --git a/llvm/lib/IR/LLVMContextImpl.h b/llvm/lib/IR/LLVMContextImpl.h
index dc5fa1cdf33f..8f1b2de1e61b 100644
--- a/llvm/lib/IR/LLVMContextImpl.h
+++ b/llvm/lib/IR/LLVMContextImpl.h
@@ -1380,6 +1380,9 @@ class LLVMContextImpl {
DenseMap<std::pair<const Function *, const BasicBlock *>, BlockAddress *>
BlockAddresses;
+
+ DenseMap<const GlobalValue *, DSOLocalEquivalent *> DSOLocalEquivalents;
+
ConstantUniqueMap<ConstantExpr> ExprConstants;
ConstantUniqueMap<InlineAsm> InlineAsms;
diff --git a/llvm/test/CodeGen/X86/dso_local_equivalent.ll b/llvm/test/CodeGen/X86/dso_local_equivalent.ll
new file mode 100644
index 000000000000..e4a5fdfa9046
--- /dev/null
+++ b/llvm/test/CodeGen/X86/dso_local_equivalent.ll
@@ -0,0 +1,99 @@
+; RUN: llc -mtriple=x86_64-linux-gnu -relocation-model=pic -data-sections -o - %s --asm-verbose=0 | FileCheck %s -check-prefixes=CHECK
+
+; Just ensure that we can write to an object file without error.
+; RUN: llc -filetype=obj -mtriple=x86_64-linux-gnu -relocation-model=pic -data-sections -o /dev/null %s
+
+declare void @extern_func()
+
+; CHECK: call_extern_func:
+; CHECK: callq extern_func at PLT
+define void @call_extern_func() {
+ call void dso_local_equivalent @extern_func()
+ ret void
+}
+
+declare hidden void @hidden_func()
+declare protected void @protected_func()
+declare dso_local void @dso_local_func()
+define internal void @internal_func() {
+entry:
+ ret void
+}
+define private void @private_func() {
+entry:
+ ret void
+}
+
+; CHECK: call_hidden_func:
+; CHECK: callq hidden_func{{$}}
+define void @call_hidden_func() {
+ call void dso_local_equivalent @hidden_func()
+ ret void
+}
+
+; CHECK: call_protected_func:
+; CHECK: callq protected_func{{$}}
+define void @call_protected_func() {
+ call void dso_local_equivalent @protected_func()
+ ret void
+}
+
+; CHECK: call_dso_local_func:
+; CHECK: callq dso_local_func{{$}}
+define void @call_dso_local_func() {
+ call void dso_local_equivalent @dso_local_func()
+ ret void
+}
+
+; CHECK: call_internal_func:
+; CHECK: callq internal_func{{$}}
+define void @call_internal_func() {
+ call void dso_local_equivalent @internal_func()
+ ret void
+}
+
+define void @aliasee_func() {
+entry:
+ ret void
+}
+
+ at alias_func = alias void (), void ()* @aliasee_func
+ at dso_local_alias_func = dso_local alias void (), void ()* @aliasee_func
+
+; CHECK: call_alias_func:
+; CHECK: callq alias_func at PLT
+define void @call_alias_func() {
+ call void dso_local_equivalent @alias_func()
+ ret void
+}
+
+; CHECK: call_dso_local_alias_func:
+; CHECK: callq .Ldso_local_alias_func$local{{$}}
+define void @call_dso_local_alias_func() {
+ call void dso_local_equivalent @dso_local_alias_func()
+ ret void
+}
+
+ at ifunc_func = ifunc void (), i64 ()* @resolver
+ at dso_local_ifunc_func = dso_local ifunc void (), i64 ()* @resolver
+
+define internal i64 @resolver() {
+entry:
+ ret i64 0
+}
+
+; If an ifunc is not dso_local already, then we should still emit a stub for it
+; to ensure it will be dso_local.
+; CHECK: call_ifunc_func:
+; CHECK: callq ifunc_func at PLT
+define void @call_ifunc_func() {
+ call void dso_local_equivalent @ifunc_func()
+ ret void
+}
+
+; CHECK: call_dso_local_ifunc_func:
+; CHECK: callq dso_local_ifunc_func{{$}}
+define void @call_dso_local_ifunc_func() {
+ call void dso_local_equivalent @dso_local_ifunc_func()
+ ret void
+}
diff --git a/llvm/test/CodeGen/X86/relptr-rodata.ll b/llvm/test/CodeGen/X86/relptr-rodata.ll
index b2635abab7dd..6dc2ed914a96 100644
--- a/llvm/test/CodeGen/X86/relptr-rodata.ll
+++ b/llvm/test/CodeGen/X86/relptr-rodata.ll
@@ -19,3 +19,27 @@ target triple = "x86_64-unknown-linux-gnu"
; CHECK: relro2:
; CHECK: .long hidden-relro2
@relro2 = constant i32 trunc (i64 sub (i64 ptrtoint (i8* @hidden to i64), i64 ptrtoint (i32* @relro2 to i64)) to i32)
+
+; CHECK: .section .rodata.cst8
+; CHECK-NEXT: .globl obj
+; CHECK: obj:
+; CHECK: .long 0
+; CHECK: .long (hidden_func-obj)-4
+
+declare hidden void @hidden_func()
+
+; Ensure that inbound GEPs with constant offsets are also resolved.
+ at obj = dso_local unnamed_addr constant { { i32, i32 } } {
+ { i32, i32 } {
+ i32 0,
+ i32 trunc (i64 sub (i64 ptrtoint (void ()* dso_local_equivalent @hidden_func to i64), i64 ptrtoint (i32* getelementptr inbounds ({ { i32, i32 } }, { { i32, i32 } }* @obj, i32 0, i32 0, i32 1) to i64)) to i32)
+ } }, align 4
+
+; CHECK: .section .rodata.rodata2
+; CHECK-NEXT: .globl rodata2
+; CHECK: rodata2:
+; CHECK: .long extern_func at PLT-rodata2
+
+declare void @extern_func()
+
+ at rodata2 = dso_local constant i32 trunc (i64 sub (i64 ptrtoint (void ()* dso_local_equivalent @extern_func to i64), i64 ptrtoint (i32* @rodata2 to i64)) to i32)
More information about the llvm-commits
mailing list