r246985 - Compute and preserve alignment more faithfully in IR-generation.
John McCall via cfe-commits
cfe-commits at lists.llvm.org
Tue Sep 8 17:35:38 PDT 2015
> On Sep 8, 2015, at 5:14 PM, Steven Wu <stevenwu at apple.com> wrote:
> Hi John
>
> It seems this commit breaks arm “vld1_lane” intrinsics. When emitting “vld1_lane” Ops[0] gets updated so PtrOp0 no longer have the same type as Ops[0].
Sorry about that.
> Here a patch for the fix. Can you review it?
Slight tweak:
> From daea3a26c6df07530407318e4f0819e6e2ff9aea Mon Sep 17 00:00:00 2001
> From: Steven Wu <stevenwu at apple.com>
> Date: Tue, 8 Sep 2015 17:10:25 -0700
> Subject: [PATCH] Fix assertion when emiting NEON builtin
>
> ---
> lib/CodeGen/CGBuiltin.cpp | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/lib/CodeGen/CGBuiltin.cpp b/lib/CodeGen/CGBuiltin.cpp
> index 729c0a1..ed3f9f6 100644
> --- a/lib/CodeGen/CGBuiltin.cpp
> +++ b/lib/CodeGen/CGBuiltin.cpp
> @@ -3772,6 +3772,7 @@ Value *CodeGenFunction::EmitARMBuiltinExpr(unsigned BuiltinID,
> Ops[1] = Builder.CreateBitCast(Ops[1], Ty);
> Ty = llvm::PointerType::getUnqual(VTy->getElementType());
> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
Please remove these two lines and replace them with:
PtrOp0 = Builder.CreateElementBitCast(PtrOp0, VTy);
You then shouldn’t need this line:
> + PtrOp0 = Address(Ops[0], PtrOp0.getAlignment());
Thanks!
John.
> Value *Ld = Builder.CreateLoad(PtrOp0);
> return Builder.CreateInsertElement(Ops[1], Ld, Ops[2], "vld1_lane");
> }
> --
> 2.3.8 (Apple Git-58)
>
>
>
>> On Sep 8, 2015, at 1:06 AM, John McCall via cfe-commits <cfe-commits at lists.llvm.org> wrote:
>>
>> Author: rjmccall
>> Date: Tue Sep 8 03:05:57 2015
>> New Revision: 246985
>>
>> URL: http://llvm.org/viewvc/llvm-project?rev=246985&view=rev
>> Log:
>> Compute and preserve alignment more faithfully in IR-generation.
>>
>> Introduce an Address type to bundle a pointer value with an
>> alignment. Introduce APIs on CGBuilderTy to work with Address
>> values. Change core APIs on CGF/CGM to traffic in Address where
>> appropriate. Require alignments to be non-zero. Update a ton
>> of code to compute and propagate alignment information.
>>
>> As part of this, I've promoted CGBuiltin's EmitPointerWithAlignment
>> helper function to CGF and made use of it in a number of places in
>> the expression emitter.
>>
>> The end result is that we should now be significantly more correct
>> when performing operations on objects that are locally known to
>> be under-aligned. Since alignment is not reliably tracked in the
>> type system, there are inherent limits to this, but at least we
>> are no longer confused by standard operations like derived-to-base
>> conversions and array-to-pointer decay. I've also fixed a large
>> number of bugs where we were applying the complete-object alignment
>> to a pointer instead of the non-virtual alignment, although most of
>> these were hidden by the very conservative approach we took with
>> member alignment.
>>
>> Also, because IRGen now reliably asserts on zero alignments, we
>> should no longer be subject to an absurd but frustrating recurring
>> bug where an incomplete type would report a zero alignment and then
>> we'd naively do a alignmentAtOffset on it and emit code using an
>> alignment equal to the largest power-of-two factor of the offset.
>>
>> We should also now be emitting much more aggressive alignment
>> attributes in the presence of over-alignment. In particular,
>> field access now uses alignmentAtOffset instead of min.
>>
>> Several times in this patch, I had to change the existing
>> code-generation pattern in order to more effectively use
>> the Address APIs. For the most part, this seems to be a strict
>> improvement, like doing pointer arithmetic with GEPs instead of
>> ptrtoint. That said, I've tried very hard to not change semantics,
>> but it is likely that I've failed in a few places, for which I
>> apologize.
>>
>> ABIArgInfo now always carries the assumed alignment of indirect and
>> indirect byval arguments. In order to cut down on what was already
>> a dauntingly large patch, I changed the code to never set align
>> attributes in the IR on non-byval indirect arguments. That is,
>> we still generate code which assumes that indirect arguments have
>> the given alignment, but we don't express this information to the
>> backend except where it's semantically required (i.e. on byvals).
>> This is likely a minor regression for those targets that did provide
>> this information, but it'll be trivial to add it back in a later
>> patch.
>>
>> I partially punted on applying this work to CGBuiltin. Please
>> do not add more uses of the CreateDefaultAligned{Load,Store}
>> APIs; they will be going away eventually.
>>
>> Added:
>> cfe/trunk/lib/CodeGen/Address.h
>> cfe/trunk/lib/CodeGen/CodeGenTypeCache.h
>> cfe/trunk/test/CodeGenCXX/alignment.cpp
>> Modified:
>> cfe/trunk/include/clang/AST/CharUnits.h
>> cfe/trunk/include/clang/CodeGen/CGFunctionInfo.h
>> cfe/trunk/lib/CodeGen/ABIInfo.h
>> cfe/trunk/lib/CodeGen/CGAtomic.cpp
>> cfe/trunk/lib/CodeGen/CGBlocks.cpp
>> cfe/trunk/lib/CodeGen/CGBlocks.h
>> cfe/trunk/lib/CodeGen/CGBuilder.h
>> cfe/trunk/lib/CodeGen/CGBuiltin.cpp
>> cfe/trunk/lib/CodeGen/CGCUDANV.cpp
>> cfe/trunk/lib/CodeGen/CGCXX.cpp
>> cfe/trunk/lib/CodeGen/CGCXXABI.cpp
>> cfe/trunk/lib/CodeGen/CGCXXABI.h
>> cfe/trunk/lib/CodeGen/CGCall.cpp
>> cfe/trunk/lib/CodeGen/CGCall.h
>> cfe/trunk/lib/CodeGen/CGClass.cpp
>> cfe/trunk/lib/CodeGen/CGCleanup.cpp
>> cfe/trunk/lib/CodeGen/CGCleanup.h
>> cfe/trunk/lib/CodeGen/CGDecl.cpp
>> cfe/trunk/lib/CodeGen/CGDeclCXX.cpp
>> cfe/trunk/lib/CodeGen/CGException.cpp
>> cfe/trunk/lib/CodeGen/CGExpr.cpp
>> cfe/trunk/lib/CodeGen/CGExprAgg.cpp
>> cfe/trunk/lib/CodeGen/CGExprCXX.cpp
>> cfe/trunk/lib/CodeGen/CGExprComplex.cpp
>> cfe/trunk/lib/CodeGen/CGExprConstant.cpp
>> cfe/trunk/lib/CodeGen/CGExprScalar.cpp
>> cfe/trunk/lib/CodeGen/CGObjC.cpp
>> cfe/trunk/lib/CodeGen/CGObjCGNU.cpp
>> cfe/trunk/lib/CodeGen/CGObjCMac.cpp
>> cfe/trunk/lib/CodeGen/CGObjCRuntime.cpp
>> cfe/trunk/lib/CodeGen/CGObjCRuntime.h
>> cfe/trunk/lib/CodeGen/CGOpenMPRuntime.cpp
>> cfe/trunk/lib/CodeGen/CGOpenMPRuntime.h
>> cfe/trunk/lib/CodeGen/CGStmt.cpp
>> cfe/trunk/lib/CodeGen/CGStmtOpenMP.cpp
>> cfe/trunk/lib/CodeGen/CGVTables.cpp
>> cfe/trunk/lib/CodeGen/CGValue.h
>> cfe/trunk/lib/CodeGen/CodeGenFunction.cpp
>> cfe/trunk/lib/CodeGen/CodeGenFunction.h
>> cfe/trunk/lib/CodeGen/CodeGenModule.cpp
>> cfe/trunk/lib/CodeGen/CodeGenModule.h
>> cfe/trunk/lib/CodeGen/ItaniumCXXABI.cpp
>> cfe/trunk/lib/CodeGen/MicrosoftCXXABI.cpp
>> cfe/trunk/lib/CodeGen/TargetInfo.cpp
>> cfe/trunk/test/CodeGen/aarch64-varargs.c
>> cfe/trunk/test/CodeGen/arm-abi-vector.c
>> cfe/trunk/test/CodeGen/arm-arguments.c
>> cfe/trunk/test/CodeGen/arm64-abi-vector.c
>> cfe/trunk/test/CodeGen/arm64-arguments.c
>> cfe/trunk/test/CodeGen/arm64-be-hfa-vararg.c
>> cfe/trunk/test/CodeGen/atomic-arm64.c
>> cfe/trunk/test/CodeGen/block-byref-aggr.c
>> cfe/trunk/test/CodeGen/c11atomics-ios.c
>> cfe/trunk/test/CodeGen/c11atomics.c
>> cfe/trunk/test/CodeGen/catch-undef-behavior.c
>> cfe/trunk/test/CodeGen/exprs.c
>> cfe/trunk/test/CodeGen/ext-vector-member-alignment.c
>> cfe/trunk/test/CodeGen/mips-varargs.c
>> cfe/trunk/test/CodeGen/object-size.c
>> cfe/trunk/test/CodeGen/packed-arrays.c
>> cfe/trunk/test/CodeGen/packed-structure.c
>> cfe/trunk/test/CodeGen/ppc-varargs-struct.c
>> cfe/trunk/test/CodeGen/ppc64-align-struct.c
>> cfe/trunk/test/CodeGen/ppc64-complex-parms.c
>> cfe/trunk/test/CodeGen/ppc64-struct-onefloat.c
>> cfe/trunk/test/CodeGen/ppc64-varargs-complex.c
>> cfe/trunk/test/CodeGen/ppc64le-varargs-complex.c
>> cfe/trunk/test/CodeGen/sparcv9-abi.c
>> cfe/trunk/test/CodeGen/tbaa-class.cpp
>> cfe/trunk/test/CodeGen/tbaa.cpp
>> cfe/trunk/test/CodeGen/vectorcall.c
>> cfe/trunk/test/CodeGen/xcore-abi.c
>> cfe/trunk/test/CodeGenCXX/arm.cpp
>> cfe/trunk/test/CodeGenCXX/cxx0x-initializer-stdinitializerlist.cpp
>> cfe/trunk/test/CodeGenCXX/cxx11-initializer-array-new.cpp
>> cfe/trunk/test/CodeGenCXX/delete-two-arg.cpp
>> cfe/trunk/test/CodeGenCXX/lambda-expressions.cpp
>> cfe/trunk/test/CodeGenCXX/microsoft-abi-array-cookies.cpp
>> cfe/trunk/test/CodeGenCXX/microsoft-abi-eh-cleanups.cpp
>> cfe/trunk/test/CodeGenCXX/microsoft-abi-structors.cpp
>> cfe/trunk/test/CodeGenCXX/microsoft-abi-try-throw.cpp
>> cfe/trunk/test/CodeGenCXX/microsoft-abi-virtual-inheritance-vtordisps.cpp
>> cfe/trunk/test/CodeGenCXX/microsoft-abi-virtual-inheritance.cpp
>> cfe/trunk/test/CodeGenCXX/static-init-wasm.cpp
>> cfe/trunk/test/CodeGenCXX/static-init.cpp
>> cfe/trunk/test/CodeGenCXX/vararg-non-pod-ms-compat.cpp
>> cfe/trunk/test/CodeGenCXX/wasm-args-returns.cpp
>> cfe/trunk/test/CodeGenObjC/arc-captured-32bit-block-var-layout-2.m
>> cfe/trunk/test/CodeGenObjC/arc-captured-32bit-block-var-layout.m
>> cfe/trunk/test/CodeGenObjC/arc-captured-block-var-inlined-layout.m
>> cfe/trunk/test/CodeGenObjC/arc-captured-block-var-layout.m
>> cfe/trunk/test/CodeGenObjC/arc-literals.m
>> cfe/trunk/test/CodeGenObjC/arc.m
>> cfe/trunk/test/CodeGenObjC/debug-info-block-captured-self.m
>> cfe/trunk/test/CodeGenObjC/ivar-base-as-invariant-load.m
>> cfe/trunk/test/CodeGenObjC/ivar-invariant.m
>> cfe/trunk/test/CodeGenObjC/mrr-captured-block-var-inlined-layout.m
>> cfe/trunk/test/CodeGenObjC/selector-ref-invariance.m
>> cfe/trunk/test/CodeGenObjCXX/arc-new-delete.mm
>> cfe/trunk/test/CodeGenObjCXX/literals.mm
>> cfe/trunk/test/CodeGenObjCXX/property-lvalue-capture.mm
>> cfe/trunk/test/CodeGenObjCXX/property-object-conditional-exp.mm
>> cfe/trunk/test/OpenMP/for_reduction_codegen.cpp
>> cfe/trunk/test/OpenMP/parallel_codegen.cpp
>> cfe/trunk/test/OpenMP/parallel_reduction_codegen.cpp
>> cfe/trunk/test/OpenMP/sections_reduction_codegen.cpp
>> cfe/trunk/test/OpenMP/task_codegen.cpp
>>
>> Modified: cfe/trunk/include/clang/AST/CharUnits.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/AST/CharUnits.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/include/clang/AST/CharUnits.h (original)
>> +++ cfe/trunk/include/clang/AST/CharUnits.h Tue Sep 8 03:05:57 2015
>> @@ -130,6 +130,14 @@ namespace clang {
>> return (Quantity & -Quantity) == Quantity;
>> }
>>
>> + /// Test whether this is a multiple of the other value.
>> + ///
>> + /// Among other things, this promises that
>> + /// self.RoundUpToAlignment(N) will just return self.
>> + bool isMultipleOf(CharUnits N) const {
>> + return (*this % N) == 0;
>> + }
>> +
>> // Arithmetic operators.
>> CharUnits operator* (QuantityType N) const {
>> return CharUnits(Quantity * N);
>> @@ -172,10 +180,20 @@ namespace clang {
>>
>> /// Given that this is a non-zero alignment value, what is the
>> /// alignment at the given offset?
>> - CharUnits alignmentAtOffset(CharUnits offset) {
>> + CharUnits alignmentAtOffset(CharUnits offset) const {
>> + assert(Quantity != 0 && "offsetting from unknown alignment?");
>> return CharUnits(llvm::MinAlign(Quantity, offset.Quantity));
>> }
>>
>> + /// Given that this is the alignment of the first element of an
>> + /// array, return the minimum alignment of any element in the array.
>> + CharUnits alignmentOfArrayElement(CharUnits elementSize) const {
>> + // Since we don't track offsetted alignments, the alignment of
>> + // the second element (or any odd element) will be minimally
>> + // aligned.
>> + return alignmentAtOffset(elementSize);
>> + }
>> +
>>
>> }; // class CharUnit
>> } // namespace clang
>>
>> Modified: cfe/trunk/include/clang/CodeGen/CGFunctionInfo.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/CodeGen/CGFunctionInfo.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/include/clang/CodeGen/CGFunctionInfo.h (original)
>> +++ cfe/trunk/include/clang/CodeGen/CGFunctionInfo.h Tue Sep 8 03:05:57 2015
>> @@ -17,6 +17,7 @@
>> #define LLVM_CLANG_CODEGEN_CGFUNCTIONINFO_H
>>
>> #include "clang/AST/CanonicalType.h"
>> +#include "clang/AST/CharUnits.h"
>> #include "clang/AST/Type.h"
>> #include "llvm/ADT/FoldingSet.h"
>> #include <cassert>
>> @@ -126,7 +127,7 @@ public:
>> static ABIArgInfo getIgnore() {
>> return ABIArgInfo(Ignore);
>> }
>> - static ABIArgInfo getIndirect(unsigned Alignment, bool ByVal = true,
>> + static ABIArgInfo getIndirect(CharUnits Alignment, bool ByVal = true,
>> bool Realign = false,
>> llvm::Type *Padding = nullptr) {
>> auto AI = ABIArgInfo(Indirect);
>> @@ -137,7 +138,7 @@ public:
>> AI.setPaddingType(Padding);
>> return AI;
>> }
>> - static ABIArgInfo getIndirectInReg(unsigned Alignment, bool ByVal = true,
>> + static ABIArgInfo getIndirectInReg(CharUnits Alignment, bool ByVal = true,
>> bool Realign = false) {
>> auto AI = getIndirect(Alignment, ByVal, Realign);
>> AI.setInReg(true);
>> @@ -211,20 +212,20 @@ public:
>> }
>>
>> // Indirect accessors
>> - unsigned getIndirectAlign() const {
>> + CharUnits getIndirectAlign() const {
>> assert(isIndirect() && "Invalid kind!");
>> - return IndirectAlign;
>> + return CharUnits::fromQuantity(IndirectAlign);
>> }
>> - void setIndirectAlign(unsigned IA) {
>> + void setIndirectAlign(CharUnits IA) {
>> assert(isIndirect() && "Invalid kind!");
>> - IndirectAlign = IA;
>> + IndirectAlign = IA.getQuantity();
>> }
>>
>> bool getIndirectByVal() const {
>> assert(isIndirect() && "Invalid kind!");
>> return IndirectByVal;
>> }
>> - void setIndirectByVal(unsigned IBV) {
>> + void setIndirectByVal(bool IBV) {
>> assert(isIndirect() && "Invalid kind!");
>> IndirectByVal = IBV;
>> }
>> @@ -370,6 +371,7 @@ class CGFunctionInfo : public llvm::Fold
>> /// The struct representing all arguments passed in memory. Only used when
>> /// passing non-trivial types with inalloca. Not part of the profile.
>> llvm::StructType *ArgStruct;
>> + unsigned ArgStructAlign;
>>
>> unsigned NumArgs;
>> ArgInfo *getArgsBuffer() {
>> @@ -463,7 +465,13 @@ public:
>>
>> /// \brief Get the struct type used to represent all the arguments in memory.
>> llvm::StructType *getArgStruct() const { return ArgStruct; }
>> - void setArgStruct(llvm::StructType *Ty) { ArgStruct = Ty; }
>> + CharUnits getArgStructAlignment() const {
>> + return CharUnits::fromQuantity(ArgStructAlign);
>> + }
>> + void setArgStruct(llvm::StructType *Ty, CharUnits Align) {
>> + ArgStruct = Ty;
>> + ArgStructAlign = Align.getQuantity();
>> + }
>>
>> void Profile(llvm::FoldingSetNodeID &ID) {
>> ID.AddInteger(getASTCallingConvention());
>>
>> Modified: cfe/trunk/lib/CodeGen/ABIInfo.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/ABIInfo.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/ABIInfo.h (original)
>> +++ cfe/trunk/lib/CodeGen/ABIInfo.h Tue Sep 8 03:05:57 2015
>> @@ -25,6 +25,8 @@ namespace clang {
>> class TargetInfo;
>>
>> namespace CodeGen {
>> + class ABIArgInfo;
>> + class Address;
>> class CGCXXABI;
>> class CGFunctionInfo;
>> class CodeGenFunction;
>> @@ -79,8 +81,9 @@ namespace clang {
>> // the ABI information any lower than CodeGen. Of course, for
>> // VAArg handling it has to be at this level; there is no way to
>> // abstract this out.
>> - virtual llvm::Value *EmitVAArg(llvm::Value *VAListAddr, QualType Ty,
>> - CodeGen::CodeGenFunction &CGF) const = 0;
>> + virtual CodeGen::Address EmitVAArg(CodeGen::CodeGenFunction &CGF,
>> + CodeGen::Address VAListAddr,
>> + QualType Ty) const = 0;
>>
>> virtual bool isHomogeneousAggregateBaseType(QualType Ty) const;
>>
>> @@ -92,6 +95,15 @@ namespace clang {
>> bool isHomogeneousAggregate(QualType Ty, const Type *&Base,
>> uint64_t &Members) const;
>>
>> + /// A convenience method to return an indirect ABIArgInfo with an
>> + /// expected alignment equal to the ABI alignment of the given type.
>> + CodeGen::ABIArgInfo
>> + getNaturalAlignIndirect(QualType Ty, bool ByRef = true,
>> + bool Realign = false,
>> + llvm::Type *Padding = nullptr) const;
>> +
>> + CodeGen::ABIArgInfo
>> + getNaturalAlignIndirectInReg(QualType Ty, bool Realign = false) const;
>> };
>> } // end namespace clang
>>
>>
>> Added: cfe/trunk/lib/CodeGen/Address.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/Address.h?rev=246985&view=auto
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/Address.h (added)
>> +++ cfe/trunk/lib/CodeGen/Address.h Tue Sep 8 03:05:57 2015
>> @@ -0,0 +1,119 @@
>> +//===-- Address.h - An aligned address -------------------------*- C++ -*-===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// This class provides a simple wrapper for a pair of a pointer and an
>> +// alignment.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef LLVM_CLANG_LIB_CODEGEN_ADDRESS_H
>> +#define LLVM_CLANG_LIB_CODEGEN_ADDRESS_H
>> +
>> +#include "llvm/IR/Constants.h"
>> +#include "clang/AST/CharUnits.h"
>> +
>> +namespace clang {
>> +namespace CodeGen {
>> +
>> +/// An aligned address.
>> +class Address {
>> + llvm::Value *Pointer;
>> + CharUnits Alignment;
>> +public:
>> + Address(llvm::Value *pointer, CharUnits alignment)
>> + : Pointer(pointer), Alignment(alignment) {
>> + assert((!alignment.isZero() || pointer == nullptr) &&
>> + "creating valid address with invalid alignment");
>> + }
>> +
>> + static Address invalid() { return Address(nullptr, CharUnits()); }
>> + bool isValid() const { return Pointer != nullptr; }
>> +
>> + llvm::Value *getPointer() const {
>> + assert(isValid());
>> + return Pointer;
>> + }
>> +
>> + /// Return the type of the pointer value.
>> + llvm::PointerType *getType() const {
>> + return llvm::cast<llvm::PointerType>(getPointer()->getType());
>> + }
>> +
>> + /// Return the type of the values stored in this address.
>> + ///
>> + /// When IR pointer types lose their element type, we should simply
>> + /// store it in Address instead for the convenience of writing code.
>> + llvm::Type *getElementType() const {
>> + return getType()->getElementType();
>> + }
>> +
>> + /// Return the address space that this address resides in.
>> + unsigned getAddressSpace() const {
>> + return getType()->getAddressSpace();
>> + }
>> +
>> + /// Return the IR name of the pointer value.
>> + llvm::StringRef getName() const {
>> + return getPointer()->getName();
>> + }
>> +
>> + /// Return the alignment of this pointer.
>> + CharUnits getAlignment() const {
>> + assert(isValid());
>> + return Alignment;
>> + }
>> +};
>> +
>> +/// A specialization of Address that requires the address to be an
>> +/// LLVM Constant.
>> +class ConstantAddress : public Address {
>> +public:
>> + ConstantAddress(llvm::Constant *pointer, CharUnits alignment)
>> + : Address(pointer, alignment) {}
>> +
>> + static ConstantAddress invalid() {
>> + return ConstantAddress(nullptr, CharUnits());
>> + }
>> +
>> + llvm::Constant *getPointer() const {
>> + return llvm::cast<llvm::Constant>(Address::getPointer());
>> + }
>> +
>> + ConstantAddress getBitCast(llvm::Type *ty) const {
>> + return ConstantAddress(llvm::ConstantExpr::getBitCast(getPointer(), ty),
>> + getAlignment());
>> + }
>> +
>> + ConstantAddress getElementBitCast(llvm::Type *ty) const {
>> + return getBitCast(ty->getPointerTo(getAddressSpace()));
>> + }
>> +
>> + static bool isaImpl(Address addr) {
>> + return llvm::isa<llvm::Constant>(addr.getPointer());
>> + }
>> + static ConstantAddress castImpl(Address addr) {
>> + return ConstantAddress(llvm::cast<llvm::Constant>(addr.getPointer()),
>> + addr.getAlignment());
>> + }
>> +};
>> +
>> +}
>> +}
>> +
>> +namespace llvm {
>> + // Present a minimal LLVM-like casting interface.
>> + template <class U> inline U cast(clang::CodeGen::Address addr) {
>> + return U::castImpl(addr);
>> + }
>> + template <class U> inline bool isa(clang::CodeGen::Address addr) {
>> + return U::isaImpl(addr);
>> + }
>> +}
>> +
>> +#endif
>>
>> Modified: cfe/trunk/lib/CodeGen/CGAtomic.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGAtomic.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGAtomic.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGAtomic.cpp Tue Sep 8 03:05:57 2015
>> @@ -80,7 +80,7 @@ namespace {
>> AtomicSizeInBits = C.toBits(
>> C.toCharUnitsFromBits(Offset + OrigBFI.Size + C.getCharWidth() - 1)
>> .RoundUpToAlignment(lvalue.getAlignment()));
>> - auto VoidPtrAddr = CGF.EmitCastToVoidPtr(lvalue.getBitFieldAddr());
>> + auto VoidPtrAddr = CGF.EmitCastToVoidPtr(lvalue.getBitFieldPointer());
>> auto OffsetInChars =
>> (C.toCharUnitsFromBits(OrigBFI.Offset) / lvalue.getAlignment()) *
>> lvalue.getAlignment();
>> @@ -94,8 +94,9 @@ namespace {
>> BFI.Offset = Offset;
>> BFI.StorageSize = AtomicSizeInBits;
>> BFI.StorageOffset += OffsetInChars;
>> - LVal = LValue::MakeBitfield(Addr, BFI, lvalue.getType(),
>> - lvalue.getAlignment());
>> + LVal = LValue::MakeBitfield(Address(Addr, lvalue.getAlignment()),
>> + BFI, lvalue.getType(),
>> + lvalue.getAlignmentSource());
>> LVal.setTBAAInfo(lvalue.getTBAAInfo());
>> AtomicTy = C.getIntTypeForBitwidth(AtomicSizeInBits, OrigBFI.IsSigned);
>> if (AtomicTy.isNull()) {
>> @@ -118,10 +119,8 @@ namespace {
>> ValueTy = lvalue.getType();
>> ValueSizeInBits = C.getTypeSize(ValueTy);
>> AtomicTy = ValueTy = CGF.getContext().getExtVectorType(
>> - lvalue.getType(), lvalue.getExtVectorAddr()
>> - ->getType()
>> - ->getPointerElementType()
>> - ->getVectorNumElements());
>> + lvalue.getType(), lvalue.getExtVectorAddress()
>> + .getElementType()->getVectorNumElements());
>> AtomicSizeInBits = C.getTypeSize(AtomicTy);
>> AtomicAlign = ValueAlign = lvalue.getAlignment();
>> LVal = lvalue;
>> @@ -139,15 +138,22 @@ namespace {
>> TypeEvaluationKind getEvaluationKind() const { return EvaluationKind; }
>> bool shouldUseLibcall() const { return UseLibcall; }
>> const LValue &getAtomicLValue() const { return LVal; }
>> - llvm::Value *getAtomicAddress() const {
>> + llvm::Value *getAtomicPointer() const {
>> if (LVal.isSimple())
>> - return LVal.getAddress();
>> + return LVal.getPointer();
>> else if (LVal.isBitField())
>> - return LVal.getBitFieldAddr();
>> + return LVal.getBitFieldPointer();
>> else if (LVal.isVectorElt())
>> - return LVal.getVectorAddr();
>> + return LVal.getVectorPointer();
>> assert(LVal.isExtVectorElt());
>> - return LVal.getExtVectorAddr();
>> + return LVal.getExtVectorPointer();
>> + }
>> + Address getAtomicAddress() const {
>> + return Address(getAtomicPointer(), getAtomicAlignment());
>> + }
>> +
>> + Address getAtomicAddressAsAtomicIntPointer() const {
>> + return emitCastToAtomicIntPointer(getAtomicAddress());
>> }
>>
>> /// Is the atomic size larger than the underlying value type?
>> @@ -169,11 +175,11 @@ namespace {
>>
>> /// Cast the given pointer to an integer pointer suitable for
>> /// atomic operations.
>> - llvm::Value *emitCastToAtomicIntPointer(llvm::Value *addr) const;
>> + Address emitCastToAtomicIntPointer(Address addr) const;
>>
>> /// Turn an atomic-layout object into an r-value.
>> - RValue convertTempToRValue(llvm::Value *addr, AggValueSlot resultSlot,
>> - SourceLocation loc, bool AsValue) const;
>> + RValue convertAtomicTempToRValue(Address addr, AggValueSlot resultSlot,
>> + SourceLocation loc, bool AsValue) const;
>>
>> /// \brief Converts a rvalue to integer value.
>> llvm::Value *convertRValueToInt(RValue RVal) const;
>> @@ -188,12 +194,12 @@ namespace {
>> /// Project an l-value down to the value field.
>> LValue projectValue() const {
>> assert(LVal.isSimple());
>> - llvm::Value *addr = getAtomicAddress();
>> + Address addr = getAtomicAddress();
>> if (hasPadding())
>> - addr = CGF.Builder.CreateStructGEP(nullptr, addr, 0);
>> + addr = CGF.Builder.CreateStructGEP(addr, 0, CharUnits());
>>
>> - return LValue::MakeAddr(addr, getValueType(), LVal.getAlignment(),
>> - CGF.getContext(), LVal.getTBAAInfo());
>> + return LValue::MakeAddr(addr, getValueType(), CGF.getContext(),
>> + LVal.getAlignmentSource(), LVal.getTBAAInfo());
>> }
>>
>> /// \brief Emits atomic load.
>> @@ -228,7 +234,7 @@ namespace {
>> bool IsVolatile);
>>
>> /// Materialize an atomic r-value in atomic-layout memory.
>> - llvm::Value *materializeRValue(RValue rvalue) const;
>> + Address materializeRValue(RValue rvalue) const;
>>
>> /// \brief Translates LLVM atomic ordering to GNU atomic ordering for
>> /// libcalls.
>> @@ -239,7 +245,7 @@ namespace {
>> bool requiresMemSetZero(llvm::Type *type) const;
>>
>> /// \brief Creates temp alloca for intermediate operations on atomic value.
>> - llvm::Value *CreateTempAlloca() const;
>> + Address CreateTempAlloca() const;
>>
>> /// \brief Emits atomic load as a libcall.
>> void EmitAtomicLoadLibcall(llvm::Value *AddForLoaded,
>> @@ -294,16 +300,16 @@ AtomicInfo::translateAtomicOrdering(cons
>> llvm_unreachable("Unhandled AtomicOrdering");
>> }
>>
>> -llvm::Value *AtomicInfo::CreateTempAlloca() const {
>> - auto *TempAlloca = CGF.CreateMemTemp(
>> +Address AtomicInfo::CreateTempAlloca() const {
>> + Address TempAlloca = CGF.CreateMemTemp(
>> (LVal.isBitField() && ValueSizeInBits > AtomicSizeInBits) ? ValueTy
>> : AtomicTy,
>> + getAtomicAlignment(),
>> "atomic-temp");
>> - TempAlloca->setAlignment(getAtomicAlignment().getQuantity());
>> // Cast to pointer to value type for bitfields.
>> if (LVal.isBitField())
>> return CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - TempAlloca, getAtomicAddress()->getType());
>> + TempAlloca, getAtomicAddress().getType());
>> return TempAlloca;
>> }
>>
>> @@ -351,7 +357,7 @@ bool AtomicInfo::requiresMemSetZero(llvm
>>
>> bool AtomicInfo::emitMemSetZeroIfNecessary() const {
>> assert(LVal.isSimple());
>> - llvm::Value *addr = LVal.getAddress();
>> + llvm::Value *addr = LVal.getPointer();
>> if (!requiresMemSetZero(addr->getType()->getPointerElementType()))
>> return false;
>>
>> @@ -363,19 +369,17 @@ bool AtomicInfo::emitMemSetZeroIfNecessa
>> }
>>
>> static void emitAtomicCmpXchg(CodeGenFunction &CGF, AtomicExpr *E, bool IsWeak,
>> - llvm::Value *Dest, llvm::Value *Ptr,
>> - llvm::Value *Val1, llvm::Value *Val2,
>> - uint64_t Size, unsigned Align,
>> + Address Dest, Address Ptr,
>> + Address Val1, Address Val2,
>> + uint64_t Size,
>> llvm::AtomicOrdering SuccessOrder,
>> llvm::AtomicOrdering FailureOrder) {
>> // Note that cmpxchg doesn't support weak cmpxchg, at least at the moment.
>> - llvm::LoadInst *Expected = CGF.Builder.CreateLoad(Val1);
>> - Expected->setAlignment(Align);
>> - llvm::LoadInst *Desired = CGF.Builder.CreateLoad(Val2);
>> - Desired->setAlignment(Align);
>> + llvm::Value *Expected = CGF.Builder.CreateLoad(Val1);
>> + llvm::Value *Desired = CGF.Builder.CreateLoad(Val2);
>>
>> llvm::AtomicCmpXchgInst *Pair = CGF.Builder.CreateAtomicCmpXchg(
>> - Ptr, Expected, Desired, SuccessOrder, FailureOrder);
>> + Ptr.getPointer(), Expected, Desired, SuccessOrder, FailureOrder);
>> Pair->setVolatile(E->isVolatile());
>> Pair->setWeak(IsWeak);
>>
>> @@ -400,26 +404,24 @@ static void emitAtomicCmpXchg(CodeGenFun
>>
>> CGF.Builder.SetInsertPoint(StoreExpectedBB);
>> // Update the memory at Expected with Old's value.
>> - llvm::StoreInst *StoreExpected = CGF.Builder.CreateStore(Old, Val1);
>> - StoreExpected->setAlignment(Align);
>> + CGF.Builder.CreateStore(Old, Val1);
>> // Finally, branch to the exit point.
>> CGF.Builder.CreateBr(ContinueBB);
>>
>> CGF.Builder.SetInsertPoint(ContinueBB);
>> // Update the memory at Dest with Cmp's value.
>> CGF.EmitStoreOfScalar(Cmp, CGF.MakeAddrLValue(Dest, E->getType()));
>> - return;
>> }
>>
>> /// Given an ordering required on success, emit all possible cmpxchg
>> /// instructions to cope with the provided (but possibly only dynamically known)
>> /// FailureOrder.
>> static void emitAtomicCmpXchgFailureSet(CodeGenFunction &CGF, AtomicExpr *E,
>> - bool IsWeak, llvm::Value *Dest,
>> - llvm::Value *Ptr, llvm::Value *Val1,
>> - llvm::Value *Val2,
>> + bool IsWeak, Address Dest,
>> + Address Ptr, Address Val1,
>> + Address Val2,
>> llvm::Value *FailureOrderVal,
>> - uint64_t Size, unsigned Align,
>> + uint64_t Size,
>> llvm::AtomicOrdering SuccessOrder) {
>> llvm::AtomicOrdering FailureOrder;
>> if (llvm::ConstantInt *FO = dyn_cast<llvm::ConstantInt>(FailureOrderVal)) {
>> @@ -440,7 +442,7 @@ static void emitAtomicCmpXchgFailureSet(
>> FailureOrder =
>> llvm::AtomicCmpXchgInst::getStrongestFailureOrdering(SuccessOrder);
>> }
>> - emitAtomicCmpXchg(CGF, E, IsWeak, Dest, Ptr, Val1, Val2, Size, Align,
>> + emitAtomicCmpXchg(CGF, E, IsWeak, Dest, Ptr, Val1, Val2, Size,
>> SuccessOrder, FailureOrder);
>> return;
>> }
>> @@ -465,13 +467,13 @@ static void emitAtomicCmpXchgFailureSet(
>> // doesn't fold to a constant for the ordering.
>> CGF.Builder.SetInsertPoint(MonotonicBB);
>> emitAtomicCmpXchg(CGF, E, IsWeak, Dest, Ptr, Val1, Val2,
>> - Size, Align, SuccessOrder, llvm::Monotonic);
>> + Size, SuccessOrder, llvm::Monotonic);
>> CGF.Builder.CreateBr(ContBB);
>>
>> if (AcquireBB) {
>> CGF.Builder.SetInsertPoint(AcquireBB);
>> emitAtomicCmpXchg(CGF, E, IsWeak, Dest, Ptr, Val1, Val2,
>> - Size, Align, SuccessOrder, llvm::Acquire);
>> + Size, SuccessOrder, llvm::Acquire);
>> CGF.Builder.CreateBr(ContBB);
>> SI->addCase(CGF.Builder.getInt32(AtomicExpr::AO_ABI_memory_order_consume),
>> AcquireBB);
>> @@ -481,7 +483,7 @@ static void emitAtomicCmpXchgFailureSet(
>> if (SeqCstBB) {
>> CGF.Builder.SetInsertPoint(SeqCstBB);
>> emitAtomicCmpXchg(CGF, E, IsWeak, Dest, Ptr, Val1, Val2,
>> - Size, Align, SuccessOrder, llvm::SequentiallyConsistent);
>> + Size, SuccessOrder, llvm::SequentiallyConsistent);
>> CGF.Builder.CreateBr(ContBB);
>> SI->addCase(CGF.Builder.getInt32(AtomicExpr::AO_ABI_memory_order_seq_cst),
>> SeqCstBB);
>> @@ -490,11 +492,10 @@ static void emitAtomicCmpXchgFailureSet(
>> CGF.Builder.SetInsertPoint(ContBB);
>> }
>>
>> -static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr *E, llvm::Value *Dest,
>> - llvm::Value *Ptr, llvm::Value *Val1, llvm::Value *Val2,
>> +static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr *E, Address Dest,
>> + Address Ptr, Address Val1, Address Val2,
>> llvm::Value *IsWeak, llvm::Value *FailureOrder,
>> - uint64_t Size, unsigned Align,
>> - llvm::AtomicOrdering Order) {
>> + uint64_t Size, llvm::AtomicOrdering Order) {
>> llvm::AtomicRMWInst::BinOp Op = llvm::AtomicRMWInst::Add;
>> llvm::Instruction::BinaryOps PostOp = (llvm::Instruction::BinaryOps)0;
>>
>> @@ -504,17 +505,17 @@ static void EmitAtomicOp(CodeGenFunction
>>
>> case AtomicExpr::AO__c11_atomic_compare_exchange_strong:
>> emitAtomicCmpXchgFailureSet(CGF, E, false, Dest, Ptr, Val1, Val2,
>> - FailureOrder, Size, Align, Order);
>> + FailureOrder, Size, Order);
>> return;
>> case AtomicExpr::AO__c11_atomic_compare_exchange_weak:
>> emitAtomicCmpXchgFailureSet(CGF, E, true, Dest, Ptr, Val1, Val2,
>> - FailureOrder, Size, Align, Order);
>> + FailureOrder, Size, Order);
>> return;
>> case AtomicExpr::AO__atomic_compare_exchange:
>> case AtomicExpr::AO__atomic_compare_exchange_n: {
>> if (llvm::ConstantInt *IsWeakC = dyn_cast<llvm::ConstantInt>(IsWeak)) {
>> emitAtomicCmpXchgFailureSet(CGF, E, IsWeakC->getZExtValue(), Dest, Ptr,
>> - Val1, Val2, FailureOrder, Size, Align, Order);
>> + Val1, Val2, FailureOrder, Size, Order);
>> } else {
>> // Create all the relevant BB's
>> llvm::BasicBlock *StrongBB =
>> @@ -528,12 +529,12 @@ static void EmitAtomicOp(CodeGenFunction
>>
>> CGF.Builder.SetInsertPoint(StrongBB);
>> emitAtomicCmpXchgFailureSet(CGF, E, false, Dest, Ptr, Val1, Val2,
>> - FailureOrder, Size, Align, Order);
>> + FailureOrder, Size, Order);
>> CGF.Builder.CreateBr(ContBB);
>>
>> CGF.Builder.SetInsertPoint(WeakBB);
>> emitAtomicCmpXchgFailureSet(CGF, E, true, Dest, Ptr, Val1, Val2,
>> - FailureOrder, Size, Align, Order);
>> + FailureOrder, Size, Order);
>> CGF.Builder.CreateBr(ContBB);
>>
>> CGF.Builder.SetInsertPoint(ContBB);
>> @@ -545,22 +546,18 @@ static void EmitAtomicOp(CodeGenFunction
>> case AtomicExpr::AO__atomic_load: {
>> llvm::LoadInst *Load = CGF.Builder.CreateLoad(Ptr);
>> Load->setAtomic(Order);
>> - Load->setAlignment(Size);
>> Load->setVolatile(E->isVolatile());
>> - llvm::StoreInst *StoreDest = CGF.Builder.CreateStore(Load, Dest);
>> - StoreDest->setAlignment(Align);
>> + CGF.Builder.CreateStore(Load, Dest);
>> return;
>> }
>>
>> case AtomicExpr::AO__c11_atomic_store:
>> case AtomicExpr::AO__atomic_store:
>> case AtomicExpr::AO__atomic_store_n: {
>> - assert(!Dest && "Store does not return a value");
>> - llvm::LoadInst *LoadVal1 = CGF.Builder.CreateLoad(Val1);
>> - LoadVal1->setAlignment(Align);
>> + assert(!Dest.isValid() && "Store does not return a value");
>> + llvm::Value *LoadVal1 = CGF.Builder.CreateLoad(Val1);
>> llvm::StoreInst *Store = CGF.Builder.CreateStore(LoadVal1, Ptr);
>> Store->setAtomic(Order);
>> - Store->setAlignment(Size);
>> Store->setVolatile(E->isVolatile());
>> return;
>> }
>> @@ -619,10 +616,9 @@ static void EmitAtomicOp(CodeGenFunction
>> break;
>> }
>>
>> - llvm::LoadInst *LoadVal1 = CGF.Builder.CreateLoad(Val1);
>> - LoadVal1->setAlignment(Align);
>> + llvm::Value *LoadVal1 = CGF.Builder.CreateLoad(Val1);
>> llvm::AtomicRMWInst *RMWI =
>> - CGF.Builder.CreateAtomicRMW(Op, Ptr, LoadVal1, Order);
>> + CGF.Builder.CreateAtomicRMW(Op, Ptr.getPointer(), LoadVal1, Order);
>> RMWI->setVolatile(E->isVolatile());
>>
>> // For __atomic_*_fetch operations, perform the operation again to
>> @@ -632,15 +628,14 @@ static void EmitAtomicOp(CodeGenFunction
>> Result = CGF.Builder.CreateBinOp(PostOp, RMWI, LoadVal1);
>> if (E->getOp() == AtomicExpr::AO__atomic_nand_fetch)
>> Result = CGF.Builder.CreateNot(Result);
>> - llvm::StoreInst *StoreDest = CGF.Builder.CreateStore(Result, Dest);
>> - StoreDest->setAlignment(Align);
>> + CGF.Builder.CreateStore(Result, Dest);
>> }
>>
>> // This function emits any expression (scalar, complex, or aggregate)
>> // into a temporary alloca.
>> -static llvm::Value *
>> +static Address
>> EmitValToTemp(CodeGenFunction &CGF, Expr *E) {
>> - llvm::Value *DeclPtr = CGF.CreateMemTemp(E->getType(), ".atomictmp");
>> + Address DeclPtr = CGF.CreateMemTemp(E->getType(), ".atomictmp");
>> CGF.EmitAnyExprToMem(E, DeclPtr, E->getType().getQualifiers(),
>> /*Init*/ true);
>> return DeclPtr;
>> @@ -652,14 +647,15 @@ AddDirectArgument(CodeGenFunction &CGF,
>> SourceLocation Loc, CharUnits SizeInChars) {
>> if (UseOptimizedLibcall) {
>> // Load value and pass it to the function directly.
>> - unsigned Align = CGF.getContext().getTypeAlignInChars(ValTy).getQuantity();
>> + CharUnits Align = CGF.getContext().getTypeAlignInChars(ValTy);
>> int64_t SizeInBits = CGF.getContext().toBits(SizeInChars);
>> ValTy =
>> CGF.getContext().getIntTypeForBitwidth(SizeInBits, /*Signed=*/false);
>> llvm::Type *IPtrTy = llvm::IntegerType::get(CGF.getLLVMContext(),
>> SizeInBits)->getPointerTo();
>> - Val = CGF.EmitLoadOfScalar(CGF.Builder.CreateBitCast(Val, IPtrTy), false,
>> - Align, CGF.getContext().getPointerType(ValTy),
>> + Address Ptr = Address(CGF.Builder.CreateBitCast(Val, IPtrTy), Align);
>> + Val = CGF.EmitLoadOfScalar(Ptr, false,
>> + CGF.getContext().getPointerType(ValTy),
>> Loc);
>> // Coerce the value into an appropriately sized integer type.
>> Args.add(RValue::get(Val), ValTy);
>> @@ -670,27 +666,27 @@ AddDirectArgument(CodeGenFunction &CGF,
>> }
>> }
>>
>> -RValue CodeGenFunction::EmitAtomicExpr(AtomicExpr *E, llvm::Value *Dest) {
>> +RValue CodeGenFunction::EmitAtomicExpr(AtomicExpr *E, Address Dest) {
>> QualType AtomicTy = E->getPtr()->getType()->getPointeeType();
>> QualType MemTy = AtomicTy;
>> if (const AtomicType *AT = AtomicTy->getAs<AtomicType>())
>> MemTy = AT->getValueType();
>> - CharUnits sizeChars = getContext().getTypeSizeInChars(AtomicTy);
>> + CharUnits sizeChars, alignChars;
>> + std::tie(sizeChars, alignChars) = getContext().getTypeInfoInChars(AtomicTy);
>> uint64_t Size = sizeChars.getQuantity();
>> - CharUnits alignChars = getContext().getTypeAlignInChars(AtomicTy);
>> - unsigned Align = alignChars.getQuantity();
>> - unsigned MaxInlineWidthInBits =
>> - getTarget().getMaxAtomicInlineWidth();
>> - bool UseLibcall = (Size != Align ||
>> + unsigned MaxInlineWidthInBits = getTarget().getMaxAtomicInlineWidth();
>> + bool UseLibcall = (sizeChars != alignChars ||
>> getContext().toBits(sizeChars) > MaxInlineWidthInBits);
>>
>> - llvm::Value *IsWeak = nullptr, *OrderFail = nullptr, *Val1 = nullptr,
>> - *Val2 = nullptr;
>> - llvm::Value *Ptr = EmitScalarExpr(E->getPtr());
>> + llvm::Value *IsWeak = nullptr, *OrderFail = nullptr;
>> +
>> + Address Val1 = Address::invalid();
>> + Address Val2 = Address::invalid();
>> + Address Ptr(EmitScalarExpr(E->getPtr()), alignChars);
>>
>> if (E->getOp() == AtomicExpr::AO__c11_atomic_init) {
>> - assert(!Dest && "Init does not return a value");
>> - LValue lvalue = LValue::MakeAddr(Ptr, AtomicTy, alignChars, getContext());
>> + assert(!Dest.isValid() && "Init does not return a value");
>> + LValue lvalue = MakeAddrLValue(Ptr, AtomicTy);
>> EmitAtomicInit(E->getVal1(), lvalue);
>> return RValue::get(nullptr);
>> }
>> @@ -706,25 +702,25 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> break;
>>
>> case AtomicExpr::AO__atomic_load:
>> - Dest = EmitScalarExpr(E->getVal1());
>> + Dest = EmitPointerWithAlignment(E->getVal1());
>> break;
>>
>> case AtomicExpr::AO__atomic_store:
>> - Val1 = EmitScalarExpr(E->getVal1());
>> + Val1 = EmitPointerWithAlignment(E->getVal1());
>> break;
>>
>> case AtomicExpr::AO__atomic_exchange:
>> - Val1 = EmitScalarExpr(E->getVal1());
>> - Dest = EmitScalarExpr(E->getVal2());
>> + Val1 = EmitPointerWithAlignment(E->getVal1());
>> + Dest = EmitPointerWithAlignment(E->getVal2());
>> break;
>>
>> case AtomicExpr::AO__c11_atomic_compare_exchange_strong:
>> case AtomicExpr::AO__c11_atomic_compare_exchange_weak:
>> case AtomicExpr::AO__atomic_compare_exchange_n:
>> case AtomicExpr::AO__atomic_compare_exchange:
>> - Val1 = EmitScalarExpr(E->getVal1());
>> + Val1 = EmitPointerWithAlignment(E->getVal1());
>> if (E->getOp() == AtomicExpr::AO__atomic_compare_exchange)
>> - Val2 = EmitScalarExpr(E->getVal2());
>> + Val2 = EmitPointerWithAlignment(E->getVal2());
>> else
>> Val2 = EmitValToTemp(*this, E->getVal2());
>> OrderFail = EmitScalarExpr(E->getOrderFail());
>> @@ -744,8 +740,9 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> CharUnits PointeeIncAmt =
>> getContext().getTypeSizeInChars(MemTy->getPointeeType());
>> Val1Scalar = Builder.CreateMul(Val1Scalar, CGM.getSize(PointeeIncAmt));
>> - Val1 = CreateMemTemp(Val1Ty, ".atomictmp");
>> - EmitStoreOfScalar(Val1Scalar, MakeAddrLValue(Val1, Val1Ty));
>> + auto Temp = CreateMemTemp(Val1Ty, ".atomictmp");
>> + Val1 = Temp;
>> + EmitStoreOfScalar(Val1Scalar, MakeAddrLValue(Temp, Val1Ty));
>> break;
>> }
>> // Fall through.
>> @@ -775,7 +772,7 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> QualType RValTy = E->getType().getUnqualifiedType();
>>
>> auto GetDest = [&] {
>> - if (!RValTy->isVoidType() && !Dest) {
>> + if (!RValTy->isVoidType() && !Dest.isValid()) {
>> Dest = CreateMemTemp(RValTy, ".atomicdst");
>> }
>> return Dest;
>> @@ -835,7 +832,8 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> getContext().getSizeType());
>> }
>> // Atomic address is the first or second parameter
>> - Args.add(RValue::get(EmitCastToVoidPtr(Ptr)), getContext().VoidPtrTy);
>> + Args.add(RValue::get(EmitCastToVoidPtr(Ptr.getPointer())),
>> + getContext().VoidPtrTy);
>>
>> std::string LibCallName;
>> QualType LoweredMemTy =
>> @@ -860,9 +858,10 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> LibCallName = "__atomic_compare_exchange";
>> RetTy = getContext().BoolTy;
>> HaveRetTy = true;
>> - Args.add(RValue::get(EmitCastToVoidPtr(Val1)), getContext().VoidPtrTy);
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val2, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + Args.add(RValue::get(EmitCastToVoidPtr(Val1.getPointer())),
>> + getContext().VoidPtrTy);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val2.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> Args.add(RValue::get(Order), getContext().IntTy);
>> Order = OrderFail;
>> break;
>> @@ -873,8 +872,8 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> case AtomicExpr::AO__atomic_exchange_n:
>> case AtomicExpr::AO__atomic_exchange:
>> LibCallName = "__atomic_exchange";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // void __atomic_store(size_t size, void *mem, void *val, int order)
>> // void __atomic_store_N(T *mem, T val, int order)
>> @@ -884,8 +883,8 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> LibCallName = "__atomic_store";
>> RetTy = getContext().VoidTy;
>> HaveRetTy = true;
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // void __atomic_load(size_t size, void *mem, void *return, int order)
>> // T __atomic_load_N(T *mem, int order)
>> @@ -898,79 +897,79 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> case AtomicExpr::AO__c11_atomic_fetch_add:
>> case AtomicExpr::AO__atomic_fetch_add:
>> LibCallName = "__atomic_fetch_add";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, LoweredMemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + LoweredMemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_fetch_and_N(T *mem, T val, int order)
>> case AtomicExpr::AO__c11_atomic_fetch_and:
>> case AtomicExpr::AO__atomic_fetch_and:
>> LibCallName = "__atomic_fetch_and";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_fetch_or_N(T *mem, T val, int order)
>> case AtomicExpr::AO__c11_atomic_fetch_or:
>> case AtomicExpr::AO__atomic_fetch_or:
>> LibCallName = "__atomic_fetch_or";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_fetch_sub_N(T *mem, T val, int order)
>> case AtomicExpr::AO__c11_atomic_fetch_sub:
>> case AtomicExpr::AO__atomic_fetch_sub:
>> LibCallName = "__atomic_fetch_sub";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, LoweredMemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + LoweredMemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_fetch_xor_N(T *mem, T val, int order)
>> case AtomicExpr::AO__c11_atomic_fetch_xor:
>> case AtomicExpr::AO__atomic_fetch_xor:
>> LibCallName = "__atomic_fetch_xor";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_fetch_nand_N(T *mem, T val, int order)
>> case AtomicExpr::AO__atomic_fetch_nand:
>> LibCallName = "__atomic_fetch_nand";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>>
>> // T __atomic_add_fetch_N(T *mem, T val, int order)
>> case AtomicExpr::AO__atomic_add_fetch:
>> LibCallName = "__atomic_add_fetch";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, LoweredMemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + LoweredMemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_and_fetch_N(T *mem, T val, int order)
>> case AtomicExpr::AO__atomic_and_fetch:
>> LibCallName = "__atomic_and_fetch";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_or_fetch_N(T *mem, T val, int order)
>> case AtomicExpr::AO__atomic_or_fetch:
>> LibCallName = "__atomic_or_fetch";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_sub_fetch_N(T *mem, T val, int order)
>> case AtomicExpr::AO__atomic_sub_fetch:
>> LibCallName = "__atomic_sub_fetch";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, LoweredMemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + LoweredMemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_xor_fetch_N(T *mem, T val, int order)
>> case AtomicExpr::AO__atomic_xor_fetch:
>> LibCallName = "__atomic_xor_fetch";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> // T __atomic_nand_fetch_N(T *mem, T val, int order)
>> case AtomicExpr::AO__atomic_nand_fetch:
>> LibCallName = "__atomic_nand_fetch";
>> - AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1, MemTy,
>> - E->getExprLoc(), sizeChars);
>> + AddDirectArgument(*this, Args, UseOptimizedLibcall, Val1.getPointer(),
>> + MemTy, E->getExprLoc(), sizeChars);
>> break;
>> }
>>
>> @@ -987,7 +986,8 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> } else {
>> // Value is returned through parameter before the order.
>> RetTy = getContext().VoidTy;
>> - Args.add(RValue::get(EmitCastToVoidPtr(Dest)), getContext().VoidPtrTy);
>> + Args.add(RValue::get(EmitCastToVoidPtr(Dest.getPointer())),
>> + getContext().VoidPtrTy);
>> }
>> }
>> // order is always the last parameter
>> @@ -1005,10 +1005,8 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> // expected an out-param.
>> if (UseOptimizedLibcall) {
>> llvm::Value *ResVal = Res.getScalarVal();
>> - llvm::StoreInst *StoreDest = Builder.CreateStore(
>> - ResVal,
>> + Builder.CreateStore(ResVal,
>> Builder.CreateBitCast(GetDest(), ResVal->getType()->getPointerTo()));
>> - StoreDest->setAlignment(Align);
>> }
>> return convertTempToRValue(Dest, RValTy, E->getExprLoc());
>> }
>> @@ -1022,12 +1020,12 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>>
>> llvm::Type *ITy =
>> llvm::IntegerType::get(getLLVMContext(), Size * 8);
>> - llvm::Value *OrigDest = GetDest();
>> + Address OrigDest = GetDest();
>> Ptr = Builder.CreateBitCast(
>> - Ptr, ITy->getPointerTo(Ptr->getType()->getPointerAddressSpace()));
>> - if (Val1) Val1 = Builder.CreateBitCast(Val1, ITy->getPointerTo());
>> - if (Val2) Val2 = Builder.CreateBitCast(Val2, ITy->getPointerTo());
>> - if (Dest && !E->isCmpXChg())
>> + Ptr, ITy->getPointerTo(Ptr.getType()->getPointerAddressSpace()));
>> + if (Val1.isValid()) Val1 = Builder.CreateBitCast(Val1, ITy->getPointerTo());
>> + if (Val2.isValid()) Val2 = Builder.CreateBitCast(Val2, ITy->getPointerTo());
>> + if (Dest.isValid() && !E->isCmpXChg())
>> Dest = Builder.CreateBitCast(Dest, ITy->getPointerTo());
>>
>> if (isa<llvm::ConstantInt>(Order)) {
>> @@ -1035,30 +1033,30 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> switch (ord) {
>> case AtomicExpr::AO_ABI_memory_order_relaxed:
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::Monotonic);
>> + Size, llvm::Monotonic);
>> break;
>> case AtomicExpr::AO_ABI_memory_order_consume:
>> case AtomicExpr::AO_ABI_memory_order_acquire:
>> if (IsStore)
>> break; // Avoid crashing on code with undefined behavior
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::Acquire);
>> + Size, llvm::Acquire);
>> break;
>> case AtomicExpr::AO_ABI_memory_order_release:
>> if (IsLoad)
>> break; // Avoid crashing on code with undefined behavior
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::Release);
>> + Size, llvm::Release);
>> break;
>> case AtomicExpr::AO_ABI_memory_order_acq_rel:
>> if (IsLoad || IsStore)
>> break; // Avoid crashing on code with undefined behavior
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::AcquireRelease);
>> + Size, llvm::AcquireRelease);
>> break;
>> case AtomicExpr::AO_ABI_memory_order_seq_cst:
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::SequentiallyConsistent);
>> + Size, llvm::SequentiallyConsistent);
>> break;
>> default: // invalid order
>> // We should not ever get here normally, but it's hard to
>> @@ -1096,12 +1094,12 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> // Emit all the different atomics
>> Builder.SetInsertPoint(MonotonicBB);
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::Monotonic);
>> + Size, llvm::Monotonic);
>> Builder.CreateBr(ContBB);
>> if (!IsStore) {
>> Builder.SetInsertPoint(AcquireBB);
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::Acquire);
>> + Size, llvm::Acquire);
>> Builder.CreateBr(ContBB);
>> SI->addCase(Builder.getInt32(AtomicExpr::AO_ABI_memory_order_consume),
>> AcquireBB);
>> @@ -1111,7 +1109,7 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> if (!IsLoad) {
>> Builder.SetInsertPoint(ReleaseBB);
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::Release);
>> + Size, llvm::Release);
>> Builder.CreateBr(ContBB);
>> SI->addCase(Builder.getInt32(AtomicExpr::AO_ABI_memory_order_release),
>> ReleaseBB);
>> @@ -1119,14 +1117,14 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> if (!IsLoad && !IsStore) {
>> Builder.SetInsertPoint(AcqRelBB);
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::AcquireRelease);
>> + Size, llvm::AcquireRelease);
>> Builder.CreateBr(ContBB);
>> SI->addCase(Builder.getInt32(AtomicExpr::AO_ABI_memory_order_acq_rel),
>> AcqRelBB);
>> }
>> Builder.SetInsertPoint(SeqCstBB);
>> EmitAtomicOp(*this, E, Dest, Ptr, Val1, Val2, IsWeak, OrderFail,
>> - Size, Align, llvm::SequentiallyConsistent);
>> + Size, llvm::SequentiallyConsistent);
>> Builder.CreateBr(ContBB);
>> SI->addCase(Builder.getInt32(AtomicExpr::AO_ABI_memory_order_seq_cst),
>> SeqCstBB);
>> @@ -1138,44 +1136,45 @@ RValue CodeGenFunction::EmitAtomicExpr(A
>> return convertTempToRValue(OrigDest, RValTy, E->getExprLoc());
>> }
>>
>> -llvm::Value *AtomicInfo::emitCastToAtomicIntPointer(llvm::Value *addr) const {
>> +Address AtomicInfo::emitCastToAtomicIntPointer(Address addr) const {
>> unsigned addrspace =
>> - cast<llvm::PointerType>(addr->getType())->getAddressSpace();
>> + cast<llvm::PointerType>(addr.getPointer()->getType())->getAddressSpace();
>> llvm::IntegerType *ty =
>> llvm::IntegerType::get(CGF.getLLVMContext(), AtomicSizeInBits);
>> return CGF.Builder.CreateBitCast(addr, ty->getPointerTo(addrspace));
>> }
>>
>> -RValue AtomicInfo::convertTempToRValue(llvm::Value *addr,
>> - AggValueSlot resultSlot,
>> - SourceLocation loc, bool AsValue) const {
>> +RValue AtomicInfo::convertAtomicTempToRValue(Address addr,
>> + AggValueSlot resultSlot,
>> + SourceLocation loc,
>> + bool asValue) const {
>> if (LVal.isSimple()) {
>> if (EvaluationKind == TEK_Aggregate)
>> return resultSlot.asRValue();
>>
>> // Drill into the padding structure if we have one.
>> if (hasPadding())
>> - addr = CGF.Builder.CreateStructGEP(nullptr, addr, 0);
>> + addr = CGF.Builder.CreateStructGEP(addr, 0, CharUnits());
>>
>> // Otherwise, just convert the temporary to an r-value using the
>> // normal conversion routine.
>> return CGF.convertTempToRValue(addr, getValueType(), loc);
>> }
>> - if (!AsValue)
>> + if (!asValue)
>> // Get RValue from temp memory as atomic for non-simple lvalues
>> - return RValue::get(
>> - CGF.Builder.CreateAlignedLoad(addr, AtomicAlign.getQuantity()));
>> + return RValue::get(CGF.Builder.CreateLoad(addr));
>> if (LVal.isBitField())
>> - return CGF.EmitLoadOfBitfieldLValue(LValue::MakeBitfield(
>> - addr, LVal.getBitFieldInfo(), LVal.getType(), LVal.getAlignment()));
>> + return CGF.EmitLoadOfBitfieldLValue(
>> + LValue::MakeBitfield(addr, LVal.getBitFieldInfo(), LVal.getType(),
>> + LVal.getAlignmentSource()));
>> if (LVal.isVectorElt())
>> - return CGF.EmitLoadOfLValue(LValue::MakeVectorElt(addr, LVal.getVectorIdx(),
>> - LVal.getType(),
>> - LVal.getAlignment()),
>> - loc);
>> + return CGF.EmitLoadOfLValue(
>> + LValue::MakeVectorElt(addr, LVal.getVectorIdx(), LVal.getType(),
>> + LVal.getAlignmentSource()), loc);
>> assert(LVal.isExtVectorElt());
>> return CGF.EmitLoadOfExtVectorElementLValue(LValue::MakeExtVectorElt(
>> - addr, LVal.getExtVectorElts(), LVal.getType(), LVal.getAlignment()));
>> + addr, LVal.getExtVectorElts(), LVal.getType(),
>> + LVal.getAlignmentSource()));
>> }
>>
>> RValue AtomicInfo::ConvertIntToValueOrAtomic(llvm::Value *IntVal,
>> @@ -1191,7 +1190,7 @@ RValue AtomicInfo::ConvertIntToValueOrAt
>> !AsValue)) {
>> auto *ValTy = AsValue
>> ? CGF.ConvertTypeForMem(ValueTy)
>> - : getAtomicAddress()->getType()->getPointerElementType();
>> + : getAtomicAddress().getType()->getPointerElementType();
>> if (ValTy->isIntegerTy()) {
>> assert(IntVal->getType() == ValTy && "Different integer types.");
>> return RValue::get(CGF.EmitFromMemory(IntVal, ValueTy));
>> @@ -1203,25 +1202,22 @@ RValue AtomicInfo::ConvertIntToValueOrAt
>>
>> // Create a temporary. This needs to be big enough to hold the
>> // atomic integer.
>> - llvm::Value *Temp;
>> + Address Temp = Address::invalid();
>> bool TempIsVolatile = false;
>> - CharUnits TempAlignment;
>> if (AsValue && getEvaluationKind() == TEK_Aggregate) {
>> assert(!ResultSlot.isIgnored());
>> - Temp = ResultSlot.getAddr();
>> - TempAlignment = getValueAlignment();
>> + Temp = ResultSlot.getAddress();
>> TempIsVolatile = ResultSlot.isVolatile();
>> } else {
>> Temp = CreateTempAlloca();
>> - TempAlignment = getAtomicAlignment();
>> }
>>
>> // Slam the integer into the temporary.
>> - llvm::Value *CastTemp = emitCastToAtomicIntPointer(Temp);
>> - CGF.Builder.CreateAlignedStore(IntVal, CastTemp, TempAlignment.getQuantity())
>> + Address CastTemp = emitCastToAtomicIntPointer(Temp);
>> + CGF.Builder.CreateStore(IntVal, CastTemp)
>> ->setVolatile(TempIsVolatile);
>>
>> - return convertTempToRValue(Temp, ResultSlot, Loc, AsValue);
>> + return convertAtomicTempToRValue(Temp, ResultSlot, Loc, AsValue);
>> }
>>
>> void AtomicInfo::EmitAtomicLoadLibcall(llvm::Value *AddForLoaded,
>> @@ -1229,7 +1225,7 @@ void AtomicInfo::EmitAtomicLoadLibcall(l
>> // void __atomic_load(size_t size, void *mem, void *return, int order);
>> CallArgList Args;
>> Args.add(RValue::get(getAtomicSizeValue()), CGF.getContext().getSizeType());
>> - Args.add(RValue::get(CGF.EmitCastToVoidPtr(getAtomicAddress())),
>> + Args.add(RValue::get(CGF.EmitCastToVoidPtr(getAtomicPointer())),
>> CGF.getContext().VoidPtrTy);
>> Args.add(RValue::get(CGF.EmitCastToVoidPtr(AddForLoaded)),
>> CGF.getContext().VoidPtrTy);
>> @@ -1242,12 +1238,11 @@ void AtomicInfo::EmitAtomicLoadLibcall(l
>> llvm::Value *AtomicInfo::EmitAtomicLoadOp(llvm::AtomicOrdering AO,
>> bool IsVolatile) {
>> // Okay, we're doing this natively.
>> - llvm::Value *Addr = emitCastToAtomicIntPointer(getAtomicAddress());
>> + Address Addr = getAtomicAddressAsAtomicIntPointer();
>> llvm::LoadInst *Load = CGF.Builder.CreateLoad(Addr, "atomic-load");
>> Load->setAtomic(AO);
>>
>> // Other decoration.
>> - Load->setAlignment(getAtomicAlignment().getQuantity());
>> if (IsVolatile)
>> Load->setVolatile(true);
>> if (LVal.getTBAAInfo())
>> @@ -1259,11 +1254,12 @@ llvm::Value *AtomicInfo::EmitAtomicLoadO
>> /// we are operating under /volatile:ms *and* the LValue itself is volatile and
>> /// performing such an operation can be performed without a libcall.
>> bool CodeGenFunction::LValueIsSuitableForInlineAtomic(LValue LV) {
>> + if (!CGM.getCodeGenOpts().MSVolatile) return false;
>> AtomicInfo AI(*this, LV);
>> bool IsVolatile = LV.isVolatile() || hasVolatileMember(LV.getType());
>> // An atomic is inline if we don't need to use a libcall.
>> bool AtomicIsInline = !AI.shouldUseLibcall();
>> - return CGM.getCodeGenOpts().MSVolatile && IsVolatile && AtomicIsInline;
>> + return IsVolatile && AtomicIsInline;
>> }
>>
>> /// An type is a candidate for having its loads and stores be made atomic if
>> @@ -1295,18 +1291,18 @@ RValue AtomicInfo::EmitAtomicLoad(AggVal
>> bool IsVolatile) {
>> // Check whether we should use a library call.
>> if (shouldUseLibcall()) {
>> - llvm::Value *TempAddr;
>> + Address TempAddr = Address::invalid();
>> if (LVal.isSimple() && !ResultSlot.isIgnored()) {
>> assert(getEvaluationKind() == TEK_Aggregate);
>> - TempAddr = ResultSlot.getAddr();
>> + TempAddr = ResultSlot.getAddress();
>> } else
>> TempAddr = CreateTempAlloca();
>>
>> - EmitAtomicLoadLibcall(TempAddr, AO, IsVolatile);
>> + EmitAtomicLoadLibcall(TempAddr.getPointer(), AO, IsVolatile);
>>
>> // Okay, turn that back into the original value or whole atomic (for
>> // non-simple lvalues) type.
>> - return convertTempToRValue(TempAddr, ResultSlot, Loc, AsValue);
>> + return convertAtomicTempToRValue(TempAddr, ResultSlot, Loc, AsValue);
>> }
>>
>> // Okay, we're doing this natively.
>> @@ -1314,7 +1310,7 @@ RValue AtomicInfo::EmitAtomicLoad(AggVal
>>
>> // If we're ignoring an aggregate return, don't do anything.
>> if (getEvaluationKind() == TEK_Aggregate && ResultSlot.isIgnored())
>> - return RValue::getAggregate(nullptr, false);
>> + return RValue::getAggregate(Address::invalid(), false);
>>
>> // Okay, turn that back into the original value or atomic (for non-simple
>> // lvalues) type.
>> @@ -1340,11 +1336,10 @@ void AtomicInfo::emitCopyIntoMemory(RVal
>> // any padding. Just do an aggregate copy of that type.
>> if (rvalue.isAggregate()) {
>> CGF.EmitAggregateCopy(getAtomicAddress(),
>> - rvalue.getAggregateAddr(),
>> + rvalue.getAggregateAddress(),
>> getAtomicType(),
>> (rvalue.isVolatileQualified()
>> - || LVal.isVolatileQualified()),
>> - LVal.getAlignment());
>> + || LVal.isVolatileQualified()));
>> return;
>> }
>>
>> @@ -1367,15 +1362,14 @@ void AtomicInfo::emitCopyIntoMemory(RVal
>>
>> /// Materialize an r-value into memory for the purposes of storing it
>> /// to an atomic type.
>> -llvm::Value *AtomicInfo::materializeRValue(RValue rvalue) const {
>> +Address AtomicInfo::materializeRValue(RValue rvalue) const {
>> // Aggregate r-values are already in memory, and EmitAtomicStore
>> // requires them to be values of the atomic type.
>> if (rvalue.isAggregate())
>> - return rvalue.getAggregateAddr();
>> + return rvalue.getAggregateAddress();
>>
>> // Otherwise, make a temporary and materialize into it.
>> - LValue TempLV = CGF.MakeAddrLValue(CreateTempAlloca(), getAtomicType(),
>> - getAtomicAlignment());
>> + LValue TempLV = CGF.MakeAddrLValue(CreateTempAlloca(), getAtomicType());
>> AtomicInfo Atomics(CGF, TempLV);
>> Atomics.emitCopyIntoMemory(rvalue);
>> return TempLV.getAddress();
>> @@ -1400,20 +1394,20 @@ llvm::Value *AtomicInfo::convertRValueTo
>> }
>> // Otherwise, we need to go through memory.
>> // Put the r-value in memory.
>> - llvm::Value *Addr = materializeRValue(RVal);
>> + Address Addr = materializeRValue(RVal);
>>
>> // Cast the temporary to the atomic int type and pull a value out.
>> Addr = emitCastToAtomicIntPointer(Addr);
>> - return CGF.Builder.CreateAlignedLoad(Addr,
>> - getAtomicAlignment().getQuantity());
>> + return CGF.Builder.CreateLoad(Addr);
>> }
>>
>> std::pair<llvm::Value *, llvm::Value *> AtomicInfo::EmitAtomicCompareExchangeOp(
>> llvm::Value *ExpectedVal, llvm::Value *DesiredVal,
>> llvm::AtomicOrdering Success, llvm::AtomicOrdering Failure, bool IsWeak) {
>> // Do the atomic store.
>> - auto *Addr = emitCastToAtomicIntPointer(getAtomicAddress());
>> - auto *Inst = CGF.Builder.CreateAtomicCmpXchg(Addr, ExpectedVal, DesiredVal,
>> + Address Addr = getAtomicAddressAsAtomicIntPointer();
>> + auto *Inst = CGF.Builder.CreateAtomicCmpXchg(Addr.getPointer(),
>> + ExpectedVal, DesiredVal,
>> Success, Failure);
>> // Other decoration.
>> Inst->setVolatile(LVal.isVolatileQualified());
>> @@ -1434,7 +1428,7 @@ AtomicInfo::EmitAtomicCompareExchangeLib
>> // void *desired, int success, int failure);
>> CallArgList Args;
>> Args.add(RValue::get(getAtomicSizeValue()), CGF.getContext().getSizeType());
>> - Args.add(RValue::get(CGF.EmitCastToVoidPtr(getAtomicAddress())),
>> + Args.add(RValue::get(CGF.EmitCastToVoidPtr(getAtomicPointer())),
>> CGF.getContext().VoidPtrTy);
>> Args.add(RValue::get(CGF.EmitCastToVoidPtr(ExpectedAddr)),
>> CGF.getContext().VoidPtrTy);
>> @@ -1462,13 +1456,14 @@ std::pair<RValue, llvm::Value *> AtomicI
>> // Check whether we should use a library call.
>> if (shouldUseLibcall()) {
>> // Produce a source address.
>> - auto *ExpectedAddr = materializeRValue(Expected);
>> - auto *DesiredAddr = materializeRValue(Desired);
>> - auto *Res = EmitAtomicCompareExchangeLibcall(ExpectedAddr, DesiredAddr,
>> + Address ExpectedAddr = materializeRValue(Expected);
>> + Address DesiredAddr = materializeRValue(Desired);
>> + auto *Res = EmitAtomicCompareExchangeLibcall(ExpectedAddr.getPointer(),
>> + DesiredAddr.getPointer(),
>> Success, Failure);
>> return std::make_pair(
>> - convertTempToRValue(ExpectedAddr, AggValueSlot::ignored(),
>> - SourceLocation(), /*AsValue=*/false),
>> + convertAtomicTempToRValue(ExpectedAddr, AggValueSlot::ignored(),
>> + SourceLocation(), /*AsValue=*/false),
>> Res);
>> }
>>
>> @@ -1487,42 +1482,41 @@ std::pair<RValue, llvm::Value *> AtomicI
>> static void
>> EmitAtomicUpdateValue(CodeGenFunction &CGF, AtomicInfo &Atomics, RValue OldRVal,
>> const llvm::function_ref<RValue(RValue)> &UpdateOp,
>> - llvm::Value *DesiredAddr) {
>> - llvm::Value *Ptr = nullptr;
>> - LValue UpdateLVal;
>> + Address DesiredAddr) {
>> RValue UpRVal;
>> LValue AtomicLVal = Atomics.getAtomicLValue();
>> LValue DesiredLVal;
>> if (AtomicLVal.isSimple()) {
>> UpRVal = OldRVal;
>> - DesiredLVal =
>> - LValue::MakeAddr(DesiredAddr, AtomicLVal.getType(),
>> - AtomicLVal.getAlignment(), CGF.CGM.getContext());
>> + DesiredLVal = CGF.MakeAddrLValue(DesiredAddr, AtomicLVal.getType());
>> } else {
>> // Build new lvalue for temp address
>> - Ptr = Atomics.materializeRValue(OldRVal);
>> + Address Ptr = Atomics.materializeRValue(OldRVal);
>> + LValue UpdateLVal;
>> if (AtomicLVal.isBitField()) {
>> UpdateLVal =
>> LValue::MakeBitfield(Ptr, AtomicLVal.getBitFieldInfo(),
>> - AtomicLVal.getType(), AtomicLVal.getAlignment());
>> + AtomicLVal.getType(),
>> + AtomicLVal.getAlignmentSource());
>> DesiredLVal =
>> LValue::MakeBitfield(DesiredAddr, AtomicLVal.getBitFieldInfo(),
>> - AtomicLVal.getType(), AtomicLVal.getAlignment());
>> + AtomicLVal.getType(),
>> + AtomicLVal.getAlignmentSource());
>> } else if (AtomicLVal.isVectorElt()) {
>> UpdateLVal = LValue::MakeVectorElt(Ptr, AtomicLVal.getVectorIdx(),
>> AtomicLVal.getType(),
>> - AtomicLVal.getAlignment());
>> + AtomicLVal.getAlignmentSource());
>> DesiredLVal = LValue::MakeVectorElt(
>> DesiredAddr, AtomicLVal.getVectorIdx(), AtomicLVal.getType(),
>> - AtomicLVal.getAlignment());
>> + AtomicLVal.getAlignmentSource());
>> } else {
>> assert(AtomicLVal.isExtVectorElt());
>> UpdateLVal = LValue::MakeExtVectorElt(Ptr, AtomicLVal.getExtVectorElts(),
>> AtomicLVal.getType(),
>> - AtomicLVal.getAlignment());
>> + AtomicLVal.getAlignmentSource());
>> DesiredLVal = LValue::MakeExtVectorElt(
>> DesiredAddr, AtomicLVal.getExtVectorElts(), AtomicLVal.getType(),
>> - AtomicLVal.getAlignment());
>> + AtomicLVal.getAlignmentSource());
>> }
>> UpdateLVal.setTBAAInfo(AtomicLVal.getTBAAInfo());
>> DesiredLVal.setTBAAInfo(AtomicLVal.getTBAAInfo());
>> @@ -1544,26 +1538,26 @@ void AtomicInfo::EmitAtomicUpdateLibcall
>> bool IsVolatile) {
>> auto Failure = llvm::AtomicCmpXchgInst::getStrongestFailureOrdering(AO);
>>
>> - llvm::Value *ExpectedAddr = CreateTempAlloca();
>> + Address ExpectedAddr = CreateTempAlloca();
>>
>> - EmitAtomicLoadLibcall(ExpectedAddr, AO, IsVolatile);
>> + EmitAtomicLoadLibcall(ExpectedAddr.getPointer(), AO, IsVolatile);
>> auto *ContBB = CGF.createBasicBlock("atomic_cont");
>> auto *ExitBB = CGF.createBasicBlock("atomic_exit");
>> CGF.EmitBlock(ContBB);
>> - auto *DesiredAddr = CreateTempAlloca();
>> + Address DesiredAddr = CreateTempAlloca();
>> if ((LVal.isBitField() && BFI.Size != ValueSizeInBits) ||
>> - requiresMemSetZero(
>> - getAtomicAddress()->getType()->getPointerElementType())) {
>> - auto *OldVal = CGF.Builder.CreateAlignedLoad(
>> - ExpectedAddr, getAtomicAlignment().getQuantity());
>> - CGF.Builder.CreateAlignedStore(OldVal, DesiredAddr,
>> - getAtomicAlignment().getQuantity());
>> + requiresMemSetZero(getAtomicAddress().getElementType())) {
>> + auto *OldVal = CGF.Builder.CreateLoad(ExpectedAddr);
>> + CGF.Builder.CreateStore(OldVal, DesiredAddr);
>> }
>> - auto OldRVal = convertTempToRValue(ExpectedAddr, AggValueSlot::ignored(),
>> - SourceLocation(), /*AsValue=*/false);
>> + auto OldRVal = convertAtomicTempToRValue(ExpectedAddr,
>> + AggValueSlot::ignored(),
>> + SourceLocation(), /*AsValue=*/false);
>> EmitAtomicUpdateValue(CGF, *this, OldRVal, UpdateOp, DesiredAddr);
>> auto *Res =
>> - EmitAtomicCompareExchangeLibcall(ExpectedAddr, DesiredAddr, AO, Failure);
>> + EmitAtomicCompareExchangeLibcall(ExpectedAddr.getPointer(),
>> + DesiredAddr.getPointer(),
>> + AO, Failure);
>> CGF.Builder.CreateCondBr(Res, ExitBB, ContBB);
>> CGF.EmitBlock(ExitBB, /*IsFinished=*/true);
>> }
>> @@ -1583,19 +1577,16 @@ void AtomicInfo::EmitAtomicUpdateOp(
>> llvm::PHINode *PHI = CGF.Builder.CreatePHI(OldVal->getType(),
>> /*NumReservedValues=*/2);
>> PHI->addIncoming(OldVal, CurBB);
>> - auto *NewAtomicAddr = CreateTempAlloca();
>> - auto *NewAtomicIntAddr = emitCastToAtomicIntPointer(NewAtomicAddr);
>> + Address NewAtomicAddr = CreateTempAlloca();
>> + Address NewAtomicIntAddr = emitCastToAtomicIntPointer(NewAtomicAddr);
>> if ((LVal.isBitField() && BFI.Size != ValueSizeInBits) ||
>> - requiresMemSetZero(
>> - getAtomicAddress()->getType()->getPointerElementType())) {
>> - CGF.Builder.CreateAlignedStore(PHI, NewAtomicIntAddr,
>> - getAtomicAlignment().getQuantity());
>> + requiresMemSetZero(getAtomicAddress().getElementType())) {
>> + CGF.Builder.CreateStore(PHI, NewAtomicIntAddr);
>> }
>> auto OldRVal = ConvertIntToValueOrAtomic(PHI, AggValueSlot::ignored(),
>> SourceLocation(), /*AsValue=*/false);
>> EmitAtomicUpdateValue(CGF, *this, OldRVal, UpdateOp, NewAtomicAddr);
>> - auto *DesiredVal = CGF.Builder.CreateAlignedLoad(
>> - NewAtomicIntAddr, getAtomicAlignment().getQuantity());
>> + auto *DesiredVal = CGF.Builder.CreateLoad(NewAtomicIntAddr);
>> // Try to write new value using cmpxchg operation
>> auto Res = EmitAtomicCompareExchangeOp(PHI, DesiredVal, AO, Failure);
>> PHI->addIncoming(Res.first, CGF.Builder.GetInsertBlock());
>> @@ -1604,23 +1595,25 @@ void AtomicInfo::EmitAtomicUpdateOp(
>> }
>>
>> static void EmitAtomicUpdateValue(CodeGenFunction &CGF, AtomicInfo &Atomics,
>> - RValue UpdateRVal, llvm::Value *DesiredAddr) {
>> + RValue UpdateRVal, Address DesiredAddr) {
>> LValue AtomicLVal = Atomics.getAtomicLValue();
>> LValue DesiredLVal;
>> // Build new lvalue for temp address
>> if (AtomicLVal.isBitField()) {
>> DesiredLVal =
>> LValue::MakeBitfield(DesiredAddr, AtomicLVal.getBitFieldInfo(),
>> - AtomicLVal.getType(), AtomicLVal.getAlignment());
>> + AtomicLVal.getType(),
>> + AtomicLVal.getAlignmentSource());
>> } else if (AtomicLVal.isVectorElt()) {
>> DesiredLVal =
>> LValue::MakeVectorElt(DesiredAddr, AtomicLVal.getVectorIdx(),
>> - AtomicLVal.getType(), AtomicLVal.getAlignment());
>> + AtomicLVal.getType(),
>> + AtomicLVal.getAlignmentSource());
>> } else {
>> assert(AtomicLVal.isExtVectorElt());
>> DesiredLVal = LValue::MakeExtVectorElt(
>> DesiredAddr, AtomicLVal.getExtVectorElts(), AtomicLVal.getType(),
>> - AtomicLVal.getAlignment());
>> + AtomicLVal.getAlignmentSource());
>> }
>> DesiredLVal.setTBAAInfo(AtomicLVal.getTBAAInfo());
>> // Store new value in the corresponding memory area
>> @@ -1632,24 +1625,23 @@ void AtomicInfo::EmitAtomicUpdateLibcall
>> RValue UpdateRVal, bool IsVolatile) {
>> auto Failure = llvm::AtomicCmpXchgInst::getStrongestFailureOrdering(AO);
>>
>> - llvm::Value *ExpectedAddr = CreateTempAlloca();
>> + Address ExpectedAddr = CreateTempAlloca();
>>
>> - EmitAtomicLoadLibcall(ExpectedAddr, AO, IsVolatile);
>> + EmitAtomicLoadLibcall(ExpectedAddr.getPointer(), AO, IsVolatile);
>> auto *ContBB = CGF.createBasicBlock("atomic_cont");
>> auto *ExitBB = CGF.createBasicBlock("atomic_exit");
>> CGF.EmitBlock(ContBB);
>> - auto *DesiredAddr = CreateTempAlloca();
>> + Address DesiredAddr = CreateTempAlloca();
>> if ((LVal.isBitField() && BFI.Size != ValueSizeInBits) ||
>> - requiresMemSetZero(
>> - getAtomicAddress()->getType()->getPointerElementType())) {
>> - auto *OldVal = CGF.Builder.CreateAlignedLoad(
>> - ExpectedAddr, getAtomicAlignment().getQuantity());
>> - CGF.Builder.CreateAlignedStore(OldVal, DesiredAddr,
>> - getAtomicAlignment().getQuantity());
>> + requiresMemSetZero(getAtomicAddress().getElementType())) {
>> + auto *OldVal = CGF.Builder.CreateLoad(ExpectedAddr);
>> + CGF.Builder.CreateStore(OldVal, DesiredAddr);
>> }
>> EmitAtomicUpdateValue(CGF, *this, UpdateRVal, DesiredAddr);
>> auto *Res =
>> - EmitAtomicCompareExchangeLibcall(ExpectedAddr, DesiredAddr, AO, Failure);
>> + EmitAtomicCompareExchangeLibcall(ExpectedAddr.getPointer(),
>> + DesiredAddr.getPointer(),
>> + AO, Failure);
>> CGF.Builder.CreateCondBr(Res, ExitBB, ContBB);
>> CGF.EmitBlock(ExitBB, /*IsFinished=*/true);
>> }
>> @@ -1668,17 +1660,14 @@ void AtomicInfo::EmitAtomicUpdateOp(llvm
>> llvm::PHINode *PHI = CGF.Builder.CreatePHI(OldVal->getType(),
>> /*NumReservedValues=*/2);
>> PHI->addIncoming(OldVal, CurBB);
>> - auto *NewAtomicAddr = CreateTempAlloca();
>> - auto *NewAtomicIntAddr = emitCastToAtomicIntPointer(NewAtomicAddr);
>> + Address NewAtomicAddr = CreateTempAlloca();
>> + Address NewAtomicIntAddr = emitCastToAtomicIntPointer(NewAtomicAddr);
>> if ((LVal.isBitField() && BFI.Size != ValueSizeInBits) ||
>> - requiresMemSetZero(
>> - getAtomicAddress()->getType()->getPointerElementType())) {
>> - CGF.Builder.CreateAlignedStore(PHI, NewAtomicIntAddr,
>> - getAtomicAlignment().getQuantity());
>> + requiresMemSetZero(getAtomicAddress().getElementType())) {
>> + CGF.Builder.CreateStore(PHI, NewAtomicIntAddr);
>> }
>> EmitAtomicUpdateValue(CGF, *this, UpdateRVal, NewAtomicAddr);
>> - auto *DesiredVal = CGF.Builder.CreateAlignedLoad(
>> - NewAtomicIntAddr, getAtomicAlignment().getQuantity());
>> + auto *DesiredVal = CGF.Builder.CreateLoad(NewAtomicIntAddr);
>> // Try to write new value using cmpxchg operation
>> auto Res = EmitAtomicCompareExchangeOp(PHI, DesiredVal, AO, Failure);
>> PHI->addIncoming(Res.first, CGF.Builder.GetInsertBlock());
>> @@ -1729,8 +1718,8 @@ void CodeGenFunction::EmitAtomicStore(RV
>> // If this is an aggregate r-value, it should agree in type except
>> // maybe for address-space qualification.
>> assert(!rvalue.isAggregate() ||
>> - rvalue.getAggregateAddr()->getType()->getPointerElementType()
>> - == dest.getAddress()->getType()->getPointerElementType());
>> + rvalue.getAggregateAddress().getElementType()
>> + == dest.getAddress().getElementType());
>>
>> AtomicInfo atomics(*this, dest);
>> LValue LVal = atomics.getAtomicLValue();
>> @@ -1745,15 +1734,16 @@ void CodeGenFunction::EmitAtomicStore(RV
>> // Check whether we should use a library call.
>> if (atomics.shouldUseLibcall()) {
>> // Produce a source address.
>> - llvm::Value *srcAddr = atomics.materializeRValue(rvalue);
>> + Address srcAddr = atomics.materializeRValue(rvalue);
>>
>> // void __atomic_store(size_t size, void *mem, void *val, int order)
>> CallArgList args;
>> args.add(RValue::get(atomics.getAtomicSizeValue()),
>> getContext().getSizeType());
>> - args.add(RValue::get(EmitCastToVoidPtr(atomics.getAtomicAddress())),
>> + args.add(RValue::get(EmitCastToVoidPtr(atomics.getAtomicPointer())),
>> + getContext().VoidPtrTy);
>> + args.add(RValue::get(EmitCastToVoidPtr(srcAddr.getPointer())),
>> getContext().VoidPtrTy);
>> - args.add(RValue::get(EmitCastToVoidPtr(srcAddr)), getContext().VoidPtrTy);
>> args.add(RValue::get(llvm::ConstantInt::get(
>> IntTy, AtomicInfo::translateAtomicOrdering(AO))),
>> getContext().IntTy);
>> @@ -1765,10 +1755,10 @@ void CodeGenFunction::EmitAtomicStore(RV
>> llvm::Value *intValue = atomics.convertRValueToInt(rvalue);
>>
>> // Do the atomic store.
>> - llvm::Value *addr =
>> + Address addr =
>> atomics.emitCastToAtomicIntPointer(atomics.getAtomicAddress());
>> intValue = Builder.CreateIntCast(
>> - intValue, addr->getType()->getPointerElementType(), /*isSigned=*/false);
>> + intValue, addr.getElementType(), /*isSigned=*/false);
>> llvm::StoreInst *store = Builder.CreateStore(intValue, addr);
>>
>> // Initializations don't need to be atomic.
>> @@ -1776,7 +1766,6 @@ void CodeGenFunction::EmitAtomicStore(RV
>> store->setAtomic(AO);
>>
>> // Other decoration.
>> - store->setAlignment(dest.getAlignment().getQuantity());
>> if (IsVolatile)
>> store->setVolatile(true);
>> if (dest.getTBAAInfo())
>> @@ -1797,11 +1786,11 @@ std::pair<RValue, llvm::Value *> CodeGen
>> // If this is an aggregate r-value, it should agree in type except
>> // maybe for address-space qualification.
>> assert(!Expected.isAggregate() ||
>> - Expected.getAggregateAddr()->getType()->getPointerElementType() ==
>> - Obj.getAddress()->getType()->getPointerElementType());
>> + Expected.getAggregateAddress().getElementType() ==
>> + Obj.getAddress().getElementType());
>> assert(!Desired.isAggregate() ||
>> - Desired.getAggregateAddr()->getType()->getPointerElementType() ==
>> - Obj.getAddress()->getType()->getPointerElementType());
>> + Desired.getAggregateAddress().getElementType() ==
>> + Obj.getAddress().getElementType());
>> AtomicInfo Atomics(*this, Obj);
>>
>> return Atomics.EmitAtomicCompareExchange(Expected, Desired, Success, Failure,
>>
>> Modified: cfe/trunk/lib/CodeGen/CGBlocks.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGBlocks.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGBlocks.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGBlocks.cpp Tue Sep 8 03:05:57 2015
>> @@ -30,7 +30,7 @@ using namespace CodeGen;
>> CGBlockInfo::CGBlockInfo(const BlockDecl *block, StringRef name)
>> : Name(name), CXXThisIndex(0), CanBeGlobal(false), NeedsCopyDispose(false),
>> HasCXXObject(false), UsesStret(false), HasCapturedVariableLayout(false),
>> - StructureType(nullptr), Block(block),
>> + LocalAddress(Address::invalid()), StructureType(nullptr), Block(block),
>> DominatingIP(nullptr) {
>>
>> // Skip asm prefix, if any. 'name' is usually taken directly from
>> @@ -40,7 +40,7 @@ CGBlockInfo::CGBlockInfo(const BlockDecl
>> }
>>
>> // Anchor the vtable to this translation unit.
>> -CodeGenModule::ByrefHelpers::~ByrefHelpers() {}
>> +BlockByrefHelpers::~BlockByrefHelpers() {}
>>
>> /// Build the given block as a global block.
>> static llvm::Constant *buildGlobalBlock(CodeGenModule &CGM,
>> @@ -111,7 +111,7 @@ static llvm::Constant *buildBlockDescrip
>> std::string typeAtEncoding =
>> CGM.getContext().getObjCEncodingForBlock(blockInfo.getBlockExpr());
>> elements.push_back(llvm::ConstantExpr::getBitCast(
>> - CGM.GetAddrOfConstantCString(typeAtEncoding), i8p));
>> + CGM.GetAddrOfConstantCString(typeAtEncoding).getPointer(), i8p));
>>
>> // GC layout.
>> if (C.getLangOpts().ObjC1) {
>> @@ -203,44 +203,34 @@ namespace {
>> Capture(capture), Type(type) {}
>>
>> /// Tell the block info that this chunk has the given field index.
>> - void setIndex(CGBlockInfo &info, unsigned index) {
>> - if (!Capture)
>> + void setIndex(CGBlockInfo &info, unsigned index, CharUnits offset) {
>> + if (!Capture) {
>> info.CXXThisIndex = index;
>> - else
>> - info.Captures[Capture->getVariable()]
>> - = CGBlockInfo::Capture::makeIndex(index);
>> + info.CXXThisOffset = offset;
>> + } else {
>> + info.Captures.insert({Capture->getVariable(),
>> + CGBlockInfo::Capture::makeIndex(index, offset)});
>> + }
>> }
>> };
>>
>> /// Order by 1) all __strong together 2) next, all byfref together 3) next,
>> /// all __weak together. Preserve descending alignment in all situations.
>> bool operator<(const BlockLayoutChunk &left, const BlockLayoutChunk &right) {
>> - CharUnits LeftValue, RightValue;
>> - bool LeftByref = left.Capture ? left.Capture->isByRef() : false;
>> - bool RightByref = right.Capture ? right.Capture->isByRef() : false;
>> -
>> - if (left.Lifetime == Qualifiers::OCL_Strong &&
>> - left.Alignment >= right.Alignment)
>> - LeftValue = CharUnits::fromQuantity(64);
>> - else if (LeftByref && left.Alignment >= right.Alignment)
>> - LeftValue = CharUnits::fromQuantity(32);
>> - else if (left.Lifetime == Qualifiers::OCL_Weak &&
>> - left.Alignment >= right.Alignment)
>> - LeftValue = CharUnits::fromQuantity(16);
>> - else
>> - LeftValue = left.Alignment;
>> - if (right.Lifetime == Qualifiers::OCL_Strong &&
>> - right.Alignment >= left.Alignment)
>> - RightValue = CharUnits::fromQuantity(64);
>> - else if (RightByref && right.Alignment >= left.Alignment)
>> - RightValue = CharUnits::fromQuantity(32);
>> - else if (right.Lifetime == Qualifiers::OCL_Weak &&
>> - right.Alignment >= left.Alignment)
>> - RightValue = CharUnits::fromQuantity(16);
>> - else
>> - RightValue = right.Alignment;
>> -
>> - return LeftValue > RightValue;
>> + if (left.Alignment != right.Alignment)
>> + return left.Alignment > right.Alignment;
>> +
>> + auto getPrefOrder = [](const BlockLayoutChunk &chunk) {
>> + if (chunk.Capture->isByRef())
>> + return 1;
>> + if (chunk.Lifetime == Qualifiers::OCL_Strong)
>> + return 0;
>> + if (chunk.Lifetime == Qualifiers::OCL_Weak)
>> + return 2;
>> + return 3;
>> + };
>> +
>> + return getPrefOrder(left) < getPrefOrder(right);
>> }
>> }
>>
>> @@ -302,31 +292,20 @@ static CharUnits getLowBit(CharUnits v)
>>
>> static void initializeForBlockHeader(CodeGenModule &CGM, CGBlockInfo &info,
>> SmallVectorImpl<llvm::Type*> &elementTypes) {
>> - ASTContext &C = CGM.getContext();
>> -
>> - // The header is basically a 'struct { void *; int; int; void *; void *; }'.
>> - CharUnits ptrSize, ptrAlign, intSize, intAlign;
>> - std::tie(ptrSize, ptrAlign) = C.getTypeInfoInChars(C.VoidPtrTy);
>> - std::tie(intSize, intAlign) = C.getTypeInfoInChars(C.IntTy);
>> -
>> - // Are there crazy embedded platforms where this isn't true?
>> - assert(intSize <= ptrSize && "layout assumptions horribly violated");
>> -
>> - CharUnits headerSize = ptrSize;
>> - if (2 * intSize < ptrAlign) headerSize += ptrSize;
>> - else headerSize += 2 * intSize;
>> - headerSize += 2 * ptrSize;
>> + // The header is basically 'struct { void *; int; int; void *; void *; }'.
>> + // Assert that that struct is packed.
>> + assert(CGM.getIntSize() <= CGM.getPointerSize());
>> + assert(CGM.getIntAlign() <= CGM.getPointerAlign());
>> + assert((2 * CGM.getIntSize()).isMultipleOf(CGM.getPointerAlign()));
>>
>> - info.BlockAlign = ptrAlign;
>> - info.BlockSize = headerSize;
>> + info.BlockAlign = CGM.getPointerAlign();
>> + info.BlockSize = 3 * CGM.getPointerSize() + 2 * CGM.getIntSize();
>>
>> assert(elementTypes.empty());
>> - llvm::Type *i8p = CGM.getTypes().ConvertType(C.VoidPtrTy);
>> - llvm::Type *intTy = CGM.getTypes().ConvertType(C.IntTy);
>> - elementTypes.push_back(i8p);
>> - elementTypes.push_back(intTy);
>> - elementTypes.push_back(intTy);
>> - elementTypes.push_back(i8p);
>> + elementTypes.push_back(CGM.VoidPtrTy);
>> + elementTypes.push_back(CGM.IntTy);
>> + elementTypes.push_back(CGM.IntTy);
>> + elementTypes.push_back(CGM.VoidPtrTy);
>> elementTypes.push_back(CGM.getBlockDescriptorType());
>>
>> assert(elementTypes.size() == BlockHeaderSize);
>> @@ -365,6 +344,8 @@ static void computeBlockInfo(CodeGenModu
>> "Can't capture 'this' outside a method");
>> QualType thisType = cast<CXXMethodDecl>(CGF->CurFuncDecl)->getThisType(C);
>>
>> + // Theoretically, this could be in a different address space, so
>> + // don't assume standard pointer size/align.
>> llvm::Type *llvmType = CGM.getTypes().ConvertType(thisType);
>> std::pair<CharUnits,CharUnits> tinfo
>> = CGM.getContext().getTypeInfoInChars(thisType);
>> @@ -384,15 +365,12 @@ static void computeBlockInfo(CodeGenModu
>> info.NeedsCopyDispose = true;
>>
>> // Just use void* instead of a pointer to the byref type.
>> - QualType byRefPtrTy = C.VoidPtrTy;
>> + CharUnits align = CGM.getPointerAlign();
>> + maxFieldAlign = std::max(maxFieldAlign, align);
>>
>> - llvm::Type *llvmType = CGM.getTypes().ConvertType(byRefPtrTy);
>> - std::pair<CharUnits,CharUnits> tinfo
>> - = CGM.getContext().getTypeInfoInChars(byRefPtrTy);
>> - maxFieldAlign = std::max(maxFieldAlign, tinfo.second);
>> -
>> - layout.push_back(BlockLayoutChunk(tinfo.second, tinfo.first,
>> - Qualifiers::OCL_None, &CI, llvmType));
>> + layout.push_back(BlockLayoutChunk(align, CGM.getPointerSize(),
>> + Qualifiers::OCL_None, &CI,
>> + CGM.VoidPtrTy));
>> continue;
>> }
>>
>> @@ -504,18 +482,13 @@ static void computeBlockInfo(CodeGenModu
>> for (; li != le; ++li) {
>> assert(endAlign >= li->Alignment);
>>
>> - li->setIndex(info, elementTypes.size());
>> + li->setIndex(info, elementTypes.size(), blockSize);
>> elementTypes.push_back(li->Type);
>> blockSize += li->Size;
>> endAlign = getLowBit(blockSize);
>>
>> // ...until we get to the alignment of the maximum field.
>> if (endAlign >= maxFieldAlign) {
>> - if (li == first) {
>> - // No user field was appended. So, a gap was added.
>> - // Save total gap size for use in block layout bit map.
>> - info.BlockHeaderForcedGapSize = li->Size;
>> - }
>> break;
>> }
>> }
>> @@ -532,6 +505,12 @@ static void computeBlockInfo(CodeGenModu
>> CharUnits newBlockSize = blockSize.RoundUpToAlignment(maxFieldAlign);
>> CharUnits padding = newBlockSize - blockSize;
>>
>> + // If we haven't yet added any fields, remember that there was an
>> + // initial gap; this need to go into the block layout bit map.
>> + if (blockSize == info.BlockHeaderForcedGapOffset) {
>> + info.BlockHeaderForcedGapSize = padding;
>> + }
>> +
>> elementTypes.push_back(llvm::ArrayType::get(CGM.Int8Ty,
>> padding.getQuantity()));
>> blockSize = newBlockSize;
>> @@ -556,7 +535,7 @@ static void computeBlockInfo(CodeGenModu
>> endAlign = getLowBit(blockSize);
>> }
>> assert(endAlign >= li->Alignment);
>> - li->setIndex(info, elementTypes.size());
>> + li->setIndex(info, elementTypes.size(), blockSize);
>> elementTypes.push_back(li->Type);
>> blockSize += li->Size;
>> endAlign = getLowBit(blockSize);
>> @@ -586,9 +565,8 @@ static void enterBlockScope(CodeGenFunct
>> if (blockInfo.CanBeGlobal) return;
>>
>> // Make the allocation for the block.
>> - blockInfo.Address =
>> - CGF.CreateTempAlloca(blockInfo.StructureType, "block");
>> - blockInfo.Address->setAlignment(blockInfo.BlockAlign.getQuantity());
>> + blockInfo.LocalAddress = CGF.CreateTempAlloca(blockInfo.StructureType,
>> + blockInfo.BlockAlign, "block");
>>
>> // If there are cleanups to emit, enter them (but inactive).
>> if (!blockInfo.NeedsCopyDispose) return;
>> @@ -621,12 +599,13 @@ static void enterBlockScope(CodeGenFunct
>> }
>>
>> // GEP down to the address.
>> - llvm::Value *addr = CGF.Builder.CreateStructGEP(
>> - blockInfo.StructureType, blockInfo.Address, capture.getIndex());
>> + Address addr = CGF.Builder.CreateStructGEP(blockInfo.LocalAddress,
>> + capture.getIndex(),
>> + capture.getOffset());
>>
>> // We can use that GEP as the dominating IP.
>> if (!blockInfo.DominatingIP)
>> - blockInfo.DominatingIP = cast<llvm::Instruction>(addr);
>> + blockInfo.DominatingIP = cast<llvm::Instruction>(addr.getPointer());
>>
>> CleanupKind cleanupKind = InactiveNormalCleanup;
>> bool useArrayEHCleanup = CGF.needsEHCleanup(dtorKind);
>> @@ -721,9 +700,8 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> // Build the block descriptor.
>> llvm::Constant *descriptor = buildBlockDescriptor(CGM, blockInfo);
>>
>> - llvm::Type *blockTy = blockInfo.StructureType;
>> - llvm::AllocaInst *blockAddr = blockInfo.Address;
>> - assert(blockAddr && "block has no address!");
>> + Address blockAddr = blockInfo.LocalAddress;
>> + assert(blockAddr.isValid() && "block has no address!");
>>
>> // Compute the initial on-stack block flags.
>> BlockFlags flags = BLOCK_HAS_SIGNATURE;
>> @@ -732,27 +710,44 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> if (blockInfo.HasCXXObject) flags |= BLOCK_HAS_CXX_OBJ;
>> if (blockInfo.UsesStret) flags |= BLOCK_USE_STRET;
>>
>> - // Initialize the block literal.
>> - Builder.CreateStore(
>> - isa, Builder.CreateStructGEP(blockTy, blockAddr, 0, "block.isa"));
>> - Builder.CreateStore(
>> - llvm::ConstantInt::get(IntTy, flags.getBitMask()),
>> - Builder.CreateStructGEP(blockTy, blockAddr, 1, "block.flags"));
>> - Builder.CreateStore(
>> - llvm::ConstantInt::get(IntTy, 0),
>> - Builder.CreateStructGEP(blockTy, blockAddr, 2, "block.reserved"));
>> - Builder.CreateStore(
>> - blockFn, Builder.CreateStructGEP(blockTy, blockAddr, 3, "block.invoke"));
>> - Builder.CreateStore(descriptor, Builder.CreateStructGEP(blockTy, blockAddr, 4,
>> - "block.descriptor"));
>> + auto projectField =
>> + [&](unsigned index, CharUnits offset, const Twine &name) -> Address {
>> + return Builder.CreateStructGEP(blockAddr, index, offset, name);
>> + };
>> + auto storeField =
>> + [&](llvm::Value *value, unsigned index, CharUnits offset,
>> + const Twine &name) {
>> + Builder.CreateStore(value, projectField(index, offset, name));
>> + };
>> +
>> + // Initialize the block header.
>> + {
>> + // We assume all the header fields are densely packed.
>> + unsigned index = 0;
>> + CharUnits offset;
>> + auto addHeaderField =
>> + [&](llvm::Value *value, CharUnits size, const Twine &name) {
>> + storeField(value, index, offset, name);
>> + offset += size;
>> + index++;
>> + };
>> +
>> + addHeaderField(isa, getPointerSize(), "block.isa");
>> + addHeaderField(llvm::ConstantInt::get(IntTy, flags.getBitMask()),
>> + getIntSize(), "block.flags");
>> + addHeaderField(llvm::ConstantInt::get(IntTy, 0),
>> + getIntSize(), "block.reserved");
>> + addHeaderField(blockFn, getPointerSize(), "block.invoke");
>> + addHeaderField(descriptor, getPointerSize(), "block.descriptor");
>> + }
>>
>> // Finally, capture all the values into the block.
>> const BlockDecl *blockDecl = blockInfo.getBlockDecl();
>>
>> // First, 'this'.
>> if (blockDecl->capturesCXXThis()) {
>> - llvm::Value *addr = Builder.CreateStructGEP(
>> - blockTy, blockAddr, blockInfo.CXXThisIndex, "block.captured-this.addr");
>> + Address addr = projectField(blockInfo.CXXThisIndex, blockInfo.CXXThisOffset,
>> + "block.captured-this.addr");
>> Builder.CreateStore(LoadCXXThis(), addr);
>> }
>>
>> @@ -765,35 +760,37 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> if (capture.isConstant()) continue;
>>
>> QualType type = variable->getType();
>> - CharUnits align = getContext().getDeclAlign(variable);
>>
>> // This will be a [[type]]*, except that a byref entry will just be
>> // an i8**.
>> - llvm::Value *blockField = Builder.CreateStructGEP(
>> - blockTy, blockAddr, capture.getIndex(), "block.captured");
>> + Address blockField =
>> + projectField(capture.getIndex(), capture.getOffset(), "block.captured");
>>
>> // Compute the address of the thing we're going to move into the
>> // block literal.
>> - llvm::Value *src;
>> + Address src = Address::invalid();
>> if (BlockInfo && CI.isNested()) {
>> // We need to use the capture from the enclosing block.
>> const CGBlockInfo::Capture &enclosingCapture =
>> BlockInfo->getCapture(variable);
>>
>> // This is a [[type]]*, except that a byref entry wil just be an i8**.
>> - src = Builder.CreateStructGEP(BlockInfo->StructureType, LoadBlockStruct(),
>> + src = Builder.CreateStructGEP(LoadBlockStruct(),
>> enclosingCapture.getIndex(),
>> + enclosingCapture.getOffset(),
>> "block.capture.addr");
>> } else if (blockDecl->isConversionFromLambda()) {
>> // The lambda capture in a lambda's conversion-to-block-pointer is
>> // special; we'll simply emit it directly.
>> - src = nullptr;
>> + src = Address::invalid();
>> } else {
>> // Just look it up in the locals map, which will give us back a
>> // [[type]]*. If that doesn't work, do the more elaborate DRE
>> // emission.
>> - src = LocalDeclMap.lookup(variable);
>> - if (!src) {
>> + auto it = LocalDeclMap.find(variable);
>> + if (it != LocalDeclMap.end()) {
>> + src = it->second;
>> + } else {
>> DeclRefExpr declRef(
>> const_cast<VarDecl *>(variable),
>> /*RefersToEnclosingVariableOrCapture*/ CI.isNested(), type,
>> @@ -808,14 +805,14 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> // live a shorter life than the stack byref anyway.
>> if (CI.isByRef()) {
>> // Get a void* that points to the byref struct.
>> + llvm::Value *byrefPointer;
>> if (CI.isNested())
>> - src = Builder.CreateAlignedLoad(src, align.getQuantity(),
>> - "byref.capture");
>> + byrefPointer = Builder.CreateLoad(src, "byref.capture");
>> else
>> - src = Builder.CreateBitCast(src, VoidPtrTy);
>> + byrefPointer = Builder.CreateBitCast(src.getPointer(), VoidPtrTy);
>>
>> // Write that void* into the capture field.
>> - Builder.CreateAlignedStore(src, blockField, align.getQuantity());
>> + Builder.CreateStore(byrefPointer, blockField);
>>
>> // If we have a copy constructor, evaluate that into the block field.
>> } else if (const Expr *copyExpr = CI.getCopyExpr()) {
>> @@ -823,7 +820,7 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> // If we have a lambda conversion, emit the expression
>> // directly into the block instead.
>> AggValueSlot Slot =
>> - AggValueSlot::forAddr(blockField, align, Qualifiers(),
>> + AggValueSlot::forAddr(blockField, Qualifiers(),
>> AggValueSlot::IsDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsNotAliased);
>> @@ -834,9 +831,8 @@ llvm::Value *CodeGenFunction::EmitBlockL
>>
>> // If it's a reference variable, copy the reference into the block field.
>> } else if (type->isReferenceType()) {
>> - llvm::Value *ref =
>> - Builder.CreateAlignedLoad(src, align.getQuantity(), "ref.val");
>> - Builder.CreateAlignedStore(ref, blockField, align.getQuantity());
>> + llvm::Value *ref = Builder.CreateLoad(src, "ref.val");
>> + Builder.CreateStore(ref, blockField);
>>
>> // If this is an ARC __strong block-pointer variable, don't do a
>> // block copy.
>> @@ -848,13 +844,11 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> } else if (type.getObjCLifetime() == Qualifiers::OCL_Strong &&
>> type->isBlockPointerType()) {
>> // Load the block and do a simple retain.
>> - LValue srcLV = MakeAddrLValue(src, type, align);
>> - llvm::Value *value = EmitLoadOfScalar(srcLV, SourceLocation());
>> + llvm::Value *value = Builder.CreateLoad(src, "block.captured_block");
>> value = EmitARCRetainNonBlock(value);
>>
>> // Do a primitive store to the block field.
>> - LValue destLV = MakeAddrLValue(blockField, type, align);
>> - EmitStoreOfScalar(value, destLV, /*init*/ true);
>> + Builder.CreateStore(value, blockField);
>>
>> // Otherwise, fake up a POD copy into the block field.
>> } else {
>> @@ -876,7 +870,7 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> // attributed to a reasonable location - otherwise it may be attributed to
>> // locations of subexpressions in the initialization.
>> EmitExprAsInit(&l2r, &blockFieldPseudoVar,
>> - MakeAddrLValue(blockField, type, align),
>> + MakeAddrLValue(blockField, type, AlignmentSource::Decl),
>> /*captured by init*/ false);
>> }
>>
>> @@ -891,7 +885,7 @@ llvm::Value *CodeGenFunction::EmitBlockL
>> // Cast to the converted block-pointer type, which happens (somewhat
>> // unfortunately) to be a pointer to function type.
>> llvm::Value *result =
>> - Builder.CreateBitCast(blockAddr,
>> + Builder.CreateBitCast(blockAddr.getPointer(),
>> ConvertType(blockInfo.getBlockExpr()->getType()));
>>
>> return result;
>> @@ -966,8 +960,8 @@ RValue CodeGenFunction::EmitBlockCallExp
>> Builder.CreateBitCast(Callee, BlockLiteralTy, "block.literal");
>>
>> // Get the function pointer from the literal.
>> - llvm::Value *FuncPtr = Builder.CreateStructGEP(
>> - CGM.getGenericBlockLiteralType(), BlockLiteral, 3);
>> + llvm::Value *FuncPtr =
>> + Builder.CreateStructGEP(CGM.getGenericBlockLiteralType(), BlockLiteral, 3);
>>
>> BlockLiteral = Builder.CreateBitCast(BlockLiteral, VoidPtrTy);
>>
>> @@ -981,7 +975,7 @@ RValue CodeGenFunction::EmitBlockCallExp
>> EmitCallArgs(Args, FnType->getAs<FunctionProtoType>(), E->arguments());
>>
>> // Load the function.
>> - llvm::Value *Func = Builder.CreateLoad(FuncPtr);
>> + llvm::Value *Func = Builder.CreateAlignedLoad(FuncPtr, getPointerAlign());
>>
>> const FunctionType *FuncTy = FnType->castAs<FunctionType>();
>> const CGFunctionInfo &FnInfo =
>> @@ -997,41 +991,35 @@ RValue CodeGenFunction::EmitBlockCallExp
>> return EmitCall(FnInfo, Func, ReturnValue, Args);
>> }
>>
>> -llvm::Value *CodeGenFunction::GetAddrOfBlockDecl(const VarDecl *variable,
>> - bool isByRef) {
>> +Address CodeGenFunction::GetAddrOfBlockDecl(const VarDecl *variable,
>> + bool isByRef) {
>> assert(BlockInfo && "evaluating block ref without block information?");
>> const CGBlockInfo::Capture &capture = BlockInfo->getCapture(variable);
>>
>> // Handle constant captures.
>> - if (capture.isConstant()) return LocalDeclMap[variable];
>> + if (capture.isConstant()) return LocalDeclMap.find(variable)->second;
>>
>> - llvm::Value *addr =
>> - Builder.CreateStructGEP(BlockInfo->StructureType, LoadBlockStruct(),
>> - capture.getIndex(), "block.capture.addr");
>> + Address addr =
>> + Builder.CreateStructGEP(LoadBlockStruct(), capture.getIndex(),
>> + capture.getOffset(), "block.capture.addr");
>>
>> if (isByRef) {
>> // addr should be a void** right now. Load, then cast the result
>> // to byref*.
>>
>> - addr = Builder.CreateLoad(addr);
>> - auto *byrefType = BuildByRefType(variable);
>> - llvm::PointerType *byrefPointerType = llvm::PointerType::get(byrefType, 0);
>> - addr = Builder.CreateBitCast(addr, byrefPointerType,
>> - "byref.addr");
>> -
>> - // Follow the forwarding pointer.
>> - addr = Builder.CreateStructGEP(byrefType, addr, 1, "byref.forwarding");
>> - addr = Builder.CreateLoad(addr, "byref.addr.forwarded");
>> -
>> - // Cast back to byref* and GEP over to the actual object.
>> - addr = Builder.CreateBitCast(addr, byrefPointerType);
>> - addr = Builder.CreateStructGEP(byrefType, addr,
>> - getByRefValueLLVMField(variable).second,
>> - variable->getNameAsString());
>> + auto &byrefInfo = getBlockByrefInfo(variable);
>> + addr = Address(Builder.CreateLoad(addr), byrefInfo.ByrefAlignment);
>> +
>> + auto byrefPointerType = llvm::PointerType::get(byrefInfo.Type, 0);
>> + addr = Builder.CreateBitCast(addr, byrefPointerType, "byref.addr");
>> +
>> + addr = emitBlockByrefAddress(addr, byrefInfo, /*follow*/ true,
>> + variable->getName());
>> }
>>
>> - if (variable->getType()->isReferenceType())
>> - addr = Builder.CreateLoad(addr, "ref.tmp");
>> + if (auto refType = variable->getType()->getAs<ReferenceType>()) {
>> + addr = EmitLoadOfReference(addr, refType);
>> + }
>>
>> return addr;
>> }
>> @@ -1048,7 +1036,7 @@ CodeGenModule::GetAddrOfGlobalBlock(cons
>> // Using that metadata, generate the actual block function.
>> llvm::Constant *blockFn;
>> {
>> - llvm::DenseMap<const Decl*, llvm::Value*> LocalDeclMap;
>> + CodeGenFunction::DeclMapTy LocalDeclMap;
>> blockFn = CodeGenFunction(*this).GenerateBlockFunction(GlobalDecl(),
>> blockInfo,
>> LocalDeclMap,
>> @@ -1102,6 +1090,44 @@ static llvm::Constant *buildGlobalBlock(
>> return llvm::ConstantExpr::getBitCast(literal, requiredType);
>> }
>>
>> +void CodeGenFunction::setBlockContextParameter(const ImplicitParamDecl *D,
>> + unsigned argNum,
>> + llvm::Value *arg) {
>> + assert(BlockInfo && "not emitting prologue of block invocation function?!");
>> +
>> + llvm::Value *localAddr = nullptr;
>> + if (CGM.getCodeGenOpts().OptimizationLevel == 0) {
>> + // Allocate a stack slot to let the debug info survive the RA.
>> + Address alloc = CreateMemTemp(D->getType(), D->getName() + ".addr");
>> + Builder.CreateStore(arg, alloc);
>> + localAddr = Builder.CreateLoad(alloc);
>> + }
>> +
>> + if (CGDebugInfo *DI = getDebugInfo()) {
>> + if (CGM.getCodeGenOpts().getDebugInfo()
>> + >= CodeGenOptions::LimitedDebugInfo) {
>> + DI->setLocation(D->getLocation());
>> + DI->EmitDeclareOfBlockLiteralArgVariable(*BlockInfo, arg, argNum,
>> + localAddr, Builder);
>> + }
>> + }
>> +
>> + SourceLocation StartLoc = BlockInfo->getBlockExpr()->getBody()->getLocStart();
>> + ApplyDebugLocation Scope(*this, StartLoc);
>> +
>> + // Instead of messing around with LocalDeclMap, just set the value
>> + // directly as BlockPointer.
>> + BlockPointer = Builder.CreateBitCast(arg,
>> + BlockInfo->StructureType->getPointerTo(),
>> + "block");
>> +}
>> +
>> +Address CodeGenFunction::LoadBlockStruct() {
>> + assert(BlockInfo && "not in a block invocation function!");
>> + assert(BlockPointer && "no block pointer set!");
>> + return Address(BlockPointer, BlockInfo->BlockAlign);
>> +}
>> +
>> llvm::Function *
>> CodeGenFunction::GenerateBlockFunction(GlobalDecl GD,
>> const CGBlockInfo &blockInfo,
>> @@ -1121,7 +1147,7 @@ CodeGenFunction::GenerateBlockFunction(G
>> for (DeclMapTy::const_iterator i = ldm.begin(), e = ldm.end(); i != e; ++i) {
>> const auto *var = dyn_cast<VarDecl>(i->first);
>> if (var && !var->hasLocalStorage())
>> - LocalDeclMap[var] = i->second;
>> + setAddrOfLocalVar(var, i->second);
>> }
>>
>> // Begin building the function declaration.
>> @@ -1162,35 +1188,28 @@ CodeGenFunction::GenerateBlockFunction(G
>> blockInfo.getBlockExpr()->getBody()->getLocStart());
>>
>> // Okay. Undo some of what StartFunction did.
>> -
>> - // Pull the 'self' reference out of the local decl map.
>> - llvm::Value *blockAddr = LocalDeclMap[&selfDecl];
>> - LocalDeclMap.erase(&selfDecl);
>> - BlockPointer = Builder.CreateBitCast(blockAddr,
>> - blockInfo.StructureType->getPointerTo(),
>> - "block");
>> +
>> // At -O0 we generate an explicit alloca for the BlockPointer, so the RA
>> // won't delete the dbg.declare intrinsics for captured variables.
>> llvm::Value *BlockPointerDbgLoc = BlockPointer;
>> if (CGM.getCodeGenOpts().OptimizationLevel == 0) {
>> // Allocate a stack slot for it, so we can point the debugger to it
>> - llvm::AllocaInst *Alloca = CreateTempAlloca(BlockPointer->getType(),
>> - "block.addr");
>> - unsigned Align = getContext().getDeclAlign(&selfDecl).getQuantity();
>> - Alloca->setAlignment(Align);
>> + Address Alloca = CreateTempAlloca(BlockPointer->getType(),
>> + getPointerAlign(),
>> + "block.addr");
>> // Set the DebugLocation to empty, so the store is recognized as a
>> // frame setup instruction by llvm::DwarfDebug::beginFunction().
>> auto NL = ApplyDebugLocation::CreateEmpty(*this);
>> - Builder.CreateAlignedStore(BlockPointer, Alloca, Align);
>> - BlockPointerDbgLoc = Alloca;
>> + Builder.CreateStore(BlockPointer, Alloca);
>> + BlockPointerDbgLoc = Alloca.getPointer();
>> }
>>
>> // If we have a C++ 'this' reference, go ahead and force it into
>> // existence now.
>> if (blockDecl->capturesCXXThis()) {
>> - llvm::Value *addr =
>> - Builder.CreateStructGEP(blockInfo.StructureType, BlockPointer,
>> - blockInfo.CXXThisIndex, "block.captured-this");
>> + Address addr =
>> + Builder.CreateStructGEP(LoadBlockStruct(), blockInfo.CXXThisIndex,
>> + blockInfo.CXXThisOffset, "block.captured-this");
>> CXXThisValue = Builder.CreateLoad(addr, "this");
>> }
>>
>> @@ -1200,15 +1219,13 @@ CodeGenFunction::GenerateBlockFunction(G
>> const CGBlockInfo::Capture &capture = blockInfo.getCapture(variable);
>> if (!capture.isConstant()) continue;
>>
>> - unsigned align = getContext().getDeclAlign(variable).getQuantity();
>> -
>> - llvm::AllocaInst *alloca =
>> - CreateMemTemp(variable->getType(), "block.captured-const");
>> - alloca->setAlignment(align);
>> + CharUnits align = getContext().getDeclAlign(variable);
>> + Address alloca =
>> + CreateMemTemp(variable->getType(), align, "block.captured-const");
>>
>> - Builder.CreateAlignedStore(capture.getConstant(), alloca, align);
>> + Builder.CreateStore(capture.getConstant(), alloca);
>>
>> - LocalDeclMap[variable] = alloca;
>> + setAddrOfLocalVar(variable, alloca);
>> }
>>
>> // Save a spot to insert the debug information for all the DeclRefExprs.
>> @@ -1242,7 +1259,8 @@ CodeGenFunction::GenerateBlockFunction(G
>>> = CodeGenOptions::LimitedDebugInfo) {
>> const CGBlockInfo::Capture &capture = blockInfo.getCapture(variable);
>> if (capture.isConstant()) {
>> - DI->EmitDeclareOfAutoVariable(variable, LocalDeclMap[variable],
>> + auto addr = LocalDeclMap.find(variable)->second;
>> + DI->EmitDeclareOfAutoVariable(variable, addr.getPointer(),
>> Builder);
>> continue;
>> }
>> @@ -1335,12 +1353,12 @@ CodeGenFunction::GenerateCopyHelperFunct
>> auto AL = ApplyDebugLocation::CreateArtificial(*this);
>> llvm::Type *structPtrTy = blockInfo.StructureType->getPointerTo();
>>
>> - llvm::Value *src = GetAddrOfLocalVar(&srcDecl);
>> - src = Builder.CreateLoad(src);
>> + Address src = GetAddrOfLocalVar(&srcDecl);
>> + src = Address(Builder.CreateLoad(src), blockInfo.BlockAlign);
>> src = Builder.CreateBitCast(src, structPtrTy, "block.source");
>>
>> - llvm::Value *dst = GetAddrOfLocalVar(&dstDecl);
>> - dst = Builder.CreateLoad(dst);
>> + Address dst = GetAddrOfLocalVar(&dstDecl);
>> + dst = Address(Builder.CreateLoad(dst), blockInfo.BlockAlign);
>> dst = Builder.CreateBitCast(dst, structPtrTy, "block.dest");
>>
>> const BlockDecl *blockDecl = blockInfo.getBlockDecl();
>> @@ -1404,10 +1422,8 @@ CodeGenFunction::GenerateCopyHelperFunct
>> }
>>
>> unsigned index = capture.getIndex();
>> - llvm::Value *srcField =
>> - Builder.CreateStructGEP(blockInfo.StructureType, src, index);
>> - llvm::Value *dstField =
>> - Builder.CreateStructGEP(blockInfo.StructureType, dst, index);
>> + Address srcField = Builder.CreateStructGEP(src, index, capture.getOffset());
>> + Address dstField = Builder.CreateStructGEP(dst, index, capture.getOffset());
>>
>> // If there's an explicit copy expression, we do that.
>> if (copyExpr) {
>> @@ -1434,11 +1450,12 @@ CodeGenFunction::GenerateCopyHelperFunct
>>
>> // We don't need this anymore, so kill it. It's not quite
>> // worth the annoyance to avoid creating it in the first place.
>> - cast<llvm::Instruction>(dstField)->eraseFromParent();
>> + cast<llvm::Instruction>(dstField.getPointer())->eraseFromParent();
>> }
>> } else {
>> srcValue = Builder.CreateBitCast(srcValue, VoidPtrTy);
>> - llvm::Value *dstAddr = Builder.CreateBitCast(dstField, VoidPtrTy);
>> + llvm::Value *dstAddr =
>> + Builder.CreateBitCast(dstField.getPointer(), VoidPtrTy);
>> llvm::Value *args[] = {
>> dstAddr, srcValue, llvm::ConstantInt::get(Int32Ty, flags.getBitMask())
>> };
>> @@ -1508,8 +1525,8 @@ CodeGenFunction::GenerateDestroyHelperFu
>>
>> llvm::Type *structPtrTy = blockInfo.StructureType->getPointerTo();
>>
>> - llvm::Value *src = GetAddrOfLocalVar(&srcDecl);
>> - src = Builder.CreateLoad(src);
>> + Address src = GetAddrOfLocalVar(&srcDecl);
>> + src = Address(Builder.CreateLoad(src), blockInfo.BlockAlign);
>> src = Builder.CreateBitCast(src, structPtrTy, "block");
>>
>> const BlockDecl *blockDecl = blockInfo.getBlockDecl();
>> @@ -1563,9 +1580,8 @@ CodeGenFunction::GenerateDestroyHelperFu
>> continue;
>> }
>>
>> - unsigned index = capture.getIndex();
>> - llvm::Value *srcField =
>> - Builder.CreateStructGEP(blockInfo.StructureType, src, index);
>> + Address srcField =
>> + Builder.CreateStructGEP(src, capture.getIndex(), capture.getOffset());
>>
>> // If there's an explicit copy expression, we do that.
>> if (dtor) {
>> @@ -1599,15 +1615,15 @@ CodeGenFunction::GenerateDestroyHelperFu
>> namespace {
>>
>> /// Emits the copy/dispose helper functions for a __block object of id type.
>> -class ObjectByrefHelpers final : public CodeGenModule::ByrefHelpers {
>> +class ObjectByrefHelpers final : public BlockByrefHelpers {
>> BlockFieldFlags Flags;
>>
>> public:
>> ObjectByrefHelpers(CharUnits alignment, BlockFieldFlags flags)
>> - : ByrefHelpers(alignment), Flags(flags) {}
>> + : BlockByrefHelpers(alignment), Flags(flags) {}
>>
>> - void emitCopy(CodeGenFunction &CGF, llvm::Value *destField,
>> - llvm::Value *srcField) override {
>> + void emitCopy(CodeGenFunction &CGF, Address destField,
>> + Address srcField) override {
>> destField = CGF.Builder.CreateBitCast(destField, CGF.VoidPtrTy);
>>
>> srcField = CGF.Builder.CreateBitCast(srcField, CGF.VoidPtrPtrTy);
>> @@ -1618,11 +1634,11 @@ public:
>> llvm::Value *flagsVal = llvm::ConstantInt::get(CGF.Int32Ty, flags);
>> llvm::Value *fn = CGF.CGM.getBlockObjectAssign();
>>
>> - llvm::Value *args[] = { destField, srcValue, flagsVal };
>> + llvm::Value *args[] = { destField.getPointer(), srcValue, flagsVal };
>> CGF.EmitNounwindRuntimeCall(fn, args);
>> }
>>
>> - void emitDispose(CodeGenFunction &CGF, llvm::Value *field) override {
>> + void emitDispose(CodeGenFunction &CGF, Address field) override {
>> field = CGF.Builder.CreateBitCast(field, CGF.Int8PtrTy->getPointerTo(0));
>> llvm::Value *value = CGF.Builder.CreateLoad(field);
>>
>> @@ -1635,16 +1651,16 @@ public:
>> };
>>
>> /// Emits the copy/dispose helpers for an ARC __block __weak variable.
>> -class ARCWeakByrefHelpers final : public CodeGenModule::ByrefHelpers {
>> +class ARCWeakByrefHelpers final : public BlockByrefHelpers {
>> public:
>> - ARCWeakByrefHelpers(CharUnits alignment) : ByrefHelpers(alignment) {}
>> + ARCWeakByrefHelpers(CharUnits alignment) : BlockByrefHelpers(alignment) {}
>>
>> - void emitCopy(CodeGenFunction &CGF, llvm::Value *destField,
>> - llvm::Value *srcField) override {
>> + void emitCopy(CodeGenFunction &CGF, Address destField,
>> + Address srcField) override {
>> CGF.EmitARCMoveWeak(destField, srcField);
>> }
>>
>> - void emitDispose(CodeGenFunction &CGF, llvm::Value *field) override {
>> + void emitDispose(CodeGenFunction &CGF, Address field) override {
>> CGF.EmitARCDestroyWeak(field);
>> }
>>
>> @@ -1656,36 +1672,31 @@ public:
>>
>> /// Emits the copy/dispose helpers for an ARC __block __strong variable
>> /// that's not of block-pointer type.
>> -class ARCStrongByrefHelpers final : public CodeGenModule::ByrefHelpers {
>> +class ARCStrongByrefHelpers final : public BlockByrefHelpers {
>> public:
>> - ARCStrongByrefHelpers(CharUnits alignment) : ByrefHelpers(alignment) {}
>> + ARCStrongByrefHelpers(CharUnits alignment) : BlockByrefHelpers(alignment) {}
>>
>> - void emitCopy(CodeGenFunction &CGF, llvm::Value *destField,
>> - llvm::Value *srcField) override {
>> + void emitCopy(CodeGenFunction &CGF, Address destField,
>> + Address srcField) override {
>> // Do a "move" by copying the value and then zeroing out the old
>> // variable.
>>
>> - llvm::LoadInst *value = CGF.Builder.CreateLoad(srcField);
>> - value->setAlignment(Alignment.getQuantity());
>> + llvm::Value *value = CGF.Builder.CreateLoad(srcField);
>>
>> llvm::Value *null =
>> llvm::ConstantPointerNull::get(cast<llvm::PointerType>(value->getType()));
>>
>> if (CGF.CGM.getCodeGenOpts().OptimizationLevel == 0) {
>> - llvm::StoreInst *store = CGF.Builder.CreateStore(null, destField);
>> - store->setAlignment(Alignment.getQuantity());
>> + CGF.Builder.CreateStore(null, destField);
>> CGF.EmitARCStoreStrongCall(destField, value, /*ignored*/ true);
>> CGF.EmitARCStoreStrongCall(srcField, null, /*ignored*/ true);
>> return;
>> }
>> - llvm::StoreInst *store = CGF.Builder.CreateStore(value, destField);
>> - store->setAlignment(Alignment.getQuantity());
>> -
>> - store = CGF.Builder.CreateStore(null, srcField);
>> - store->setAlignment(Alignment.getQuantity());
>> + CGF.Builder.CreateStore(value, destField);
>> + CGF.Builder.CreateStore(null, srcField);
>> }
>>
>> - void emitDispose(CodeGenFunction &CGF, llvm::Value *field) override {
>> + void emitDispose(CodeGenFunction &CGF, Address field) override {
>> CGF.EmitARCDestroyStrong(field, ARCImpreciseLifetime);
>> }
>>
>> @@ -1697,25 +1708,22 @@ public:
>>
>> /// Emits the copy/dispose helpers for an ARC __block __strong
>> /// variable that's of block-pointer type.
>> -class ARCStrongBlockByrefHelpers final : public CodeGenModule::ByrefHelpers {
>> +class ARCStrongBlockByrefHelpers final : public BlockByrefHelpers {
>> public:
>> - ARCStrongBlockByrefHelpers(CharUnits alignment) : ByrefHelpers(alignment) {}
>> + ARCStrongBlockByrefHelpers(CharUnits alignment)
>> + : BlockByrefHelpers(alignment) {}
>>
>> - void emitCopy(CodeGenFunction &CGF, llvm::Value *destField,
>> - llvm::Value *srcField) override {
>> + void emitCopy(CodeGenFunction &CGF, Address destField,
>> + Address srcField) override {
>> // Do the copy with objc_retainBlock; that's all that
>> // _Block_object_assign would do anyway, and we'd have to pass the
>> // right arguments to make sure it doesn't get no-op'ed.
>> - llvm::LoadInst *oldValue = CGF.Builder.CreateLoad(srcField);
>> - oldValue->setAlignment(Alignment.getQuantity());
>> -
>> + llvm::Value *oldValue = CGF.Builder.CreateLoad(srcField);
>> llvm::Value *copy = CGF.EmitARCRetainBlock(oldValue, /*mandatory*/ true);
>> -
>> - llvm::StoreInst *store = CGF.Builder.CreateStore(copy, destField);
>> - store->setAlignment(Alignment.getQuantity());
>> + CGF.Builder.CreateStore(copy, destField);
>> }
>>
>> - void emitDispose(CodeGenFunction &CGF, llvm::Value *field) override {
>> + void emitDispose(CodeGenFunction &CGF, Address field) override {
>> CGF.EmitARCDestroyStrong(field, ARCImpreciseLifetime);
>> }
>>
>> @@ -1727,23 +1735,23 @@ public:
>>
>> /// Emits the copy/dispose helpers for a __block variable with a
>> /// nontrivial copy constructor or destructor.
>> -class CXXByrefHelpers final : public CodeGenModule::ByrefHelpers {
>> +class CXXByrefHelpers final : public BlockByrefHelpers {
>> QualType VarType;
>> const Expr *CopyExpr;
>>
>> public:
>> CXXByrefHelpers(CharUnits alignment, QualType type,
>> const Expr *copyExpr)
>> - : ByrefHelpers(alignment), VarType(type), CopyExpr(copyExpr) {}
>> + : BlockByrefHelpers(alignment), VarType(type), CopyExpr(copyExpr) {}
>>
>> bool needsCopy() const override { return CopyExpr != nullptr; }
>> - void emitCopy(CodeGenFunction &CGF, llvm::Value *destField,
>> - llvm::Value *srcField) override {
>> + void emitCopy(CodeGenFunction &CGF, Address destField,
>> + Address srcField) override {
>> if (!CopyExpr) return;
>> CGF.EmitSynthesizedCXXCopyCtor(destField, srcField, CopyExpr);
>> }
>>
>> - void emitDispose(CodeGenFunction &CGF, llvm::Value *field) override {
>> + void emitDispose(CodeGenFunction &CGF, Address field) override {
>> EHScopeStack::stable_iterator cleanupDepth = CGF.EHStack.stable_begin();
>> CGF.PushDestructorCleanup(VarType, field);
>> CGF.PopCleanupBlocks(cleanupDepth);
>> @@ -1756,10 +1764,8 @@ public:
>> } // end anonymous namespace
>>
>> static llvm::Constant *
>> -generateByrefCopyHelper(CodeGenFunction &CGF,
>> - llvm::StructType &byrefType,
>> - unsigned valueFieldIndex,
>> - CodeGenModule::ByrefHelpers &byrefInfo) {
>> +generateByrefCopyHelper(CodeGenFunction &CGF, const BlockByrefInfo &byrefInfo,
>> + BlockByrefHelpers &generator) {
>> ASTContext &Context = CGF.getContext();
>>
>> QualType R = Context.VoidTy;
>> @@ -1776,8 +1782,7 @@ generateByrefCopyHelper(CodeGenFunction
>> const CGFunctionInfo &FI = CGF.CGM.getTypes().arrangeFreeFunctionDeclaration(
>> R, args, FunctionType::ExtInfo(), /*variadic=*/false);
>>
>> - CodeGenTypes &Types = CGF.CGM.getTypes();
>> - llvm::FunctionType *LTy = Types.GetFunctionType(FI);
>> + llvm::FunctionType *LTy = CGF.CGM.getTypes().GetFunctionType(FI);
>>
>> // FIXME: We'd like to put these into a mergable by content, with
>> // internal linkage.
>> @@ -1797,24 +1802,26 @@ generateByrefCopyHelper(CodeGenFunction
>>
>> CGF.StartFunction(FD, R, Fn, FI, args);
>>
>> - if (byrefInfo.needsCopy()) {
>> - llvm::Type *byrefPtrType = byrefType.getPointerTo(0);
>> + if (generator.needsCopy()) {
>> + llvm::Type *byrefPtrType = byrefInfo.Type->getPointerTo(0);
>>
>> // dst->x
>> - llvm::Value *destField = CGF.GetAddrOfLocalVar(&dst);
>> - destField = CGF.Builder.CreateLoad(destField);
>> + Address destField = CGF.GetAddrOfLocalVar(&dst);
>> + destField = Address(CGF.Builder.CreateLoad(destField),
>> + byrefInfo.ByrefAlignment);
>> destField = CGF.Builder.CreateBitCast(destField, byrefPtrType);
>> - destField = CGF.Builder.CreateStructGEP(&byrefType, destField,
>> - valueFieldIndex, "x");
>> + destField = CGF.emitBlockByrefAddress(destField, byrefInfo, false,
>> + "dest-object");
>>
>> // src->x
>> - llvm::Value *srcField = CGF.GetAddrOfLocalVar(&src);
>> - srcField = CGF.Builder.CreateLoad(srcField);
>> + Address srcField = CGF.GetAddrOfLocalVar(&src);
>> + srcField = Address(CGF.Builder.CreateLoad(srcField),
>> + byrefInfo.ByrefAlignment);
>> srcField = CGF.Builder.CreateBitCast(srcField, byrefPtrType);
>> - srcField =
>> - CGF.Builder.CreateStructGEP(&byrefType, srcField, valueFieldIndex, "x");
>> + srcField = CGF.emitBlockByrefAddress(srcField, byrefInfo, false,
>> + "src-object");
>>
>> - byrefInfo.emitCopy(CGF, destField, srcField);
>> + generator.emitCopy(CGF, destField, srcField);
>> }
>>
>> CGF.FinishFunction();
>> @@ -1824,19 +1831,17 @@ generateByrefCopyHelper(CodeGenFunction
>>
>> /// Build the copy helper for a __block variable.
>> static llvm::Constant *buildByrefCopyHelper(CodeGenModule &CGM,
>> - llvm::StructType &byrefType,
>> - unsigned byrefValueIndex,
>> - CodeGenModule::ByrefHelpers &info) {
>> + const BlockByrefInfo &byrefInfo,
>> + BlockByrefHelpers &generator) {
>> CodeGenFunction CGF(CGM);
>> - return generateByrefCopyHelper(CGF, byrefType, byrefValueIndex, info);
>> + return generateByrefCopyHelper(CGF, byrefInfo, generator);
>> }
>>
>> /// Generate code for a __block variable's dispose helper.
>> static llvm::Constant *
>> generateByrefDisposeHelper(CodeGenFunction &CGF,
>> - llvm::StructType &byrefType,
>> - unsigned byrefValueIndex,
>> - CodeGenModule::ByrefHelpers &byrefInfo) {
>> + const BlockByrefInfo &byrefInfo,
>> + BlockByrefHelpers &generator) {
>> ASTContext &Context = CGF.getContext();
>> QualType R = Context.VoidTy;
>>
>> @@ -1848,8 +1853,7 @@ generateByrefDisposeHelper(CodeGenFuncti
>> const CGFunctionInfo &FI = CGF.CGM.getTypes().arrangeFreeFunctionDeclaration(
>> R, args, FunctionType::ExtInfo(), /*variadic=*/false);
>>
>> - CodeGenTypes &Types = CGF.CGM.getTypes();
>> - llvm::FunctionType *LTy = Types.GetFunctionType(FI);
>> + llvm::FunctionType *LTy = CGF.CGM.getTypes().GetFunctionType(FI);
>>
>> // FIXME: We'd like to put these into a mergable by content, with
>> // internal linkage.
>> @@ -1869,13 +1873,14 @@ generateByrefDisposeHelper(CodeGenFuncti
>> false, false);
>> CGF.StartFunction(FD, R, Fn, FI, args);
>>
>> - if (byrefInfo.needsDispose()) {
>> - llvm::Value *V = CGF.GetAddrOfLocalVar(&src);
>> - V = CGF.Builder.CreateLoad(V);
>> - V = CGF.Builder.CreateBitCast(V, byrefType.getPointerTo(0));
>> - V = CGF.Builder.CreateStructGEP(&byrefType, V, byrefValueIndex, "x");
>> + if (generator.needsDispose()) {
>> + Address addr = CGF.GetAddrOfLocalVar(&src);
>> + addr = Address(CGF.Builder.CreateLoad(addr), byrefInfo.ByrefAlignment);
>> + auto byrefPtrType = byrefInfo.Type->getPointerTo(0);
>> + addr = CGF.Builder.CreateBitCast(addr, byrefPtrType);
>> + addr = CGF.emitBlockByrefAddress(addr, byrefInfo, false, "object");
>>
>> - byrefInfo.emitDispose(CGF, V);
>> + generator.emitDispose(CGF, addr);
>> }
>>
>> CGF.FinishFunction();
>> @@ -1885,37 +1890,29 @@ generateByrefDisposeHelper(CodeGenFuncti
>>
>> /// Build the dispose helper for a __block variable.
>> static llvm::Constant *buildByrefDisposeHelper(CodeGenModule &CGM,
>> - llvm::StructType &byrefType,
>> - unsigned byrefValueIndex,
>> - CodeGenModule::ByrefHelpers &info) {
>> + const BlockByrefInfo &byrefInfo,
>> + BlockByrefHelpers &generator) {
>> CodeGenFunction CGF(CGM);
>> - return generateByrefDisposeHelper(CGF, byrefType, byrefValueIndex, info);
>> + return generateByrefDisposeHelper(CGF, byrefInfo, generator);
>> }
>>
>> /// Lazily build the copy and dispose helpers for a __block variable
>> /// with the given information.
>> template <class T>
>> -static T *buildByrefHelpers(CodeGenModule &CGM, llvm::StructType &byrefTy,
>> - unsigned byrefValueIndex, T byrefInfo) {
>> - // Increase the field's alignment to be at least pointer alignment,
>> - // since the layout of the byref struct will guarantee at least that.
>> - byrefInfo.Alignment = std::max(byrefInfo.Alignment,
>> - CharUnits::fromQuantity(CGM.PointerAlignInBytes));
>> -
>> +static T *buildByrefHelpers(CodeGenModule &CGM, const BlockByrefInfo &byrefInfo,
>> + T &&generator) {
>> llvm::FoldingSetNodeID id;
>> - byrefInfo.Profile(id);
>> + generator.Profile(id);
>>
>> void *insertPos;
>> - CodeGenModule::ByrefHelpers *node
>> + BlockByrefHelpers *node
>> = CGM.ByrefHelpersCache.FindNodeOrInsertPos(id, insertPos);
>> if (node) return static_cast<T*>(node);
>>
>> - byrefInfo.CopyHelper =
>> - buildByrefCopyHelper(CGM, byrefTy, byrefValueIndex, byrefInfo);
>> - byrefInfo.DisposeHelper =
>> - buildByrefDisposeHelper(CGM, byrefTy, byrefValueIndex,byrefInfo);
>> + generator.CopyHelper = buildByrefCopyHelper(CGM, byrefInfo, generator);
>> + generator.DisposeHelper = buildByrefDisposeHelper(CGM, byrefInfo, generator);
>>
>> - T *copy = new (CGM.getContext()) T(std::move(byrefInfo));
>> + T *copy = new (CGM.getContext()) T(std::move(generator));
>> CGM.ByrefHelpersCache.InsertNode(copy, insertPos);
>> return copy;
>> }
>> @@ -1923,21 +1920,25 @@ static T *buildByrefHelpers(CodeGenModul
>> /// Build the copy and dispose helpers for the given __block variable
>> /// emission. Places the helpers in the global cache. Returns null
>> /// if no helpers are required.
>> -CodeGenModule::ByrefHelpers *
>> +BlockByrefHelpers *
>> CodeGenFunction::buildByrefHelpers(llvm::StructType &byrefType,
>> const AutoVarEmission &emission) {
>> const VarDecl &var = *emission.Variable;
>> QualType type = var.getType();
>>
>> - unsigned byrefValueIndex = getByRefValueLLVMField(&var).second;
>> + auto &byrefInfo = getBlockByrefInfo(&var);
>> +
>> + // The alignment we care about for the purposes of uniquing byref
>> + // helpers is the alignment of the actual byref value field.
>> + CharUnits valueAlignment =
>> + byrefInfo.ByrefAlignment.alignmentAtOffset(byrefInfo.FieldOffset);
>>
>> if (const CXXRecordDecl *record = type->getAsCXXRecordDecl()) {
>> const Expr *copyExpr = CGM.getContext().getBlockVarCopyInits(&var);
>> if (!copyExpr && record->hasTrivialDestructor()) return nullptr;
>>
>> return ::buildByrefHelpers(
>> - CGM, byrefType, byrefValueIndex,
>> - CXXByrefHelpers(emission.Alignment, type, copyExpr));
>> + CGM, byrefInfo, CXXByrefHelpers(valueAlignment, type, copyExpr));
>> }
>>
>> // Otherwise, if we don't have a retainable type, there's nothing to do.
>> @@ -1961,23 +1962,22 @@ CodeGenFunction::buildByrefHelpers(llvm:
>> // Tell the runtime that this is ARC __weak, called by the
>> // byref routines.
>> case Qualifiers::OCL_Weak:
>> - return ::buildByrefHelpers(CGM, byrefType, byrefValueIndex,
>> - ARCWeakByrefHelpers(emission.Alignment));
>> + return ::buildByrefHelpers(CGM, byrefInfo,
>> + ARCWeakByrefHelpers(valueAlignment));
>>
>> // ARC __strong __block variables need to be retained.
>> case Qualifiers::OCL_Strong:
>> // Block pointers need to be copied, and there's no direct
>> // transfer possible.
>> if (type->isBlockPointerType()) {
>> - return ::buildByrefHelpers(
>> - CGM, byrefType, byrefValueIndex,
>> - ARCStrongBlockByrefHelpers(emission.Alignment));
>> + return ::buildByrefHelpers(CGM, byrefInfo,
>> + ARCStrongBlockByrefHelpers(valueAlignment));
>>
>> // Otherwise, we transfer ownership of the retain from the stack
>> // to the heap.
>> } else {
>> - return ::buildByrefHelpers(CGM, byrefType, byrefValueIndex,
>> - ARCStrongByrefHelpers(emission.Alignment));
>> + return ::buildByrefHelpers(CGM, byrefInfo,
>> + ARCStrongByrefHelpers(valueAlignment));
>> }
>> }
>> llvm_unreachable("fell out of lifetime switch!");
>> @@ -1996,28 +1996,33 @@ CodeGenFunction::buildByrefHelpers(llvm:
>> if (type.isObjCGCWeak())
>> flags |= BLOCK_FIELD_IS_WEAK;
>>
>> - return ::buildByrefHelpers(CGM, byrefType, byrefValueIndex,
>> - ObjectByrefHelpers(emission.Alignment, flags));
>> + return ::buildByrefHelpers(CGM, byrefInfo,
>> + ObjectByrefHelpers(valueAlignment, flags));
>> }
>>
>> -std::pair<llvm::Type *, unsigned>
>> -CodeGenFunction::getByRefValueLLVMField(const ValueDecl *VD) const {
>> - assert(ByRefValueInfo.count(VD) && "Did not find value!");
>> -
>> - return ByRefValueInfo.find(VD)->second;
>> +Address CodeGenFunction::emitBlockByrefAddress(Address baseAddr,
>> + const VarDecl *var,
>> + bool followForward) {
>> + auto &info = getBlockByrefInfo(var);
>> + return emitBlockByrefAddress(baseAddr, info, followForward, var->getName());
>> }
>>
>> -llvm::Value *CodeGenFunction::BuildBlockByrefAddress(llvm::Value *BaseAddr,
>> - const VarDecl *V) {
>> - auto P = getByRefValueLLVMField(V);
>> - llvm::Value *Loc =
>> - Builder.CreateStructGEP(P.first, BaseAddr, 1, "forwarding");
>> - Loc = Builder.CreateLoad(Loc);
>> - Loc = Builder.CreateStructGEP(P.first, Loc, P.second, V->getNameAsString());
>> - return Loc;
>> +Address CodeGenFunction::emitBlockByrefAddress(Address baseAddr,
>> + const BlockByrefInfo &info,
>> + bool followForward,
>> + const llvm::Twine &name) {
>> + // Chase the forwarding address if requested.
>> + if (followForward) {
>> + Address forwardingAddr =
>> + Builder.CreateStructGEP(baseAddr, 1, getPointerSize(), "forwarding");
>> + baseAddr = Address(Builder.CreateLoad(forwardingAddr), info.ByrefAlignment);
>> + }
>> +
>> + return Builder.CreateStructGEP(baseAddr, info.FieldIndex,
>> + info.FieldOffset, name);
>> }
>>
>> -/// BuildByRefType - This routine changes a __block variable declared as T x
>> +/// BuildByrefInfo - This routine changes a __block variable declared as T x
>> /// into:
>> ///
>> /// struct {
>> @@ -2032,108 +2037,116 @@ llvm::Value *CodeGenFunction::BuildBlock
>> /// T x;
>> /// } x
>> ///
>> -llvm::Type *CodeGenFunction::BuildByRefType(const VarDecl *D) {
>> - std::pair<llvm::Type *, unsigned> &Info = ByRefValueInfo[D];
>> - if (Info.first)
>> - return Info.first;
>> +const BlockByrefInfo &CodeGenFunction::getBlockByrefInfo(const VarDecl *D) {
>> + auto it = BlockByrefInfos.find(D);
>> + if (it != BlockByrefInfos.end())
>> + return it->second;
>> +
>> + llvm::StructType *byrefType =
>> + llvm::StructType::create(getLLVMContext(),
>> + "struct.__block_byref_" + D->getNameAsString());
>>
>> QualType Ty = D->getType();
>>
>> + CharUnits size;
>> SmallVector<llvm::Type *, 8> types;
>>
>> - llvm::StructType *ByRefType =
>> - llvm::StructType::create(getLLVMContext(),
>> - "struct.__block_byref_" + D->getNameAsString());
>> -
>> // void *__isa;
>> types.push_back(Int8PtrTy);
>> + size += getPointerSize();
>>
>> // void *__forwarding;
>> - types.push_back(llvm::PointerType::getUnqual(ByRefType));
>> + types.push_back(llvm::PointerType::getUnqual(byrefType));
>> + size += getPointerSize();
>>
>> // int32_t __flags;
>> types.push_back(Int32Ty);
>> + size += CharUnits::fromQuantity(4);
>>
>> // int32_t __size;
>> types.push_back(Int32Ty);
>> + size += CharUnits::fromQuantity(4);
>> +
>> // Note that this must match *exactly* the logic in buildByrefHelpers.
>> - bool HasCopyAndDispose = getContext().BlockRequiresCopying(Ty, D);
>> - if (HasCopyAndDispose) {
>> + bool hasCopyAndDispose = getContext().BlockRequiresCopying(Ty, D);
>> + if (hasCopyAndDispose) {
>> /// void *__copy_helper;
>> types.push_back(Int8PtrTy);
>> + size += getPointerSize();
>>
>> /// void *__destroy_helper;
>> types.push_back(Int8PtrTy);
>> + size += getPointerSize();
>> }
>> +
>> bool HasByrefExtendedLayout = false;
>> Qualifiers::ObjCLifetime Lifetime;
>> if (getContext().getByrefLifetime(Ty, Lifetime, HasByrefExtendedLayout) &&
>> - HasByrefExtendedLayout)
>> + HasByrefExtendedLayout) {
>> /// void *__byref_variable_layout;
>> types.push_back(Int8PtrTy);
>> -
>> - bool Packed = false;
>> - CharUnits Align = getContext().getDeclAlign(D);
>> - if (Align >
>> - getContext().toCharUnitsFromBits(getTarget().getPointerAlign(0))) {
>> - // We have to insert padding.
>> -
>> - // The struct above has 2 32-bit integers.
>> - unsigned CurrentOffsetInBytes = 4 * 2;
>> -
>> - // And either 2, 3, 4 or 5 pointers.
>> - unsigned noPointers = 2;
>> - if (HasCopyAndDispose)
>> - noPointers += 2;
>> - if (HasByrefExtendedLayout)
>> - noPointers += 1;
>> -
>> - CurrentOffsetInBytes += noPointers * CGM.getDataLayout().getTypeAllocSize(Int8PtrTy);
>> -
>> - // Align the offset.
>> - unsigned AlignedOffsetInBytes =
>> - llvm::RoundUpToAlignment(CurrentOffsetInBytes, Align.getQuantity());
>> -
>> - unsigned NumPaddingBytes = AlignedOffsetInBytes - CurrentOffsetInBytes;
>> - if (NumPaddingBytes > 0) {
>> - llvm::Type *Ty = Int8Ty;
>> - // FIXME: We need a sema error for alignment larger than the minimum of
>> - // the maximal stack alignment and the alignment of malloc on the system.
>> - if (NumPaddingBytes > 1)
>> - Ty = llvm::ArrayType::get(Ty, NumPaddingBytes);
>> -
>> - types.push_back(Ty);
>> -
>> - // We want a packed struct.
>> - Packed = true;
>> - }
>> + size += CharUnits::fromQuantity(PointerSizeInBytes);
>> }
>>
>> // T x;
>> - types.push_back(ConvertTypeForMem(Ty));
>> -
>> - ByRefType->setBody(types, Packed);
>> -
>> - Info.first = ByRefType;
>> -
>> - Info.second = types.size() - 1;
>> -
>> - return Info.first;
>> + llvm::Type *varTy = ConvertTypeForMem(Ty);
>> +
>> + bool packed = false;
>> + CharUnits varAlign = getContext().getDeclAlign(D);
>> + CharUnits varOffset = size.RoundUpToAlignment(varAlign);
>> +
>> + // We may have to insert padding.
>> + if (varOffset != size) {
>> + llvm::Type *paddingTy =
>> + llvm::ArrayType::get(Int8Ty, (varOffset - size).getQuantity());
>> +
>> + types.push_back(paddingTy);
>> + size = varOffset;
>> +
>> + // Conversely, we might have to prevent LLVM from inserting padding.
>> + } else if (CGM.getDataLayout().getABITypeAlignment(varTy)
>> + > varAlign.getQuantity()) {
>> + packed = true;
>> + }
>> + types.push_back(varTy);
>> +
>> + byrefType->setBody(types, packed);
>> +
>> + BlockByrefInfo info;
>> + info.Type = byrefType;
>> + info.FieldIndex = types.size() - 1;
>> + info.FieldOffset = varOffset;
>> + info.ByrefAlignment = std::max(varAlign, getPointerAlign());
>> +
>> + auto pair = BlockByrefInfos.insert({D, info});
>> + assert(pair.second && "info was inserted recursively?");
>> + return pair.first->second;
>> }
>>
>> /// Initialize the structural components of a __block variable, i.e.
>> /// everything but the actual object.
>> void CodeGenFunction::emitByrefStructureInit(const AutoVarEmission &emission) {
>> // Find the address of the local.
>> - llvm::Value *addr = emission.Address;
>> + Address addr = emission.Addr;
>>
>> // That's an alloca of the byref structure type.
>> llvm::StructType *byrefType = cast<llvm::StructType>(
>> - cast<llvm::PointerType>(addr->getType())->getElementType());
>> + cast<llvm::PointerType>(addr.getPointer()->getType())->getElementType());
>> +
>> + unsigned nextHeaderIndex = 0;
>> + CharUnits nextHeaderOffset;
>> + auto storeHeaderField = [&](llvm::Value *value, CharUnits fieldSize,
>> + const Twine &name) {
>> + auto fieldAddr = Builder.CreateStructGEP(addr, nextHeaderIndex,
>> + nextHeaderOffset, name);
>> + Builder.CreateStore(value, fieldAddr);
>> +
>> + nextHeaderIndex++;
>> + nextHeaderOffset += fieldSize;
>> + };
>>
>> // Build the byref helpers if necessary. This is null if we don't need any.
>> - CodeGenModule::ByrefHelpers *helpers =
>> - buildByrefHelpers(*byrefType, emission);
>> + BlockByrefHelpers *helpers = buildByrefHelpers(*byrefType, emission);
>>
>> const VarDecl &D = *emission.Variable;
>> QualType type = D.getType();
>> @@ -2142,7 +2155,7 @@ void CodeGenFunction::emitByrefStructure
>> Qualifiers::ObjCLifetime ByrefLifetime;
>> bool ByRefHasLifetime =
>> getContext().getByrefLifetime(type, ByrefLifetime, HasByrefExtendedLayout);
>> -
>> +
>> llvm::Value *V;
>>
>> // Initialize the 'isa', which is just 0 or 1.
>> @@ -2150,12 +2163,10 @@ void CodeGenFunction::emitByrefStructure
>> if (type.isObjCGCWeak())
>> isa = 1;
>> V = Builder.CreateIntToPtr(Builder.getInt32(isa), Int8PtrTy, "isa");
>> - Builder.CreateStore(V,
>> - Builder.CreateStructGEP(nullptr, addr, 0, "byref.isa"));
>> + storeHeaderField(V, getPointerSize(), "byref.isa");
>>
>> // Store the address of the variable into its own forwarding pointer.
>> - Builder.CreateStore(
>> - addr, Builder.CreateStructGEP(nullptr, addr, 1, "byref.forwarding"));
>> + storeHeaderField(addr.getPointer(), getPointerSize(), "byref.forwarding");
>>
>> // Blocks ABI:
>> // c) the flags field is set to either 0 if no helper functions are
>> @@ -2201,31 +2212,23 @@ void CodeGenFunction::emitByrefStructure
>> printf("\n");
>> }
>> }
>> -
>> - Builder.CreateStore(llvm::ConstantInt::get(IntTy, flags.getBitMask()),
>> - Builder.CreateStructGEP(nullptr, addr, 2, "byref.flags"));
>> + storeHeaderField(llvm::ConstantInt::get(IntTy, flags.getBitMask()),
>> + getIntSize(), "byref.flags");
>>
>> CharUnits byrefSize = CGM.GetTargetTypeStoreSize(byrefType);
>> V = llvm::ConstantInt::get(IntTy, byrefSize.getQuantity());
>> - Builder.CreateStore(V,
>> - Builder.CreateStructGEP(nullptr, addr, 3, "byref.size"));
>> + storeHeaderField(V, getIntSize(), "byref.size");
>>
>> if (helpers) {
>> - llvm::Value *copy_helper = Builder.CreateStructGEP(nullptr, addr, 4);
>> - Builder.CreateStore(helpers->CopyHelper, copy_helper);
>> -
>> - llvm::Value *destroy_helper = Builder.CreateStructGEP(nullptr, addr, 5);
>> - Builder.CreateStore(helpers->DisposeHelper, destroy_helper);
>> + storeHeaderField(helpers->CopyHelper, getPointerSize(),
>> + "byref.copyHelper");
>> + storeHeaderField(helpers->DisposeHelper, getPointerSize(),
>> + "byref.disposeHelper");
>> }
>> +
>> if (ByRefHasLifetime && HasByrefExtendedLayout) {
>> - llvm::Constant* ByrefLayoutInfo = CGM.getObjCRuntime().BuildByrefLayout(CGM, type);
>> - llvm::Value *ByrefInfoAddr =
>> - Builder.CreateStructGEP(nullptr, addr, helpers ? 6 : 4, "byref.layout");
>> - // cast destination to pointer to source type.
>> - llvm::Type *DesTy = ByrefLayoutInfo->getType();
>> - DesTy = DesTy->getPointerTo();
>> - llvm::Value *BC = Builder.CreatePointerCast(ByrefInfoAddr, DesTy);
>> - Builder.CreateStore(ByrefLayoutInfo, BC);
>> + auto layoutInfo = CGM.getObjCRuntime().BuildByrefLayout(CGM, type);
>> + storeHeaderField(layoutInfo, getPointerSize(), "byref.layout");
>> }
>> }
>>
>> @@ -2239,6 +2242,7 @@ void CodeGenFunction::BuildBlockRelease(
>> }
>>
>> namespace {
>> + /// Release a __block variable.
>> struct CallBlockRelease final : EHScopeStack::Cleanup {
>> llvm::Value *Addr;
>> CallBlockRelease(llvm::Value *Addr) : Addr(Addr) {}
>> @@ -2259,7 +2263,8 @@ void CodeGenFunction::enterByrefCleanup(
>> if (CGM.getLangOpts().getGC() == LangOptions::GCOnly)
>> return;
>>
>> - EHStack.pushCleanup<CallBlockRelease>(NormalAndEHCleanup, emission.Address);
>> + EHStack.pushCleanup<CallBlockRelease>(NormalAndEHCleanup,
>> + emission.Addr.getPointer());
>> }
>>
>> /// Adjust the declaration of something from the blocks API.
>>
>> Modified: cfe/trunk/lib/CodeGen/CGBlocks.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGBlocks.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGBlocks.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGBlocks.h Tue Sep 8 03:05:57 2015
>> @@ -140,6 +140,43 @@ inline BlockFieldFlags operator|(BlockFi
>> return BlockFieldFlags(l) | BlockFieldFlags(r);
>> }
>>
>> +/// Information about the layout of a __block variable.
>> +class BlockByrefInfo {
>> +public:
>> + llvm::StructType *Type;
>> + unsigned FieldIndex;
>> + CharUnits ByrefAlignment;
>> + CharUnits FieldOffset;
>> +};
>> +
>> +/// A pair of helper functions for a __block variable.
>> +class BlockByrefHelpers : public llvm::FoldingSetNode {
>> +public:
>> + llvm::Constant *CopyHelper;
>> + llvm::Constant *DisposeHelper;
>> +
>> + /// The alignment of the field. This is important because
>> + /// different offsets to the field within the byref struct need to
>> + /// have different helper functions.
>> + CharUnits Alignment;
>> +
>> + BlockByrefHelpers(CharUnits alignment) : Alignment(alignment) {}
>> + BlockByrefHelpers(const BlockByrefHelpers &) = default;
>> + virtual ~BlockByrefHelpers();
>> +
>> + void Profile(llvm::FoldingSetNodeID &id) const {
>> + id.AddInteger(Alignment.getQuantity());
>> + profileImpl(id);
>> + }
>> + virtual void profileImpl(llvm::FoldingSetNodeID &id) const = 0;
>> +
>> + virtual bool needsCopy() const { return true; }
>> + virtual void emitCopy(CodeGenFunction &CGF, Address dest, Address src) = 0;
>> +
>> + virtual bool needsDispose() const { return true; }
>> + virtual void emitDispose(CodeGenFunction &CGF, Address field) = 0;
>> +};
>> +
>> /// CGBlockInfo - Information to generate a block literal.
>> class CGBlockInfo {
>> public:
>> @@ -152,14 +189,19 @@ public:
>> class Capture {
>> uintptr_t Data;
>> EHScopeStack::stable_iterator Cleanup;
>> + CharUnits::QuantityType Offset;
>>
>> public:
>> bool isIndex() const { return (Data & 1) != 0; }
>> bool isConstant() const { return !isIndex(); }
>> - unsigned getIndex() const { assert(isIndex()); return Data >> 1; }
>> - llvm::Value *getConstant() const {
>> - assert(isConstant());
>> - return reinterpret_cast<llvm::Value*>(Data);
>> +
>> + unsigned getIndex() const {
>> + assert(isIndex());
>> + return Data >> 1;
>> + }
>> + CharUnits getOffset() const {
>> + assert(isIndex());
>> + return CharUnits::fromQuantity(Offset);
>> }
>> EHScopeStack::stable_iterator getCleanup() const {
>> assert(isIndex());
>> @@ -170,9 +212,15 @@ public:
>> Cleanup = cleanup;
>> }
>>
>> - static Capture makeIndex(unsigned index) {
>> + llvm::Value *getConstant() const {
>> + assert(isConstant());
>> + return reinterpret_cast<llvm::Value*>(Data);
>> + }
>> +
>> + static Capture makeIndex(unsigned index, CharUnits offset) {
>> Capture v;
>> v.Data = (index << 1) | 1;
>> + v.Offset = offset.getQuantity();
>> return v;
>> }
>>
>> @@ -205,12 +253,13 @@ public:
>> /// The mapping of allocated indexes within the block.
>> llvm::DenseMap<const VarDecl*, Capture> Captures;
>>
>> - llvm::AllocaInst *Address;
>> + Address LocalAddress;
>> llvm::StructType *StructureType;
>> const BlockDecl *Block;
>> const BlockExpr *BlockExpression;
>> CharUnits BlockSize;
>> CharUnits BlockAlign;
>> + CharUnits CXXThisOffset;
>>
>> // Offset of the gap caused by block header having a smaller
>> // alignment than the alignment of the block descriptor. This
>>
>> Modified: cfe/trunk/lib/CodeGen/CGBuilder.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGBuilder.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGBuilder.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGBuilder.h Tue Sep 8 03:05:57 2015
>> @@ -11,6 +11,8 @@
>> #define LLVM_CLANG_LIB_CODEGEN_CGBUILDER_H
>>
>> #include "llvm/IR/IRBuilder.h"
>> +#include "Address.h"
>> +#include "CodeGenTypeCache.h"
>>
>> namespace clang {
>> namespace CodeGen {
>> @@ -42,9 +44,260 @@ private:
>> #else
>> #define PreserveNames true
>> #endif
>> +
>> typedef CGBuilderInserter<PreserveNames> CGBuilderInserterTy;
>> +
>> typedef llvm::IRBuilder<PreserveNames, llvm::ConstantFolder,
>> - CGBuilderInserterTy> CGBuilderTy;
>> + CGBuilderInserterTy> CGBuilderBaseTy;
>> +
>> +class CGBuilderTy : public CGBuilderBaseTy {
>> + /// Storing a reference to the type cache here makes it a lot easier
>> + /// to build natural-feeling, target-specific IR.
>> + const CodeGenTypeCache &TypeCache;
>> +public:
>> + CGBuilderTy(const CodeGenTypeCache &TypeCache, llvm::LLVMContext &C)
>> + : CGBuilderBaseTy(C), TypeCache(TypeCache) {}
>> + CGBuilderTy(const CodeGenTypeCache &TypeCache,
>> + llvm::LLVMContext &C, const llvm::ConstantFolder &F,
>> + const CGBuilderInserterTy &Inserter)
>> + : CGBuilderBaseTy(C, F, Inserter), TypeCache(TypeCache) {}
>> + CGBuilderTy(const CodeGenTypeCache &TypeCache, llvm::Instruction *I)
>> + : CGBuilderBaseTy(I), TypeCache(TypeCache) {}
>> + CGBuilderTy(const CodeGenTypeCache &TypeCache, llvm::BasicBlock *BB)
>> + : CGBuilderBaseTy(BB), TypeCache(TypeCache) {}
>> +
>> + llvm::ConstantInt *getSize(CharUnits N) {
>> + return llvm::ConstantInt::get(TypeCache.SizeTy, N.getQuantity());
>> + }
>> + llvm::ConstantInt *getSize(uint64_t N) {
>> + return llvm::ConstantInt::get(TypeCache.SizeTy, N);
>> + }
>> +
>> + // Note that we intentionally hide the CreateLoad APIs that don't
>> + // take an alignment.
>> + llvm::LoadInst *CreateLoad(Address Addr, const llvm::Twine &Name = "") {
>> + return CreateAlignedLoad(Addr.getPointer(),
>> + Addr.getAlignment().getQuantity(),
>> + Name);
>> + }
>> + llvm::LoadInst *CreateLoad(Address Addr, const char *Name) {
>> + // This overload is required to prevent string literals from
>> + // ending up in the IsVolatile overload.
>> + return CreateAlignedLoad(Addr.getPointer(),
>> + Addr.getAlignment().getQuantity(),
>> + Name);
>> + }
>> + llvm::LoadInst *CreateLoad(Address Addr, bool IsVolatile,
>> + const llvm::Twine &Name = "") {
>> + return CreateAlignedLoad(Addr.getPointer(),
>> + Addr.getAlignment().getQuantity(),
>> + IsVolatile,
>> + Name);
>> + }
>> +
>> + using CGBuilderBaseTy::CreateAlignedLoad;
>> + llvm::LoadInst *CreateAlignedLoad(llvm::Value *Addr, CharUnits Align,
>> + const llvm::Twine &Name = "") {
>> + return CreateAlignedLoad(Addr, Align.getQuantity(), Name);
>> + }
>> + llvm::LoadInst *CreateAlignedLoad(llvm::Value *Addr, CharUnits Align,
>> + const char *Name) {
>> + return CreateAlignedLoad(Addr, Align.getQuantity(), Name);
>> + }
>> + llvm::LoadInst *CreateAlignedLoad(llvm::Type *Ty, llvm::Value *Addr,
>> + CharUnits Align,
>> + const llvm::Twine &Name = "") {
>> + assert(Addr->getType()->getPointerElementType() == Ty);
>> + return CreateAlignedLoad(Addr, Align.getQuantity(), Name);
>> + }
>> + llvm::LoadInst *CreateAlignedLoad(llvm::Value *Addr, CharUnits Align,
>> + bool IsVolatile,
>> + const llvm::Twine &Name = "") {
>> + return CreateAlignedLoad(Addr, Align.getQuantity(), IsVolatile, Name);
>> + }
>> +
>> + // Note that we intentionally hide the CreateStore APIs that don't
>> + // take an alignment.
>> + llvm::StoreInst *CreateStore(llvm::Value *Val, Address Addr,
>> + bool IsVolatile = false) {
>> + return CreateAlignedStore(Val, Addr.getPointer(),
>> + Addr.getAlignment().getQuantity(), IsVolatile);
>> + }
>> +
>> + using CGBuilderBaseTy::CreateAlignedStore;
>> + llvm::StoreInst *CreateAlignedStore(llvm::Value *Val, llvm::Value *Addr,
>> + CharUnits Align, bool IsVolatile = false) {
>> + return CreateAlignedStore(Val, Addr, Align.getQuantity(), IsVolatile);
>> + }
>> +
>> + // FIXME: these "default-aligned" APIs should be removed,
>> + // but I don't feel like fixing all the builtin code right now.
>> + llvm::LoadInst *CreateDefaultAlignedLoad(llvm::Value *Addr,
>> + const llvm::Twine &Name = "") {
>> + return CGBuilderBaseTy::CreateLoad(Addr, false, Name);
>> + }
>> + llvm::LoadInst *CreateDefaultAlignedLoad(llvm::Value *Addr,
>> + const char *Name) {
>> + return CGBuilderBaseTy::CreateLoad(Addr, false, Name);
>> + }
>> + llvm::LoadInst *CreateDefaultAlignedLoad(llvm::Value *Addr, bool IsVolatile,
>> + const llvm::Twine &Name = "") {
>> + return CGBuilderBaseTy::CreateLoad(Addr, IsVolatile, Name);
>> + }
>> +
>> + llvm::StoreInst *CreateDefaultAlignedStore(llvm::Value *Val,
>> + llvm::Value *Addr,
>> + bool IsVolatile = false) {
>> + return CGBuilderBaseTy::CreateStore(Val, Addr, IsVolatile);
>> + }
>> +
>> + /// Emit a load from an i1 flag variable.
>> + llvm::LoadInst *CreateFlagLoad(llvm::Value *Addr,
>> + const llvm::Twine &Name = "") {
>> + assert(Addr->getType()->getPointerElementType() == getInt1Ty());
>> + return CreateAlignedLoad(getInt1Ty(), Addr, CharUnits::One(), Name);
>> + }
>> +
>> + /// Emit a store to an i1 flag variable.
>> + llvm::StoreInst *CreateFlagStore(bool Value, llvm::Value *Addr) {
>> + assert(Addr->getType()->getPointerElementType() == getInt1Ty());
>> + return CreateAlignedStore(getInt1(Value), Addr, CharUnits::One());
>> + }
>> +
>> + using CGBuilderBaseTy::CreateBitCast;
>> + Address CreateBitCast(Address Addr, llvm::Type *Ty,
>> + const llvm::Twine &Name = "") {
>> + return Address(CreateBitCast(Addr.getPointer(), Ty, Name),
>> + Addr.getAlignment());
>> + }
>> +
>> + /// Cast the element type of the given address to a different type,
>> + /// preserving information like the alignment and address space.
>> + Address CreateElementBitCast(Address Addr, llvm::Type *Ty,
>> + const llvm::Twine &Name = "") {
>> + auto PtrTy = Ty->getPointerTo(Addr.getAddressSpace());
>> + return CreateBitCast(Addr, PtrTy, Name);
>> + }
>> +
>> + using CGBuilderBaseTy::CreatePointerBitCastOrAddrSpaceCast;
>> + Address CreatePointerBitCastOrAddrSpaceCast(Address Addr, llvm::Type *Ty,
>> + const llvm::Twine &Name = "") {
>> + llvm::Value *Ptr =
>> + CreatePointerBitCastOrAddrSpaceCast(Addr.getPointer(), Ty, Name);
>> + return Address(Ptr, Addr.getAlignment());
>> + }
>> +
>> + using CGBuilderBaseTy::CreateStructGEP;
>> + Address CreateStructGEP(Address Addr, unsigned Index, CharUnits Offset,
>> + const llvm::Twine &Name = "") {
>> + return Address(CreateStructGEP(Addr.getElementType(),
>> + Addr.getPointer(), Index, Name),
>> + Addr.getAlignment().alignmentAtOffset(Offset));
>> + }
>> +
>> + /// Given
>> + /// %addr = [n x T]* ...
>> + /// produce
>> + /// %name = getelementptr inbounds %addr, i64 0, i64 index
>> + /// where i64 is actually the target word size.
>> + ///
>> + /// This API assumes that drilling into an array like this is always
>> + /// an inbounds operation.
>> + ///
>> + /// \param EltSize - the size of the type T in bytes
>> + Address CreateConstArrayGEP(Address Addr, uint64_t Index, CharUnits EltSize,
>> + const llvm::Twine &Name = "") {
>> + return Address(CreateInBoundsGEP(Addr.getPointer(),
>> + {getSize(CharUnits::Zero()),
>> + getSize(Index)},
>> + Name),
>> + Addr.getAlignment().alignmentAtOffset(Index * EltSize));
>> + }
>> +
>> + /// Given
>> + /// %addr = T* ...
>> + /// produce
>> + /// %name = getelementptr inbounds %addr, i64 index
>> + /// where i64 is actually the target word size.
>> + ///
>> + /// \param EltSize - the size of the type T in bytes
>> + Address CreateConstInBoundsGEP(Address Addr, uint64_t Index,
>> + CharUnits EltSize,
>> + const llvm::Twine &Name = "") {
>> + return Address(CreateInBoundsGEP(Addr.getElementType(), Addr.getPointer(),
>> + {getSize(Index)}, Name),
>> + Addr.getAlignment().alignmentAtOffset(Index * EltSize));
>> + }
>> +
>> + /// Given
>> + /// %addr = T* ...
>> + /// produce
>> + /// %name = getelementptr inbounds %addr, i64 index
>> + /// where i64 is actually the target word size.
>> + ///
>> + /// \param EltSize - the size of the type T in bytes
>> + Address CreateConstGEP(Address Addr, uint64_t Index, CharUnits EltSize,
>> + const llvm::Twine &Name = "") {
>> + return Address(CreateGEP(Addr.getElementType(), Addr.getPointer(),
>> + {getSize(Index)}, Name),
>> + Addr.getAlignment().alignmentAtOffset(Index * EltSize));
>> + }
>> +
>> + /// Given a pointer to i8, adjust it by a given constant offset.
>> + Address CreateConstInBoundsByteGEP(Address Addr, CharUnits Offset,
>> + const llvm::Twine &Name = "") {
>> + assert(Addr.getElementType() == TypeCache.Int8Ty);
>> + return Address(CreateInBoundsGEP(Addr.getPointer(), getSize(Offset), Name),
>> + Addr.getAlignment().alignmentAtOffset(Offset));
>> + }
>> + Address CreateConstByteGEP(Address Addr, CharUnits Offset,
>> + const llvm::Twine &Name = "") {
>> + assert(Addr.getElementType() == TypeCache.Int8Ty);
>> + return Address(CreateGEP(Addr.getPointer(), getSize(Offset), Name),
>> + Addr.getAlignment().alignmentAtOffset(Offset));
>> + }
>> +
>> + llvm::Value *CreateConstInBoundsByteGEP(llvm::Value *Ptr, CharUnits Offset,
>> + const llvm::Twine &Name = "") {
>> + assert(Ptr->getType()->getPointerElementType() == TypeCache.Int8Ty);
>> + return CreateInBoundsGEP(Ptr, getSize(Offset), Name);
>> + }
>> + llvm::Value *CreateConstByteGEP(llvm::Value *Ptr, CharUnits Offset,
>> + const llvm::Twine &Name = "") {
>> + assert(Ptr->getType()->getPointerElementType() == TypeCache.Int8Ty);
>> + return CreateGEP(Ptr, getSize(Offset), Name);
>> + }
>> +
>> + using CGBuilderBaseTy::CreateMemCpy;
>> + llvm::CallInst *CreateMemCpy(Address Dest, Address Src, llvm::Value *Size,
>> + bool IsVolatile = false) {
>> + auto Align = std::min(Dest.getAlignment(), Src.getAlignment());
>> + return CreateMemCpy(Dest.getPointer(), Src.getPointer(), Size,
>> + Align.getQuantity(), IsVolatile);
>> + }
>> + llvm::CallInst *CreateMemCpy(Address Dest, Address Src, uint64_t Size,
>> + bool IsVolatile = false) {
>> + auto Align = std::min(Dest.getAlignment(), Src.getAlignment());
>> + return CreateMemCpy(Dest.getPointer(), Src.getPointer(), Size,
>> + Align.getQuantity(), IsVolatile);
>> + }
>> +
>> + using CGBuilderBaseTy::CreateMemMove;
>> + llvm::CallInst *CreateMemMove(Address Dest, Address Src, llvm::Value *Size,
>> + bool IsVolatile = false) {
>> + auto Align = std::min(Dest.getAlignment(), Src.getAlignment());
>> + return CreateMemMove(Dest.getPointer(), Src.getPointer(), Size,
>> + Align.getQuantity(), IsVolatile);
>> + }
>> +
>> + using CGBuilderBaseTy::CreateMemSet;
>> + llvm::CallInst *CreateMemSet(Address Dest, llvm::Value *Value,
>> + llvm::Value *Size, bool IsVolatile = false) {
>> + return CreateMemSet(Dest.getPointer(), Value, Size,
>> + Dest.getAlignment().getQuantity(), IsVolatile);
>> + }
>> +};
>> +
>> #undef PreserveNames
>>
>> } // end namespace CodeGen
>>
>> Modified: cfe/trunk/lib/CodeGen/CGBuiltin.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGBuiltin.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGBuiltin.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGBuiltin.cpp Tue Sep 8 03:05:57 2015
>> @@ -282,7 +282,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> case Builtin::BI__builtin_va_end: {
>> Value *ArgValue = (BuiltinID == Builtin::BI__va_start)
>> ? EmitScalarExpr(E->getArg(0))
>> - : EmitVAListRef(E->getArg(0));
>> + : EmitVAListRef(E->getArg(0)).getPointer();
>> llvm::Type *DestType = Int8PtrTy;
>> if (ArgValue->getType() != DestType)
>> ArgValue = Builder.CreateBitCast(ArgValue, DestType,
>> @@ -293,8 +293,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> return RValue::get(Builder.CreateCall(CGM.getIntrinsic(inst), ArgValue));
>> }
>> case Builtin::BI__builtin_va_copy: {
>> - Value *DstPtr = EmitVAListRef(E->getArg(0));
>> - Value *SrcPtr = EmitVAListRef(E->getArg(1));
>> + Value *DstPtr = EmitVAListRef(E->getArg(0)).getPointer();
>> + Value *SrcPtr = EmitVAListRef(E->getArg(1)).getPointer();
>>
>> llvm::Type *Type = Int8PtrTy;
>>
>> @@ -743,29 +743,24 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> }
>> case Builtin::BIbzero:
>> case Builtin::BI__builtin_bzero: {
>> - std::pair<llvm::Value*, unsigned> Dest =
>> - EmitPointerWithAlignment(E->getArg(0));
>> + Address Dest = EmitPointerWithAlignment(E->getArg(0));
>> Value *SizeVal = EmitScalarExpr(E->getArg(1));
>> - EmitNonNullArgCheck(RValue::get(Dest.first), E->getArg(0)->getType(),
>> + EmitNonNullArgCheck(RValue::get(Dest.getPointer()), E->getArg(0)->getType(),
>> E->getArg(0)->getExprLoc(), FD, 0);
>> - Builder.CreateMemSet(Dest.first, Builder.getInt8(0), SizeVal,
>> - Dest.second, false);
>> - return RValue::get(Dest.first);
>> + Builder.CreateMemSet(Dest, Builder.getInt8(0), SizeVal, false);
>> + return RValue::get(Dest.getPointer());
>> }
>> case Builtin::BImemcpy:
>> case Builtin::BI__builtin_memcpy: {
>> - std::pair<llvm::Value*, unsigned> Dest =
>> - EmitPointerWithAlignment(E->getArg(0));
>> - std::pair<llvm::Value*, unsigned> Src =
>> - EmitPointerWithAlignment(E->getArg(1));
>> + Address Dest = EmitPointerWithAlignment(E->getArg(0));
>> + Address Src = EmitPointerWithAlignment(E->getArg(1));
>> Value *SizeVal = EmitScalarExpr(E->getArg(2));
>> - unsigned Align = std::min(Dest.second, Src.second);
>> - EmitNonNullArgCheck(RValue::get(Dest.first), E->getArg(0)->getType(),
>> + EmitNonNullArgCheck(RValue::get(Dest.getPointer()), E->getArg(0)->getType(),
>> E->getArg(0)->getExprLoc(), FD, 0);
>> - EmitNonNullArgCheck(RValue::get(Src.first), E->getArg(1)->getType(),
>> + EmitNonNullArgCheck(RValue::get(Src.getPointer()), E->getArg(1)->getType(),
>> E->getArg(1)->getExprLoc(), FD, 1);
>> - Builder.CreateMemCpy(Dest.first, Src.first, SizeVal, Align, false);
>> - return RValue::get(Dest.first);
>> + Builder.CreateMemCpy(Dest, Src, SizeVal, false);
>> + return RValue::get(Dest.getPointer());
>> }
>>
>> case Builtin::BI__builtin___memcpy_chk: {
>> @@ -776,23 +771,20 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> break;
>> if (Size.ugt(DstSize))
>> break;
>> - std::pair<llvm::Value*, unsigned> Dest =
>> - EmitPointerWithAlignment(E->getArg(0));
>> - std::pair<llvm::Value*, unsigned> Src =
>> - EmitPointerWithAlignment(E->getArg(1));
>> + Address Dest = EmitPointerWithAlignment(E->getArg(0));
>> + Address Src = EmitPointerWithAlignment(E->getArg(1));
>> Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
>> - unsigned Align = std::min(Dest.second, Src.second);
>> - Builder.CreateMemCpy(Dest.first, Src.first, SizeVal, Align, false);
>> - return RValue::get(Dest.first);
>> + Builder.CreateMemCpy(Dest, Src, SizeVal, false);
>> + return RValue::get(Dest.getPointer());
>> }
>>
>> case Builtin::BI__builtin_objc_memmove_collectable: {
>> - Value *Address = EmitScalarExpr(E->getArg(0));
>> - Value *SrcAddr = EmitScalarExpr(E->getArg(1));
>> + Address DestAddr = EmitPointerWithAlignment(E->getArg(0));
>> + Address SrcAddr = EmitPointerWithAlignment(E->getArg(1));
>> Value *SizeVal = EmitScalarExpr(E->getArg(2));
>> CGM.getObjCRuntime().EmitGCMemmoveCollectable(*this,
>> - Address, SrcAddr, SizeVal);
>> - return RValue::get(Address);
>> + DestAddr, SrcAddr, SizeVal);
>> + return RValue::get(DestAddr.getPointer());
>> }
>>
>> case Builtin::BI__builtin___memmove_chk: {
>> @@ -803,42 +795,35 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> break;
>> if (Size.ugt(DstSize))
>> break;
>> - std::pair<llvm::Value*, unsigned> Dest =
>> - EmitPointerWithAlignment(E->getArg(0));
>> - std::pair<llvm::Value*, unsigned> Src =
>> - EmitPointerWithAlignment(E->getArg(1));
>> + Address Dest = EmitPointerWithAlignment(E->getArg(0));
>> + Address Src = EmitPointerWithAlignment(E->getArg(1));
>> Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
>> - unsigned Align = std::min(Dest.second, Src.second);
>> - Builder.CreateMemMove(Dest.first, Src.first, SizeVal, Align, false);
>> - return RValue::get(Dest.first);
>> + Builder.CreateMemMove(Dest, Src, SizeVal, false);
>> + return RValue::get(Dest.getPointer());
>> }
>>
>> case Builtin::BImemmove:
>> case Builtin::BI__builtin_memmove: {
>> - std::pair<llvm::Value*, unsigned> Dest =
>> - EmitPointerWithAlignment(E->getArg(0));
>> - std::pair<llvm::Value*, unsigned> Src =
>> - EmitPointerWithAlignment(E->getArg(1));
>> + Address Dest = EmitPointerWithAlignment(E->getArg(0));
>> + Address Src = EmitPointerWithAlignment(E->getArg(1));
>> Value *SizeVal = EmitScalarExpr(E->getArg(2));
>> - unsigned Align = std::min(Dest.second, Src.second);
>> - EmitNonNullArgCheck(RValue::get(Dest.first), E->getArg(0)->getType(),
>> + EmitNonNullArgCheck(RValue::get(Dest.getPointer()), E->getArg(0)->getType(),
>> E->getArg(0)->getExprLoc(), FD, 0);
>> - EmitNonNullArgCheck(RValue::get(Src.first), E->getArg(1)->getType(),
>> + EmitNonNullArgCheck(RValue::get(Src.getPointer()), E->getArg(1)->getType(),
>> E->getArg(1)->getExprLoc(), FD, 1);
>> - Builder.CreateMemMove(Dest.first, Src.first, SizeVal, Align, false);
>> - return RValue::get(Dest.first);
>> + Builder.CreateMemMove(Dest, Src, SizeVal, false);
>> + return RValue::get(Dest.getPointer());
>> }
>> case Builtin::BImemset:
>> case Builtin::BI__builtin_memset: {
>> - std::pair<llvm::Value*, unsigned> Dest =
>> - EmitPointerWithAlignment(E->getArg(0));
>> + Address Dest = EmitPointerWithAlignment(E->getArg(0));
>> Value *ByteVal = Builder.CreateTrunc(EmitScalarExpr(E->getArg(1)),
>> Builder.getInt8Ty());
>> Value *SizeVal = EmitScalarExpr(E->getArg(2));
>> - EmitNonNullArgCheck(RValue::get(Dest.first), E->getArg(0)->getType(),
>> + EmitNonNullArgCheck(RValue::get(Dest.getPointer()), E->getArg(0)->getType(),
>> E->getArg(0)->getExprLoc(), FD, 0);
>> - Builder.CreateMemSet(Dest.first, ByteVal, SizeVal, Dest.second, false);
>> - return RValue::get(Dest.first);
>> + Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
>> + return RValue::get(Dest.getPointer());
>> }
>> case Builtin::BI__builtin___memset_chk: {
>> // fold __builtin_memset_chk(x, y, cst1, cst2) to memset iff cst1<=cst2.
>> @@ -848,13 +833,12 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> break;
>> if (Size.ugt(DstSize))
>> break;
>> - std::pair<llvm::Value*, unsigned> Dest =
>> - EmitPointerWithAlignment(E->getArg(0));
>> + Address Dest = EmitPointerWithAlignment(E->getArg(0));
>> Value *ByteVal = Builder.CreateTrunc(EmitScalarExpr(E->getArg(1)),
>> Builder.getInt8Ty());
>> Value *SizeVal = llvm::ConstantInt::get(Builder.getContext(), Size);
>> - Builder.CreateMemSet(Dest.first, ByteVal, SizeVal, Dest.second, false);
>> - return RValue::get(Dest.first);
>> + Builder.CreateMemSet(Dest, ByteVal, SizeVal, false);
>> + return RValue::get(Dest.getPointer());
>> }
>> case Builtin::BI__builtin_dwarf_cfa: {
>> // The offset in bytes from the first argument to the CFA.
>> @@ -958,7 +942,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> }
>> case Builtin::BI__builtin_setjmp: {
>> // Buffer is a void**.
>> - Value *Buf = EmitScalarExpr(E->getArg(0));
>> + Address Buf = EmitPointerWithAlignment(E->getArg(0));
>>
>> // Store the frame pointer to the setjmp buffer.
>> Value *FrameAddr =
>> @@ -969,14 +953,14 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> // Store the stack pointer to the setjmp buffer.
>> Value *StackAddr =
>> Builder.CreateCall(CGM.getIntrinsic(Intrinsic::stacksave));
>> - Value *StackSaveSlot =
>> - Builder.CreateGEP(Buf, ConstantInt::get(Int32Ty, 2));
>> + Address StackSaveSlot =
>> + Builder.CreateConstInBoundsGEP(Buf, 2, getPointerSize());
>> Builder.CreateStore(StackAddr, StackSaveSlot);
>>
>> // Call LLVM's EH setjmp, which is lightweight.
>> Value *F = CGM.getIntrinsic(Intrinsic::eh_sjlj_setjmp);
>> Buf = Builder.CreateBitCast(Buf, Int8PtrTy);
>> - return RValue::get(Builder.CreateCall(F, Buf));
>> + return RValue::get(Builder.CreateCall(F, Buf.getPointer()));
>> }
>> case Builtin::BI__builtin_longjmp: {
>> Value *Buf = EmitScalarExpr(E->getArg(0));
>> @@ -1141,8 +1125,8 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> StoreSize.getQuantity() * 8);
>> Ptr = Builder.CreateBitCast(Ptr, ITy->getPointerTo());
>> llvm::StoreInst *Store =
>> - Builder.CreateStore(llvm::Constant::getNullValue(ITy), Ptr);
>> - Store->setAlignment(StoreSize.getQuantity());
>> + Builder.CreateAlignedStore(llvm::Constant::getNullValue(ITy), Ptr,
>> + StoreSize);
>> Store->setAtomic(llvm::Release);
>> return RValue::get(nullptr);
>> }
>> @@ -1276,15 +1260,14 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> bool Volatile =
>> PtrTy->castAs<PointerType>()->getPointeeType().isVolatileQualified();
>>
>> - Value *Ptr = EmitScalarExpr(E->getArg(0));
>> - unsigned AddrSpace = Ptr->getType()->getPointerAddressSpace();
>> + Address Ptr = EmitPointerWithAlignment(E->getArg(0));
>> + unsigned AddrSpace = Ptr.getPointer()->getType()->getPointerAddressSpace();
>> Ptr = Builder.CreateBitCast(Ptr, Int8Ty->getPointerTo(AddrSpace));
>> Value *NewVal = Builder.getInt8(0);
>> Value *Order = EmitScalarExpr(E->getArg(1));
>> if (isa<llvm::ConstantInt>(Order)) {
>> int ord = cast<llvm::ConstantInt>(Order)->getZExtValue();
>> StoreInst *Store = Builder.CreateStore(NewVal, Ptr, Volatile);
>> - Store->setAlignment(1);
>> switch (ord) {
>> case 0: // memory_order_relaxed
>> default: // invalid order
>> @@ -1317,7 +1300,6 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> for (unsigned i = 0; i < 3; ++i) {
>> Builder.SetInsertPoint(BBs[i]);
>> StoreInst *Store = Builder.CreateStore(NewVal, Ptr, Volatile);
>> - Store->setAlignment(1);
>> Store->setOrdering(Orders[i]);
>> Builder.CreateBr(ContBB);
>> }
>> @@ -1499,8 +1481,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> llvm::Value *X = EmitScalarExpr(E->getArg(0));
>> llvm::Value *Y = EmitScalarExpr(E->getArg(1));
>> llvm::Value *Carryin = EmitScalarExpr(E->getArg(2));
>> - std::pair<llvm::Value*, unsigned> CarryOutPtr =
>> - EmitPointerWithAlignment(E->getArg(3));
>> + Address CarryOutPtr = EmitPointerWithAlignment(E->getArg(3));
>>
>> // Decide if we are lowering to a uadd.with.overflow or usub.with.overflow.
>> llvm::Intrinsic::ID IntrinsicId;
>> @@ -1531,9 +1512,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> Sum1, Carryin, Carry2);
>> llvm::Value *CarryOut = Builder.CreateZExt(Builder.CreateOr(Carry1, Carry2),
>> X->getType());
>> - llvm::StoreInst *CarryOutStore = Builder.CreateStore(CarryOut,
>> - CarryOutPtr.first);
>> - CarryOutStore->setAlignment(CarryOutPtr.second);
>> + Builder.CreateStore(CarryOut, CarryOutPtr);
>> return RValue::get(Sum2);
>> }
>> case Builtin::BI__builtin_uadd_overflow:
>> @@ -1560,8 +1539,7 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>> // Scalarize our inputs.
>> llvm::Value *X = EmitScalarExpr(E->getArg(0));
>> llvm::Value *Y = EmitScalarExpr(E->getArg(1));
>> - std::pair<llvm::Value *, unsigned> SumOutPtr =
>> - EmitPointerWithAlignment(E->getArg(2));
>> + Address SumOutPtr = EmitPointerWithAlignment(E->getArg(2));
>>
>> // Decide which of the overflow intrinsics we are lowering to:
>> llvm::Intrinsic::ID IntrinsicId;
>> @@ -1602,13 +1580,12 @@ RValue CodeGenFunction::EmitBuiltinExpr(
>>
>> llvm::Value *Carry;
>> llvm::Value *Sum = EmitOverflowIntrinsic(*this, IntrinsicId, X, Y, Carry);
>> - llvm::StoreInst *SumOutStore = Builder.CreateStore(Sum, SumOutPtr.first);
>> - SumOutStore->setAlignment(SumOutPtr.second);
>> + Builder.CreateStore(Sum, SumOutPtr);
>>
>> return RValue::get(Carry);
>> }
>> case Builtin::BI__builtin_addressof:
>> - return RValue::get(EmitLValue(E->getArg(0)).getAddress());
>> + return RValue::get(EmitLValue(E->getArg(0)).getPointer());
>> case Builtin::BI__builtin_operator_new:
>> return EmitBuiltinNewDeleteCall(FD->getType()->castAs<FunctionProtoType>(),
>> E->getArg(0), false);
>> @@ -1997,61 +1974,6 @@ Value *CodeGenFunction::EmitNeonRShiftIm
>> return Builder.CreateAShr(Vec, Shift, name);
>> }
>>
>> -/// GetPointeeAlignment - Given an expression with a pointer type, find the
>> -/// alignment of the type referenced by the pointer. Skip over implicit
>> -/// casts.
>> -std::pair<llvm::Value*, unsigned>
>> -CodeGenFunction::EmitPointerWithAlignment(const Expr *Addr) {
>> - assert(Addr->getType()->isPointerType());
>> - Addr = Addr->IgnoreParens();
>> - if (const ImplicitCastExpr *ICE = dyn_cast<ImplicitCastExpr>(Addr)) {
>> - if ((ICE->getCastKind() == CK_BitCast || ICE->getCastKind() == CK_NoOp) &&
>> - ICE->getSubExpr()->getType()->isPointerType()) {
>> - std::pair<llvm::Value*, unsigned> Ptr =
>> - EmitPointerWithAlignment(ICE->getSubExpr());
>> - Ptr.first = Builder.CreateBitCast(Ptr.first,
>> - ConvertType(Addr->getType()));
>> - return Ptr;
>> - } else if (ICE->getCastKind() == CK_ArrayToPointerDecay) {
>> - LValue LV = EmitLValue(ICE->getSubExpr());
>> - unsigned Align = LV.getAlignment().getQuantity();
>> - if (!Align) {
>> - // FIXME: Once LValues are fixed to always set alignment,
>> - // zap this code.
>> - QualType PtTy = ICE->getSubExpr()->getType();
>> - if (!PtTy->isIncompleteType())
>> - Align = getContext().getTypeAlignInChars(PtTy).getQuantity();
>> - else
>> - Align = 1;
>> - }
>> - return std::make_pair(LV.getAddress(), Align);
>> - }
>> - }
>> - if (const UnaryOperator *UO = dyn_cast<UnaryOperator>(Addr)) {
>> - if (UO->getOpcode() == UO_AddrOf) {
>> - LValue LV = EmitLValue(UO->getSubExpr());
>> - unsigned Align = LV.getAlignment().getQuantity();
>> - if (!Align) {
>> - // FIXME: Once LValues are fixed to always set alignment,
>> - // zap this code.
>> - QualType PtTy = UO->getSubExpr()->getType();
>> - if (!PtTy->isIncompleteType())
>> - Align = getContext().getTypeAlignInChars(PtTy).getQuantity();
>> - else
>> - Align = 1;
>> - }
>> - return std::make_pair(LV.getAddress(), Align);
>> - }
>> - }
>> -
>> - unsigned Align = 1;
>> - QualType PtTy = Addr->getType()->getPointeeType();
>> - if (!PtTy->isIncompleteType())
>> - Align = getContext().getTypeAlignInChars(PtTy).getQuantity();
>> -
>> - return std::make_pair(EmitScalarExpr(Addr), Align);
>> -}
>> -
>> enum {
>> AddRetType = (1 << 0),
>> Add1ArgType = (1 << 1),
>> @@ -2762,7 +2684,7 @@ static Value *EmitCommonNeonSISDBuiltinE
>> Value *CodeGenFunction::EmitCommonNeonBuiltinExpr(
>> unsigned BuiltinID, unsigned LLVMIntrinsic, unsigned AltLLVMIntrinsic,
>> const char *NameHint, unsigned Modifier, const CallExpr *E,
>> - SmallVectorImpl<llvm::Value *> &Ops, llvm::Value *Align) {
>> + SmallVectorImpl<llvm::Value *> &Ops, Address PtrOp0, Address PtrOp1) {
>> // Get the last argument, which specifies the vector type.
>> llvm::APSInt NeonTypeConst;
>> const Expr *Arg = E->getArg(E->getNumArgs() - 1);
>> @@ -2779,6 +2701,10 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> if (!Ty)
>> return nullptr;
>>
>> + auto getAlignmentValue32 = [&](Address addr) -> Value* {
>> + return Builder.getInt32(addr.getAlignment().getQuantity());
>> + };
>> +
>> unsigned Int = LLVMIntrinsic;
>> if ((Modifier & UnsignedAlts) && !Usgn)
>> Int = AltLLVMIntrinsic;
>> @@ -2927,7 +2853,7 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> }
>> case NEON::BI__builtin_neon_vld1_v:
>> case NEON::BI__builtin_neon_vld1q_v:
>> - Ops.push_back(Align);
>> + Ops.push_back(getAlignmentValue32(PtrOp0));
>> return EmitNeonCall(CGM.getIntrinsic(LLVMIntrinsic, Ty), Ops, "vld1");
>> case NEON::BI__builtin_neon_vld2_v:
>> case NEON::BI__builtin_neon_vld2q_v:
>> @@ -2936,18 +2862,18 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> case NEON::BI__builtin_neon_vld4_v:
>> case NEON::BI__builtin_neon_vld4q_v: {
>> Function *F = CGM.getIntrinsic(LLVMIntrinsic, Ty);
>> + Value *Align = getAlignmentValue32(PtrOp1);
>> Ops[1] = Builder.CreateCall(F, {Ops[1], Align}, NameHint);
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld1_dup_v:
>> case NEON::BI__builtin_neon_vld1q_dup_v: {
>> Value *V = UndefValue::get(Ty);
>> Ty = llvm::PointerType::getUnqual(VTy->getElementType());
>> - Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - LoadInst *Ld = Builder.CreateLoad(Ops[0]);
>> - Ld->setAlignment(cast<ConstantInt>(Align)->getZExtValue());
>> + PtrOp0 = Builder.CreateBitCast(PtrOp0, Ty);
>> + LoadInst *Ld = Builder.CreateLoad(PtrOp0);
>> llvm::Constant *CI = ConstantInt::get(SizeTy, 0);
>> Ops[0] = Builder.CreateInsertElement(V, Ld, CI);
>> return EmitNeonSplat(Ops[0], CI);
>> @@ -2961,11 +2887,11 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> Function *F = CGM.getIntrinsic(LLVMIntrinsic, Ty);
>> for (unsigned I = 2; I < Ops.size() - 1; ++I)
>> Ops[I] = Builder.CreateBitCast(Ops[I], Ty);
>> - Ops.push_back(Align);
>> + Ops.push_back(getAlignmentValue32(PtrOp1));
>> Ops[1] = Builder.CreateCall(F, makeArrayRef(Ops).slice(1), NameHint);
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vmovl_v: {
>> llvm::Type *DTy =llvm::VectorType::getTruncatedElementVectorType(VTy);
>> @@ -3078,7 +3004,7 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> case NEON::BI__builtin_neon_vst3q_lane_v:
>> case NEON::BI__builtin_neon_vst4_lane_v:
>> case NEON::BI__builtin_neon_vst4q_lane_v:
>> - Ops.push_back(Align);
>> + Ops.push_back(getAlignmentValue32(PtrOp0));
>> return EmitNeonCall(CGM.getIntrinsic(Int, Ty), Ops, "");
>> case NEON::BI__builtin_neon_vsubhn_v: {
>> llvm::VectorType *SrcTy =
>> @@ -3113,7 +3039,7 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> Value *Addr = Builder.CreateConstInBoundsGEP1_32(Ty, Ops[0], vi);
>> SV = llvm::ConstantVector::get(Indices);
>> SV = Builder.CreateShuffleVector(Ops[1], Ops[2], SV, "vtrn");
>> - SV = Builder.CreateStore(SV, Addr);
>> + SV = Builder.CreateDefaultAlignedStore(SV, Addr);
>> }
>> return SV;
>> }
>> @@ -3141,7 +3067,7 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> Value *Addr = Builder.CreateConstInBoundsGEP1_32(Ty, Ops[0], vi);
>> SV = llvm::ConstantVector::get(Indices);
>> SV = Builder.CreateShuffleVector(Ops[1], Ops[2], SV, "vuzp");
>> - SV = Builder.CreateStore(SV, Addr);
>> + SV = Builder.CreateDefaultAlignedStore(SV, Addr);
>> }
>> return SV;
>> }
>> @@ -3161,7 +3087,7 @@ Value *CodeGenFunction::EmitCommonNeonBu
>> Value *Addr = Builder.CreateConstInBoundsGEP1_32(Ty, Ops[0], vi);
>> SV = llvm::ConstantVector::get(Indices);
>> SV = Builder.CreateShuffleVector(Ops[1], Ops[2], SV, "vzip");
>> - SV = Builder.CreateStore(SV, Addr);
>> + SV = Builder.CreateDefaultAlignedStore(SV, Addr);
>> }
>> return SV;
>> }
>> @@ -3496,11 +3422,11 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> : Intrinsic::arm_strexd);
>> llvm::Type *STy = llvm::StructType::get(Int32Ty, Int32Ty, nullptr);
>>
>> - Value *Tmp = CreateMemTemp(E->getArg(0)->getType());
>> + Address Tmp = CreateMemTemp(E->getArg(0)->getType());
>> Value *Val = EmitScalarExpr(E->getArg(0));
>> Builder.CreateStore(Val, Tmp);
>>
>> - Value *LdPtr = Builder.CreateBitCast(Tmp,llvm::PointerType::getUnqual(STy));
>> + Address LdPtr = Builder.CreateBitCast(Tmp,llvm::PointerType::getUnqual(STy));
>> Val = Builder.CreateLoad(LdPtr);
>>
>> Value *Arg0 = Builder.CreateExtractValue(Val, 0);
>> @@ -3619,8 +3545,13 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> getContext().GetBuiltinType(BuiltinID, Error, &ICEArguments);
>> assert(Error == ASTContext::GE_None && "Should not codegen an error");
>>
>> + auto getAlignmentValue32 = [&](Address addr) -> Value* {
>> + return Builder.getInt32(addr.getAlignment().getQuantity());
>> + };
>> +
>> + Address PtrOp0 = Address::invalid();
>> + Address PtrOp1 = Address::invalid();
>> SmallVector<Value*, 4> Ops;
>> - llvm::Value *Align = nullptr;
>> bool HasExtraArg = HasExtraNeonArgument(BuiltinID);
>> unsigned NumArgs = E->getNumArgs() - (HasExtraArg ? 1 : 0);
>> for (unsigned i = 0, e = NumArgs; i != e; i++) {
>> @@ -3650,10 +3581,8 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> case NEON::BI__builtin_neon_vst4q_lane_v:
>> // Get the alignment for the argument in addition to the value;
>> // we'll use it later.
>> - std::pair<llvm::Value*, unsigned> Src =
>> - EmitPointerWithAlignment(E->getArg(0));
>> - Ops.push_back(Src.first);
>> - Align = Builder.getInt32(Src.second);
>> + PtrOp0 = EmitPointerWithAlignment(E->getArg(0));
>> + Ops.push_back(PtrOp0.getPointer());
>> continue;
>> }
>> }
>> @@ -3676,10 +3605,8 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> case NEON::BI__builtin_neon_vld4_dup_v:
>> // Get the alignment for the argument in addition to the value;
>> // we'll use it later.
>> - std::pair<llvm::Value*, unsigned> Src =
>> - EmitPointerWithAlignment(E->getArg(1));
>> - Ops.push_back(Src.first);
>> - Align = Builder.getInt32(Src.second);
>> + PtrOp1 = EmitPointerWithAlignment(E->getArg(1));
>> + Ops.push_back(PtrOp1.getPointer());
>> continue;
>> }
>> }
>> @@ -3790,7 +3717,7 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> if (Builtin)
>> return EmitCommonNeonBuiltinExpr(
>> Builtin->BuiltinID, Builtin->LLVMIntrinsic, Builtin->AltLLVMIntrinsic,
>> - Builtin->NameHint, Builtin->TypeModifier, E, Ops, Align);
>> + Builtin->NameHint, Builtin->TypeModifier, E, Ops, PtrOp0, PtrOp1);
>>
>> unsigned Int;
>> switch (BuiltinID) {
>> @@ -3807,6 +3734,7 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> // Load the value as a one-element vector.
>> Ty = llvm::VectorType::get(VTy->getElementType(), 1);
>> Function *F = CGM.getIntrinsic(Intrinsic::arm_neon_vld1, Ty);
>> + Value *Align = getAlignmentValue32(PtrOp0);
>> Value *Ld = Builder.CreateCall(F, {Ops[0], Align});
>> // Combine them.
>> uint32_t Indices[] = {1 - Lane, Lane};
>> @@ -3818,8 +3746,7 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> Ops[1] = Builder.CreateBitCast(Ops[1], Ty);
>> Ty = llvm::PointerType::getUnqual(VTy->getElementType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - LoadInst *Ld = Builder.CreateLoad(Ops[0]);
>> - Ld->setAlignment(cast<ConstantInt>(Align)->getZExtValue());
>> + Value *Ld = Builder.CreateLoad(PtrOp0);
>> return Builder.CreateInsertElement(Ops[1], Ld, Ops[2], "vld1_lane");
>> }
>> case NEON::BI__builtin_neon_vld2_dup_v:
>> @@ -3840,10 +3767,11 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> default: llvm_unreachable("unknown vld_dup intrinsic?");
>> }
>> Function *F = CGM.getIntrinsic(Int, Ty);
>> + llvm::Value *Align = getAlignmentValue32(PtrOp1);
>> Ops[1] = Builder.CreateCall(F, {Ops[1], Align}, "vld_dup");
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> switch (BuiltinID) {
>> case NEON::BI__builtin_neon_vld2_dup_v:
>> @@ -3866,7 +3794,7 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>>
>> llvm::Constant *CI = ConstantInt::get(Int32Ty, 0);
>> Args.push_back(CI);
>> - Args.push_back(Align);
>> + Args.push_back(getAlignmentValue32(PtrOp1));
>>
>> Ops[1] = Builder.CreateCall(F, Args, "vld_dup");
>> // splat lane 0 to all elts in each vector of the result.
>> @@ -3879,7 +3807,7 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> }
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vqrshrn_n_v:
>> Int =
>> @@ -3931,7 +3859,7 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> Ops[1] = Builder.CreateBitCast(Ops[1], Ty);
>> Value *SV = llvm::ConstantVector::get(cast<llvm::Constant>(Ops[2]));
>> Ops[1] = Builder.CreateShuffleVector(Ops[1], Ops[1], SV);
>> - Ops[2] = Align;
>> + Ops[2] = getAlignmentValue32(PtrOp0);
>> return Builder.CreateCall(CGM.getIntrinsic(Intrinsic::arm_neon_vst1,
>> Ops[1]->getType()), Ops);
>> }
>> @@ -3940,9 +3868,7 @@ Value *CodeGenFunction::EmitARMBuiltinEx
>> Ops[1] = Builder.CreateBitCast(Ops[1], Ty);
>> Ops[1] = Builder.CreateExtractElement(Ops[1], Ops[2]);
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> - StoreInst *St = Builder.CreateStore(Ops[1],
>> - Builder.CreateBitCast(Ops[0], Ty));
>> - St->setAlignment(cast<ConstantInt>(Align)->getZExtValue());
>> + auto St = Builder.CreateStore(Ops[1], Builder.CreateBitCast(PtrOp0, Ty));
>> return St;
>> }
>> case NEON::BI__builtin_neon_vtbl1_v:
>> @@ -4269,14 +4195,11 @@ Value *CodeGenFunction::EmitAArch64Built
>> : Intrinsic::aarch64_stxp);
>> llvm::Type *STy = llvm::StructType::get(Int64Ty, Int64Ty, nullptr);
>>
>> - Value *One = llvm::ConstantInt::get(Int32Ty, 1);
>> - Value *Tmp = Builder.CreateAlloca(ConvertType(E->getArg(0)->getType()),
>> - One);
>> - Value *Val = EmitScalarExpr(E->getArg(0));
>> - Builder.CreateStore(Val, Tmp);
>> + Address Tmp = CreateMemTemp(E->getArg(0)->getType());
>> + EmitAnyExprToMem(E->getArg(0), Tmp, Qualifiers(), /*init*/ true);
>>
>> - Value *LdPtr = Builder.CreateBitCast(Tmp,llvm::PointerType::getUnqual(STy));
>> - Val = Builder.CreateLoad(LdPtr);
>> + Tmp = Builder.CreateBitCast(Tmp, llvm::PointerType::getUnqual(STy));
>> + llvm::Value *Val = Builder.CreateLoad(Tmp);
>>
>> Value *Arg0 = Builder.CreateExtractValue(Val, 0);
>> Value *Arg1 = Builder.CreateExtractValue(Val, 1);
>> @@ -4430,12 +4353,12 @@ Value *CodeGenFunction::EmitAArch64Built
>> case NEON::BI__builtin_neon_vldrq_p128: {
>> llvm::Type *Int128PTy = llvm::Type::getIntNPtrTy(getLLVMContext(), 128);
>> Value *Ptr = Builder.CreateBitCast(EmitScalarExpr(E->getArg(0)), Int128PTy);
>> - return Builder.CreateLoad(Ptr);
>> + return Builder.CreateDefaultAlignedLoad(Ptr);
>> }
>> case NEON::BI__builtin_neon_vstrq_p128: {
>> llvm::Type *Int128PTy = llvm::Type::getIntNPtrTy(getLLVMContext(), 128);
>> Value *Ptr = Builder.CreateBitCast(Ops[0], Int128PTy);
>> - return Builder.CreateStore(EmitScalarExpr(E->getArg(1)), Ptr);
>> + return Builder.CreateDefaultAlignedStore(EmitScalarExpr(E->getArg(1)), Ptr);
>> }
>> case NEON::BI__builtin_neon_vcvts_u32_f32:
>> case NEON::BI__builtin_neon_vcvtd_u64_f64:
>> @@ -4895,7 +4818,8 @@ Value *CodeGenFunction::EmitAArch64Built
>> if (Builtin)
>> return EmitCommonNeonBuiltinExpr(
>> Builtin->BuiltinID, Builtin->LLVMIntrinsic, Builtin->AltLLVMIntrinsic,
>> - Builtin->NameHint, Builtin->TypeModifier, E, Ops, nullptr);
>> + Builtin->NameHint, Builtin->TypeModifier, E, Ops,
>> + /*never use addresses*/ Address::invalid(), Address::invalid());
>>
>> if (Value *V = EmitAArch64TblBuiltinExpr(*this, BuiltinID, E, Ops))
>> return V;
>> @@ -5589,7 +5513,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, Ops[1], "vld1xN");
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vst1_x2_v:
>> case NEON::BI__builtin_neon_vst1q_x2_v:
>> @@ -5620,25 +5544,25 @@ Value *CodeGenFunction::EmitAArch64Built
>> case NEON::BI__builtin_neon_vld1_v:
>> case NEON::BI__builtin_neon_vld1q_v:
>> Ops[0] = Builder.CreateBitCast(Ops[0], llvm::PointerType::getUnqual(VTy));
>> - return Builder.CreateLoad(Ops[0]);
>> + return Builder.CreateDefaultAlignedLoad(Ops[0]);
>> case NEON::BI__builtin_neon_vst1_v:
>> case NEON::BI__builtin_neon_vst1q_v:
>> Ops[0] = Builder.CreateBitCast(Ops[0], llvm::PointerType::getUnqual(VTy));
>> Ops[1] = Builder.CreateBitCast(Ops[1], VTy);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> case NEON::BI__builtin_neon_vld1_lane_v:
>> case NEON::BI__builtin_neon_vld1q_lane_v:
>> Ops[1] = Builder.CreateBitCast(Ops[1], Ty);
>> Ty = llvm::PointerType::getUnqual(VTy->getElementType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - Ops[0] = Builder.CreateLoad(Ops[0]);
>> + Ops[0] = Builder.CreateDefaultAlignedLoad(Ops[0]);
>> return Builder.CreateInsertElement(Ops[1], Ops[0], Ops[2], "vld1_lane");
>> case NEON::BI__builtin_neon_vld1_dup_v:
>> case NEON::BI__builtin_neon_vld1q_dup_v: {
>> Value *V = UndefValue::get(Ty);
>> Ty = llvm::PointerType::getUnqual(VTy->getElementType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - Ops[0] = Builder.CreateLoad(Ops[0]);
>> + Ops[0] = Builder.CreateDefaultAlignedLoad(Ops[0]);
>> llvm::Constant *CI = ConstantInt::get(Int32Ty, 0);
>> Ops[0] = Builder.CreateInsertElement(V, Ops[0], CI);
>> return EmitNeonSplat(Ops[0], CI);
>> @@ -5648,7 +5572,8 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateBitCast(Ops[1], Ty);
>> Ops[1] = Builder.CreateExtractElement(Ops[1], Ops[2]);
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> - return Builder.CreateStore(Ops[1], Builder.CreateBitCast(Ops[0], Ty));
>> + return Builder.CreateDefaultAlignedStore(Ops[1],
>> + Builder.CreateBitCast(Ops[0], Ty));
>> case NEON::BI__builtin_neon_vld2_v:
>> case NEON::BI__builtin_neon_vld2q_v: {
>> llvm::Type *PTy = llvm::PointerType::getUnqual(VTy);
>> @@ -5658,7 +5583,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, Ops[1], "vld2");
>> Ops[0] = Builder.CreateBitCast(Ops[0],
>> llvm::PointerType::getUnqual(Ops[1]->getType()));
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld3_v:
>> case NEON::BI__builtin_neon_vld3q_v: {
>> @@ -5669,7 +5594,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, Ops[1], "vld3");
>> Ops[0] = Builder.CreateBitCast(Ops[0],
>> llvm::PointerType::getUnqual(Ops[1]->getType()));
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld4_v:
>> case NEON::BI__builtin_neon_vld4q_v: {
>> @@ -5680,7 +5605,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, Ops[1], "vld4");
>> Ops[0] = Builder.CreateBitCast(Ops[0],
>> llvm::PointerType::getUnqual(Ops[1]->getType()));
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld2_dup_v:
>> case NEON::BI__builtin_neon_vld2q_dup_v: {
>> @@ -5692,7 +5617,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, Ops[1], "vld2");
>> Ops[0] = Builder.CreateBitCast(Ops[0],
>> llvm::PointerType::getUnqual(Ops[1]->getType()));
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld3_dup_v:
>> case NEON::BI__builtin_neon_vld3q_dup_v: {
>> @@ -5704,7 +5629,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, Ops[1], "vld3");
>> Ops[0] = Builder.CreateBitCast(Ops[0],
>> llvm::PointerType::getUnqual(Ops[1]->getType()));
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld4_dup_v:
>> case NEON::BI__builtin_neon_vld4q_dup_v: {
>> @@ -5716,7 +5641,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, Ops[1], "vld4");
>> Ops[0] = Builder.CreateBitCast(Ops[0],
>> llvm::PointerType::getUnqual(Ops[1]->getType()));
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld2_lane_v:
>> case NEON::BI__builtin_neon_vld2q_lane_v: {
>> @@ -5730,7 +5655,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, makeArrayRef(Ops).slice(1), "vld2_lane");
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld3_lane_v:
>> case NEON::BI__builtin_neon_vld3q_lane_v: {
>> @@ -5745,7 +5670,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, makeArrayRef(Ops).slice(1), "vld3_lane");
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vld4_lane_v:
>> case NEON::BI__builtin_neon_vld4q_lane_v: {
>> @@ -5761,7 +5686,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Ops[1] = Builder.CreateCall(F, makeArrayRef(Ops).slice(1), "vld4_lane");
>> Ty = llvm::PointerType::getUnqual(Ops[1]->getType());
>> Ops[0] = Builder.CreateBitCast(Ops[0], Ty);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case NEON::BI__builtin_neon_vst2_v:
>> case NEON::BI__builtin_neon_vst2q_v: {
>> @@ -5830,7 +5755,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Value *Addr = Builder.CreateConstInBoundsGEP1_32(Ty, Ops[0], vi);
>> SV = llvm::ConstantVector::get(Indices);
>> SV = Builder.CreateShuffleVector(Ops[1], Ops[2], SV, "vtrn");
>> - SV = Builder.CreateStore(SV, Addr);
>> + SV = Builder.CreateDefaultAlignedStore(SV, Addr);
>> }
>> return SV;
>> }
>> @@ -5849,7 +5774,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Value *Addr = Builder.CreateConstInBoundsGEP1_32(Ty, Ops[0], vi);
>> SV = llvm::ConstantVector::get(Indices);
>> SV = Builder.CreateShuffleVector(Ops[1], Ops[2], SV, "vuzp");
>> - SV = Builder.CreateStore(SV, Addr);
>> + SV = Builder.CreateDefaultAlignedStore(SV, Addr);
>> }
>> return SV;
>> }
>> @@ -5869,7 +5794,7 @@ Value *CodeGenFunction::EmitAArch64Built
>> Value *Addr = Builder.CreateConstInBoundsGEP1_32(Ty, Ops[0], vi);
>> SV = llvm::ConstantVector::get(Indices);
>> SV = Builder.CreateShuffleVector(Ops[1], Ops[2], SV, "vzip");
>> - SV = Builder.CreateStore(SV, Addr);
>> + SV = Builder.CreateDefaultAlignedStore(SV, Addr);
>> }
>> return SV;
>> }
>> @@ -6041,7 +5966,8 @@ Value *CodeGenFunction::EmitX86BuiltinEx
>> ConstantInt::get(Int32Ty, 0)
>> };
>> Value *CpuFeatures = Builder.CreateGEP(STy, CpuModel, Idxs);
>> - Value *Features = Builder.CreateLoad(CpuFeatures);
>> + Value *Features = Builder.CreateAlignedLoad(CpuFeatures,
>> + CharUnits::fromQuantity(4));
>>
>> // Check the value of the bit corresponding to the feature requested.
>> Value *Bitset = Builder.CreateAnd(
>> @@ -6049,9 +5975,9 @@ Value *CodeGenFunction::EmitX86BuiltinEx
>> return Builder.CreateICmpNE(Bitset, llvm::ConstantInt::get(Int32Ty, 0));
>> }
>> case X86::BI_mm_prefetch: {
>> - Value *Address = EmitScalarExpr(E->getArg(0));
>> + Value *Address = Ops[0];
>> Value *RW = ConstantInt::get(Int32Ty, 0);
>> - Value *Locality = EmitScalarExpr(E->getArg(1));
>> + Value *Locality = Ops[1];
>> Value *Data = ConstantInt::get(Int32Ty, 1);
>> Value *F = CGM.getIntrinsic(Intrinsic::prefetch);
>> return Builder.CreateCall(F, {Address, RW, Locality, Data});
>> @@ -6069,15 +5995,15 @@ Value *CodeGenFunction::EmitX86BuiltinEx
>> return Builder.CreateExtractElement(Ops[0],
>> llvm::ConstantInt::get(Ops[1]->getType(), 0));
>> case X86::BI__builtin_ia32_ldmxcsr: {
>> - Value *Tmp = CreateMemTemp(E->getArg(0)->getType());
>> + Address Tmp = CreateMemTemp(E->getArg(0)->getType());
>> Builder.CreateStore(Ops[0], Tmp);
>> return Builder.CreateCall(CGM.getIntrinsic(Intrinsic::x86_sse_ldmxcsr),
>> - Builder.CreateBitCast(Tmp, Int8PtrTy));
>> + Builder.CreateBitCast(Tmp.getPointer(), Int8PtrTy));
>> }
>> case X86::BI__builtin_ia32_stmxcsr: {
>> - Value *Tmp = CreateMemTemp(E->getType());
>> + Address Tmp = CreateMemTemp(E->getType());
>> Builder.CreateCall(CGM.getIntrinsic(Intrinsic::x86_sse_stmxcsr),
>> - Builder.CreateBitCast(Tmp, Int8PtrTy));
>> + Builder.CreateBitCast(Tmp.getPointer(), Int8PtrTy));
>> return Builder.CreateLoad(Tmp, "stmxcsr");
>> }
>> case X86::BI__builtin_ia32_storehps:
>> @@ -6095,7 +6021,7 @@ Value *CodeGenFunction::EmitX86BuiltinEx
>>
>> // cast pointer to i64 & store
>> Ops[0] = Builder.CreateBitCast(Ops[0], PtrTy);
>> - return Builder.CreateStore(Ops[1], Ops[0]);
>> + return Builder.CreateDefaultAlignedStore(Ops[1], Ops[0]);
>> }
>> case X86::BI__builtin_ia32_palignr128:
>> case X86::BI__builtin_ia32_palignr256: {
>> @@ -6204,7 +6130,7 @@ Value *CodeGenFunction::EmitX86BuiltinEx
>> Value *BC = Builder.CreateBitCast(Ops[0],
>> llvm::PointerType::getUnqual(Ops[1]->getType()),
>> "cast");
>> - StoreInst *SI = Builder.CreateStore(Ops[1], BC);
>> + StoreInst *SI = Builder.CreateDefaultAlignedStore(Ops[1], BC);
>> SI->setMetadata(CGM.getModule().getMDKindID("nontemporal"), Node);
>>
>> // If the operand is an integer, we can't assume alignment. Otherwise,
>> @@ -6256,7 +6182,8 @@ Value *CodeGenFunction::EmitX86BuiltinEx
>> }
>>
>> Value *Call = Builder.CreateCall(CGM.getIntrinsic(ID));
>> - Builder.CreateStore(Builder.CreateExtractValue(Call, 0), Ops[0]);
>> + Builder.CreateDefaultAlignedStore(Builder.CreateExtractValue(Call, 0),
>> + Ops[0]);
>> return Builder.CreateExtractValue(Call, 1);
>> }
>> // SSE comparison intrisics
>> @@ -6659,8 +6586,7 @@ Value *CodeGenFunction::EmitAMDGPUBuilti
>> // Translate from the intrinsics's struct return to the builtin's out
>> // argument.
>>
>> - std::pair<llvm::Value *, unsigned> FlagOutPtr
>> - = EmitPointerWithAlignment(E->getArg(3));
>> + Address FlagOutPtr = EmitPointerWithAlignment(E->getArg(3));
>>
>> llvm::Value *X = EmitScalarExpr(E->getArg(0));
>> llvm::Value *Y = EmitScalarExpr(E->getArg(1));
>> @@ -6675,11 +6601,10 @@ Value *CodeGenFunction::EmitAMDGPUBuilti
>> llvm::Value *Flag = Builder.CreateExtractValue(Tmp, 1);
>>
>> llvm::Type *RealFlagType
>> - = FlagOutPtr.first->getType()->getPointerElementType();
>> + = FlagOutPtr.getPointer()->getType()->getPointerElementType();
>>
>> llvm::Value *FlagExt = Builder.CreateZExt(Flag, RealFlagType);
>> - llvm::StoreInst *FlagStore = Builder.CreateStore(FlagExt, FlagOutPtr.first);
>> - FlagStore->setAlignment(FlagOutPtr.second);
>> + Builder.CreateStore(FlagExt, FlagOutPtr);
>> return Result;
>> }
>> case AMDGPU::BI__builtin_amdgpu_div_fmas:
>> @@ -6730,7 +6655,7 @@ static Value *EmitSystemZIntrinsicWithCC
>> SmallVector<Value *, 8> Args(NumArgs);
>> for (unsigned I = 0; I < NumArgs; ++I)
>> Args[I] = CGF.EmitScalarExpr(E->getArg(I));
>> - Value *CCPtr = CGF.EmitScalarExpr(E->getArg(NumArgs));
>> + Address CCPtr = CGF.EmitPointerWithAlignment(E->getArg(NumArgs));
>> Value *F = CGF.CGM.getIntrinsic(IntrinsicID);
>> Value *Call = CGF.Builder.CreateCall(F, Args);
>> Value *CC = CGF.Builder.CreateExtractValue(Call, 1);
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCUDANV.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCUDANV.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCUDANV.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGCUDANV.cpp Tue Sep 8 03:05:57 2015
>> @@ -57,9 +57,9 @@ private:
>> unsigned Alignment = 0) {
>> llvm::Constant *Zeros[] = {llvm::ConstantInt::get(SizeTy, 0),
>> llvm::ConstantInt::get(SizeTy, 0)};
>> - auto *ConstStr = CGM.GetAddrOfConstantCString(Str, Name.c_str());
>> - return llvm::ConstantExpr::getGetElementPtr(ConstStr->getValueType(),
>> - ConstStr, Zeros);
>> + auto ConstStr = CGM.GetAddrOfConstantCString(Str, Name.c_str());
>> + return llvm::ConstantExpr::getGetElementPtr(ConstStr.getElementType(),
>> + ConstStr.getPointer(), Zeros);
>> }
>>
>> void emitDeviceStubBody(CodeGenFunction &CGF, FunctionArgList &Args);
>> @@ -121,7 +121,7 @@ void CGNVCUDARuntime::emitDeviceStubBody
>> std::vector<llvm::Type *> ArgTypes;
>> for (FunctionArgList::const_iterator I = Args.begin(), E = Args.end();
>> I != E; ++I) {
>> - llvm::Value *V = CGF.GetAddrOfLocalVar(*I);
>> + llvm::Value *V = CGF.GetAddrOfLocalVar(*I).getPointer();
>> ArgValues.push_back(V);
>> assert(isa<llvm::PointerType>(V->getType()) && "Arg type not PointerType");
>> ArgTypes.push_back(cast<llvm::PointerType>(V->getType())->getElementType());
>> @@ -173,7 +173,7 @@ llvm::Function *CGNVCUDARuntime::makeReg
>> llvm::GlobalValue::InternalLinkage, "__cuda_register_kernels", &TheModule);
>> llvm::BasicBlock *EntryBB =
>> llvm::BasicBlock::Create(Context, "entry", RegisterKernelsFunc);
>> - CGBuilderTy Builder(Context);
>> + CGBuilderTy Builder(CGM, Context);
>> Builder.SetInsertPoint(EntryBB);
>>
>> // void __cudaRegisterFunction(void **, const char *, char *, const char *,
>> @@ -230,7 +230,7 @@ llvm::Function *CGNVCUDARuntime::makeMod
>> llvm::GlobalValue::InternalLinkage, "__cuda_module_ctor", &TheModule);
>> llvm::BasicBlock *CtorEntryBB =
>> llvm::BasicBlock::Create(Context, "entry", ModuleCtorFunc);
>> - CGBuilderTy CtorBuilder(Context);
>> + CGBuilderTy CtorBuilder(CGM, Context);
>>
>> CtorBuilder.SetInsertPoint(CtorEntryBB);
>>
>> @@ -267,7 +267,8 @@ llvm::Function *CGNVCUDARuntime::makeMod
>> llvm::GlobalVariable *GpuBinaryHandle = new llvm::GlobalVariable(
>> TheModule, VoidPtrPtrTy, false, llvm::GlobalValue::InternalLinkage,
>> llvm::ConstantPointerNull::get(VoidPtrPtrTy), "__cuda_gpubin_handle");
>> - CtorBuilder.CreateStore(RegisterFatbinCall, GpuBinaryHandle, false);
>> + CtorBuilder.CreateAlignedStore(RegisterFatbinCall, GpuBinaryHandle,
>> + CGM.getPointerAlign());
>>
>> // Call __cuda_register_kernels(GpuBinaryHandle);
>> CtorBuilder.CreateCall(RegisterKernelsFunc, RegisterFatbinCall);
>> @@ -300,12 +301,13 @@ llvm::Function *CGNVCUDARuntime::makeMod
>> llvm::GlobalValue::InternalLinkage, "__cuda_module_dtor", &TheModule);
>> llvm::BasicBlock *DtorEntryBB =
>> llvm::BasicBlock::Create(Context, "entry", ModuleDtorFunc);
>> - CGBuilderTy DtorBuilder(Context);
>> + CGBuilderTy DtorBuilder(CGM, Context);
>> DtorBuilder.SetInsertPoint(DtorEntryBB);
>>
>> for (llvm::GlobalVariable *GpuBinaryHandle : GpuBinaryHandles) {
>> - DtorBuilder.CreateCall(UnregisterFatbinFunc,
>> - DtorBuilder.CreateLoad(GpuBinaryHandle, false));
>> + auto HandleValue =
>> + DtorBuilder.CreateAlignedLoad(GpuBinaryHandle, CGM.getPointerAlign());
>> + DtorBuilder.CreateCall(UnregisterFatbinFunc, HandleValue);
>> }
>>
>> DtorBuilder.CreateRetVoid();
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCXX.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCXX.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCXX.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGCXX.cpp Tue Sep 8 03:05:57 2015
>> @@ -28,6 +28,7 @@
>> using namespace clang;
>> using namespace CodeGen;
>>
>> +
>> /// Try to emit a base destructor as an alias to its primary
>> /// base-class destructor.
>> bool CodeGenModule::TryEmitBaseDestructorAsAlias(const CXXDestructorDecl *D) {
>> @@ -273,7 +274,7 @@ static llvm::Value *BuildAppleKextVirtua
>> VTableIndex += AddressPoint;
>> llvm::Value *VFuncPtr =
>> CGF.Builder.CreateConstInBoundsGEP1_64(VTable, VTableIndex, "vfnkxt");
>> - return CGF.Builder.CreateLoad(VFuncPtr);
>> + return CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.PointerAlignInBytes);
>> }
>>
>> /// BuildAppleKextVirtualCall - This routine is to support gcc's kext ABI making
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCXXABI.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCXXABI.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCXXABI.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGCXXABI.cpp Tue Sep 8 03:05:57 2015
>> @@ -73,10 +73,12 @@ CGCXXABI::ConvertMemberPointerType(const
>> }
>>
>> llvm::Value *CGCXXABI::EmitLoadOfMemberFunctionPointer(
>> - CodeGenFunction &CGF, const Expr *E, llvm::Value *&This,
>> + CodeGenFunction &CGF, const Expr *E, Address This,
>> + llvm::Value *&ThisPtrForCall,
>> llvm::Value *MemPtr, const MemberPointerType *MPT) {
>> ErrorUnsupportedABI(CGF, "calls through member pointers");
>>
>> + ThisPtrForCall = This.getPointer();
>> const FunctionProtoType *FPT =
>> MPT->getPointeeType()->getAs<FunctionProtoType>();
>> const CXXRecordDecl *RD =
>> @@ -88,10 +90,11 @@ llvm::Value *CGCXXABI::EmitLoadOfMemberF
>>
>> llvm::Value *
>> CGCXXABI::EmitMemberDataPointerAddress(CodeGenFunction &CGF, const Expr *E,
>> - llvm::Value *Base, llvm::Value *MemPtr,
>> + Address Base, llvm::Value *MemPtr,
>> const MemberPointerType *MPT) {
>> ErrorUnsupportedABI(CGF, "loads of member pointers");
>> - llvm::Type *Ty = CGF.ConvertType(MPT->getPointeeType())->getPointerTo();
>> + llvm::Type *Ty = CGF.ConvertType(MPT->getPointeeType())
>> + ->getPointerTo(Base.getAddressSpace());
>> return llvm::Constant::getNullValue(Ty);
>> }
>>
>> @@ -159,13 +162,24 @@ void CGCXXABI::buildThisParam(CodeGenFun
>> &CGM.getContext().Idents.get("this"),
>> MD->getThisType(CGM.getContext()));
>> params.push_back(ThisDecl);
>> - getThisDecl(CGF) = ThisDecl;
>> + CGF.CXXABIThisDecl = ThisDecl;
>> +
>> + // Compute the presumed alignment of 'this', which basically comes
>> + // down to whether we know it's a complete object or not.
>> + auto &Layout = CGF.getContext().getASTRecordLayout(MD->getParent());
>> + if (MD->getParent()->getNumVBases() == 0 || // avoid vcall in common case
>> + MD->getParent()->hasAttr<FinalAttr>() ||
>> + !isThisCompleteObject(CGF.CurGD)) {
>> + CGF.CXXABIThisAlignment = Layout.getAlignment();
>> + } else {
>> + CGF.CXXABIThisAlignment = Layout.getNonVirtualAlignment();
>> + }
>> }
>>
>> void CGCXXABI::EmitThisParam(CodeGenFunction &CGF) {
>> /// Initialize the 'this' slot.
>> assert(getThisDecl(CGF) && "no 'this' variable for function");
>> - getThisValue(CGF)
>> + CGF.CXXABIThisValue
>> = CGF.Builder.CreateLoad(CGF.GetAddrOfLocalVar(getThisDecl(CGF)),
>> "this");
>> }
>> @@ -186,14 +200,14 @@ CharUnits CGCXXABI::getArrayCookieSizeIm
>> return CharUnits::Zero();
>> }
>>
>> -llvm::Value *CGCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *NewPtr,
>> - llvm::Value *NumElements,
>> - const CXXNewExpr *expr,
>> - QualType ElementType) {
>> +Address CGCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address NewPtr,
>> + llvm::Value *NumElements,
>> + const CXXNewExpr *expr,
>> + QualType ElementType) {
>> // Should never be called.
>> ErrorUnsupportedABI(CGF, "array cookie initialization");
>> - return nullptr;
>> + return Address::invalid();
>> }
>>
>> bool CGCXXABI::requiresArrayCookie(const CXXDeleteExpr *expr,
>> @@ -215,31 +229,30 @@ bool CGCXXABI::requiresArrayCookie(const
>> return expr->getAllocatedType().isDestructedType();
>> }
>>
>> -void CGCXXABI::ReadArrayCookie(CodeGenFunction &CGF, llvm::Value *ptr,
>> +void CGCXXABI::ReadArrayCookie(CodeGenFunction &CGF, Address ptr,
>> const CXXDeleteExpr *expr, QualType eltTy,
>> llvm::Value *&numElements,
>> llvm::Value *&allocPtr, CharUnits &cookieSize) {
>> // Derive a char* in the same address space as the pointer.
>> - unsigned AS = ptr->getType()->getPointerAddressSpace();
>> - llvm::Type *charPtrTy = CGF.Int8Ty->getPointerTo(AS);
>> - ptr = CGF.Builder.CreateBitCast(ptr, charPtrTy);
>> + ptr = CGF.Builder.CreateElementBitCast(ptr, CGF.Int8Ty);
>>
>> // If we don't need an array cookie, bail out early.
>> if (!requiresArrayCookie(expr, eltTy)) {
>> - allocPtr = ptr;
>> + allocPtr = ptr.getPointer();
>> numElements = nullptr;
>> cookieSize = CharUnits::Zero();
>> return;
>> }
>>
>> cookieSize = getArrayCookieSizeImpl(eltTy);
>> - allocPtr = CGF.Builder.CreateConstInBoundsGEP1_64(ptr,
>> - -cookieSize.getQuantity());
>> - numElements = readArrayCookieImpl(CGF, allocPtr, cookieSize);
>> + Address allocAddr =
>> + CGF.Builder.CreateConstInBoundsByteGEP(ptr, -cookieSize);
>> + allocPtr = allocAddr.getPointer();
>> + numElements = readArrayCookieImpl(CGF, allocAddr, cookieSize);
>> }
>>
>> llvm::Value *CGCXXABI::readArrayCookieImpl(CodeGenFunction &CGF,
>> - llvm::Value *ptr,
>> + Address ptr,
>> CharUnits cookieSize) {
>> ErrorUnsupportedABI(CGF, "reading a new[] cookie");
>> return llvm::ConstantInt::get(CGF.SizeTy, 0);
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCXXABI.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCXXABI.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCXXABI.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGCXXABI.h Tue Sep 8 03:05:57 2015
>> @@ -48,12 +48,15 @@ protected:
>> : CGM(CGM), MangleCtx(CGM.getContext().createMangleContext()) {}
>>
>> protected:
>> - ImplicitParamDecl *&getThisDecl(CodeGenFunction &CGF) {
>> + ImplicitParamDecl *getThisDecl(CodeGenFunction &CGF) {
>> return CGF.CXXABIThisDecl;
>> }
>> - llvm::Value *&getThisValue(CodeGenFunction &CGF) {
>> + llvm::Value *getThisValue(CodeGenFunction &CGF) {
>> return CGF.CXXABIThisValue;
>> }
>> + Address getThisAddress(CodeGenFunction &CGF) {
>> + return Address(CGF.CXXABIThisValue, CGF.CXXABIThisAlignment);
>> + }
>>
>> /// Issue a diagnostic about unsupported features in the ABI.
>> void ErrorUnsupportedABI(CodeGenFunction &CGF, StringRef S);
>> @@ -77,6 +80,12 @@ protected:
>> virtual bool requiresArrayCookie(const CXXDeleteExpr *E, QualType eltType);
>> virtual bool requiresArrayCookie(const CXXNewExpr *E);
>>
>> + /// Determine whether there's something special about the rules of
>> + /// the ABI tell us that 'this' is a complete object within the
>> + /// given function. Obvious common logic like being defined on a
>> + /// final class will have been taken care of by the caller.
>> + virtual bool isThisCompleteObject(GlobalDecl GD) const = 0;
>> +
>> public:
>>
>> virtual ~CGCXXABI();
>> @@ -135,13 +144,14 @@ public:
>> /// pointer. Apply the this-adjustment and set 'This' to the
>> /// adjusted value.
>> virtual llvm::Value *EmitLoadOfMemberFunctionPointer(
>> - CodeGenFunction &CGF, const Expr *E, llvm::Value *&This,
>> - llvm::Value *MemPtr, const MemberPointerType *MPT);
>> + CodeGenFunction &CGF, const Expr *E, Address This,
>> + llvm::Value *&ThisPtrForCall, llvm::Value *MemPtr,
>> + const MemberPointerType *MPT);
>>
>> /// Calculate an l-value from an object and a data member pointer.
>> virtual llvm::Value *
>> EmitMemberDataPointerAddress(CodeGenFunction &CGF, const Expr *E,
>> - llvm::Value *Base, llvm::Value *MemPtr,
>> + Address Base, llvm::Value *MemPtr,
>> const MemberPointerType *MPT);
>>
>> /// Perform a derived-to-base, base-to-derived, or bitcast member
>> @@ -212,7 +222,7 @@ protected:
>> public:
>> virtual void emitVirtualObjectDelete(CodeGenFunction &CGF,
>> const CXXDeleteExpr *DE,
>> - llvm::Value *Ptr, QualType ElementType,
>> + Address Ptr, QualType ElementType,
>> const CXXDestructorDecl *Dtor) = 0;
>> virtual void emitRethrow(CodeGenFunction &CGF, bool isNoReturn) = 0;
>> virtual void emitThrow(CodeGenFunction &CGF, const CXXThrowExpr *E) = 0;
>> @@ -235,26 +245,26 @@ public:
>> QualType SrcRecordTy) = 0;
>> virtual void EmitBadTypeidCall(CodeGenFunction &CGF) = 0;
>> virtual llvm::Value *EmitTypeid(CodeGenFunction &CGF, QualType SrcRecordTy,
>> - llvm::Value *ThisPtr,
>> + Address ThisPtr,
>> llvm::Type *StdTypeInfoPtrTy) = 0;
>>
>> virtual bool shouldDynamicCastCallBeNullChecked(bool SrcIsPtr,
>> QualType SrcRecordTy) = 0;
>>
>> virtual llvm::Value *
>> - EmitDynamicCastCall(CodeGenFunction &CGF, llvm::Value *Value,
>> + EmitDynamicCastCall(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy, QualType DestTy,
>> QualType DestRecordTy, llvm::BasicBlock *CastEnd) = 0;
>>
>> virtual llvm::Value *EmitDynamicCastToVoid(CodeGenFunction &CGF,
>> - llvm::Value *Value,
>> + Address Value,
>> QualType SrcRecordTy,
>> QualType DestTy) = 0;
>>
>> virtual bool EmitBadCastCall(CodeGenFunction &CGF) = 0;
>>
>> virtual llvm::Value *GetVirtualBaseClassOffset(CodeGenFunction &CGF,
>> - llvm::Value *This,
>> + Address This,
>> const CXXRecordDecl *ClassDecl,
>> const CXXRecordDecl *BaseClassDecl) = 0;
>>
>> @@ -297,10 +307,9 @@ public:
>> /// Perform ABI-specific "this" argument adjustment required prior to
>> /// a call of a virtual function.
>> /// The "VirtualCall" argument is true iff the call itself is virtual.
>> - virtual llvm::Value *
>> + virtual Address
>> adjustThisArgumentForVirtualFunctionCall(CodeGenFunction &CGF, GlobalDecl GD,
>> - llvm::Value *This,
>> - bool VirtualCall) {
>> + Address This, bool VirtualCall) {
>> return This;
>> }
>>
>> @@ -340,7 +349,7 @@ public:
>> virtual void EmitDestructorCall(CodeGenFunction &CGF,
>> const CXXDestructorDecl *DD, CXXDtorType Type,
>> bool ForVirtualBase, bool Delegating,
>> - llvm::Value *This) = 0;
>> + Address This) = 0;
>>
>> /// Emits the VTable definitions required for the given record type.
>> virtual void emitVTableDefinitions(CodeGenVTables &CGVT,
>> @@ -368,14 +377,14 @@ public:
>> /// Build a virtual function pointer in the ABI-specific way.
>> virtual llvm::Value *getVirtualFunctionPointer(CodeGenFunction &CGF,
>> GlobalDecl GD,
>> - llvm::Value *This,
>> + Address This,
>> llvm::Type *Ty,
>> SourceLocation Loc) = 0;
>>
>> /// Emit the ABI-specific virtual destructor call.
>> virtual llvm::Value *
>> EmitVirtualDestructorCall(CodeGenFunction &CGF, const CXXDestructorDecl *Dtor,
>> - CXXDtorType DtorType, llvm::Value *This,
>> + CXXDtorType DtorType, Address This,
>> const CXXMemberCallExpr *CE) = 0;
>>
>> virtual void adjustCallArgsForDestructorThunk(CodeGenFunction &CGF,
>> @@ -391,11 +400,11 @@ public:
>> GlobalDecl GD, bool ReturnAdjustment) = 0;
>>
>> virtual llvm::Value *performThisAdjustment(CodeGenFunction &CGF,
>> - llvm::Value *This,
>> + Address This,
>> const ThisAdjustment &TA) = 0;
>>
>> virtual llvm::Value *performReturnAdjustment(CodeGenFunction &CGF,
>> - llvm::Value *Ret,
>> + Address Ret,
>> const ReturnAdjustment &RA) = 0;
>>
>> virtual void EmitReturnFromThunk(CodeGenFunction &CGF,
>> @@ -432,11 +441,11 @@ public:
>> /// always a size_t
>> /// \param ElementType - the base element allocated type,
>> /// i.e. the allocated type after stripping all array types
>> - virtual llvm::Value *InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *NewPtr,
>> - llvm::Value *NumElements,
>> - const CXXNewExpr *expr,
>> - QualType ElementType);
>> + virtual Address InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address NewPtr,
>> + llvm::Value *NumElements,
>> + const CXXNewExpr *expr,
>> + QualType ElementType);
>>
>> /// Reads the array cookie associated with the given pointer,
>> /// if it has one.
>> @@ -451,7 +460,7 @@ public:
>> /// function
>> /// \param CookieSize - an out parameter which will be initialized
>> /// with the size of the cookie, or zero if there is no cookie
>> - virtual void ReadArrayCookie(CodeGenFunction &CGF, llvm::Value *Ptr,
>> + virtual void ReadArrayCookie(CodeGenFunction &CGF, Address Ptr,
>> const CXXDeleteExpr *expr,
>> QualType ElementType, llvm::Value *&NumElements,
>> llvm::Value *&AllocPtr, CharUnits &CookieSize);
>> @@ -474,8 +483,7 @@ protected:
>> /// Other parameters are as above.
>> ///
>> /// \return a size_t
>> - virtual llvm::Value *readArrayCookieImpl(CodeGenFunction &IGF,
>> - llvm::Value *ptr,
>> + virtual llvm::Value *readArrayCookieImpl(CodeGenFunction &IGF, Address ptr,
>> CharUnits cookieSize);
>>
>> public:
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCall.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCall.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCall.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGCall.cpp Tue Sep 8 03:05:57 2015
>> @@ -553,6 +553,7 @@ CGFunctionInfo *CGFunctionInfo::create(u
>> FI->HasRegParm = info.getHasRegParm();
>> FI->RegParm = info.getRegParm();
>> FI->ArgStruct = nullptr;
>> + FI->ArgStructAlign = 0;
>> FI->NumArgs = argTypes.size();
>> FI->getArgsBuffer()[0].type = resultType;
>> for (unsigned i = 0, e = argTypes.size(); i != e; ++i)
>> @@ -726,6 +727,21 @@ CodeGenTypes::getExpandedTypes(QualType
>> }
>> }
>>
>> +static void forConstantArrayExpansion(CodeGenFunction &CGF,
>> + ConstantArrayExpansion *CAE,
>> + Address BaseAddr,
>> + llvm::function_ref<void(Address)> Fn) {
>> + CharUnits EltSize = CGF.getContext().getTypeSizeInChars(CAE->EltTy);
>> + CharUnits EltAlign =
>> + BaseAddr.getAlignment().alignmentOfArrayElement(EltSize);
>> +
>> + for (int i = 0, n = CAE->NumElts; i < n; i++) {
>> + llvm::Value *EltAddr =
>> + CGF.Builder.CreateConstGEP2_32(nullptr, BaseAddr.getPointer(), 0, i);
>> + Fn(Address(EltAddr, EltAlign));
>> + }
>> +}
>> +
>> void CodeGenFunction::ExpandTypeFromArgs(
>> QualType Ty, LValue LV, SmallVectorImpl<llvm::Argument *>::iterator &AI) {
>> assert(LV.isSimple() &&
>> @@ -733,17 +749,16 @@ void CodeGenFunction::ExpandTypeFromArgs
>>
>> auto Exp = getTypeExpansion(Ty, getContext());
>> if (auto CAExp = dyn_cast<ConstantArrayExpansion>(Exp.get())) {
>> - for (int i = 0, n = CAExp->NumElts; i < n; i++) {
>> - llvm::Value *EltAddr =
>> - Builder.CreateConstGEP2_32(nullptr, LV.getAddress(), 0, i);
>> + forConstantArrayExpansion(*this, CAExp, LV.getAddress(),
>> + [&](Address EltAddr) {
>> LValue LV = MakeAddrLValue(EltAddr, CAExp->EltTy);
>> ExpandTypeFromArgs(CAExp->EltTy, LV, AI);
>> - }
>> + });
>> } else if (auto RExp = dyn_cast<RecordExpansion>(Exp.get())) {
>> - llvm::Value *This = LV.getAddress();
>> + Address This = LV.getAddress();
>> for (const CXXBaseSpecifier *BS : RExp->Bases) {
>> // Perform a single step derived-to-base conversion.
>> - llvm::Value *Base =
>> + Address Base =
>> GetAddressOfBaseClass(This, Ty->getAsCXXRecordDecl(), &BS, &BS + 1,
>> /*NullCheckValue=*/false, SourceLocation());
>> LValue SubLV = MakeAddrLValue(Base, BS->getType());
>> @@ -756,15 +771,10 @@ void CodeGenFunction::ExpandTypeFromArgs
>> LValue SubLV = EmitLValueForField(LV, FD);
>> ExpandTypeFromArgs(FD->getType(), SubLV, AI);
>> }
>> - } else if (auto CExp = dyn_cast<ComplexExpansion>(Exp.get())) {
>> - llvm::Value *RealAddr =
>> - Builder.CreateStructGEP(nullptr, LV.getAddress(), 0, "real");
>> - EmitStoreThroughLValue(RValue::get(*AI++),
>> - MakeAddrLValue(RealAddr, CExp->EltTy));
>> - llvm::Value *ImagAddr =
>> - Builder.CreateStructGEP(nullptr, LV.getAddress(), 1, "imag");
>> - EmitStoreThroughLValue(RValue::get(*AI++),
>> - MakeAddrLValue(ImagAddr, CExp->EltTy));
>> + } else if (isa<ComplexExpansion>(Exp.get())) {
>> + auto realValue = *AI++;
>> + auto imagValue = *AI++;
>> + EmitStoreOfComplex(ComplexPairTy(realValue, imagValue), LV, /*init*/ true);
>> } else {
>> assert(isa<NoExpansion>(Exp.get()));
>> EmitStoreThroughLValue(RValue::get(*AI++), LV);
>> @@ -776,18 +786,17 @@ void CodeGenFunction::ExpandTypeToArgs(
>> SmallVectorImpl<llvm::Value *> &IRCallArgs, unsigned &IRCallArgPos) {
>> auto Exp = getTypeExpansion(Ty, getContext());
>> if (auto CAExp = dyn_cast<ConstantArrayExpansion>(Exp.get())) {
>> - llvm::Value *Addr = RV.getAggregateAddr();
>> - for (int i = 0, n = CAExp->NumElts; i < n; i++) {
>> - llvm::Value *EltAddr = Builder.CreateConstGEP2_32(nullptr, Addr, 0, i);
>> + forConstantArrayExpansion(*this, CAExp, RV.getAggregateAddress(),
>> + [&](Address EltAddr) {
>> RValue EltRV =
>> convertTempToRValue(EltAddr, CAExp->EltTy, SourceLocation());
>> ExpandTypeToArgs(CAExp->EltTy, EltRV, IRFuncTy, IRCallArgs, IRCallArgPos);
>> - }
>> + });
>> } else if (auto RExp = dyn_cast<RecordExpansion>(Exp.get())) {
>> - llvm::Value *This = RV.getAggregateAddr();
>> + Address This = RV.getAggregateAddress();
>> for (const CXXBaseSpecifier *BS : RExp->Bases) {
>> // Perform a single step derived-to-base conversion.
>> - llvm::Value *Base =
>> + Address Base =
>> GetAddressOfBaseClass(This, Ty->getAsCXXRecordDecl(), &BS, &BS + 1,
>> /*NullCheckValue=*/false, SourceLocation());
>> RValue BaseRV = RValue::getAggregate(Base);
>> @@ -822,12 +831,22 @@ void CodeGenFunction::ExpandTypeToArgs(
>> }
>> }
>>
>> +/// Create a temporary allocation for the purposes of coercion.
>> +static Address CreateTempAllocaForCoercion(CodeGenFunction &CGF, llvm::Type *Ty,
>> + CharUnits MinAlign) {
>> + // Don't use an alignment that's worse than what LLVM would prefer.
>> + auto PrefAlign = CGF.CGM.getDataLayout().getPrefTypeAlignment(Ty);
>> + CharUnits Align = std::max(MinAlign, CharUnits::fromQuantity(PrefAlign));
>> +
>> + return CGF.CreateTempAlloca(Ty, Align);
>> +}
>> +
>> /// EnterStructPointerForCoercedAccess - Given a struct pointer that we are
>> /// accessing some number of bytes out of it, try to gep into the struct to get
>> /// at its inner goodness. Dive as deep as possible without entering an element
>> /// with an in-memory size smaller than DstSize.
>> -static llvm::Value *
>> -EnterStructPointerForCoercedAccess(llvm::Value *SrcPtr,
>> +static Address
>> +EnterStructPointerForCoercedAccess(Address SrcPtr,
>> llvm::StructType *SrcSTy,
>> uint64_t DstSize, CodeGenFunction &CGF) {
>> // We can't dive into a zero-element struct.
>> @@ -846,11 +865,10 @@ EnterStructPointerForCoercedAccess(llvm:
>> return SrcPtr;
>>
>> // GEP into the first element.
>> - SrcPtr = CGF.Builder.CreateConstGEP2_32(SrcSTy, SrcPtr, 0, 0, "coerce.dive");
>> + SrcPtr = CGF.Builder.CreateStructGEP(SrcPtr, 0, CharUnits(), "coerce.dive");
>>
>> // If the first element is a struct, recurse.
>> - llvm::Type *SrcTy =
>> - cast<llvm::PointerType>(SrcPtr->getType())->getElementType();
>> + llvm::Type *SrcTy = SrcPtr.getElementType();
>> if (llvm::StructType *SrcSTy = dyn_cast<llvm::StructType>(SrcTy))
>> return EnterStructPointerForCoercedAccess(SrcPtr, SrcSTy, DstSize, CGF);
>>
>> @@ -918,21 +936,19 @@ static llvm::Value *CoerceIntOrPtrToIntO
>> /// This safely handles the case when the src type is smaller than the
>> /// destination type; in this situation the values of bits which not
>> /// present in the src are undefined.
>> -static llvm::Value *CreateCoercedLoad(llvm::Value *SrcPtr,
>> - llvm::Type *Ty, CharUnits SrcAlign,
>> +static llvm::Value *CreateCoercedLoad(Address Src, llvm::Type *Ty,
>> CodeGenFunction &CGF) {
>> - llvm::Type *SrcTy =
>> - cast<llvm::PointerType>(SrcPtr->getType())->getElementType();
>> + llvm::Type *SrcTy = Src.getElementType();
>>
>> // If SrcTy and Ty are the same, just do a load.
>> if (SrcTy == Ty)
>> - return CGF.Builder.CreateAlignedLoad(SrcPtr, SrcAlign.getQuantity());
>> + return CGF.Builder.CreateLoad(Src);
>>
>> uint64_t DstSize = CGF.CGM.getDataLayout().getTypeAllocSize(Ty);
>>
>> if (llvm::StructType *SrcSTy = dyn_cast<llvm::StructType>(SrcTy)) {
>> - SrcPtr = EnterStructPointerForCoercedAccess(SrcPtr, SrcSTy, DstSize, CGF);
>> - SrcTy = cast<llvm::PointerType>(SrcPtr->getType())->getElementType();
>> + Src = EnterStructPointerForCoercedAccess(Src, SrcSTy, DstSize, CGF);
>> + SrcTy = Src.getType()->getElementType();
>> }
>>
>> uint64_t SrcSize = CGF.CGM.getDataLayout().getTypeAllocSize(SrcTy);
>> @@ -941,8 +957,7 @@ static llvm::Value *CreateCoercedLoad(ll
>> // extension or truncation to the desired type.
>> if ((isa<llvm::IntegerType>(Ty) || isa<llvm::PointerType>(Ty)) &&
>> (isa<llvm::IntegerType>(SrcTy) || isa<llvm::PointerType>(SrcTy))) {
>> - llvm::LoadInst *Load =
>> - CGF.Builder.CreateAlignedLoad(SrcPtr, SrcAlign.getQuantity());
>> + llvm::Value *Load = CGF.Builder.CreateLoad(Src);
>> return CoerceIntOrPtrToIntOrPtr(Load, Ty, CGF);
>> }
>>
>> @@ -954,22 +969,18 @@ static llvm::Value *CreateCoercedLoad(ll
>> //
>> // FIXME: Assert that we aren't truncating non-padding bits when have access
>> // to that information.
>> - llvm::Value *Casted =
>> - CGF.Builder.CreateBitCast(SrcPtr, llvm::PointerType::getUnqual(Ty));
>> - return CGF.Builder.CreateAlignedLoad(Casted, SrcAlign.getQuantity());
>> + Src = CGF.Builder.CreateBitCast(Src, llvm::PointerType::getUnqual(Ty));
>> + return CGF.Builder.CreateLoad(Src);
>> }
>>
>> - // Otherwise do coercion through memory. This is stupid, but
>> - // simple.
>> - llvm::AllocaInst *Tmp = CGF.CreateTempAlloca(Ty);
>> - Tmp->setAlignment(SrcAlign.getQuantity());
>> - llvm::Type *I8PtrTy = CGF.Builder.getInt8PtrTy();
>> - llvm::Value *Casted = CGF.Builder.CreateBitCast(Tmp, I8PtrTy);
>> - llvm::Value *SrcCasted = CGF.Builder.CreateBitCast(SrcPtr, I8PtrTy);
>> + // Otherwise do coercion through memory. This is stupid, but simple.
>> + Address Tmp = CreateTempAllocaForCoercion(CGF, Ty, Src.getAlignment());
>> + Address Casted = CGF.Builder.CreateBitCast(Tmp, CGF.Int8PtrTy);
>> + Address SrcCasted = CGF.Builder.CreateBitCast(Src, CGF.Int8PtrTy);
>> CGF.Builder.CreateMemCpy(Casted, SrcCasted,
>> llvm::ConstantInt::get(CGF.IntPtrTy, SrcSize),
>> - SrcAlign.getQuantity(), false);
>> - return CGF.Builder.CreateAlignedLoad(Tmp, SrcAlign.getQuantity());
>> + false);
>> + return CGF.Builder.CreateLoad(Tmp);
>> }
>>
>> // Function to store a first-class aggregate into memory. We prefer to
>> @@ -977,8 +988,7 @@ static llvm::Value *CreateCoercedLoad(ll
>> // fast-isel.
>> // FIXME: Do we need to recurse here?
>> static void BuildAggStore(CodeGenFunction &CGF, llvm::Value *Val,
>> - llvm::Value *DestPtr, bool DestIsVolatile,
>> - CharUnits DestAlign) {
>> + Address Dest, bool DestIsVolatile) {
>> // Prefer scalar stores to first-class aggregate stores.
>> if (llvm::StructType *STy =
>> dyn_cast<llvm::StructType>(Val->getType())) {
>> @@ -986,17 +996,13 @@ static void BuildAggStore(CodeGenFunctio
>> CGF.CGM.getDataLayout().getStructLayout(STy);
>>
>> for (unsigned i = 0, e = STy->getNumElements(); i != e; ++i) {
>> - llvm::Value *EltPtr = CGF.Builder.CreateConstGEP2_32(STy, DestPtr, 0, i);
>> + auto EltOffset = CharUnits::fromQuantity(Layout->getElementOffset(i));
>> + Address EltPtr = CGF.Builder.CreateStructGEP(Dest, i, EltOffset);
>> llvm::Value *Elt = CGF.Builder.CreateExtractValue(Val, i);
>> - uint64_t EltOffset = Layout->getElementOffset(i);
>> - CharUnits EltAlign =
>> - DestAlign.alignmentAtOffset(CharUnits::fromQuantity(EltOffset));
>> - CGF.Builder.CreateAlignedStore(Elt, EltPtr, EltAlign.getQuantity(),
>> - DestIsVolatile);
>> + CGF.Builder.CreateStore(Elt, EltPtr, DestIsVolatile);
>> }
>> } else {
>> - CGF.Builder.CreateAlignedStore(Val, DestPtr, DestAlign.getQuantity(),
>> - DestIsVolatile);
>> + CGF.Builder.CreateStore(Val, Dest, DestIsVolatile);
>> }
>> }
>>
>> @@ -1007,24 +1013,21 @@ static void BuildAggStore(CodeGenFunctio
>> /// This safely handles the case when the src type is larger than the
>> /// destination type; the upper bits of the src will be lost.
>> static void CreateCoercedStore(llvm::Value *Src,
>> - llvm::Value *DstPtr,
>> + Address Dst,
>> bool DstIsVolatile,
>> - CharUnits DstAlign,
>> CodeGenFunction &CGF) {
>> llvm::Type *SrcTy = Src->getType();
>> - llvm::Type *DstTy =
>> - cast<llvm::PointerType>(DstPtr->getType())->getElementType();
>> + llvm::Type *DstTy = Dst.getType()->getElementType();
>> if (SrcTy == DstTy) {
>> - CGF.Builder.CreateAlignedStore(Src, DstPtr, DstAlign.getQuantity(),
>> - DstIsVolatile);
>> + CGF.Builder.CreateStore(Src, Dst, DstIsVolatile);
>> return;
>> }
>>
>> uint64_t SrcSize = CGF.CGM.getDataLayout().getTypeAllocSize(SrcTy);
>>
>> if (llvm::StructType *DstSTy = dyn_cast<llvm::StructType>(DstTy)) {
>> - DstPtr = EnterStructPointerForCoercedAccess(DstPtr, DstSTy, SrcSize, CGF);
>> - DstTy = cast<llvm::PointerType>(DstPtr->getType())->getElementType();
>> + Dst = EnterStructPointerForCoercedAccess(Dst, DstSTy, SrcSize, CGF);
>> + DstTy = Dst.getType()->getElementType();
>> }
>>
>> // If the source and destination are integer or pointer types, just do an
>> @@ -1032,8 +1035,7 @@ static void CreateCoercedStore(llvm::Val
>> if ((isa<llvm::IntegerType>(SrcTy) || isa<llvm::PointerType>(SrcTy)) &&
>> (isa<llvm::IntegerType>(DstTy) || isa<llvm::PointerType>(DstTy))) {
>> Src = CoerceIntOrPtrToIntOrPtr(Src, DstTy, CGF);
>> - CGF.Builder.CreateAlignedStore(Src, DstPtr, DstAlign.getQuantity(),
>> - DstIsVolatile);
>> + CGF.Builder.CreateStore(Src, Dst, DstIsVolatile);
>> return;
>> }
>>
>> @@ -1041,9 +1043,8 @@ static void CreateCoercedStore(llvm::Val
>>
>> // If store is legal, just bitcast the src pointer.
>> if (SrcSize <= DstSize) {
>> - llvm::Value *Casted =
>> - CGF.Builder.CreateBitCast(DstPtr, llvm::PointerType::getUnqual(SrcTy));
>> - BuildAggStore(CGF, Src, Casted, DstIsVolatile, DstAlign);
>> + Dst = CGF.Builder.CreateBitCast(Dst, llvm::PointerType::getUnqual(SrcTy));
>> + BuildAggStore(CGF, Src, Dst, DstIsVolatile);
>> } else {
>> // Otherwise do coercion through memory. This is stupid, but
>> // simple.
>> @@ -1054,18 +1055,27 @@ static void CreateCoercedStore(llvm::Val
>> //
>> // FIXME: Assert that we aren't truncating non-padding bits when have access
>> // to that information.
>> - llvm::AllocaInst *Tmp = CGF.CreateTempAlloca(SrcTy);
>> - Tmp->setAlignment(DstAlign.getQuantity());
>> - CGF.Builder.CreateAlignedStore(Src, Tmp, DstAlign.getQuantity());
>> - llvm::Type *I8PtrTy = CGF.Builder.getInt8PtrTy();
>> - llvm::Value *Casted = CGF.Builder.CreateBitCast(Tmp, I8PtrTy);
>> - llvm::Value *DstCasted = CGF.Builder.CreateBitCast(DstPtr, I8PtrTy);
>> + Address Tmp = CreateTempAllocaForCoercion(CGF, SrcTy, Dst.getAlignment());
>> + CGF.Builder.CreateStore(Src, Tmp);
>> + Address Casted = CGF.Builder.CreateBitCast(Tmp, CGF.Int8PtrTy);
>> + Address DstCasted = CGF.Builder.CreateBitCast(Dst, CGF.Int8PtrTy);
>> CGF.Builder.CreateMemCpy(DstCasted, Casted,
>> llvm::ConstantInt::get(CGF.IntPtrTy, DstSize),
>> - DstAlign.getQuantity(), false);
>> + false);
>> }
>> }
>>
>> +static Address emitAddressAtOffset(CodeGenFunction &CGF, Address addr,
>> + const ABIArgInfo &info) {
>> + if (unsigned offset = info.getDirectOffset()) {
>> + addr = CGF.Builder.CreateElementBitCast(addr, CGF.Int8Ty);
>> + addr = CGF.Builder.CreateConstInBoundsByteGEP(addr,
>> + CharUnits::fromQuantity(offset));
>> + addr = CGF.Builder.CreateElementBitCast(addr, info.getCoerceToType());
>> + }
>> + return addr;
>> +}
>> +
>> namespace {
>>
>> /// Encapsulates information about the way function arguments from
>> @@ -1279,12 +1289,7 @@ CodeGenTypes::GetFunctionType(const CGFu
>> }
>> break;
>>
>> - case ABIArgInfo::Indirect: {
>> - assert(!retAI.getIndirectAlign() && "Align unused on indirect return.");
>> - resultType = llvm::Type::getVoidTy(getLLVMContext());
>> - break;
>> - }
>> -
>> + case ABIArgInfo::Indirect:
>> case ABIArgInfo::Ignore:
>> resultType = llvm::Type::getVoidTy(getLLVMContext());
>> break;
>> @@ -1656,7 +1661,7 @@ void CodeGenModule::ConstructAttributeLi
>> if (AI.getIndirectByVal())
>> Attrs.addAttribute(llvm::Attribute::ByVal);
>>
>> - unsigned Align = AI.getIndirectAlign();
>> + CharUnits Align = AI.getIndirectAlign();
>>
>> // In a byval argument, it is important that the required
>> // alignment of the type is honored, as LLVM might be creating a
>> @@ -1668,10 +1673,12 @@ void CodeGenModule::ConstructAttributeLi
>> // This is different from indirect *not* byval, where the object
>> // exists already, and the align attribute is purely
>> // informative.
>> - if (Align == 0 && AI.getIndirectByVal())
>> - Align = getContext().getTypeAlignInChars(ParamType).getQuantity();
>> + assert(!Align.isZero());
>>
>> - Attrs.addAlignmentAttr(Align);
>> + // For now, only add this when we have a byval argument.
>> + // TODO: be less lazy about updating test cases.
>> + if (AI.getIndirectByVal())
>> + Attrs.addAlignmentAttr(Align.getQuantity());
>>
>> // byval disables readnone and readonly.
>> FuncAttrs.removeAttribute(llvm::Attribute::ReadOnly)
>> @@ -1797,10 +1804,14 @@ void CodeGenFunction::EmitFunctionProlog
>>
>> // If we're using inalloca, all the memory arguments are GEPs off of the last
>> // parameter, which is a pointer to the complete memory area.
>> - llvm::Value *ArgStruct = nullptr;
>> + Address ArgStruct = Address::invalid();
>> + const llvm::StructLayout *ArgStructLayout = nullptr;
>> if (IRFunctionArgs.hasInallocaArg()) {
>> - ArgStruct = FnArgs[IRFunctionArgs.getInallocaArgNo()];
>> - assert(ArgStruct->getType() == FI.getArgStruct()->getPointerTo());
>> + ArgStructLayout = CGM.getDataLayout().getStructLayout(FI.getArgStruct());
>> + ArgStruct = Address(FnArgs[IRFunctionArgs.getInallocaArgNo()],
>> + FI.getArgStructAlignment());
>> +
>> + assert(ArgStruct.getType() == FI.getArgStruct()->getPointerTo());
>> }
>>
>> // Name the struct return parameter.
>> @@ -1814,9 +1825,7 @@ void CodeGenFunction::EmitFunctionProlog
>> // Track if we received the parameter as a pointer (indirect, byval, or
>> // inalloca). If already have a pointer, EmitParmDecl doesn't need to copy it
>> // into a local alloca for us.
>> - enum ValOrPointer { HaveValue = 0, HavePointer = 1 };
>> - typedef llvm::PointerIntPair<llvm::Value *, 1> ValueAndIsPtr;
>> - SmallVector<ValueAndIsPtr, 16> ArgVals;
>> + SmallVector<ParamValue, 16> ArgVals;
>> ArgVals.reserve(Args.size());
>>
>> // Create a pointer value for every parameter declaration. This usually
>> @@ -1842,49 +1851,47 @@ void CodeGenFunction::EmitFunctionProlog
>> switch (ArgI.getKind()) {
>> case ABIArgInfo::InAlloca: {
>> assert(NumIRArgs == 0);
>> - llvm::Value *V =
>> - Builder.CreateStructGEP(FI.getArgStruct(), ArgStruct,
>> - ArgI.getInAllocaFieldIndex(), Arg->getName());
>> - ArgVals.push_back(ValueAndIsPtr(V, HavePointer));
>> + auto FieldIndex = ArgI.getInAllocaFieldIndex();
>> + CharUnits FieldOffset =
>> + CharUnits::fromQuantity(ArgStructLayout->getElementOffset(FieldIndex));
>> + Address V = Builder.CreateStructGEP(ArgStruct, FieldIndex, FieldOffset,
>> + Arg->getName());
>> + ArgVals.push_back(ParamValue::forIndirect(V));
>> break;
>> }
>>
>> case ABIArgInfo::Indirect: {
>> assert(NumIRArgs == 1);
>> - llvm::Value *V = FnArgs[FirstIRArg];
>> + Address ParamAddr = Address(FnArgs[FirstIRArg], ArgI.getIndirectAlign());
>>
>> if (!hasScalarEvaluationKind(Ty)) {
>> // Aggregates and complex variables are accessed by reference. All we
>> - // need to do is realign the value, if requested
>> + // need to do is realign the value, if requested.
>> + Address V = ParamAddr;
>> if (ArgI.getIndirectRealign()) {
>> - llvm::Value *AlignedTemp = CreateMemTemp(Ty, "coerce");
>> + Address AlignedTemp = CreateMemTemp(Ty, "coerce");
>>
>> // Copy from the incoming argument pointer to the temporary with the
>> // appropriate alignment.
>> //
>> // FIXME: We should have a common utility for generating an aggregate
>> // copy.
>> - llvm::Type *I8PtrTy = Builder.getInt8PtrTy();
>> CharUnits Size = getContext().getTypeSizeInChars(Ty);
>> - llvm::Value *Dst = Builder.CreateBitCast(AlignedTemp, I8PtrTy);
>> - llvm::Value *Src = Builder.CreateBitCast(V, I8PtrTy);
>> - Builder.CreateMemCpy(Dst,
>> - Src,
>> - llvm::ConstantInt::get(IntPtrTy,
>> - Size.getQuantity()),
>> - ArgI.getIndirectAlign(),
>> - false);
>> + auto SizeVal = llvm::ConstantInt::get(IntPtrTy, Size.getQuantity());
>> + Address Dst = Builder.CreateBitCast(AlignedTemp, Int8PtrTy);
>> + Address Src = Builder.CreateBitCast(ParamAddr, Int8PtrTy);
>> + Builder.CreateMemCpy(Dst, Src, SizeVal, false);
>> V = AlignedTemp;
>> }
>> - ArgVals.push_back(ValueAndIsPtr(V, HavePointer));
>> + ArgVals.push_back(ParamValue::forIndirect(V));
>> } else {
>> // Load scalar value from indirect argument.
>> - V = EmitLoadOfScalar(V, false, ArgI.getIndirectAlign(), Ty,
>> - Arg->getLocStart());
>> + llvm::Value *V =
>> + EmitLoadOfScalar(ParamAddr, false, Ty, Arg->getLocStart());
>>
>> if (isPromoted)
>> V = emitArgumentDemotion(*this, Arg, V);
>> - ArgVals.push_back(ValueAndIsPtr(V, HaveValue));
>> + ArgVals.push_back(ParamValue::forDirect(V));
>> }
>> break;
>> }
>> @@ -1989,87 +1996,66 @@ void CodeGenFunction::EmitFunctionProlog
>> if (V->getType() != LTy)
>> V = Builder.CreateBitCast(V, LTy);
>>
>> - ArgVals.push_back(ValueAndIsPtr(V, HaveValue));
>> + ArgVals.push_back(ParamValue::forDirect(V));
>> break;
>> }
>>
>> - llvm::AllocaInst *Alloca = CreateMemTemp(Ty, Arg->getName());
>> + Address Alloca = CreateMemTemp(Ty, getContext().getDeclAlign(Arg),
>> + Arg->getName());
>>
>> - // The alignment we need to use is the max of the requested alignment for
>> - // the argument plus the alignment required by our access code below.
>> - unsigned AlignmentToUse =
>> - CGM.getDataLayout().getABITypeAlignment(ArgI.getCoerceToType());
>> - AlignmentToUse = std::max(AlignmentToUse,
>> - (unsigned)getContext().getDeclAlign(Arg).getQuantity());
>> -
>> - Alloca->setAlignment(AlignmentToUse);
>> - llvm::Value *V = Alloca;
>> - llvm::Value *Ptr = V; // Pointer to store into.
>> - CharUnits PtrAlign = CharUnits::fromQuantity(AlignmentToUse);
>> -
>> - // If the value is offset in memory, apply the offset now.
>> - if (unsigned Offs = ArgI.getDirectOffset()) {
>> - Ptr = Builder.CreateBitCast(Ptr, Builder.getInt8PtrTy());
>> - Ptr = Builder.CreateConstGEP1_32(Builder.getInt8Ty(), Ptr, Offs);
>> - Ptr = Builder.CreateBitCast(Ptr,
>> - llvm::PointerType::getUnqual(ArgI.getCoerceToType()));
>> - PtrAlign = PtrAlign.alignmentAtOffset(CharUnits::fromQuantity(Offs));
>> - }
>> + // Pointer to store into.
>> + Address Ptr = emitAddressAtOffset(*this, Alloca, ArgI);
>>
>> // Fast-isel and the optimizer generally like scalar values better than
>> // FCAs, so we flatten them if this is safe to do for this argument.
>> llvm::StructType *STy = dyn_cast<llvm::StructType>(ArgI.getCoerceToType());
>> if (ArgI.isDirect() && ArgI.getCanBeFlattened() && STy &&
>> STy->getNumElements() > 1) {
>> + auto SrcLayout = CGM.getDataLayout().getStructLayout(STy);
>> uint64_t SrcSize = CGM.getDataLayout().getTypeAllocSize(STy);
>> - llvm::Type *DstTy =
>> - cast<llvm::PointerType>(Ptr->getType())->getElementType();
>> + llvm::Type *DstTy = Ptr.getElementType();
>> uint64_t DstSize = CGM.getDataLayout().getTypeAllocSize(DstTy);
>>
>> + Address AddrToStoreInto = Address::invalid();
>> if (SrcSize <= DstSize) {
>> - Ptr = Builder.CreateBitCast(Ptr, llvm::PointerType::getUnqual(STy));
>> -
>> - assert(STy->getNumElements() == NumIRArgs);
>> - for (unsigned i = 0, e = STy->getNumElements(); i != e; ++i) {
>> - auto AI = FnArgs[FirstIRArg + i];
>> - AI->setName(Arg->getName() + ".coerce" + Twine(i));
>> - llvm::Value *EltPtr = Builder.CreateConstGEP2_32(STy, Ptr, 0, i);
>> - Builder.CreateStore(AI, EltPtr);
>> - }
>> + AddrToStoreInto =
>> + Builder.CreateBitCast(Ptr, llvm::PointerType::getUnqual(STy));
>> } else {
>> - llvm::AllocaInst *TempAlloca =
>> - CreateTempAlloca(ArgI.getCoerceToType(), "coerce");
>> - TempAlloca->setAlignment(AlignmentToUse);
>> - llvm::Value *TempV = TempAlloca;
>> -
>> - assert(STy->getNumElements() == NumIRArgs);
>> - for (unsigned i = 0, e = STy->getNumElements(); i != e; ++i) {
>> - auto AI = FnArgs[FirstIRArg + i];
>> - AI->setName(Arg->getName() + ".coerce" + Twine(i));
>> - llvm::Value *EltPtr =
>> - Builder.CreateConstGEP2_32(ArgI.getCoerceToType(), TempV, 0, i);
>> - Builder.CreateStore(AI, EltPtr);
>> - }
>> + AddrToStoreInto =
>> + CreateTempAlloca(STy, Alloca.getAlignment(), "coerce");
>> + }
>>
>> - Builder.CreateMemCpy(Ptr, TempV, DstSize, AlignmentToUse);
>> + assert(STy->getNumElements() == NumIRArgs);
>> + for (unsigned i = 0, e = STy->getNumElements(); i != e; ++i) {
>> + auto AI = FnArgs[FirstIRArg + i];
>> + AI->setName(Arg->getName() + ".coerce" + Twine(i));
>> + auto Offset = CharUnits::fromQuantity(SrcLayout->getElementOffset(i));
>> + Address EltPtr =
>> + Builder.CreateStructGEP(AddrToStoreInto, i, Offset);
>> + Builder.CreateStore(AI, EltPtr);
>> + }
>> +
>> + if (SrcSize > DstSize) {
>> + Builder.CreateMemCpy(Ptr, AddrToStoreInto, DstSize);
>> }
>> +
>> } else {
>> // Simple case, just do a coerced store of the argument into the alloca.
>> assert(NumIRArgs == 1);
>> auto AI = FnArgs[FirstIRArg];
>> AI->setName(Arg->getName() + ".coerce");
>> - CreateCoercedStore(AI, Ptr, /*DestIsVolatile=*/false, PtrAlign, *this);
>> + CreateCoercedStore(AI, Ptr, /*DestIsVolatile=*/false, *this);
>> }
>>
>> -
>> // Match to what EmitParmDecl is expecting for this type.
>> if (CodeGenFunction::hasScalarEvaluationKind(Ty)) {
>> - V = EmitLoadOfScalar(V, false, AlignmentToUse, Ty, Arg->getLocStart());
>> + llvm::Value *V =
>> + EmitLoadOfScalar(Alloca, false, Ty, Arg->getLocStart());
>> if (isPromoted)
>> V = emitArgumentDemotion(*this, Arg, V);
>> - ArgVals.push_back(ValueAndIsPtr(V, HaveValue));
>> + ArgVals.push_back(ParamValue::forDirect(V));
>> } else {
>> - ArgVals.push_back(ValueAndIsPtr(V, HavePointer));
>> + ArgVals.push_back(ParamValue::forIndirect(Alloca));
>> }
>> break;
>> }
>> @@ -2078,11 +2064,9 @@ void CodeGenFunction::EmitFunctionProlog
>> // If this structure was expanded into multiple arguments then
>> // we need to create a temporary and reconstruct it from the
>> // arguments.
>> - llvm::AllocaInst *Alloca = CreateMemTemp(Ty);
>> - CharUnits Align = getContext().getDeclAlign(Arg);
>> - Alloca->setAlignment(Align.getQuantity());
>> - LValue LV = MakeAddrLValue(Alloca, Ty, Align);
>> - ArgVals.push_back(ValueAndIsPtr(Alloca, HavePointer));
>> + Address Alloca = CreateMemTemp(Ty, getContext().getDeclAlign(Arg));
>> + LValue LV = MakeAddrLValue(Alloca, Ty);
>> + ArgVals.push_back(ParamValue::forIndirect(Alloca));
>>
>> auto FnArgIter = FnArgs.begin() + FirstIRArg;
>> ExpandTypeFromArgs(Ty, LV, FnArgIter);
>> @@ -2098,10 +2082,10 @@ void CodeGenFunction::EmitFunctionProlog
>> assert(NumIRArgs == 0);
>> // Initialize the local variable appropriately.
>> if (!hasScalarEvaluationKind(Ty)) {
>> - ArgVals.push_back(ValueAndIsPtr(CreateMemTemp(Ty), HavePointer));
>> + ArgVals.push_back(ParamValue::forIndirect(CreateMemTemp(Ty)));
>> } else {
>> llvm::Value *U = llvm::UndefValue::get(ConvertType(Arg->getType()));
>> - ArgVals.push_back(ValueAndIsPtr(U, HaveValue));
>> + ArgVals.push_back(ParamValue::forDirect(U));
>> }
>> break;
>> }
>> @@ -2109,12 +2093,10 @@ void CodeGenFunction::EmitFunctionProlog
>>
>> if (getTarget().getCXXABI().areArgsDestroyedLeftToRightInCallee()) {
>> for (int I = Args.size() - 1; I >= 0; --I)
>> - EmitParmDecl(*Args[I], ArgVals[I].getPointer(), ArgVals[I].getInt(),
>> - I + 1);
>> + EmitParmDecl(*Args[I], ArgVals[I], I + 1);
>> } else {
>> for (unsigned I = 0, E = Args.size(); I != E; ++I)
>> - EmitParmDecl(*Args[I], ArgVals[I].getPointer(), ArgVals[I].getInt(),
>> - I + 1);
>> + EmitParmDecl(*Args[I], ArgVals[I], I + 1);
>> }
>> }
>>
>> @@ -2240,7 +2222,7 @@ static llvm::Value *tryRemoveRetainOfSel
>> llvm::LoadInst *load =
>> dyn_cast<llvm::LoadInst>(retainedValue->stripPointerCasts());
>> if (!load || load->isAtomic() || load->isVolatile() ||
>> - load->getPointerOperand() != CGF.GetAddrOfLocalVar(self))
>> + load->getPointerOperand() != CGF.GetAddrOfLocalVar(self).getPointer())
>> return nullptr;
>>
>> // Okay! Burn it all down. This relies for correctness on the
>> @@ -2281,7 +2263,7 @@ static llvm::StoreInst *findDominatingSt
>> // for something immediately preceding the IP. Sometimes this can
>> // happen with how we generate implicit-returns; it can also happen
>> // with noreturn cleanups.
>> - if (!CGF.ReturnValue->hasOneUse()) {
>> + if (!CGF.ReturnValue.getPointer()->hasOneUse()) {
>> llvm::BasicBlock *IP = CGF.Builder.GetInsertBlock();
>> if (IP->empty()) return nullptr;
>> llvm::Instruction *I = &IP->back();
>> @@ -2307,13 +2289,14 @@ static llvm::StoreInst *findDominatingSt
>>
>> llvm::StoreInst *store = dyn_cast<llvm::StoreInst>(I);
>> if (!store) return nullptr;
>> - if (store->getPointerOperand() != CGF.ReturnValue) return nullptr;
>> + if (store->getPointerOperand() != CGF.ReturnValue.getPointer())
>> + return nullptr;
>> assert(!store->isAtomic() && !store->isVolatile()); // see below
>> return store;
>> }
>>
>> llvm::StoreInst *store =
>> - dyn_cast<llvm::StoreInst>(CGF.ReturnValue->user_back());
>> + dyn_cast<llvm::StoreInst>(CGF.ReturnValue.getPointer()->user_back());
>> if (!store) return nullptr;
>>
>> // These aren't actually possible for non-coerced returns, and we
>> @@ -2344,7 +2327,7 @@ void CodeGenFunction::EmitFunctionEpilog
>> }
>>
>> // Functions with no result always return void.
>> - if (!ReturnValue) {
>> + if (!ReturnValue.isValid()) {
>> Builder.CreateRetVoid();
>> return;
>> }
>> @@ -2365,7 +2348,7 @@ void CodeGenFunction::EmitFunctionEpilog
>> llvm::Value *ArgStruct = EI;
>> llvm::Value *SRet = Builder.CreateStructGEP(
>> nullptr, ArgStruct, RetAI.getInAllocaFieldIndex());
>> - RV = Builder.CreateLoad(SRet, "sret");
>> + RV = Builder.CreateAlignedLoad(SRet, getPointerAlign(), "sret");
>> }
>> break;
>>
>> @@ -2376,8 +2359,7 @@ void CodeGenFunction::EmitFunctionEpilog
>> switch (getEvaluationKind(RetTy)) {
>> case TEK_Complex: {
>> ComplexPairTy RT =
>> - EmitLoadOfComplex(MakeNaturalAlignAddrLValue(ReturnValue, RetTy),
>> - EndLoc);
>> + EmitLoadOfComplex(MakeAddrLValue(ReturnValue, RetTy), EndLoc);
>> EmitStoreOfComplex(RT, MakeNaturalAlignAddrLValue(AI, RetTy),
>> /*isInit*/ true);
>> break;
>> @@ -2415,9 +2397,12 @@ void CodeGenFunction::EmitFunctionEpilog
>> SI->eraseFromParent();
>>
>> // If that was the only use of the return value, nuke it as well now.
>> - if (ReturnValue->use_empty() && isa<llvm::AllocaInst>(ReturnValue)) {
>> - cast<llvm::AllocaInst>(ReturnValue)->eraseFromParent();
>> - ReturnValue = nullptr;
>> + auto returnValueInst = ReturnValue.getPointer();
>> + if (returnValueInst->use_empty()) {
>> + if (auto alloca = dyn_cast<llvm::AllocaInst>(returnValueInst)) {
>> + alloca->eraseFromParent();
>> + ReturnValue = Address::invalid();
>> + }
>> }
>>
>> // Otherwise, we have to do a simple load.
>> @@ -2425,18 +2410,10 @@ void CodeGenFunction::EmitFunctionEpilog
>> RV = Builder.CreateLoad(ReturnValue);
>> }
>> } else {
>> - llvm::Value *V = ReturnValue;
>> - CharUnits Align = getContext().getTypeAlignInChars(RetTy);
>> // If the value is offset in memory, apply the offset now.
>> - if (unsigned Offs = RetAI.getDirectOffset()) {
>> - V = Builder.CreateBitCast(V, Builder.getInt8PtrTy());
>> - V = Builder.CreateConstGEP1_32(Builder.getInt8Ty(), V, Offs);
>> - V = Builder.CreateBitCast(V,
>> - llvm::PointerType::getUnqual(RetAI.getCoerceToType()));
>> - Align = Align.alignmentAtOffset(CharUnits::fromQuantity(Offs));
>> - }
>> + Address V = emitAddressAtOffset(*this, ReturnValue, RetAI);
>>
>> - RV = CreateCoercedLoad(V, RetAI.getCoerceToType(), Align, *this);
>> + RV = CreateCoercedLoad(V, RetAI.getCoerceToType(), *this);
>> }
>>
>> // In ARC, end functions that return a retainable type with a call
>> @@ -2486,14 +2463,20 @@ static bool isInAllocaArgument(CGCXXABI
>> return RD && ABI.getRecordArgABI(RD) == CGCXXABI::RAA_DirectInMemory;
>> }
>>
>> -static AggValueSlot createPlaceholderSlot(CodeGenFunction &CGF, QualType Ty) {
>> +static AggValueSlot createPlaceholderSlot(CodeGenFunction &CGF,
>> + QualType Ty) {
>> // FIXME: Generate IR in one pass, rather than going back and fixing up these
>> // placeholders.
>> llvm::Type *IRTy = CGF.ConvertTypeForMem(Ty);
>> llvm::Value *Placeholder =
>> - llvm::UndefValue::get(IRTy->getPointerTo()->getPointerTo());
>> - Placeholder = CGF.Builder.CreateLoad(Placeholder);
>> - return AggValueSlot::forAddr(Placeholder, CharUnits::Zero(),
>> + llvm::UndefValue::get(IRTy->getPointerTo()->getPointerTo());
>> + Placeholder = CGF.Builder.CreateDefaultAlignedLoad(Placeholder);
>> +
>> + // FIXME: When we generate this IR in one pass, we shouldn't need
>> + // this win32-specific alignment hack.
>> + CharUnits Align = CharUnits::fromQuantity(4);
>> +
>> + return AggValueSlot::forAddr(Address(Placeholder, Align),
>> Ty.getQualifiers(),
>> AggValueSlot::IsNotDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> @@ -2506,7 +2489,7 @@ void CodeGenFunction::EmitDelegateCallAr
>> // StartFunction converted the ABI-lowered parameter(s) into a
>> // local alloca. We need to turn that into an r-value suitable
>> // for EmitCall.
>> - llvm::Value *local = GetAddrOfLocalVar(param);
>> + Address local = GetAddrOfLocalVar(param);
>>
>> QualType type = param->getType();
>>
>> @@ -2541,20 +2524,21 @@ static bool isProvablyNonNull(llvm::Valu
>> static void emitWriteback(CodeGenFunction &CGF,
>> const CallArgList::Writeback &writeback) {
>> const LValue &srcLV = writeback.Source;
>> - llvm::Value *srcAddr = srcLV.getAddress();
>> - assert(!isProvablyNull(srcAddr) &&
>> + Address srcAddr = srcLV.getAddress();
>> + assert(!isProvablyNull(srcAddr.getPointer()) &&
>> "shouldn't have writeback for provably null argument");
>>
>> llvm::BasicBlock *contBB = nullptr;
>>
>> // If the argument wasn't provably non-null, we need to null check
>> // before doing the store.
>> - bool provablyNonNull = isProvablyNonNull(srcAddr);
>> + bool provablyNonNull = isProvablyNonNull(srcAddr.getPointer());
>> if (!provablyNonNull) {
>> llvm::BasicBlock *writebackBB = CGF.createBasicBlock("icr.writeback");
>> contBB = CGF.createBasicBlock("icr.done");
>>
>> - llvm::Value *isNull = CGF.Builder.CreateIsNull(srcAddr, "icr.isnull");
>> + llvm::Value *isNull =
>> + CGF.Builder.CreateIsNull(srcAddr.getPointer(), "icr.isnull");
>> CGF.Builder.CreateCondBr(isNull, contBB, writebackBB);
>> CGF.EmitBlock(writebackBB);
>> }
>> @@ -2563,9 +2547,8 @@ static void emitWriteback(CodeGenFunctio
>> llvm::Value *value = CGF.Builder.CreateLoad(writeback.Temporary);
>>
>> // Cast it back, in case we're writing an id to a Foo* or something.
>> - value = CGF.Builder.CreateBitCast(value,
>> - cast<llvm::PointerType>(srcAddr->getType())->getElementType(),
>> - "icr.writeback-cast");
>> + value = CGF.Builder.CreateBitCast(value, srcAddr.getElementType(),
>> + "icr.writeback-cast");
>>
>> // Perform the writeback.
>>
>> @@ -2629,7 +2612,9 @@ static const Expr *maybeGetUnaryAddrOfOp
>> }
>>
>> /// Emit an argument that's being passed call-by-writeback. That is,
>> -/// we are passing the address of
>> +/// we are passing the address of an __autoreleased temporary; it
>> +/// might be copy-initialized with the current value of the given
>> +/// address, but it will definitely be copied out of after the call.
>> static void emitWritebackArg(CodeGenFunction &CGF, CallArgList &args,
>> const ObjCIndirectCopyRestoreExpr *CRE) {
>> LValue srcLV;
>> @@ -2641,13 +2626,13 @@ static void emitWritebackArg(CodeGenFunc
>>
>> // Otherwise, just emit it as a scalar.
>> } else {
>> - llvm::Value *srcAddr = CGF.EmitScalarExpr(CRE->getSubExpr());
>> + Address srcAddr = CGF.EmitPointerWithAlignment(CRE->getSubExpr());
>>
>> QualType srcAddrType =
>> CRE->getSubExpr()->getType()->castAs<PointerType>()->getPointeeType();
>> - srcLV = CGF.MakeNaturalAlignAddrLValue(srcAddr, srcAddrType);
>> + srcLV = CGF.MakeAddrLValue(srcAddr, srcAddrType);
>> }
>> - llvm::Value *srcAddr = srcLV.getAddress();
>> + Address srcAddr = srcLV.getAddress();
>>
>> // The dest and src types don't necessarily match in LLVM terms
>> // because of the crazy ObjC compatibility rules.
>> @@ -2656,15 +2641,16 @@ static void emitWritebackArg(CodeGenFunc
>> cast<llvm::PointerType>(CGF.ConvertType(CRE->getType()));
>>
>> // If the address is a constant null, just pass the appropriate null.
>> - if (isProvablyNull(srcAddr)) {
>> + if (isProvablyNull(srcAddr.getPointer())) {
>> args.add(RValue::get(llvm::ConstantPointerNull::get(destType)),
>> CRE->getType());
>> return;
>> }
>>
>> // Create the temporary.
>> - llvm::Value *temp = CGF.CreateTempAlloca(destType->getElementType(),
>> - "icr.temp");
>> + Address temp = CGF.CreateTempAlloca(destType->getElementType(),
>> + CGF.getPointerAlign(),
>> + "icr.temp");
>> // Loading an l-value can introduce a cleanup if the l-value is __weak,
>> // and that cleanup will be conditional if we can't prove that the l-value
>> // isn't null, so we need to register a dominating point so that the cleanups
>> @@ -2686,15 +2672,16 @@ static void emitWritebackArg(CodeGenFunc
>> // If the address is *not* known to be non-null, we need to switch.
>> llvm::Value *finalArgument;
>>
>> - bool provablyNonNull = isProvablyNonNull(srcAddr);
>> + bool provablyNonNull = isProvablyNonNull(srcAddr.getPointer());
>> if (provablyNonNull) {
>> - finalArgument = temp;
>> + finalArgument = temp.getPointer();
>> } else {
>> - llvm::Value *isNull = CGF.Builder.CreateIsNull(srcAddr, "icr.isnull");
>> + llvm::Value *isNull =
>> + CGF.Builder.CreateIsNull(srcAddr.getPointer(), "icr.isnull");
>>
>> finalArgument = CGF.Builder.CreateSelect(isNull,
>> llvm::ConstantPointerNull::get(destType),
>> - temp, "icr.argument");
>> + temp.getPointer(), "icr.argument");
>>
>> // If we need to copy, then the load has to be conditional, which
>> // means we need control flow.
>> @@ -2766,7 +2753,8 @@ void CallArgList::allocateArgumentMemory
>> // stacksave to an alloca to avoid violating SSA form.
>> // TODO: This is dead if we never emit the cleanup. We should create the
>> // alloca and store lazily on the first cleanup emission.
>> - StackBaseMem = CGF.CreateTempAlloca(CGF.Int8PtrTy, "inalloca.spmem");
>> + StackBaseMem = CGF.CreateTempAlloca(CGF.Int8PtrTy, CGF.getPointerAlign(),
>> + "inalloca.spmem");
>> CGF.Builder.CreateStore(StackBase, StackBaseMem);
>> CGF.pushStackRestore(EHCleanup, StackBaseMem);
>> StackCleanup = CGF.EHStack.getInnermostEHScope();
>> @@ -2853,10 +2841,10 @@ void CodeGenFunction::EmitCallArgs(
>> namespace {
>>
>> struct DestroyUnpassedArg final : EHScopeStack::Cleanup {
>> - DestroyUnpassedArg(llvm::Value *Addr, QualType Ty)
>> + DestroyUnpassedArg(Address Addr, QualType Ty)
>> : Addr(Addr), Ty(Ty) {}
>>
>> - llvm::Value *Addr;
>> + Address Addr;
>> QualType Ty;
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> @@ -2930,7 +2918,8 @@ void CodeGenFunction::EmitCallArg(CallAr
>> // Create a no-op GEP between the placeholder and the cleanup so we can
>> // RAUW it successfully. It also serves as a marker of the first
>> // instruction where the cleanup is active.
>> - pushFullExprCleanup<DestroyUnpassedArg>(EHCleanup, Slot.getAddr(), type);
>> + pushFullExprCleanup<DestroyUnpassedArg>(EHCleanup, Slot.getAddress(),
>> + type);
>> // This unreachable is a temporary marker which will be removed later.
>> llvm::Instruction *IsActive = Builder.CreateUnreachable();
>> args.addArgCleanupDeactivation(EHStack.getInnermostEHScope(), IsActive);
>> @@ -2947,9 +2936,8 @@ void CodeGenFunction::EmitCallArg(CallAr
>> } else {
>> // We can't represent a misaligned lvalue in the CallArgList, so copy
>> // to an aligned temporary now.
>> - llvm::Value *tmp = CreateMemTemp(type);
>> - EmitAggregateCopy(tmp, L.getAddress(), type, L.isVolatile(),
>> - L.getAlignment());
>> + Address tmp = CreateMemTemp(type);
>> + EmitAggregateCopy(tmp, L.getAddress(), type, L.isVolatile());
>> args.add(RValue::getAggregate(tmp), type);
>> }
>> return;
>> @@ -3124,8 +3112,10 @@ RValue CodeGenFunction::EmitCall(const C
>>
>> // If we're using inalloca, insert the allocation after the stack save.
>> // FIXME: Do this earlier rather than hacking it in here!
>> - llvm::AllocaInst *ArgMemory = nullptr;
>> + Address ArgMemory = Address::invalid();
>> + const llvm::StructLayout *ArgMemoryLayout = nullptr;
>> if (llvm::StructType *ArgStruct = CallInfo.getArgStruct()) {
>> + ArgMemoryLayout = CGM.getDataLayout().getStructLayout(ArgStruct);
>> llvm::Instruction *IP = CallArgs.getStackBase();
>> llvm::AllocaInst *AI;
>> if (IP) {
>> @@ -3134,36 +3124,44 @@ RValue CodeGenFunction::EmitCall(const C
>> } else {
>> AI = CreateTempAlloca(ArgStruct, "argmem");
>> }
>> + auto Align = CallInfo.getArgStructAlignment();
>> + AI->setAlignment(Align.getQuantity());
>> AI->setUsedWithInAlloca(true);
>> assert(AI->isUsedWithInAlloca() && !AI->isStaticAlloca());
>> - ArgMemory = AI;
>> + ArgMemory = Address(AI, Align);
>> }
>>
>> + // Helper function to drill into the inalloca allocation.
>> + auto createInAllocaStructGEP = [&](unsigned FieldIndex) -> Address {
>> + auto FieldOffset =
>> + CharUnits::fromQuantity(ArgMemoryLayout->getElementOffset(FieldIndex));
>> + return Builder.CreateStructGEP(ArgMemory, FieldIndex, FieldOffset);
>> + };
>> +
>> ClangToLLVMArgMapping IRFunctionArgs(CGM.getContext(), CallInfo);
>> SmallVector<llvm::Value *, 16> IRCallArgs(IRFunctionArgs.totalIRArgs());
>>
>> // If the call returns a temporary with struct return, create a temporary
>> // alloca to hold the result, unless one is given to us.
>> - llvm::Value *SRetPtr = nullptr;
>> + Address SRetPtr = Address::invalid();
>> size_t UnusedReturnSize = 0;
>> if (RetAI.isIndirect() || RetAI.isInAlloca()) {
>> - SRetPtr = ReturnValue.getValue();
>> - if (!SRetPtr) {
>> + if (!ReturnValue.isNull()) {
>> + SRetPtr = ReturnValue.getValue();
>> + } else {
>> SRetPtr = CreateMemTemp(RetTy);
>> if (HaveInsertPoint() && ReturnValue.isUnused()) {
>> uint64_t size =
>> CGM.getDataLayout().getTypeAllocSize(ConvertTypeForMem(RetTy));
>> - if (EmitLifetimeStart(size, SRetPtr))
>> + if (EmitLifetimeStart(size, SRetPtr.getPointer()))
>> UnusedReturnSize = size;
>> }
>> }
>> if (IRFunctionArgs.hasSRetArg()) {
>> - IRCallArgs[IRFunctionArgs.getSRetArgNo()] = SRetPtr;
>> + IRCallArgs[IRFunctionArgs.getSRetArgNo()] = SRetPtr.getPointer();
>> } else {
>> - llvm::Value *Addr =
>> - Builder.CreateStructGEP(ArgMemory->getAllocatedType(), ArgMemory,
>> - RetAI.getInAllocaFieldIndex());
>> - Builder.CreateStore(SRetPtr, Addr);
>> + Address Addr = createInAllocaStructGEP(RetAI.getInAllocaFieldIndex());
>> + Builder.CreateStore(SRetPtr.getPointer(), Addr);
>> }
>> }
>>
>> @@ -3176,8 +3174,6 @@ RValue CodeGenFunction::EmitCall(const C
>> const ABIArgInfo &ArgInfo = info_it->info;
>> RValue RV = I->RV;
>>
>> - CharUnits TypeAlign = getContext().getTypeAlignInChars(I->Ty);
>> -
>> // Insert a padding argument to ensure proper alignment.
>> if (IRFunctionArgs.hasPaddingArg(ArgNo))
>> IRCallArgs[IRFunctionArgs.getPaddingArgNo(ArgNo)] =
>> @@ -3193,27 +3189,23 @@ RValue CodeGenFunction::EmitCall(const C
>> if (RV.isAggregate()) {
>> // Replace the placeholder with the appropriate argument slot GEP.
>> llvm::Instruction *Placeholder =
>> - cast<llvm::Instruction>(RV.getAggregateAddr());
>> + cast<llvm::Instruction>(RV.getAggregatePointer());
>> CGBuilderTy::InsertPoint IP = Builder.saveIP();
>> Builder.SetInsertPoint(Placeholder);
>> - llvm::Value *Addr =
>> - Builder.CreateStructGEP(ArgMemory->getAllocatedType(), ArgMemory,
>> - ArgInfo.getInAllocaFieldIndex());
>> + Address Addr = createInAllocaStructGEP(ArgInfo.getInAllocaFieldIndex());
>> Builder.restoreIP(IP);
>> - deferPlaceholderReplacement(Placeholder, Addr);
>> + deferPlaceholderReplacement(Placeholder, Addr.getPointer());
>> } else {
>> // Store the RValue into the argument struct.
>> - llvm::Value *Addr =
>> - Builder.CreateStructGEP(ArgMemory->getAllocatedType(), ArgMemory,
>> - ArgInfo.getInAllocaFieldIndex());
>> - unsigned AS = Addr->getType()->getPointerAddressSpace();
>> + Address Addr = createInAllocaStructGEP(ArgInfo.getInAllocaFieldIndex());
>> + unsigned AS = Addr.getType()->getPointerAddressSpace();
>> llvm::Type *MemType = ConvertTypeForMem(I->Ty)->getPointerTo(AS);
>> // There are some cases where a trivial bitcast is not avoidable. The
>> // definition of a type later in a translation unit may change it's type
>> // from {}* to (%struct.foo*)*.
>> - if (Addr->getType() != MemType)
>> + if (Addr.getType() != MemType)
>> Addr = Builder.CreateBitCast(Addr, MemType);
>> - LValue argLV = MakeAddrLValue(Addr, I->Ty, TypeAlign);
>> + LValue argLV = MakeAddrLValue(Addr, I->Ty);
>> EmitInitStoreOfNonAggregate(*this, RV, argLV);
>> }
>> break;
>> @@ -3223,12 +3215,10 @@ RValue CodeGenFunction::EmitCall(const C
>> assert(NumIRArgs == 1);
>> if (RV.isScalar() || RV.isComplex()) {
>> // Make a temporary alloca to pass the argument.
>> - llvm::AllocaInst *AI = CreateMemTemp(I->Ty);
>> - if (ArgInfo.getIndirectAlign() > AI->getAlignment())
>> - AI->setAlignment(ArgInfo.getIndirectAlign());
>> - IRCallArgs[FirstIRArg] = AI;
>> + Address Addr = CreateMemTemp(I->Ty, ArgInfo.getIndirectAlign());
>> + IRCallArgs[FirstIRArg] = Addr.getPointer();
>>
>> - LValue argLV = MakeAddrLValue(AI, I->Ty, TypeAlign);
>> + LValue argLV = MakeAddrLValue(Addr, I->Ty);
>> EmitInitStoreOfNonAggregate(*this, RV, argLV);
>> } else {
>> // We want to avoid creating an unnecessary temporary+copy here;
>> @@ -3239,27 +3229,27 @@ RValue CodeGenFunction::EmitCall(const C
>> // we cannot force it to be sufficiently aligned.
>> // 3. If the argument is byval, but RV is located in an address space
>> // different than that of the argument (0).
>> - llvm::Value *Addr = RV.getAggregateAddr();
>> - unsigned Align = ArgInfo.getIndirectAlign();
>> + Address Addr = RV.getAggregateAddress();
>> + CharUnits Align = ArgInfo.getIndirectAlign();
>> const llvm::DataLayout *TD = &CGM.getDataLayout();
>> - const unsigned RVAddrSpace = Addr->getType()->getPointerAddressSpace();
>> + const unsigned RVAddrSpace = Addr.getType()->getAddressSpace();
>> const unsigned ArgAddrSpace =
>> (FirstIRArg < IRFuncTy->getNumParams()
>> ? IRFuncTy->getParamType(FirstIRArg)->getPointerAddressSpace()
>> : 0);
>> if ((!ArgInfo.getIndirectByVal() && I->NeedsCopy) ||
>> - (ArgInfo.getIndirectByVal() && TypeAlign.getQuantity() < Align &&
>> - llvm::getOrEnforceKnownAlignment(Addr, Align, *TD) < Align) ||
>> + (ArgInfo.getIndirectByVal() && Addr.getAlignment() < Align &&
>> + llvm::getOrEnforceKnownAlignment(Addr.getPointer(),
>> + Align.getQuantity(), *TD)
>> + < Align.getQuantity()) ||
>> (ArgInfo.getIndirectByVal() && (RVAddrSpace != ArgAddrSpace))) {
>> // Create an aligned temporary, and copy to it.
>> - llvm::AllocaInst *AI = CreateMemTemp(I->Ty);
>> - if (Align > AI->getAlignment())
>> - AI->setAlignment(Align);
>> - IRCallArgs[FirstIRArg] = AI;
>> + Address AI = CreateMemTemp(I->Ty, ArgInfo.getIndirectAlign());
>> + IRCallArgs[FirstIRArg] = AI.getPointer();
>> EmitAggregateCopy(AI, Addr, I->Ty, RV.isVolatileQualified());
>> } else {
>> // Skip the extra memcpy call.
>> - IRCallArgs[FirstIRArg] = Addr;
>> + IRCallArgs[FirstIRArg] = Addr.getPointer();
>> }
>> }
>> break;
>> @@ -3279,7 +3269,7 @@ RValue CodeGenFunction::EmitCall(const C
>> if (RV.isScalar())
>> V = RV.getScalarVal();
>> else
>> - V = Builder.CreateLoad(RV.getAggregateAddr());
>> + V = Builder.CreateLoad(RV.getAggregateAddress());
>>
>> // We might have to widen integers, but we should never truncate.
>> if (ArgInfo.getCoerceToType() != V->getType() &&
>> @@ -3296,35 +3286,24 @@ RValue CodeGenFunction::EmitCall(const C
>> }
>>
>> // FIXME: Avoid the conversion through memory if possible.
>> - llvm::Value *SrcPtr;
>> - CharUnits SrcAlign;
>> + Address Src = Address::invalid();
>> if (RV.isScalar() || RV.isComplex()) {
>> - SrcPtr = CreateMemTemp(I->Ty, "coerce");
>> - SrcAlign = TypeAlign;
>> - LValue SrcLV = MakeAddrLValue(SrcPtr, I->Ty, TypeAlign);
>> + Src = CreateMemTemp(I->Ty, "coerce");
>> + LValue SrcLV = MakeAddrLValue(Src, I->Ty);
>> EmitInitStoreOfNonAggregate(*this, RV, SrcLV);
>> } else {
>> - SrcPtr = RV.getAggregateAddr();
>> - // This alignment is guaranteed by EmitCallArg.
>> - SrcAlign = TypeAlign;
>> + Src = RV.getAggregateAddress();
>> }
>>
>> // If the value is offset in memory, apply the offset now.
>> - if (unsigned Offs = ArgInfo.getDirectOffset()) {
>> - SrcPtr = Builder.CreateBitCast(SrcPtr, Builder.getInt8PtrTy());
>> - SrcPtr = Builder.CreateConstGEP1_32(Builder.getInt8Ty(), SrcPtr, Offs);
>> - SrcPtr = Builder.CreateBitCast(SrcPtr,
>> - llvm::PointerType::getUnqual(ArgInfo.getCoerceToType()));
>> - SrcAlign = SrcAlign.alignmentAtOffset(CharUnits::fromQuantity(Offs));
>> - }
>> + Src = emitAddressAtOffset(*this, Src, ArgInfo);
>>
>> // Fast-isel and the optimizer generally like scalar values better than
>> // FCAs, so we flatten them if this is safe to do for this argument.
>> llvm::StructType *STy =
>> dyn_cast<llvm::StructType>(ArgInfo.getCoerceToType());
>> if (STy && ArgInfo.isDirect() && ArgInfo.getCanBeFlattened()) {
>> - llvm::Type *SrcTy =
>> - cast<llvm::PointerType>(SrcPtr->getType())->getElementType();
>> + llvm::Type *SrcTy = Src.getType()->getElementType();
>> uint64_t SrcSize = CGM.getDataLayout().getTypeAllocSize(SrcTy);
>> uint64_t DstSize = CGM.getDataLayout().getTypeAllocSize(STy);
>>
>> @@ -3333,29 +3312,28 @@ RValue CodeGenFunction::EmitCall(const C
>> // of the destination type to allow loading all of it. The bits past
>> // the source value are left undef.
>> if (SrcSize < DstSize) {
>> - llvm::AllocaInst *TempAlloca
>> - = CreateTempAlloca(STy, SrcPtr->getName() + ".coerce");
>> - Builder.CreateMemCpy(TempAlloca, SrcPtr, SrcSize, 0);
>> - SrcPtr = TempAlloca;
>> + Address TempAlloca
>> + = CreateTempAlloca(STy, Src.getAlignment(),
>> + Src.getName() + ".coerce");
>> + Builder.CreateMemCpy(TempAlloca, Src, SrcSize);
>> + Src = TempAlloca;
>> } else {
>> - SrcPtr = Builder.CreateBitCast(SrcPtr,
>> - llvm::PointerType::getUnqual(STy));
>> + Src = Builder.CreateBitCast(Src, llvm::PointerType::getUnqual(STy));
>> }
>>
>> + auto SrcLayout = CGM.getDataLayout().getStructLayout(STy);
>> assert(NumIRArgs == STy->getNumElements());
>> for (unsigned i = 0, e = STy->getNumElements(); i != e; ++i) {
>> - llvm::Value *EltPtr = Builder.CreateConstGEP2_32(STy, SrcPtr, 0, i);
>> - llvm::LoadInst *LI = Builder.CreateLoad(EltPtr);
>> - // We don't know what we're loading from.
>> - LI->setAlignment(1);
>> + auto Offset = CharUnits::fromQuantity(SrcLayout->getElementOffset(i));
>> + Address EltPtr = Builder.CreateStructGEP(Src, i, Offset);
>> + llvm::Value *LI = Builder.CreateLoad(EltPtr);
>> IRCallArgs[FirstIRArg + i] = LI;
>> }
>> } else {
>> // In the simple case, just pass the coerced loaded value.
>> assert(NumIRArgs == 1);
>> IRCallArgs[FirstIRArg] =
>> - CreateCoercedLoad(SrcPtr, ArgInfo.getCoerceToType(),
>> - SrcAlign, *this);
>> + CreateCoercedLoad(Src, ArgInfo.getCoerceToType(), *this);
>> }
>>
>> break;
>> @@ -3369,8 +3347,8 @@ RValue CodeGenFunction::EmitCall(const C
>> }
>> }
>>
>> - if (ArgMemory) {
>> - llvm::Value *Arg = ArgMemory;
>> + if (ArgMemory.isValid()) {
>> + llvm::Value *Arg = ArgMemory.getPointer();
>> if (CallInfo.isVariadic()) {
>> // When passing non-POD arguments by value to variadic functions, we will
>> // end up with a variadic prototype and an inalloca call site. In such
>> @@ -3496,7 +3474,7 @@ RValue CodeGenFunction::EmitCall(const C
>> if (CS.doesNotReturn()) {
>> if (UnusedReturnSize)
>> EmitLifetimeEnd(llvm::ConstantInt::get(Int64Ty, UnusedReturnSize),
>> - SRetPtr);
>> + SRetPtr.getPointer());
>>
>> Builder.CreateUnreachable();
>> Builder.ClearInsertionPoint();
>> @@ -3530,7 +3508,7 @@ RValue CodeGenFunction::EmitCall(const C
>> RValue ret = convertTempToRValue(SRetPtr, RetTy, SourceLocation());
>> if (UnusedReturnSize)
>> EmitLifetimeEnd(llvm::ConstantInt::get(Int64Ty, UnusedReturnSize),
>> - SRetPtr);
>> + SRetPtr.getPointer());
>> return ret;
>> }
>>
>> @@ -3550,15 +3528,14 @@ RValue CodeGenFunction::EmitCall(const C
>> return RValue::getComplex(std::make_pair(Real, Imag));
>> }
>> case TEK_Aggregate: {
>> - llvm::Value *DestPtr = ReturnValue.getValue();
>> + Address DestPtr = ReturnValue.getValue();
>> bool DestIsVolatile = ReturnValue.isVolatile();
>> - CharUnits DestAlign = getContext().getTypeAlignInChars(RetTy);
>>
>> - if (!DestPtr) {
>> + if (!DestPtr.isValid()) {
>> DestPtr = CreateMemTemp(RetTy, "agg.tmp");
>> DestIsVolatile = false;
>> }
>> - BuildAggStore(*this, CI, DestPtr, DestIsVolatile, DestAlign);
>> + BuildAggStore(*this, CI, DestPtr, DestIsVolatile);
>> return RValue::getAggregate(DestPtr);
>> }
>> case TEK_Scalar: {
>> @@ -3573,28 +3550,17 @@ RValue CodeGenFunction::EmitCall(const C
>> llvm_unreachable("bad evaluation kind");
>> }
>>
>> - llvm::Value *DestPtr = ReturnValue.getValue();
>> + Address DestPtr = ReturnValue.getValue();
>> bool DestIsVolatile = ReturnValue.isVolatile();
>> - CharUnits DestAlign = getContext().getTypeAlignInChars(RetTy);
>>
>> - if (!DestPtr) {
>> + if (!DestPtr.isValid()) {
>> DestPtr = CreateMemTemp(RetTy, "coerce");
>> DestIsVolatile = false;
>> }
>>
>> // If the value is offset in memory, apply the offset now.
>> - llvm::Value *StorePtr = DestPtr;
>> - CharUnits StoreAlign = DestAlign;
>> - if (unsigned Offs = RetAI.getDirectOffset()) {
>> - StorePtr = Builder.CreateBitCast(StorePtr, Builder.getInt8PtrTy());
>> - StorePtr =
>> - Builder.CreateConstGEP1_32(Builder.getInt8Ty(), StorePtr, Offs);
>> - StorePtr = Builder.CreateBitCast(StorePtr,
>> - llvm::PointerType::getUnqual(RetAI.getCoerceToType()));
>> - StoreAlign =
>> - StoreAlign.alignmentAtOffset(CharUnits::fromQuantity(Offs));
>> - }
>> - CreateCoercedStore(CI, StorePtr, DestIsVolatile, StoreAlign, *this);
>> + Address StorePtr = emitAddressAtOffset(*this, DestPtr, RetAI);
>> + CreateCoercedStore(CI, StorePtr, DestIsVolatile, *this);
>>
>> return convertTempToRValue(DestPtr, RetTy, SourceLocation());
>> }
>> @@ -3624,6 +3590,6 @@ RValue CodeGenFunction::EmitCall(const C
>>
>> /* VarArg handling */
>>
>> -llvm::Value *CodeGenFunction::EmitVAArg(llvm::Value *VAListAddr, QualType Ty) {
>> - return CGM.getTypes().getABIInfo().EmitVAArg(VAListAddr, Ty, *this);
>> +Address CodeGenFunction::EmitVAArg(Address VAListAddr, QualType Ty) {
>> + return CGM.getTypes().getABIInfo().EmitVAArg(*this, VAListAddr, Ty);
>> }
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCall.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCall.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCall.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGCall.h Tue Sep 8 03:05:57 2015
>> @@ -56,7 +56,7 @@ namespace CodeGen {
>> class CallArgList :
>> public SmallVector<CallArg, 16> {
>> public:
>> - CallArgList() : StackBase(nullptr), StackBaseMem(nullptr) {}
>> + CallArgList() : StackBase(nullptr), StackBaseMem(Address::invalid()) {}
>>
>> struct Writeback {
>> /// The original argument. Note that the argument l-value
>> @@ -64,7 +64,7 @@ namespace CodeGen {
>> LValue Source;
>>
>> /// The temporary alloca.
>> - llvm::Value *Temporary;
>> + Address Temporary;
>>
>> /// A value to "use" after the writeback, or null.
>> llvm::Value *ToUse;
>> @@ -88,12 +88,9 @@ namespace CodeGen {
>> other.Writebacks.begin(), other.Writebacks.end());
>> }
>>
>> - void addWriteback(LValue srcLV, llvm::Value *temporary,
>> + void addWriteback(LValue srcLV, Address temporary,
>> llvm::Value *toUse) {
>> - Writeback writeback;
>> - writeback.Source = srcLV;
>> - writeback.Temporary = temporary;
>> - writeback.ToUse = toUse;
>> + Writeback writeback = { srcLV, temporary, toUse };
>> Writebacks.push_back(writeback);
>> }
>>
>> @@ -138,7 +135,7 @@ namespace CodeGen {
>> llvm::CallInst *StackBase;
>>
>> /// The alloca holding the stackbase. We need it to maintain SSA form.
>> - llvm::AllocaInst *StackBaseMem;
>> + Address StackBaseMem;
>>
>> /// The iterator pointing to the stack restore cleanup. We manually run and
>> /// deactivate this cleanup after the call in the unexceptional case because
>> @@ -156,6 +153,7 @@ namespace CodeGen {
>> /// function can be stored, and whether the address is volatile or not.
>> class ReturnValueSlot {
>> llvm::PointerIntPair<llvm::Value *, 2, unsigned int> Value;
>> + CharUnits Alignment;
>>
>> // Return value slot flags
>> enum Flags {
>> @@ -165,14 +163,15 @@ namespace CodeGen {
>>
>> public:
>> ReturnValueSlot() {}
>> - ReturnValueSlot(llvm::Value *Value, bool IsVolatile, bool IsUnused = false)
>> - : Value(Value,
>> - (IsVolatile ? IS_VOLATILE : 0) | (IsUnused ? IS_UNUSED : 0)) {}
>> + ReturnValueSlot(Address Addr, bool IsVolatile, bool IsUnused = false)
>> + : Value(Addr.isValid() ? Addr.getPointer() : nullptr,
>> + (IsVolatile ? IS_VOLATILE : 0) | (IsUnused ? IS_UNUSED : 0)),
>> + Alignment(Addr.isValid() ? Addr.getAlignment() : CharUnits::Zero()) {}
>>
>> - bool isNull() const { return !getValue(); }
>> + bool isNull() const { return !getValue().isValid(); }
>>
>> bool isVolatile() const { return Value.getInt() & IS_VOLATILE; }
>> - llvm::Value *getValue() const { return Value.getPointer(); }
>> + Address getValue() const { return Address(Value.getPointer(), Alignment); }
>> bool isUnused() const { return Value.getInt() & IS_UNUSED; }
>> };
>>
>>
>> Modified: cfe/trunk/lib/CodeGen/CGClass.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGClass.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGClass.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGClass.cpp Tue Sep 8 03:05:57 2015
>> @@ -29,6 +29,119 @@
>> using namespace clang;
>> using namespace CodeGen;
>>
>> +/// Return the best known alignment for an unknown pointer to a
>> +/// particular class.
>> +CharUnits CodeGenModule::getClassPointerAlignment(const CXXRecordDecl *RD) {
>> + if (!RD->isCompleteDefinition())
>> + return CharUnits::One(); // Hopefully won't be used anywhere.
>> +
>> + auto &layout = getContext().getASTRecordLayout(RD);
>> +
>> + // If the class is final, then we know that the pointer points to an
>> + // object of that type and can use the full alignment.
>> + if (RD->hasAttr<FinalAttr>()) {
>> + return layout.getAlignment();
>> +
>> + // Otherwise, we have to assume it could be a subclass.
>> + } else {
>> + return layout.getNonVirtualAlignment();
>> + }
>> +}
>> +
>> +/// Return the best known alignment for a pointer to a virtual base,
>> +/// given the alignment of a pointer to the derived class.
>> +CharUnits CodeGenModule::getVBaseAlignment(CharUnits actualDerivedAlign,
>> + const CXXRecordDecl *derivedClass,
>> + const CXXRecordDecl *vbaseClass) {
>> + // The basic idea here is that an underaligned derived pointer might
>> + // indicate an underaligned base pointer.
>> +
>> + assert(vbaseClass->isCompleteDefinition());
>> + auto &baseLayout = getContext().getASTRecordLayout(vbaseClass);
>> + CharUnits expectedVBaseAlign = baseLayout.getNonVirtualAlignment();
>> +
>> + return getDynamicOffsetAlignment(actualDerivedAlign, derivedClass,
>> + expectedVBaseAlign);
>> +}
>> +
>> +CharUnits
>> +CodeGenModule::getDynamicOffsetAlignment(CharUnits actualBaseAlign,
>> + const CXXRecordDecl *baseDecl,
>> + CharUnits expectedTargetAlign) {
>> + // If the base is an incomplete type (which is, alas, possible with
>> + // member pointers), be pessimistic.
>> + if (!baseDecl->isCompleteDefinition())
>> + return std::min(actualBaseAlign, expectedTargetAlign);
>> +
>> + auto &baseLayout = getContext().getASTRecordLayout(baseDecl);
>> + CharUnits expectedBaseAlign = baseLayout.getNonVirtualAlignment();
>> +
>> + // If the class is properly aligned, assume the target offset is, too.
>> + //
>> + // This actually isn't necessarily the right thing to do --- if the
>> + // class is a complete object, but it's only properly aligned for a
>> + // base subobject, then the alignments of things relative to it are
>> + // probably off as well. (Note that this requires the alignment of
>> + // the target to be greater than the NV alignment of the derived
>> + // class.)
>> + //
>> + // However, our approach to this kind of under-alignment can only
>> + // ever be best effort; after all, we're never going to propagate
>> + // alignments through variables or parameters. Note, in particular,
>> + // that constructing a polymorphic type in an address that's less
>> + // than pointer-aligned will generally trap in the constructor,
>> + // unless we someday add some sort of attribute to change the
>> + // assumed alignment of 'this'. So our goal here is pretty much
>> + // just to allow the user to explicitly say that a pointer is
>> + // under-aligned and then safely access its fields and v-tables.
>> + if (actualBaseAlign >= expectedBaseAlign) {
>> + return expectedTargetAlign;
>> + }
>> +
>> + // Otherwise, we might be offset by an arbitrary multiple of the
>> + // actual alignment. The correct adjustment is to take the min of
>> + // the two alignments.
>> + return std::min(actualBaseAlign, expectedTargetAlign);
>> +}
>> +
>> +Address CodeGenFunction::LoadCXXThisAddress() {
>> + assert(CurFuncDecl && "loading 'this' without a func declaration?");
>> + assert(isa<CXXMethodDecl>(CurFuncDecl));
>> +
>> + // Lazily compute CXXThisAlignment.
>> + if (CXXThisAlignment.isZero()) {
>> + // Just use the best known alignment for the parent.
>> + // TODO: if we're currently emitting a complete-object ctor/dtor,
>> + // we can always use the complete-object alignment.
>> + auto RD = cast<CXXMethodDecl>(CurFuncDecl)->getParent();
>> + CXXThisAlignment = CGM.getClassPointerAlignment(RD);
>> + }
>> +
>> + return Address(LoadCXXThis(), CXXThisAlignment);
>> +}
>> +
>> +/// Emit the address of a field using a member data pointer.
>> +///
>> +/// \param E Only used for emergency diagnostics
>> +Address
>> +CodeGenFunction::EmitCXXMemberDataPointerAddress(const Expr *E, Address base,
>> + llvm::Value *memberPtr,
>> + const MemberPointerType *memberPtrType,
>> + AlignmentSource *alignSource) {
>> + // Ask the ABI to compute the actual address.
>> + llvm::Value *ptr =
>> + CGM.getCXXABI().EmitMemberDataPointerAddress(*this, E, base,
>> + memberPtr, memberPtrType);
>> +
>> + QualType memberType = memberPtrType->getPointeeType();
>> + CharUnits memberAlign = getNaturalTypeAlignment(memberType, alignSource);
>> + memberAlign =
>> + CGM.getDynamicOffsetAlignment(base.getAlignment(),
>> + memberPtrType->getClass()->getAsCXXRecordDecl(),
>> + memberAlign);
>> + return Address(ptr, memberAlign);
>> +}
>> +
>> CharUnits CodeGenModule::computeNonVirtualBaseClassOffset(
>> const CXXRecordDecl *DerivedClass, CastExpr::path_const_iterator Start,
>> CastExpr::path_const_iterator End) {
>> @@ -78,15 +191,13 @@ CodeGenModule::GetNonVirtualBaseClassOff
>> /// when the type is known to be complete (e.g. in complete destructors).
>> ///
>> /// The object pointed to by 'This' is assumed to be non-null.
>> -llvm::Value *
>> -CodeGenFunction::GetAddressOfDirectBaseInCompleteClass(llvm::Value *This,
>> +Address
>> +CodeGenFunction::GetAddressOfDirectBaseInCompleteClass(Address This,
>> const CXXRecordDecl *Derived,
>> const CXXRecordDecl *Base,
>> bool BaseIsVirtual) {
>> // 'this' must be a pointer (in some address space) to Derived.
>> - assert(This->getType()->isPointerTy() &&
>> - cast<llvm::PointerType>(This->getType())->getElementType()
>> - == ConvertType(Derived));
>> + assert(This.getElementType() == ConvertType(Derived));
>>
>> // Compute the offset of the virtual base.
>> CharUnits Offset;
>> @@ -98,20 +209,22 @@ CodeGenFunction::GetAddressOfDirectBaseI
>>
>> // Shift and cast down to the base type.
>> // TODO: for complete types, this should be possible with a GEP.
>> - llvm::Value *V = This;
>> - if (Offset.isPositive()) {
>> - V = Builder.CreateBitCast(V, Int8PtrTy);
>> - V = Builder.CreateConstInBoundsGEP1_64(V, Offset.getQuantity());
>> + Address V = This;
>> + if (!Offset.isZero()) {
>> + V = Builder.CreateElementBitCast(V, Int8Ty);
>> + V = Builder.CreateConstInBoundsByteGEP(V, Offset);
>> }
>> - V = Builder.CreateBitCast(V, ConvertType(Base)->getPointerTo());
>> + V = Builder.CreateElementBitCast(V, ConvertType(Base));
>>
>> return V;
>> }
>>
>> -static llvm::Value *
>> -ApplyNonVirtualAndVirtualOffset(CodeGenFunction &CGF, llvm::Value *ptr,
>> +static Address
>> +ApplyNonVirtualAndVirtualOffset(CodeGenFunction &CGF, Address addr,
>> CharUnits nonVirtualOffset,
>> - llvm::Value *virtualOffset) {
>> + llvm::Value *virtualOffset,
>> + const CXXRecordDecl *derivedClass,
>> + const CXXRecordDecl *nearestVBase) {
>> // Assert that we have something to do.
>> assert(!nonVirtualOffset.isZero() || virtualOffset != nullptr);
>>
>> @@ -128,13 +241,27 @@ ApplyNonVirtualAndVirtualOffset(CodeGenF
>> }
>>
>> // Apply the base offset.
>> + llvm::Value *ptr = addr.getPointer();
>> ptr = CGF.Builder.CreateBitCast(ptr, CGF.Int8PtrTy);
>> ptr = CGF.Builder.CreateInBoundsGEP(ptr, baseOffset, "add.ptr");
>> - return ptr;
>> +
>> + // If we have a virtual component, the alignment of the result will
>> + // be relative only to the known alignment of that vbase.
>> + CharUnits alignment;
>> + if (virtualOffset) {
>> + assert(nearestVBase && "virtual offset without vbase?");
>> + alignment = CGF.CGM.getVBaseAlignment(addr.getAlignment(),
>> + derivedClass, nearestVBase);
>> + } else {
>> + alignment = addr.getAlignment();
>> + }
>> + alignment = alignment.alignmentAtOffset(nonVirtualOffset);
>> +
>> + return Address(ptr, alignment);
>> }
>>
>> -llvm::Value *CodeGenFunction::GetAddressOfBaseClass(
>> - llvm::Value *Value, const CXXRecordDecl *Derived,
>> +Address CodeGenFunction::GetAddressOfBaseClass(
>> + Address Value, const CXXRecordDecl *Derived,
>> CastExpr::path_const_iterator PathBegin,
>> CastExpr::path_const_iterator PathEnd, bool NullCheckValue,
>> SourceLocation Loc) {
>> @@ -174,14 +301,14 @@ llvm::Value *CodeGenFunction::GetAddress
>> ConvertType((PathEnd[-1])->getType())->getPointerTo();
>>
>> QualType DerivedTy = getContext().getRecordType(Derived);
>> - CharUnits DerivedAlign = getContext().getTypeAlignInChars(DerivedTy);
>> + CharUnits DerivedAlign = CGM.getClassPointerAlignment(Derived);
>>
>> // If the static offset is zero and we don't have a virtual step,
>> // just do a bitcast; null checks are unnecessary.
>> if (NonVirtualOffset.isZero() && !VBase) {
>> if (sanitizePerformTypeCheck()) {
>> - EmitTypeCheck(TCK_Upcast, Loc, Value, DerivedTy, DerivedAlign,
>> - !NullCheckValue);
>> + EmitTypeCheck(TCK_Upcast, Loc, Value.getPointer(),
>> + DerivedTy, DerivedAlign, !NullCheckValue);
>> }
>> return Builder.CreateBitCast(Value, BasePtrTy);
>> }
>> @@ -196,14 +323,14 @@ llvm::Value *CodeGenFunction::GetAddress
>> llvm::BasicBlock *notNullBB = createBasicBlock("cast.notnull");
>> endBB = createBasicBlock("cast.end");
>>
>> - llvm::Value *isNull = Builder.CreateIsNull(Value);
>> + llvm::Value *isNull = Builder.CreateIsNull(Value.getPointer());
>> Builder.CreateCondBr(isNull, endBB, notNullBB);
>> EmitBlock(notNullBB);
>> }
>>
>> if (sanitizePerformTypeCheck()) {
>> - EmitTypeCheck(VBase ? TCK_UpcastToVirtualBase : TCK_Upcast, Loc, Value,
>> - DerivedTy, DerivedAlign, true);
>> + EmitTypeCheck(VBase ? TCK_UpcastToVirtualBase : TCK_Upcast, Loc,
>> + Value.getPointer(), DerivedTy, DerivedAlign, true);
>> }
>>
>> // Compute the virtual offset.
>> @@ -214,9 +341,8 @@ llvm::Value *CodeGenFunction::GetAddress
>> }
>>
>> // Apply both offsets.
>> - Value = ApplyNonVirtualAndVirtualOffset(*this, Value,
>> - NonVirtualOffset,
>> - VirtualOffset);
>> + Value = ApplyNonVirtualAndVirtualOffset(*this, Value, NonVirtualOffset,
>> + VirtualOffset, Derived, VBase);
>>
>> // Cast to the destination type.
>> Value = Builder.CreateBitCast(Value, BasePtrTy);
>> @@ -228,16 +354,16 @@ llvm::Value *CodeGenFunction::GetAddress
>> EmitBlock(endBB);
>>
>> llvm::PHINode *PHI = Builder.CreatePHI(BasePtrTy, 2, "cast.result");
>> - PHI->addIncoming(Value, notNullBB);
>> + PHI->addIncoming(Value.getPointer(), notNullBB);
>> PHI->addIncoming(llvm::Constant::getNullValue(BasePtrTy), origBB);
>> - Value = PHI;
>> + Value = Address(PHI, Value.getAlignment());
>> }
>>
>> return Value;
>> }
>>
>> -llvm::Value *
>> -CodeGenFunction::GetAddressOfDerivedClass(llvm::Value *Value,
>> +Address
>> +CodeGenFunction::GetAddressOfDerivedClass(Address BaseAddr,
>> const CXXRecordDecl *Derived,
>> CastExpr::path_const_iterator PathBegin,
>> CastExpr::path_const_iterator PathEnd,
>> @@ -253,7 +379,7 @@ CodeGenFunction::GetAddressOfDerivedClas
>>
>> if (!NonVirtualOffset) {
>> // No offset, we can just cast back.
>> - return Builder.CreateBitCast(Value, DerivedPtrTy);
>> + return Builder.CreateBitCast(BaseAddr, DerivedPtrTy);
>> }
>>
>> llvm::BasicBlock *CastNull = nullptr;
>> @@ -265,19 +391,20 @@ CodeGenFunction::GetAddressOfDerivedClas
>> CastNotNull = createBasicBlock("cast.notnull");
>> CastEnd = createBasicBlock("cast.end");
>>
>> - llvm::Value *IsNull = Builder.CreateIsNull(Value);
>> + llvm::Value *IsNull = Builder.CreateIsNull(BaseAddr.getPointer());
>> Builder.CreateCondBr(IsNull, CastNull, CastNotNull);
>> EmitBlock(CastNotNull);
>> }
>>
>> // Apply the offset.
>> - Value = Builder.CreateBitCast(Value, Int8PtrTy);
>> + llvm::Value *Value = Builder.CreateBitCast(BaseAddr.getPointer(), Int8PtrTy);
>> Value = Builder.CreateGEP(Value, Builder.CreateNeg(NonVirtualOffset),
>> "sub.ptr");
>>
>> // Just cast.
>> Value = Builder.CreateBitCast(Value, DerivedPtrTy);
>>
>> + // Produce a PHI if we had a null-check.
>> if (NullCheckValue) {
>> Builder.CreateBr(CastEnd);
>> EmitBlock(CastNull);
>> @@ -286,12 +413,11 @@ CodeGenFunction::GetAddressOfDerivedClas
>>
>> llvm::PHINode *PHI = Builder.CreatePHI(Value->getType(), 2);
>> PHI->addIncoming(Value, CastNotNull);
>> - PHI->addIncoming(llvm::Constant::getNullValue(Value->getType()),
>> - CastNull);
>> + PHI->addIncoming(llvm::Constant::getNullValue(Value->getType()), CastNull);
>> Value = PHI;
>> }
>>
>> - return Value;
>> + return Address(Value, CGM.getClassPointerAlignment(Derived));
>> }
>>
>> llvm::Value *CodeGenFunction::GetVTTParameter(GlobalDecl GD,
>> @@ -356,8 +482,8 @@ namespace {
>> cast<CXXMethodDecl>(CGF.CurCodeDecl)->getParent();
>>
>> const CXXDestructorDecl *D = BaseClass->getDestructor();
>> - llvm::Value *Addr =
>> - CGF.GetAddressOfDirectBaseInCompleteClass(CGF.LoadCXXThis(),
>> + Address Addr =
>> + CGF.GetAddressOfDirectBaseInCompleteClass(CGF.LoadCXXThisAddress(),
>> DerivedClass, BaseClass,
>> BaseIsVirtual);
>> CGF.EmitCXXDestructorCall(D, Dtor_Base, BaseIsVirtual,
>> @@ -396,7 +522,7 @@ static void EmitBaseInitializer(CodeGenF
>> assert(BaseInit->isBaseInitializer() &&
>> "Must have base initializer!");
>>
>> - llvm::Value *ThisPtr = CGF.LoadCXXThis();
>> + Address ThisPtr = CGF.LoadCXXThisAddress();
>>
>> const Type *BaseType = BaseInit->getBaseClass();
>> CXXRecordDecl *BaseClassDecl =
>> @@ -416,13 +542,12 @@ static void EmitBaseInitializer(CodeGenF
>>
>> // We can pretend to be a complete class because it only matters for
>> // virtual bases, and we only do virtual bases for complete ctors.
>> - llvm::Value *V =
>> + Address V =
>> CGF.GetAddressOfDirectBaseInCompleteClass(ThisPtr, ClassDecl,
>> BaseClassDecl,
>> isBaseVirtual);
>> - CharUnits Alignment = CGF.getContext().getTypeAlignInChars(BaseType);
>> AggValueSlot AggSlot =
>> - AggValueSlot::forAddr(V, Alignment, Qualifiers(),
>> + AggValueSlot::forAddr(V, Qualifiers(),
>> AggValueSlot::IsDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsNotAliased);
>> @@ -438,17 +563,17 @@ static void EmitBaseInitializer(CodeGenF
>> static void EmitAggMemberInitializer(CodeGenFunction &CGF,
>> LValue LHS,
>> Expr *Init,
>> - llvm::Value *ArrayIndexVar,
>> + Address ArrayIndexVar,
>> QualType T,
>> ArrayRef<VarDecl *> ArrayIndexes,
>> unsigned Index) {
>> if (Index == ArrayIndexes.size()) {
>> LValue LV = LHS;
>>
>> - if (ArrayIndexVar) {
>> + if (ArrayIndexVar.isValid()) {
>> // If we have an array index variable, load it and use it as an offset.
>> // Then, increment the value.
>> - llvm::Value *Dest = LHS.getAddress();
>> + llvm::Value *Dest = LHS.getPointer();
>> llvm::Value *ArrayIndex = CGF.Builder.CreateLoad(ArrayIndexVar);
>> Dest = CGF.Builder.CreateInBoundsGEP(Dest, ArrayIndex, "destaddress");
>> llvm::Value *Next = llvm::ConstantInt::get(ArrayIndex->getType(), 1);
>> @@ -456,9 +581,9 @@ static void EmitAggMemberInitializer(Cod
>> CGF.Builder.CreateStore(Next, ArrayIndexVar);
>>
>> // Update the LValue.
>> - LV.setAddress(Dest);
>> - CharUnits Align = CGF.getContext().getTypeAlignInChars(T);
>> - LV.setAlignment(std::min(Align, LV.getAlignment()));
>> + CharUnits EltSize = CGF.getContext().getTypeSizeInChars(T);
>> + CharUnits Align = LV.getAlignment().alignmentOfArrayElement(EltSize);
>> + LV.setAddress(Address(Dest, Align));
>> }
>>
>> switch (CGF.getEvaluationKind(T)) {
>> @@ -485,14 +610,11 @@ static void EmitAggMemberInitializer(Cod
>>
>> const ConstantArrayType *Array = CGF.getContext().getAsConstantArrayType(T);
>> assert(Array && "Array initialization without the array type?");
>> - llvm::Value *IndexVar
>> - = CGF.GetAddrOfLocalVar(ArrayIndexes[Index]);
>> - assert(IndexVar && "Array index variable not loaded");
>> + Address IndexVar = CGF.GetAddrOfLocalVar(ArrayIndexes[Index]);
>>
>> // Initialize this index variable to zero.
>> llvm::Value* Zero
>> - = llvm::Constant::getNullValue(
>> - CGF.ConvertType(CGF.getContext().getSizeType()));
>> + = llvm::Constant::getNullValue(IndexVar.getElementType());
>> CGF.Builder.CreateStore(Zero, IndexVar);
>>
>> // Start the loop with a block that tests the condition.
>> @@ -626,9 +748,8 @@ static void EmitMemberInitializer(CodeGe
>> CGF.EmitInitializerForField(Field, LHS, MemberInit->getInit(), ArrayIndexes);
>> }
>>
>> -void CodeGenFunction::EmitInitializerForField(
>> - FieldDecl *Field, LValue LHS, Expr *Init,
>> - ArrayRef<VarDecl *> ArrayIndexes) {
>> +void CodeGenFunction::EmitInitializerForField(FieldDecl *Field, LValue LHS,
>> + Expr *Init, ArrayRef<VarDecl *> ArrayIndexes) {
>> QualType FieldType = Field->getType();
>> switch (getEvaluationKind(FieldType)) {
>> case TEK_Scalar:
>> @@ -643,26 +764,23 @@ void CodeGenFunction::EmitInitializerFor
>> EmitComplexExprIntoLValue(Init, LHS, /*isInit*/ true);
>> break;
>> case TEK_Aggregate: {
>> - llvm::Value *ArrayIndexVar = nullptr;
>> + Address ArrayIndexVar = Address::invalid();
>> if (ArrayIndexes.size()) {
>> - llvm::Type *SizeTy = ConvertType(getContext().getSizeType());
>> -
>> // The LHS is a pointer to the first object we'll be constructing, as
>> // a flat array.
>> QualType BaseElementTy = getContext().getBaseElementType(FieldType);
>> llvm::Type *BasePtr = ConvertType(BaseElementTy);
>> BasePtr = llvm::PointerType::getUnqual(BasePtr);
>> - llvm::Value *BaseAddrPtr = Builder.CreateBitCast(LHS.getAddress(),
>> - BasePtr);
>> + Address BaseAddrPtr = Builder.CreateBitCast(LHS.getAddress(), BasePtr);
>> LHS = MakeAddrLValue(BaseAddrPtr, BaseElementTy);
>>
>> // Create an array index that will be used to walk over all of the
>> // objects we're constructing.
>> - ArrayIndexVar = CreateTempAlloca(SizeTy, "object.index");
>> - llvm::Value *Zero = llvm::Constant::getNullValue(SizeTy);
>> + ArrayIndexVar = CreateMemTemp(getContext().getSizeType(), "object.index");
>> + llvm::Value *Zero =
>> + llvm::Constant::getNullValue(ArrayIndexVar.getElementType());
>> Builder.CreateStore(Zero, ArrayIndexVar);
>>
>> -
>> // Emit the block variables for the array indices, if any.
>> for (unsigned I = 0, N = ArrayIndexes.size(); I != N; ++I)
>> EmitAutoVarDecl(*ArrayIndexes[I]);
>> @@ -930,19 +1048,16 @@ namespace {
>>
>> CharUnits MemcpySize = getMemcpySize(FirstByteOffset);
>> QualType RecordTy = CGF.getContext().getTypeDeclType(ClassDecl);
>> - llvm::Value *ThisPtr = CGF.LoadCXXThis();
>> - LValue DestLV = CGF.MakeNaturalAlignAddrLValue(ThisPtr, RecordTy);
>> + Address ThisPtr = CGF.LoadCXXThisAddress();
>> + LValue DestLV = CGF.MakeAddrLValue(ThisPtr, RecordTy);
>> LValue Dest = CGF.EmitLValueForFieldInitialization(DestLV, FirstField);
>> llvm::Value *SrcPtr = CGF.Builder.CreateLoad(CGF.GetAddrOfLocalVar(SrcRec));
>> LValue SrcLV = CGF.MakeNaturalAlignAddrLValue(SrcPtr, RecordTy);
>> LValue Src = CGF.EmitLValueForFieldInitialization(SrcLV, FirstField);
>>
>> - CharUnits Offset = CGF.getContext().toCharUnitsFromBits(FirstByteOffset);
>> - CharUnits Alignment = DestLV.getAlignment().alignmentAtOffset(Offset);
>> -
>> - emitMemcpyIR(Dest.isBitField() ? Dest.getBitFieldAddr() : Dest.getAddress(),
>> - Src.isBitField() ? Src.getBitFieldAddr() : Src.getAddress(),
>> - MemcpySize, Alignment);
>> + emitMemcpyIR(Dest.isBitField() ? Dest.getBitFieldAddress() : Dest.getAddress(),
>> + Src.isBitField() ? Src.getBitFieldAddress() : Src.getAddress(),
>> + MemcpySize);
>> reset();
>> }
>>
>> @@ -956,20 +1071,18 @@ namespace {
>>
>> private:
>>
>> - void emitMemcpyIR(llvm::Value *DestPtr, llvm::Value *SrcPtr,
>> - CharUnits Size, CharUnits Alignment) {
>> - llvm::PointerType *DPT = cast<llvm::PointerType>(DestPtr->getType());
>> + void emitMemcpyIR(Address DestPtr, Address SrcPtr, CharUnits Size) {
>> + llvm::PointerType *DPT = DestPtr.getType();
>> llvm::Type *DBP =
>> llvm::Type::getInt8PtrTy(CGF.getLLVMContext(), DPT->getAddressSpace());
>> DestPtr = CGF.Builder.CreateBitCast(DestPtr, DBP);
>>
>> - llvm::PointerType *SPT = cast<llvm::PointerType>(SrcPtr->getType());
>> + llvm::PointerType *SPT = SrcPtr.getType();
>> llvm::Type *SBP =
>> llvm::Type::getInt8PtrTy(CGF.getLLVMContext(), SPT->getAddressSpace());
>> SrcPtr = CGF.Builder.CreateBitCast(SrcPtr, SBP);
>>
>> - CGF.Builder.CreateMemCpy(DestPtr, SrcPtr, Size.getQuantity(),
>> - Alignment.getQuantity());
>> + CGF.Builder.CreateMemCpy(DestPtr, SrcPtr, Size.getQuantity());
>> }
>>
>> void addInitialField(FieldDecl *F) {
>> @@ -1089,9 +1202,9 @@ namespace {
>> }
>>
>> void pushEHDestructors() {
>> - llvm::Value *ThisPtr = CGF.LoadCXXThis();
>> + Address ThisPtr = CGF.LoadCXXThisAddress();
>> QualType RecordTy = CGF.getContext().getTypeDeclType(ClassDecl);
>> - LValue LHS = CGF.MakeNaturalAlignAddrLValue(ThisPtr, RecordTy);
>> + LValue LHS = CGF.MakeAddrLValue(ThisPtr, RecordTy);
>>
>> for (unsigned i = 0; i < AggregatedInits.size(); ++i) {
>> CXXCtorInitializer *MemberInit = AggregatedInits[i];
>> @@ -1274,7 +1387,7 @@ void CodeGenFunction::EmitCtorPrologue(c
>> InitializeVTablePointers(ClassDecl);
>>
>> // And finally, initialize class members.
>> - FieldConstructionScope FCS(*this, CXXThisValue);
>> + FieldConstructionScope FCS(*this, LoadCXXThisAddress());
>> ConstructorMemcpyizer CM(*this, CD, Args);
>> for (; B != E; B++) {
>> CXXCtorInitializer *Member = (*B);
>> @@ -1383,7 +1496,7 @@ void CodeGenFunction::EmitDestructorBody
>> if (DtorType == Dtor_Deleting) {
>> EnterDtorCleanups(Dtor, Dtor_Deleting);
>> EmitCXXDestructorCall(Dtor, Dtor_Complete, /*ForVirtualBase=*/false,
>> - /*Delegating=*/false, LoadCXXThis());
>> + /*Delegating=*/false, LoadCXXThisAddress());
>> PopCleanupBlock();
>> return;
>> }
>> @@ -1418,7 +1531,7 @@ void CodeGenFunction::EmitDestructorBody
>>
>> if (!isTryBody) {
>> EmitCXXDestructorCall(Dtor, Dtor_Base, /*ForVirtualBase=*/false,
>> - /*Delegating=*/false, LoadCXXThis());
>> + /*Delegating=*/false, LoadCXXThisAddress());
>> break;
>> }
>> // Fallthrough: act like we're in the base variant.
>> @@ -1524,7 +1637,7 @@ namespace {
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> // Find the address of the field.
>> - llvm::Value *thisValue = CGF.LoadCXXThis();
>> + Address thisValue = CGF.LoadCXXThisAddress();
>> QualType RecordTy = CGF.getContext().getTagDeclType(field->getParent());
>> LValue ThisLV = CGF.MakeAddrLValue(thisValue, RecordTy);
>> LValue LV = CGF.EmitLValueForField(ThisLV, field);
>> @@ -1738,7 +1851,7 @@ void CodeGenFunction::EnterDtorCleanups(
>> /// zero-initialized before it is constructed
>> void CodeGenFunction::EmitCXXAggrConstructorCall(
>> const CXXConstructorDecl *ctor, const ConstantArrayType *arrayType,
>> - llvm::Value *arrayBegin, const CXXConstructExpr *E, bool zeroInitialize) {
>> + Address arrayBegin, const CXXConstructExpr *E, bool zeroInitialize) {
>> QualType elementType;
>> llvm::Value *numElements =
>> emitArrayLength(arrayType, elementType, arrayBegin);
>> @@ -1757,7 +1870,7 @@ void CodeGenFunction::EmitCXXAggrConstru
>> /// zero-initialized before it is constructed
>> void CodeGenFunction::EmitCXXAggrConstructorCall(const CXXConstructorDecl *ctor,
>> llvm::Value *numElements,
>> - llvm::Value *arrayBegin,
>> + Address arrayBase,
>> const CXXConstructExpr *E,
>> bool zeroInitialize) {
>>
>> @@ -1784,6 +1897,7 @@ void CodeGenFunction::EmitCXXAggrConstru
>> }
>>
>> // Find the end of the array.
>> + llvm::Value *arrayBegin = arrayBase.getPointer();
>> llvm::Value *arrayEnd = Builder.CreateInBoundsGEP(arrayBegin, numElements,
>> "arrayctor.end");
>>
>> @@ -1797,11 +1911,21 @@ void CodeGenFunction::EmitCXXAggrConstru
>>
>> // Inside the loop body, emit the constructor call on the array element.
>>
>> + // The alignment of the base, adjusted by the size of a single element,
>> + // provides a conservative estimate of the alignment of every element.
>> + // (This assumes we never start tracking offsetted alignments.)
>> + //
>> + // Note that these are complete objects and so we don't need to
>> + // use the non-virtual size or alignment.
>> QualType type = getContext().getTypeDeclType(ctor->getParent());
>> + CharUnits eltAlignment =
>> + arrayBase.getAlignment()
>> + .alignmentOfArrayElement(getContext().getTypeSizeInChars(type));
>> + Address curAddr = Address(cur, eltAlignment);
>>
>> // Zero initialize the storage, if requested.
>> if (zeroInitialize)
>> - EmitNullInitialization(cur, type);
>> + EmitNullInitialization(curAddr, type);
>>
>> // C++ [class.temporary]p4:
>> // There are two contexts in which temporaries are destroyed at a different
>> @@ -1819,11 +1943,12 @@ void CodeGenFunction::EmitCXXAggrConstru
>> if (getLangOpts().Exceptions &&
>> !ctor->getParent()->hasTrivialDestructor()) {
>> Destroyer *destroyer = destroyCXXObject;
>> - pushRegularPartialArrayCleanup(arrayBegin, cur, type, *destroyer);
>> + pushRegularPartialArrayCleanup(arrayBegin, cur, type, eltAlignment,
>> + *destroyer);
>> }
>>
>> EmitCXXConstructorCall(ctor, Ctor_Complete, /*ForVirtualBase=*/false,
>> - /*Delegating=*/false, cur, E);
>> + /*Delegating=*/false, curAddr, E);
>> }
>>
>> // Go to the next element.
>> @@ -1844,7 +1969,7 @@ void CodeGenFunction::EmitCXXAggrConstru
>> }
>>
>> void CodeGenFunction::destroyCXXObject(CodeGenFunction &CGF,
>> - llvm::Value *addr,
>> + Address addr,
>> QualType type) {
>> const RecordType *rtype = type->castAs<RecordType>();
>> const CXXRecordDecl *record = cast<CXXRecordDecl>(rtype->getDecl());
>> @@ -1857,14 +1982,14 @@ void CodeGenFunction::destroyCXXObject(C
>> void CodeGenFunction::EmitCXXConstructorCall(const CXXConstructorDecl *D,
>> CXXCtorType Type,
>> bool ForVirtualBase,
>> - bool Delegating, llvm::Value *This,
>> + bool Delegating, Address This,
>> const CXXConstructExpr *E) {
>> // C++11 [class.mfct.non-static]p2:
>> // If a non-static member function of a class X is called for an object that
>> // is not of type X, or of a type derived from X, the behavior is undefined.
>> // FIXME: Provide a source location here.
>> - EmitTypeCheck(CodeGenFunction::TCK_ConstructorCall, SourceLocation(), This,
>> - getContext().getRecordType(D->getParent()));
>> + EmitTypeCheck(CodeGenFunction::TCK_ConstructorCall, SourceLocation(),
>> + This.getPointer(), getContext().getRecordType(D->getParent()));
>>
>> if (D->isTrivial() && D->isDefaultConstructor()) {
>> assert(E->getNumArgs() == 0 && "trivial default ctor with args");
>> @@ -1879,7 +2004,7 @@ void CodeGenFunction::EmitCXXConstructor
>>
>> const Expr *Arg = E->getArg(0);
>> QualType SrcTy = Arg->getType();
>> - llvm::Value *Src = EmitLValue(Arg).getAddress();
>> + Address Src = EmitLValue(Arg).getAddress();
>> QualType DestTy = getContext().getTypeDeclType(D->getParent());
>> EmitAggregateCopyCtor(This, Src, DestTy, SrcTy);
>> return;
>> @@ -1888,7 +2013,7 @@ void CodeGenFunction::EmitCXXConstructor
>> CallArgList Args;
>>
>> // Push the this ptr.
>> - Args.add(RValue::get(This), D->getThisType(getContext()));
>> + Args.add(RValue::get(This.getPointer()), D->getThisType(getContext()));
>>
>> // Add the rest of the user-supplied arguments.
>> const FunctionProtoType *FPT = D->getType()->castAs<FunctionProtoType>();
>> @@ -1907,8 +2032,8 @@ void CodeGenFunction::EmitCXXConstructor
>>
>> void
>> CodeGenFunction::EmitSynthesizedCXXCopyCtorCall(const CXXConstructorDecl *D,
>> - llvm::Value *This, llvm::Value *Src,
>> - const CXXConstructExpr *E) {
>> + Address This, Address Src,
>> + const CXXConstructExpr *E) {
>> if (isMemcpyEquivalentSpecialMember(D)) {
>> assert(E->getNumArgs() == 1 && "unexpected argcount for trivial ctor");
>> assert(D->isCopyOrMoveConstructor() &&
>> @@ -1927,13 +2052,13 @@ CodeGenFunction::EmitSynthesizedCXXCopyC
>> CallArgList Args;
>>
>> // Push the this ptr.
>> - Args.add(RValue::get(This), D->getThisType(getContext()));
>> + Args.add(RValue::get(This.getPointer()), D->getThisType(getContext()));
>>
>> // Push the src ptr.
>> QualType QT = *(FPT->param_type_begin());
>> llvm::Type *t = CGM.getTypes().ConvertType(QT);
>> Src = Builder.CreateBitCast(Src, t);
>> - Args.add(RValue::get(Src), QT);
>> + Args.add(RValue::get(Src.getPointer()), QT);
>>
>> // Skip over first argument (Src).
>> EmitCallArgs(Args, FPT, drop_begin(E->arguments(), 1), E->getConstructor(),
>> @@ -1988,10 +2113,10 @@ CodeGenFunction::EmitDelegateCXXConstruc
>> namespace {
>> struct CallDelegatingCtorDtor final : EHScopeStack::Cleanup {
>> const CXXDestructorDecl *Dtor;
>> - llvm::Value *Addr;
>> + Address Addr;
>> CXXDtorType Type;
>>
>> - CallDelegatingCtorDtor(const CXXDestructorDecl *D, llvm::Value *Addr,
>> + CallDelegatingCtorDtor(const CXXDestructorDecl *D, Address Addr,
>> CXXDtorType Type)
>> : Dtor(D), Addr(Addr), Type(Type) {}
>>
>> @@ -2007,12 +2132,10 @@ CodeGenFunction::EmitDelegatingCXXConstr
>> const FunctionArgList &Args) {
>> assert(Ctor->isDelegatingConstructor());
>>
>> - llvm::Value *ThisPtr = LoadCXXThis();
>> + Address ThisPtr = LoadCXXThisAddress();
>>
>> - QualType Ty = getContext().getTagDeclType(Ctor->getParent());
>> - CharUnits Alignment = getContext().getTypeAlignInChars(Ty);
>> AggValueSlot AggSlot =
>> - AggValueSlot::forAddr(ThisPtr, Alignment, Qualifiers(),
>> + AggValueSlot::forAddr(ThisPtr, Qualifiers(),
>> AggValueSlot::IsDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsNotAliased);
>> @@ -2034,7 +2157,7 @@ void CodeGenFunction::EmitCXXDestructorC
>> CXXDtorType Type,
>> bool ForVirtualBase,
>> bool Delegating,
>> - llvm::Value *This) {
>> + Address This) {
>> CGM.getCXXABI().EmitDestructorCall(*this, DD, Type, ForVirtualBase,
>> Delegating, This);
>> }
>> @@ -2042,9 +2165,9 @@ void CodeGenFunction::EmitCXXDestructorC
>> namespace {
>> struct CallLocalDtor final : EHScopeStack::Cleanup {
>> const CXXDestructorDecl *Dtor;
>> - llvm::Value *Addr;
>> + Address Addr;
>>
>> - CallLocalDtor(const CXXDestructorDecl *D, llvm::Value *Addr)
>> + CallLocalDtor(const CXXDestructorDecl *D, Address Addr)
>> : Dtor(D), Addr(Addr) {}
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> @@ -2056,11 +2179,11 @@ namespace {
>> }
>>
>> void CodeGenFunction::PushDestructorCleanup(const CXXDestructorDecl *D,
>> - llvm::Value *Addr) {
>> + Address Addr) {
>> EHStack.pushCleanup<CallLocalDtor>(NormalAndEHCleanup, D, Addr);
>> }
>>
>> -void CodeGenFunction::PushDestructorCleanup(QualType T, llvm::Value *Addr) {
>> +void CodeGenFunction::PushDestructorCleanup(QualType T, Address Addr) {
>> CXXRecordDecl *ClassDecl = T->getAsCXXRecordDecl();
>> if (!ClassDecl) return;
>> if (ClassDecl->hasTrivialDestructor()) return;
>> @@ -2098,10 +2221,9 @@ CodeGenFunction::InitializeVTablePointer
>> if (NeedsVirtualOffset) {
>> // We need to use the virtual base offset offset because the virtual base
>> // might have a different offset in the most derived class.
>> - VirtualOffset = CGM.getCXXABI().GetVirtualBaseClassOffset(*this,
>> - LoadCXXThis(),
>> - VTableClass,
>> - NearestVBase);
>> + VirtualOffset =
>> + CGM.getCXXABI().GetVirtualBaseClassOffset(*this, LoadCXXThisAddress(),
>> + VTableClass, NearestVBase);
>> NonVirtualOffset = OffsetFromNearestVBase;
>> } else {
>> // We can just use the base offset in the complete class.
>> @@ -2109,12 +2231,14 @@ CodeGenFunction::InitializeVTablePointer
>> }
>>
>> // Apply the offsets.
>> - llvm::Value *VTableField = LoadCXXThis();
>> + Address VTableField = LoadCXXThisAddress();
>>
>> if (!NonVirtualOffset.isZero() || VirtualOffset)
>> VTableField = ApplyNonVirtualAndVirtualOffset(*this, VTableField,
>> NonVirtualOffset,
>> - VirtualOffset);
>> + VirtualOffset,
>> + VTableClass,
>> + NearestVBase);
>>
>> // Finally, store the address point. Use the same LLVM types as the field to
>> // support optimization.
>> @@ -2202,9 +2326,9 @@ void CodeGenFunction::InitializeVTablePo
>> CGM.getCXXABI().initializeHiddenVirtualInheritanceMembers(*this, RD);
>> }
>>
>> -llvm::Value *CodeGenFunction::GetVTablePtr(llvm::Value *This,
>> +llvm::Value *CodeGenFunction::GetVTablePtr(Address This,
>> llvm::Type *Ty) {
>> - llvm::Value *VTablePtrSrc = Builder.CreateBitCast(This, Ty->getPointerTo());
>> + Address VTablePtrSrc = Builder.CreateElementBitCast(This, Ty);
>> llvm::Instruction *VTable = Builder.CreateLoad(VTablePtrSrc, "vtable");
>> CGM.DecorateInstruction(VTable, CGM.getTBAAInfoForVTablePtr());
>> return VTable;
>> @@ -2290,7 +2414,8 @@ void CodeGenFunction::EmitVTablePtrCheck
>> EmitBlock(CheckBlock);
>> }
>>
>> - llvm::Value *VTable = GetVTablePtr(Derived, Int8PtrTy);
>> + llvm::Value *VTable =
>> + GetVTablePtr(Address(Derived, getPointerAlign()), Int8PtrTy);
>> EmitVTablePtrCheck(ClassDecl, VTable, TCK, Loc);
>>
>> if (MayBeNull) {
>> @@ -2479,8 +2604,8 @@ void CodeGenFunction::EmitLambdaBlockInv
>> CallArgList CallArgs;
>>
>> QualType ThisType = getContext().getPointerType(getContext().getRecordType(Lambda));
>> - llvm::Value *ThisPtr = GetAddrOfBlockDecl(variable, false);
>> - CallArgs.add(RValue::get(ThisPtr), ThisType);
>> + Address ThisPtr = GetAddrOfBlockDecl(variable, false);
>> + CallArgs.add(RValue::get(ThisPtr.getPointer()), ThisType);
>>
>> // Add the rest of the parameters.
>> for (auto param : BD->params())
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCleanup.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCleanup.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCleanup.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGCleanup.cpp Tue Sep 8 03:05:57 2015
>> @@ -27,7 +27,7 @@ bool DominatingValue<RValue>::saved_type
>> if (rv.isScalar())
>> return DominatingLLVMValue::needsSaving(rv.getScalarVal());
>> if (rv.isAggregate())
>> - return DominatingLLVMValue::needsSaving(rv.getAggregateAddr());
>> + return DominatingLLVMValue::needsSaving(rv.getAggregatePointer());
>> return true;
>> }
>>
>> @@ -41,9 +41,10 @@ DominatingValue<RValue>::saved_type::sav
>> return saved_type(V, ScalarLiteral);
>>
>> // Everything else needs an alloca.
>> - llvm::Value *addr = CGF.CreateTempAlloca(V->getType(), "saved-rvalue");
>> + Address addr =
>> + CGF.CreateDefaultAlignTempAlloca(V->getType(), "saved-rvalue");
>> CGF.Builder.CreateStore(V, addr);
>> - return saved_type(addr, ScalarAddress);
>> + return saved_type(addr.getPointer(), ScalarAddress);
>> }
>>
>> if (rv.isComplex()) {
>> @@ -51,42 +52,56 @@ DominatingValue<RValue>::saved_type::sav
>> llvm::Type *ComplexTy =
>> llvm::StructType::get(V.first->getType(), V.second->getType(),
>> (void*) nullptr);
>> - llvm::Value *addr = CGF.CreateTempAlloca(ComplexTy, "saved-complex");
>> + Address addr = CGF.CreateDefaultAlignTempAlloca(ComplexTy, "saved-complex");
>> CGF.Builder.CreateStore(V.first,
>> - CGF.Builder.CreateStructGEP(ComplexTy, addr, 0));
>> + CGF.Builder.CreateStructGEP(addr, 0, CharUnits()));
>> + CharUnits offset = CharUnits::fromQuantity(
>> + CGF.CGM.getDataLayout().getTypeAllocSize(V.first->getType()));
>> CGF.Builder.CreateStore(V.second,
>> - CGF.Builder.CreateStructGEP(ComplexTy, addr, 1));
>> - return saved_type(addr, ComplexAddress);
>> + CGF.Builder.CreateStructGEP(addr, 1, offset));
>> + return saved_type(addr.getPointer(), ComplexAddress);
>> }
>>
>> assert(rv.isAggregate());
>> - llvm::Value *V = rv.getAggregateAddr(); // TODO: volatile?
>> - if (!DominatingLLVMValue::needsSaving(V))
>> - return saved_type(V, AggregateLiteral);
>> -
>> - llvm::Value *addr = CGF.CreateTempAlloca(V->getType(), "saved-rvalue");
>> - CGF.Builder.CreateStore(V, addr);
>> - return saved_type(addr, AggregateAddress);
>> + Address V = rv.getAggregateAddress(); // TODO: volatile?
>> + if (!DominatingLLVMValue::needsSaving(V.getPointer()))
>> + return saved_type(V.getPointer(), AggregateLiteral,
>> + V.getAlignment().getQuantity());
>> +
>> + Address addr =
>> + CGF.CreateTempAlloca(V.getType(), CGF.getPointerAlign(), "saved-rvalue");
>> + CGF.Builder.CreateStore(V.getPointer(), addr);
>> + return saved_type(addr.getPointer(), AggregateAddress,
>> + V.getAlignment().getQuantity());
>> }
>>
>> /// Given a saved r-value produced by SaveRValue, perform the code
>> /// necessary to restore it to usability at the current insertion
>> /// point.
>> RValue DominatingValue<RValue>::saved_type::restore(CodeGenFunction &CGF) {
>> + auto getSavingAddress = [&](llvm::Value *value) {
>> + auto alignment = cast<llvm::AllocaInst>(value)->getAlignment();
>> + return Address(value, CharUnits::fromQuantity(alignment));
>> + };
>> switch (K) {
>> case ScalarLiteral:
>> return RValue::get(Value);
>> case ScalarAddress:
>> - return RValue::get(CGF.Builder.CreateLoad(Value));
>> + return RValue::get(CGF.Builder.CreateLoad(getSavingAddress(Value)));
>> case AggregateLiteral:
>> - return RValue::getAggregate(Value);
>> - case AggregateAddress:
>> - return RValue::getAggregate(CGF.Builder.CreateLoad(Value));
>> + return RValue::getAggregate(Address(Value, CharUnits::fromQuantity(Align)));
>> + case AggregateAddress: {
>> + auto addr = CGF.Builder.CreateLoad(getSavingAddress(Value));
>> + return RValue::getAggregate(Address(addr, CharUnits::fromQuantity(Align)));
>> + }
>> case ComplexAddress: {
>> - llvm::Value *real =
>> - CGF.Builder.CreateLoad(CGF.Builder.CreateStructGEP(nullptr, Value, 0));
>> - llvm::Value *imag =
>> - CGF.Builder.CreateLoad(CGF.Builder.CreateStructGEP(nullptr, Value, 1));
>> + Address address = getSavingAddress(Value);
>> + llvm::Value *real = CGF.Builder.CreateLoad(
>> + CGF.Builder.CreateStructGEP(address, 0, CharUnits()));
>> + CharUnits offset = CharUnits::fromQuantity(
>> + CGF.CGM.getDataLayout().getTypeAllocSize(real->getType()));
>> + llvm::Value *imag = CGF.Builder.CreateLoad(
>> + CGF.Builder.CreateStructGEP(address, 1, offset));
>> return RValue::getComplex(real, imag);
>> }
>> }
>> @@ -275,8 +290,8 @@ void EHScopeStack::popNullFixups() {
>>
>> void CodeGenFunction::initFullExprCleanup() {
>> // Create a variable to decide whether the cleanup needs to be run.
>> - llvm::AllocaInst *active
>> - = CreateTempAlloca(Builder.getInt1Ty(), "cleanup.cond");
>> + Address active = CreateTempAlloca(Builder.getInt1Ty(), CharUnits::One(),
>> + "cleanup.cond");
>>
>> // Initialize it to false at a site that's guaranteed to be run
>> // before each evaluation.
>> @@ -287,7 +302,7 @@ void CodeGenFunction::initFullExprCleanu
>>
>> // Set that as the active flag in the cleanup.
>> EHCleanupScope &cleanup = cast<EHCleanupScope>(*EHStack.begin());
>> - assert(!cleanup.getActiveFlag() && "cleanup already has active flag?");
>> + assert(!cleanup.hasActiveFlag() && "cleanup already has active flag?");
>> cleanup.setActiveFlag(active);
>>
>> if (cleanup.isNormalCleanup()) cleanup.setTestFlagInNormalCleanup();
>> @@ -296,6 +311,19 @@ void CodeGenFunction::initFullExprCleanu
>>
>> void EHScopeStack::Cleanup::anchor() {}
>>
>> +static void createStoreInstBefore(llvm::Value *value, Address addr,
>> + llvm::Instruction *beforeInst) {
>> + auto store = new llvm::StoreInst(value, addr.getPointer(), beforeInst);
>> + store->setAlignment(addr.getAlignment().getQuantity());
>> +}
>> +
>> +static llvm::LoadInst *createLoadInstBefore(Address addr, const Twine &name,
>> + llvm::Instruction *beforeInst) {
>> + auto load = new llvm::LoadInst(addr.getPointer(), name, beforeInst);
>> + load->setAlignment(addr.getAlignment().getQuantity());
>> + return load;
>> +}
>> +
>> /// All the branch fixups on the EH stack have propagated out past the
>> /// outermost normal cleanup; resolve them all by adding cases to the
>> /// given switch instruction.
>> @@ -318,9 +346,9 @@ static void ResolveAllBranchFixups(CodeG
>> // i.e. where there's an unresolved fixup inside a single cleanup
>> // entry which we're currently popping.
>> if (Fixup.OptimisticBranchBlock == nullptr) {
>> - new llvm::StoreInst(CGF.Builder.getInt32(Fixup.DestinationIndex),
>> - CGF.getNormalCleanupDestSlot(),
>> - Fixup.InitialBranch);
>> + createStoreInstBefore(CGF.Builder.getInt32(Fixup.DestinationIndex),
>> + CGF.getNormalCleanupDestSlot(),
>> + Fixup.InitialBranch);
>> Fixup.InitialBranch->setSuccessor(0, CleanupEntry);
>> }
>>
>> @@ -346,8 +374,8 @@ static llvm::SwitchInst *TransitionToCle
>>
>> if (llvm::BranchInst *Br = dyn_cast<llvm::BranchInst>(Term)) {
>> assert(Br->isUnconditional());
>> - llvm::LoadInst *Load =
>> - new llvm::LoadInst(CGF.getNormalCleanupDestSlot(), "cleanup.dest", Term);
>> + auto Load = createLoadInstBefore(CGF.getNormalCleanupDestSlot(),
>> + "cleanup.dest", Term);
>> llvm::SwitchInst *Switch =
>> llvm::SwitchInst::Create(Load, Br->getSuccessor(0), 4, Block);
>> Br->eraseFromParent();
>> @@ -492,7 +520,7 @@ static llvm::BasicBlock *SimplifyCleanup
>> static void EmitCleanup(CodeGenFunction &CGF,
>> EHScopeStack::Cleanup *Fn,
>> EHScopeStack::Cleanup::Flags flags,
>> - llvm::Value *ActiveFlag) {
>> + Address ActiveFlag) {
>> // Itanium EH cleanups occur within a terminate scope. Microsoft SEH doesn't
>> // have this behavior, and the Microsoft C++ runtime will call terminate for
>> // us if the cleanup throws.
>> @@ -505,7 +533,7 @@ static void EmitCleanup(CodeGenFunction
>> // If there's an active flag, load it and skip the cleanup if it's
>> // false.
>> llvm::BasicBlock *ContBB = nullptr;
>> - if (ActiveFlag) {
>> + if (ActiveFlag.isValid()) {
>> ContBB = CGF.createBasicBlock("cleanup.done");
>> llvm::BasicBlock *CleanupBB = CGF.createBasicBlock("cleanup.action");
>> llvm::Value *IsActive
>> @@ -519,7 +547,7 @@ static void EmitCleanup(CodeGenFunction
>> assert(CGF.HaveInsertPoint() && "cleanup ended with no insertion point?");
>>
>> // Emit the continuation block if there was an active flag.
>> - if (ActiveFlag)
>> + if (ActiveFlag.isValid())
>> CGF.EmitBlock(ContBB);
>>
>> // Leave the terminate scope.
>> @@ -599,10 +627,12 @@ void CodeGenFunction::PopCleanupBlock(bo
>>
>> // Remember activation information.
>> bool IsActive = Scope.isActive();
>> - llvm::Value *NormalActiveFlag =
>> - Scope.shouldTestFlagInNormalCleanup() ? Scope.getActiveFlag() : nullptr;
>> - llvm::Value *EHActiveFlag =
>> - Scope.shouldTestFlagInEHCleanup() ? Scope.getActiveFlag() : nullptr;
>> + Address NormalActiveFlag =
>> + Scope.shouldTestFlagInNormalCleanup() ? Scope.getActiveFlag()
>> + : Address::invalid();
>> + Address EHActiveFlag =
>> + Scope.shouldTestFlagInEHCleanup() ? Scope.getActiveFlag()
>> + : Address::invalid();
>>
>> // Check whether we need an EH cleanup. This is only true if we've
>> // generated a lazy EH cleanup block.
>> @@ -769,7 +799,7 @@ void CodeGenFunction::PopCleanupBlock(bo
>>
>> // Clean up the possibly dead store to the cleanup dest slot.
>> llvm::Instruction *NormalCleanupDestSlot =
>> - cast<llvm::Instruction>(getNormalCleanupDestSlot());
>> + cast<llvm::Instruction>(getNormalCleanupDestSlot().getPointer());
>> if (NormalCleanupDestSlot->hasOneUse()) {
>> NormalCleanupDestSlot->user_back()->eraseFromParent();
>> NormalCleanupDestSlot->eraseFromParent();
>> @@ -795,7 +825,8 @@ void CodeGenFunction::PopCleanupBlock(bo
>> const unsigned SwitchCapacity = 10;
>>
>> llvm::LoadInst *Load =
>> - new llvm::LoadInst(getNormalCleanupDestSlot(), "cleanup.dest");
>> + createLoadInstBefore(getNormalCleanupDestSlot(), "cleanup.dest",
>> + nullptr);
>> llvm::SwitchInst *Switch =
>> llvm::SwitchInst::Create(Load, Default, SwitchCapacity);
>>
>> @@ -841,9 +872,9 @@ void CodeGenFunction::PopCleanupBlock(bo
>> BranchFixup &Fixup = EHStack.getBranchFixup(I);
>> if (!Fixup.Destination) continue;
>> if (!Fixup.OptimisticBranchBlock) {
>> - new llvm::StoreInst(Builder.getInt32(Fixup.DestinationIndex),
>> - getNormalCleanupDestSlot(),
>> - Fixup.InitialBranch);
>> + createStoreInstBefore(Builder.getInt32(Fixup.DestinationIndex),
>> + getNormalCleanupDestSlot(),
>> + Fixup.InitialBranch);
>> Fixup.InitialBranch->setSuccessor(0, NormalEntry);
>> }
>> Fixup.OptimisticBranchBlock = NormalExit;
>> @@ -908,8 +939,7 @@ void CodeGenFunction::PopCleanupBlock(bo
>>
>> // We only actually emit the cleanup code if the cleanup is either
>> // active or was used before it was deactivated.
>> - if (EHActiveFlag || IsActive) {
>> -
>> + if (EHActiveFlag.isValid() || IsActive) {
>> cleanupFlags.setIsForEHCleanup();
>> EmitCleanup(*this, Fn, cleanupFlags, EHActiveFlag);
>> }
>> @@ -993,7 +1023,7 @@ void CodeGenFunction::EmitBranchThroughC
>>
>> // Store the index at the start.
>> llvm::ConstantInt *Index = Builder.getInt32(Dest.getDestIndex());
>> - new llvm::StoreInst(Index, getNormalCleanupDestSlot(), BI);
>> + createStoreInstBefore(Index, getNormalCleanupDestSlot(), BI);
>>
>> // Adjust BI to point to the first cleanup block.
>> {
>> @@ -1112,23 +1142,24 @@ static void SetupCleanupBlockActivation(
>> // If it hasn't yet been used as either, we're done.
>> if (!needFlag) return;
>>
>> - llvm::AllocaInst *var = Scope.getActiveFlag();
>> - if (!var) {
>> - var = CGF.CreateTempAlloca(CGF.Builder.getInt1Ty(), "cleanup.isactive");
>> + Address var = Scope.getActiveFlag();
>> + if (!var.isValid()) {
>> + var = CGF.CreateTempAlloca(CGF.Builder.getInt1Ty(), CharUnits::One(),
>> + "cleanup.isactive");
>> Scope.setActiveFlag(var);
>>
>> assert(dominatingIP && "no existing variable and no dominating IP!");
>>
>> // Initialize to true or false depending on whether it was
>> // active up to this point.
>> - llvm::Value *value = CGF.Builder.getInt1(kind == ForDeactivation);
>> + llvm::Constant *value = CGF.Builder.getInt1(kind == ForDeactivation);
>>
>> // If we're in a conditional block, ignore the dominating IP and
>> // use the outermost conditional branch.
>> if (CGF.isInConditionalBranch()) {
>> CGF.setBeforeOutermostConditional(value, var);
>> } else {
>> - new llvm::StoreInst(value, var, dominatingIP);
>> + createStoreInstBefore(value, var, dominatingIP);
>> }
>> }
>>
>> @@ -1170,17 +1201,17 @@ void CodeGenFunction::DeactivateCleanupB
>> Scope.setActive(false);
>> }
>>
>> -llvm::Value *CodeGenFunction::getNormalCleanupDestSlot() {
>> +Address CodeGenFunction::getNormalCleanupDestSlot() {
>> if (!NormalCleanupDest)
>> NormalCleanupDest =
>> CreateTempAlloca(Builder.getInt32Ty(), "cleanup.dest.slot");
>> - return NormalCleanupDest;
>> + return Address(NormalCleanupDest, CharUnits::fromQuantity(4));
>> }
>>
>> /// Emits all the code to cause the given temporary to be cleaned up.
>> void CodeGenFunction::EmitCXXTemporary(const CXXTemporary *Temporary,
>> QualType TempType,
>> - llvm::Value *Ptr) {
>> + Address Ptr) {
>> pushDestroy(NormalAndEHCleanup, Ptr, TempType, destroyCXXObject,
>> /*useEHCleanup*/ true);
>> }
>>
>> Modified: cfe/trunk/lib/CodeGen/CGCleanup.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGCleanup.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGCleanup.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGCleanup.h Tue Sep 8 03:05:57 2015
>> @@ -15,6 +15,8 @@
>> #define LLVM_CLANG_LIB_CODEGEN_CGCLEANUP_H
>>
>> #include "EHScopeStack.h"
>> +
>> +#include "Address.h"
>> #include "llvm/ADT/SmallPtrSet.h"
>> #include "llvm/ADT/SmallVector.h"
>>
>> @@ -305,8 +307,14 @@ public:
>> bool isLifetimeMarker() const { return CleanupBits.IsLifetimeMarker; }
>> void setLifetimeMarker() { CleanupBits.IsLifetimeMarker = true; }
>>
>> - llvm::AllocaInst *getActiveFlag() const { return ActiveFlag; }
>> - void setActiveFlag(llvm::AllocaInst *Var) { ActiveFlag = Var; }
>> + bool hasActiveFlag() const { return ActiveFlag != nullptr; }
>> + Address getActiveFlag() const {
>> + return Address(ActiveFlag, CharUnits::One());
>> + }
>> + void setActiveFlag(Address Var) {
>> + assert(Var.getAlignment().isOne());
>> + ActiveFlag = cast<llvm::AllocaInst>(Var.getPointer());
>> + }
>>
>> void setTestFlagInNormalCleanup() {
>> CleanupBits.TestFlagInNormalCleanup = true;
>>
>> Modified: cfe/trunk/lib/CodeGen/CGDecl.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGDecl.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGDecl.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGDecl.cpp Tue Sep 8 03:05:57 2015
>> @@ -12,6 +12,7 @@
>> //===----------------------------------------------------------------------===//
>>
>> #include "CodeGenFunction.h"
>> +#include "CGBlocks.h"
>> #include "CGCleanup.h"
>> #include "CGDebugInfo.h"
>> #include "CGOpenCLRuntime.h"
>> @@ -340,17 +341,15 @@ CodeGenFunction::AddInitializerToStaticV
>>
>> void CodeGenFunction::EmitStaticVarDecl(const VarDecl &D,
>> llvm::GlobalValue::LinkageTypes Linkage) {
>> - llvm::Value *&DMEntry = LocalDeclMap[&D];
>> - assert(!DMEntry && "Decl already exists in localdeclmap!");
>> -
>> // Check to see if we already have a global variable for this
>> // declaration. This can happen when double-emitting function
>> // bodies, e.g. with complete and base constructors.
>> llvm::Constant *addr = CGM.getOrCreateStaticVarDecl(D, Linkage);
>> + CharUnits alignment = getContext().getDeclAlign(&D);
>>
>> // Store into LocalDeclMap before generating initializer to handle
>> // circular references.
>> - DMEntry = addr;
>> + setAddrOfLocalVar(&D, Address(addr, alignment));
>>
>> // We can't have a VLA here, but we can have a pointer to a VLA,
>> // even though that doesn't really make any sense.
>> @@ -367,7 +366,7 @@ void CodeGenFunction::EmitStaticVarDecl(
>> if (D.getInit())
>> var = AddInitializerToStaticVarDecl(D, var);
>>
>> - var->setAlignment(getContext().getDeclAlign(&D).getQuantity());
>> + var->setAlignment(alignment.getQuantity());
>>
>> if (D.hasAttr<AnnotateAttr>())
>> CGM.AddGlobalAnnotations(&D, var);
>> @@ -385,7 +384,8 @@ void CodeGenFunction::EmitStaticVarDecl(
>> // RAUW's the GV uses of this constant will be invalid.
>> llvm::Constant *castedAddr =
>> llvm::ConstantExpr::getPointerBitCastOrAddrSpaceCast(var, expectedType);
>> - DMEntry = castedAddr;
>> + if (var != castedAddr)
>> + LocalDeclMap.find(&D)->second = Address(castedAddr, alignment);
>> CGM.setStaticLocalDeclAddress(&D, castedAddr);
>>
>> CGM.getSanitizerMetadata()->reportGlobalToASan(var, D);
>> @@ -401,13 +401,13 @@ void CodeGenFunction::EmitStaticVarDecl(
>>
>> namespace {
>> struct DestroyObject final : EHScopeStack::Cleanup {
>> - DestroyObject(llvm::Value *addr, QualType type,
>> + DestroyObject(Address addr, QualType type,
>> CodeGenFunction::Destroyer *destroyer,
>> bool useEHCleanupForArray)
>> : addr(addr), type(type), destroyer(destroyer),
>> useEHCleanupForArray(useEHCleanupForArray) {}
>>
>> - llvm::Value *addr;
>> + Address addr;
>> QualType type;
>> CodeGenFunction::Destroyer *destroyer;
>> bool useEHCleanupForArray;
>> @@ -422,14 +422,14 @@ namespace {
>> };
>>
>> struct DestroyNRVOVariable final : EHScopeStack::Cleanup {
>> - DestroyNRVOVariable(llvm::Value *addr,
>> + DestroyNRVOVariable(Address addr,
>> const CXXDestructorDecl *Dtor,
>> llvm::Value *NRVOFlag)
>> : Dtor(Dtor), NRVOFlag(NRVOFlag), Loc(addr) {}
>>
>> const CXXDestructorDecl *Dtor;
>> llvm::Value *NRVOFlag;
>> - llvm::Value *Loc;
>> + Address Loc;
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> // Along the exceptions path we always execute the dtor.
>> @@ -440,7 +440,8 @@ namespace {
>> // If we exited via NRVO, we skip the destructor call.
>> llvm::BasicBlock *RunDtorBB = CGF.createBasicBlock("nrvo.unused");
>> SkipDtorBB = CGF.createBasicBlock("nrvo.skipdtor");
>> - llvm::Value *DidNRVO = CGF.Builder.CreateLoad(NRVOFlag, "nrvo.val");
>> + llvm::Value *DidNRVO =
>> + CGF.Builder.CreateFlagLoad(NRVOFlag, "nrvo.val");
>> CGF.Builder.CreateCondBr(DidNRVO, SkipDtorBB, RunDtorBB);
>> CGF.EmitBlock(RunDtorBB);
>> }
>> @@ -455,8 +456,8 @@ namespace {
>> };
>>
>> struct CallStackRestore final : EHScopeStack::Cleanup {
>> - llvm::Value *Stack;
>> - CallStackRestore(llvm::Value *Stack) : Stack(Stack) {}
>> + Address Stack;
>> + CallStackRestore(Address Stack) : Stack(Stack) {}
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> llvm::Value *V = CGF.Builder.CreateLoad(Stack);
>> llvm::Value *F = CGF.CGM.getIntrinsic(llvm::Intrinsic::stackrestore);
>> @@ -493,7 +494,7 @@ namespace {
>> Var.getType(), VK_LValue, SourceLocation());
>> // Compute the address of the local variable, in case it's a byref
>> // or something.
>> - llvm::Value *Addr = CGF.EmitDeclRefLValue(&DRE).getAddress();
>> + llvm::Value *Addr = CGF.EmitDeclRefLValue(&DRE).getPointer();
>>
>> // In some cases, the type of the function argument will be different from
>> // the type of the pointer. An example of this is
>> @@ -517,8 +518,8 @@ namespace {
>> llvm::Value *Addr;
>> llvm::Value *Size;
>> public:
>> - CallLifetimeEnd(llvm::Value *addr, llvm::Value *size)
>> - : Addr(addr), Size(size) {}
>> + CallLifetimeEnd(Address addr, llvm::Value *size)
>> + : Addr(addr.getPointer()), Size(size) {}
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> CGF.EmitLifetimeEnd(Size, Addr);
>> @@ -529,7 +530,7 @@ namespace {
>> /// EmitAutoVarWithLifetime - Does the setup required for an automatic
>> /// variable with lifetime.
>> static void EmitAutoVarWithLifetime(CodeGenFunction &CGF, const VarDecl &var,
>> - llvm::Value *addr,
>> + Address addr,
>> Qualifiers::ObjCLifetime lifetime) {
>> switch (lifetime) {
>> case Qualifiers::OCL_None:
>> @@ -599,7 +600,7 @@ static bool isAccessedBy(const ValueDecl
>> static void drillIntoBlockVariable(CodeGenFunction &CGF,
>> LValue &lvalue,
>> const VarDecl *var) {
>> - lvalue.setAddress(CGF.BuildBlockByrefAddress(lvalue.getAddress(), var));
>> + lvalue.setAddress(CGF.emitBlockByrefAddress(lvalue.getAddress(), var));
>> }
>>
>> void CodeGenFunction::EmitScalarInit(const Expr *init, const ValueDecl *D,
>> @@ -637,15 +638,12 @@ void CodeGenFunction::EmitScalarInit(con
>> if (capturedByInit) {
>> // We can use a simple GEP for this because it can't have been
>> // moved yet.
>> - tempLV.setAddress(Builder.CreateStructGEP(
>> - nullptr, tempLV.getAddress(),
>> - getByRefValueLLVMField(cast<VarDecl>(D)).second));
>> + tempLV.setAddress(emitBlockByrefAddress(tempLV.getAddress(),
>> + cast<VarDecl>(D),
>> + /*follow*/ false));
>> }
>>
>> - llvm::PointerType *ty
>> - = cast<llvm::PointerType>(tempLV.getAddress()->getType());
>> - ty = cast<llvm::PointerType>(ty->getElementType());
>> -
>> + auto ty = cast<llvm::PointerType>(tempLV.getAddress().getElementType());
>> llvm::Value *zero = llvm::ConstantPointerNull::get(ty);
>>
>> // If __weak, we want to use a barrier under certain conditions.
>> @@ -789,7 +787,7 @@ static void emitStoresForInitAfterMemset
>> if (isa<llvm::ConstantInt>(Init) || isa<llvm::ConstantFP>(Init) ||
>> isa<llvm::ConstantVector>(Init) || isa<llvm::BlockAddress>(Init) ||
>> isa<llvm::ConstantExpr>(Init)) {
>> - Builder.CreateStore(Init, Loc, isVolatile);
>> + Builder.CreateDefaultAlignedStore(Init, Loc, isVolatile);
>> return;
>> }
>>
>> @@ -892,13 +890,12 @@ CodeGenFunction::EmitAutoVarAlloca(const
>> emission.IsByRef = isByRef;
>>
>> CharUnits alignment = getContext().getDeclAlign(&D);
>> - emission.Alignment = alignment;
>>
>> // If the type is variably-modified, emit all the VLA sizes for it.
>> if (Ty->isVariablyModifiedType())
>> EmitVariablyModifiedType(Ty);
>>
>> - llvm::Value *DeclPtr;
>> + Address address = Address::invalid();
>> if (Ty->isConstantSizeType()) {
>> bool NRVO = getLangOpts().ElideConstructors &&
>> D.isNRVOVariable();
>> @@ -924,7 +921,8 @@ CodeGenFunction::EmitAutoVarAlloca(const
>> CGM.isTypeConstant(Ty, true)) {
>> EmitStaticVarDecl(D, llvm::GlobalValue::InternalLinkage);
>>
>> - emission.Address = nullptr; // signal this condition to later callbacks
>> + // Signal this condition to later callbacks.
>> + emission.Addr = Address::invalid();
>> assert(emission.wasEmittedAsGlobal());
>> return emission;
>> }
>> @@ -935,13 +933,12 @@ CodeGenFunction::EmitAutoVarAlloca(const
>>
>> // A normal fixed sized variable becomes an alloca in the entry block,
>> // unless it's an NRVO variable.
>> - llvm::Type *LTy = ConvertTypeForMem(Ty);
>>
>> if (NRVO) {
>> // The named return value optimization: allocate this variable in the
>> // return slot, so that we can elide the copy when returning this
>> // variable (C++0x [class.copy]p34).
>> - DeclPtr = ReturnValue;
>> + address = ReturnValue;
>>
>> if (const RecordType *RecordTy = Ty->getAs<RecordType>()) {
>> if (!cast<CXXRecordDecl>(RecordTy->getDecl())->hasTrivialDestructor()) {
>> @@ -949,34 +946,36 @@ CodeGenFunction::EmitAutoVarAlloca(const
>> // to this variable. Set it to zero to indicate that NRVO was not
>> // applied.
>> llvm::Value *Zero = Builder.getFalse();
>> - llvm::Value *NRVOFlag = CreateTempAlloca(Zero->getType(), "nrvo");
>> + Address NRVOFlag =
>> + CreateTempAlloca(Zero->getType(), CharUnits::One(), "nrvo");
>> EnsureInsertPoint();
>> Builder.CreateStore(Zero, NRVOFlag);
>>
>> // Record the NRVO flag for this variable.
>> - NRVOFlags[&D] = NRVOFlag;
>> - emission.NRVOFlag = NRVOFlag;
>> + NRVOFlags[&D] = NRVOFlag.getPointer();
>> + emission.NRVOFlag = NRVOFlag.getPointer();
>> }
>> }
>> } else {
>> - if (isByRef)
>> - LTy = BuildByRefType(&D);
>> -
>> - llvm::AllocaInst *Alloc = CreateTempAlloca(LTy);
>> - Alloc->setName(D.getName());
>> + CharUnits allocaAlignment;
>> + llvm::Type *allocaTy;
>> + if (isByRef) {
>> + auto &byrefInfo = getBlockByrefInfo(&D);
>> + allocaTy = byrefInfo.Type;
>> + allocaAlignment = byrefInfo.ByrefAlignment;
>> + } else {
>> + allocaTy = ConvertTypeForMem(Ty);
>> + allocaAlignment = alignment;
>> + }
>>
>> - CharUnits allocaAlignment = alignment;
>> - if (isByRef)
>> - allocaAlignment = std::max(allocaAlignment,
>> - getContext().toCharUnitsFromBits(getTarget().getPointerAlign(0)));
>> - Alloc->setAlignment(allocaAlignment.getQuantity());
>> - DeclPtr = Alloc;
>> + address = CreateTempAlloca(allocaTy, allocaAlignment, D.getName());
>>
>> // Emit a lifetime intrinsic if meaningful. There's no point
>> // in doing this if we don't have a valid insertion point (?).
>> - uint64_t size = CGM.getDataLayout().getTypeAllocSize(LTy);
>> if (HaveInsertPoint()) {
>> - emission.SizeForLifetimeMarkers = EmitLifetimeStart(size, Alloc);
>> + uint64_t size = CGM.getDataLayout().getTypeAllocSize(allocaTy);
>> + emission.SizeForLifetimeMarkers =
>> + EmitLifetimeStart(size, address.getPointer());
>> } else {
>> assert(!emission.useLifetimeMarkers());
>> }
>> @@ -986,11 +985,11 @@ CodeGenFunction::EmitAutoVarAlloca(const
>>
>> if (!DidCallStackSave) {
>> // Save the stack.
>> - llvm::Value *Stack = CreateTempAlloca(Int8PtrTy, "saved_stack");
>> + Address Stack =
>> + CreateTempAlloca(Int8PtrTy, getPointerAlign(), "saved_stack");
>>
>> llvm::Value *F = CGM.getIntrinsic(llvm::Intrinsic::stacksave);
>> llvm::Value *V = Builder.CreateCall(F);
>> -
>> Builder.CreateStore(V, Stack);
>>
>> DidCallStackSave = true;
>> @@ -1010,13 +1009,11 @@ CodeGenFunction::EmitAutoVarAlloca(const
>> llvm::AllocaInst *vla = Builder.CreateAlloca(llvmTy, elementCount, "vla");
>> vla->setAlignment(alignment.getQuantity());
>>
>> - DeclPtr = vla;
>> + address = Address(vla, alignment);
>> }
>>
>> - llvm::Value *&DMEntry = LocalDeclMap[&D];
>> - assert(!DMEntry && "Decl already exists in localdeclmap!");
>> - DMEntry = DeclPtr;
>> - emission.Address = DeclPtr;
>> + setAddrOfLocalVar(&D, address);
>> + emission.Addr = address;
>>
>> // Emit debug info for local var declaration.
>> if (HaveInsertPoint())
>> @@ -1024,12 +1021,12 @@ CodeGenFunction::EmitAutoVarAlloca(const
>> if (CGM.getCodeGenOpts().getDebugInfo()
>>> = CodeGenOptions::LimitedDebugInfo) {
>> DI->setLocation(D.getLocation());
>> - DI->EmitDeclareOfAutoVariable(&D, DeclPtr, Builder);
>> + DI->EmitDeclareOfAutoVariable(&D, address.getPointer(), Builder);
>> }
>> }
>>
>> if (D.hasAttr<AnnotateAttr>())
>> - EmitVarAnnotations(&D, emission.Address);
>> + EmitVarAnnotations(&D, address.getPointer());
>>
>> return emission;
>> }
>> @@ -1125,15 +1122,13 @@ void CodeGenFunction::EmitAutoVarInit(co
>> if (isTrivialInitializer(Init))
>> return;
>>
>> - CharUnits alignment = emission.Alignment;
>> -
>> // Check whether this is a byref variable that's potentially
>> // captured and moved by its own initializer. If so, we'll need to
>> // emit the initializer first, then copy into the variable.
>> bool capturedByInit = emission.IsByRef && isCapturedBy(D, Init);
>>
>> - llvm::Value *Loc =
>> - capturedByInit ? emission.Address : emission.getObjectAddress(*this);
>> + Address Loc =
>> + capturedByInit ? emission.Addr : emission.getObjectAddress(*this);
>>
>> llvm::Constant *constant = nullptr;
>> if (emission.IsConstantAggregate || D.isConstexpr()) {
>> @@ -1142,14 +1137,14 @@ void CodeGenFunction::EmitAutoVarInit(co
>> }
>>
>> if (!constant) {
>> - LValue lv = MakeAddrLValue(Loc, type, alignment);
>> + LValue lv = MakeAddrLValue(Loc, type);
>> lv.setNonGC(true);
>> return EmitExprAsInit(Init, &D, lv, capturedByInit);
>> }
>>
>> if (!emission.IsConstantAggregate) {
>> // For simple scalar/complex initialization, store the value directly.
>> - LValue lv = MakeAddrLValue(Loc, type, alignment);
>> + LValue lv = MakeAddrLValue(Loc, type);
>> lv.setNonGC(true);
>> return EmitStoreThroughLValue(RValue::get(constant), lv, true);
>> }
>> @@ -1163,7 +1158,7 @@ void CodeGenFunction::EmitAutoVarInit(co
>> getContext().getTypeSizeInChars(type).getQuantity());
>>
>> llvm::Type *BP = Int8PtrTy;
>> - if (Loc->getType() != BP)
>> + if (Loc.getType() != BP)
>> Loc = Builder.CreateBitCast(Loc, BP);
>>
>> // If the initializer is all or mostly zeros, codegen with memset then do
>> @@ -1171,11 +1166,12 @@ void CodeGenFunction::EmitAutoVarInit(co
>> if (shouldUseMemSetPlusStoresToInitialize(constant,
>> CGM.getDataLayout().getTypeAllocSize(constant->getType()))) {
>> Builder.CreateMemSet(Loc, llvm::ConstantInt::get(Int8Ty, 0), SizeVal,
>> - alignment.getQuantity(), isVolatile);
>> + isVolatile);
>> // Zero and undef don't require a stores.
>> if (!constant->isNullValue() && !isa<llvm::UndefValue>(constant)) {
>> Loc = Builder.CreateBitCast(Loc, constant->getType()->getPointerTo());
>> - emitStoresForInitAfterMemset(constant, Loc, isVolatile, Builder);
>> + emitStoresForInitAfterMemset(constant, Loc.getPointer(),
>> + isVolatile, Builder);
>> }
>> } else {
>> // Otherwise, create a temporary global with the initializer then
>> @@ -1185,15 +1181,14 @@ void CodeGenFunction::EmitAutoVarInit(co
>> new llvm::GlobalVariable(CGM.getModule(), constant->getType(), true,
>> llvm::GlobalValue::PrivateLinkage,
>> constant, Name);
>> - GV->setAlignment(alignment.getQuantity());
>> + GV->setAlignment(Loc.getAlignment().getQuantity());
>> GV->setUnnamedAddr(true);
>>
>> - llvm::Value *SrcPtr = GV;
>> - if (SrcPtr->getType() != BP)
>> + Address SrcPtr = Address(GV, Loc.getAlignment());
>> + if (SrcPtr.getType() != BP)
>> SrcPtr = Builder.CreateBitCast(SrcPtr, BP);
>>
>> - Builder.CreateMemCpy(Loc, SrcPtr, SizeVal, alignment.getQuantity(),
>> - isVolatile);
>> + Builder.CreateMemCpy(Loc, SrcPtr, SizeVal, isVolatile);
>> }
>> }
>>
>> @@ -1254,7 +1249,7 @@ void CodeGenFunction::emitAutoVarTypeCle
>>
>> // Note that for __block variables, we want to destroy the
>> // original stack object, not the possibly forwarded object.
>> - llvm::Value *addr = emission.getObjectAddress(*this);
>> + Address addr = emission.getObjectAddress(*this);
>>
>> const VarDecl *var = emission.Variable;
>> QualType type = var->getType();
>> @@ -1272,8 +1267,8 @@ void CodeGenFunction::emitAutoVarTypeCle
>> if (emission.NRVOFlag) {
>> assert(!type->isArrayType());
>> CXXDestructorDecl *dtor = type->getAsCXXRecordDecl()->getDestructor();
>> - EHStack.pushCleanup<DestroyNRVOVariable>(cleanupKind, addr, dtor,
>> - emission.NRVOFlag);
>> + EHStack.pushCleanup<DestroyNRVOVariable>(cleanupKind, addr,
>> + dtor, emission.NRVOFlag);
>> return;
>> }
>> break;
>> @@ -1370,7 +1365,7 @@ CodeGenFunction::getDestroyer(QualType::
>> /// pushEHDestroy - Push the standard destructor for the given type as
>> /// an EH-only cleanup.
>> void CodeGenFunction::pushEHDestroy(QualType::DestructionKind dtorKind,
>> - llvm::Value *addr, QualType type) {
>> + Address addr, QualType type) {
>> assert(dtorKind && "cannot push destructor for trivial type");
>> assert(needsEHCleanup(dtorKind));
>>
>> @@ -1380,7 +1375,7 @@ void CodeGenFunction::pushEHDestroy(Qual
>> /// pushDestroy - Push the standard destructor for the given type as
>> /// at least a normal cleanup.
>> void CodeGenFunction::pushDestroy(QualType::DestructionKind dtorKind,
>> - llvm::Value *addr, QualType type) {
>> + Address addr, QualType type) {
>> assert(dtorKind && "cannot push destructor for trivial type");
>>
>> CleanupKind cleanupKind = getCleanupKind(dtorKind);
>> @@ -1388,19 +1383,19 @@ void CodeGenFunction::pushDestroy(QualTy
>> cleanupKind & EHCleanup);
>> }
>>
>> -void CodeGenFunction::pushDestroy(CleanupKind cleanupKind, llvm::Value *addr,
>> +void CodeGenFunction::pushDestroy(CleanupKind cleanupKind, Address addr,
>> QualType type, Destroyer *destroyer,
>> bool useEHCleanupForArray) {
>> pushFullExprCleanup<DestroyObject>(cleanupKind, addr, type,
>> destroyer, useEHCleanupForArray);
>> }
>>
>> -void CodeGenFunction::pushStackRestore(CleanupKind Kind, llvm::Value *SPMem) {
>> +void CodeGenFunction::pushStackRestore(CleanupKind Kind, Address SPMem) {
>> EHStack.pushCleanup<CallStackRestore>(Kind, SPMem);
>> }
>>
>> void CodeGenFunction::pushLifetimeExtendedDestroy(
>> - CleanupKind cleanupKind, llvm::Value *addr, QualType type,
>> + CleanupKind cleanupKind, Address addr, QualType type,
>> Destroyer *destroyer, bool useEHCleanupForArray) {
>> assert(!isInConditionalBranch() &&
>> "performing lifetime extension from within conditional");
>> @@ -1430,15 +1425,18 @@ void CodeGenFunction::pushLifetimeExtend
>> /// \param useEHCleanupForArray - whether an EH cleanup should be
>> /// used when destroying array elements, in case one of the
>> /// destructions throws an exception
>> -void CodeGenFunction::emitDestroy(llvm::Value *addr, QualType type,
>> +void CodeGenFunction::emitDestroy(Address addr, QualType type,
>> Destroyer *destroyer,
>> bool useEHCleanupForArray) {
>> const ArrayType *arrayType = getContext().getAsArrayType(type);
>> if (!arrayType)
>> return destroyer(*this, addr, type);
>>
>> - llvm::Value *begin = addr;
>> - llvm::Value *length = emitArrayLength(arrayType, type, begin);
>> + llvm::Value *length = emitArrayLength(arrayType, type, addr);
>> +
>> + CharUnits elementAlign =
>> + addr.getAlignment()
>> + .alignmentOfArrayElement(getContext().getTypeSizeInChars(type));
>>
>> // Normally we have to check whether the array is zero-length.
>> bool checkZeroLength = true;
>> @@ -1450,8 +1448,9 @@ void CodeGenFunction::emitDestroy(llvm::
>> checkZeroLength = false;
>> }
>>
>> + llvm::Value *begin = addr.getPointer();
>> llvm::Value *end = Builder.CreateInBoundsGEP(begin, length);
>> - emitArrayDestroy(begin, end, type, destroyer,
>> + emitArrayDestroy(begin, end, type, elementAlign, destroyer,
>> checkZeroLength, useEHCleanupForArray);
>> }
>>
>> @@ -1467,11 +1466,12 @@ void CodeGenFunction::emitDestroy(llvm::
>> /// element throws
>> void CodeGenFunction::emitArrayDestroy(llvm::Value *begin,
>> llvm::Value *end,
>> - QualType type,
>> + QualType elementType,
>> + CharUnits elementAlign,
>> Destroyer *destroyer,
>> bool checkZeroLength,
>> bool useEHCleanup) {
>> - assert(!type->isArrayType());
>> + assert(!elementType->isArrayType());
>>
>> // The basic structure here is a do-while loop, because we don't
>> // need to check for the zero-element case.
>> @@ -1497,10 +1497,11 @@ void CodeGenFunction::emitArrayDestroy(l
>> "arraydestroy.element");
>>
>> if (useEHCleanup)
>> - pushRegularPartialArrayCleanup(begin, element, type, destroyer);
>> + pushRegularPartialArrayCleanup(begin, element, elementType, elementAlign,
>> + destroyer);
>>
>> // Perform the actual destruction there.
>> - destroyer(*this, element, type);
>> + destroyer(*this, Address(element, elementAlign), elementType);
>>
>> if (useEHCleanup)
>> PopCleanupBlock();
>> @@ -1518,7 +1519,7 @@ void CodeGenFunction::emitArrayDestroy(l
>> /// emitArrayDestroy, the element type here may still be an array type.
>> static void emitPartialArrayDestroy(CodeGenFunction &CGF,
>> llvm::Value *begin, llvm::Value *end,
>> - QualType type,
>> + QualType type, CharUnits elementAlign,
>> CodeGenFunction::Destroyer *destroyer) {
>> // If the element type is itself an array, drill down.
>> unsigned arrayDepth = 0;
>> @@ -1540,7 +1541,7 @@ static void emitPartialArrayDestroy(Code
>> // Destroy the array. We don't ever need an EH cleanup because we
>> // assume that we're in an EH cleanup ourselves, so a throwing
>> // destructor causes an immediate terminate.
>> - CGF.emitArrayDestroy(begin, end, type, destroyer,
>> + CGF.emitArrayDestroy(begin, end, type, elementAlign, destroyer,
>> /*checkZeroLength*/ true, /*useEHCleanup*/ false);
>> }
>>
>> @@ -1553,16 +1554,18 @@ namespace {
>> llvm::Value *ArrayEnd;
>> QualType ElementType;
>> CodeGenFunction::Destroyer *Destroyer;
>> + CharUnits ElementAlign;
>> public:
>> RegularPartialArrayDestroy(llvm::Value *arrayBegin, llvm::Value *arrayEnd,
>> - QualType elementType,
>> + QualType elementType, CharUnits elementAlign,
>> CodeGenFunction::Destroyer *destroyer)
>> : ArrayBegin(arrayBegin), ArrayEnd(arrayEnd),
>> - ElementType(elementType), Destroyer(destroyer) {}
>> + ElementType(elementType), Destroyer(destroyer),
>> + ElementAlign(elementAlign) {}
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> emitPartialArrayDestroy(CGF, ArrayBegin, ArrayEnd,
>> - ElementType, Destroyer);
>> + ElementType, ElementAlign, Destroyer);
>> }
>> };
>>
>> @@ -1571,21 +1574,24 @@ namespace {
>> /// determined and must be loaded from a local.
>> class IrregularPartialArrayDestroy final : public EHScopeStack::Cleanup {
>> llvm::Value *ArrayBegin;
>> - llvm::Value *ArrayEndPointer;
>> + Address ArrayEndPointer;
>> QualType ElementType;
>> CodeGenFunction::Destroyer *Destroyer;
>> + CharUnits ElementAlign;
>> public:
>> IrregularPartialArrayDestroy(llvm::Value *arrayBegin,
>> - llvm::Value *arrayEndPointer,
>> + Address arrayEndPointer,
>> QualType elementType,
>> + CharUnits elementAlign,
>> CodeGenFunction::Destroyer *destroyer)
>> : ArrayBegin(arrayBegin), ArrayEndPointer(arrayEndPointer),
>> - ElementType(elementType), Destroyer(destroyer) {}
>> + ElementType(elementType), Destroyer(destroyer),
>> + ElementAlign(elementAlign) {}
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> llvm::Value *arrayEnd = CGF.Builder.CreateLoad(ArrayEndPointer);
>> emitPartialArrayDestroy(CGF, ArrayBegin, arrayEnd,
>> - ElementType, Destroyer);
>> + ElementType, ElementAlign, Destroyer);
>> }
>> };
>> }
>> @@ -1597,12 +1603,14 @@ namespace {
>> /// \param elementType - the immediate element type of the array;
>> /// possibly still an array type
>> void CodeGenFunction::pushIrregularPartialArrayCleanup(llvm::Value *arrayBegin,
>> - llvm::Value *arrayEndPointer,
>> + Address arrayEndPointer,
>> QualType elementType,
>> + CharUnits elementAlign,
>> Destroyer *destroyer) {
>> pushFullExprCleanup<IrregularPartialArrayDestroy>(EHCleanup,
>> arrayBegin, arrayEndPointer,
>> - elementType, destroyer);
>> + elementType, elementAlign,
>> + destroyer);
>> }
>>
>> /// pushRegularPartialArrayCleanup - Push an EH cleanup to destroy
>> @@ -1614,10 +1622,12 @@ void CodeGenFunction::pushIrregularParti
>> void CodeGenFunction::pushRegularPartialArrayCleanup(llvm::Value *arrayBegin,
>> llvm::Value *arrayEnd,
>> QualType elementType,
>> + CharUnits elementAlign,
>> Destroyer *destroyer) {
>> pushFullExprCleanup<RegularPartialArrayDestroy>(EHCleanup,
>> arrayBegin, arrayEnd,
>> - elementType, destroyer);
>> + elementType, elementAlign,
>> + destroyer);
>> }
>>
>> /// Lazily declare the @llvm.lifetime.start intrinsic.
>> @@ -1657,56 +1667,38 @@ namespace {
>>
>> /// Emit an alloca (or GlobalValue depending on target)
>> /// for the specified parameter and set up LocalDeclMap.
>> -void CodeGenFunction::EmitParmDecl(const VarDecl &D, llvm::Value *Arg,
>> - bool ArgIsPointer, unsigned ArgNo) {
>> +void CodeGenFunction::EmitParmDecl(const VarDecl &D, ParamValue Arg,
>> + unsigned ArgNo) {
>> // FIXME: Why isn't ImplicitParamDecl a ParmVarDecl?
>> assert((isa<ParmVarDecl>(D) || isa<ImplicitParamDecl>(D)) &&
>> "Invalid argument to EmitParmDecl");
>>
>> - Arg->setName(D.getName());
>> + Arg.getAnyValue()->setName(D.getName());
>>
>> QualType Ty = D.getType();
>>
>> // Use better IR generation for certain implicit parameters.
>> - if (isa<ImplicitParamDecl>(D)) {
>> + if (auto IPD = dyn_cast<ImplicitParamDecl>(&D)) {
>> // The only implicit argument a block has is its literal.
>> + // We assume this is always passed directly.
>> if (BlockInfo) {
>> - LocalDeclMap[&D] = Arg;
>> - llvm::Value *LocalAddr = nullptr;
>> - if (CGM.getCodeGenOpts().OptimizationLevel == 0) {
>> - // Allocate a stack slot to let the debug info survive the RA.
>> - llvm::AllocaInst *Alloc = CreateTempAlloca(ConvertTypeForMem(Ty),
>> - D.getName() + ".addr");
>> - Alloc->setAlignment(getContext().getDeclAlign(&D).getQuantity());
>> - LValue lv = MakeAddrLValue(Alloc, Ty, getContext().getDeclAlign(&D));
>> - EmitStoreOfScalar(Arg, lv, /* isInitialization */ true);
>> - LocalAddr = Builder.CreateLoad(Alloc);
>> - }
>> -
>> - if (CGDebugInfo *DI = getDebugInfo()) {
>> - if (CGM.getCodeGenOpts().getDebugInfo()
>> - >= CodeGenOptions::LimitedDebugInfo) {
>> - DI->setLocation(D.getLocation());
>> - DI->EmitDeclareOfBlockLiteralArgVariable(*BlockInfo, Arg, ArgNo,
>> - LocalAddr, Builder);
>> - }
>> - }
>> -
>> + setBlockContextParameter(IPD, ArgNo, Arg.getDirectValue());
>> return;
>> }
>> }
>>
>> - llvm::Value *DeclPtr;
>> + Address DeclPtr = Address::invalid();
>> bool DoStore = false;
>> bool IsScalar = hasScalarEvaluationKind(Ty);
>> - CharUnits Align = getContext().getDeclAlign(&D);
>> // If we already have a pointer to the argument, reuse the input pointer.
>> - if (ArgIsPointer) {
>> + if (Arg.isIndirect()) {
>> + DeclPtr = Arg.getIndirectAddress();
>> // If we have a prettier pointer type at this point, bitcast to that.
>> - unsigned AS = cast<llvm::PointerType>(Arg->getType())->getAddressSpace();
>> + unsigned AS = DeclPtr.getType()->getAddressSpace();
>> llvm::Type *IRTy = ConvertTypeForMem(Ty)->getPointerTo(AS);
>> - DeclPtr = Arg->getType() == IRTy ? Arg : Builder.CreateBitCast(Arg, IRTy,
>> - D.getName());
>> + if (DeclPtr.getType() != IRTy)
>> + DeclPtr = Builder.CreateBitCast(DeclPtr, IRTy, D.getName());
>> +
>> // Push a destructor cleanup for this parameter if the ABI requires it.
>> // Don't push a cleanup in a thunk for a method that will also emit a
>> // cleanup.
>> @@ -1718,14 +1710,14 @@ void CodeGenFunction::EmitParmDecl(const
>> }
>> } else {
>> // Otherwise, create a temporary to hold the value.
>> - llvm::AllocaInst *Alloc = CreateTempAlloca(ConvertTypeForMem(Ty),
>> - D.getName() + ".addr");
>> - Alloc->setAlignment(Align.getQuantity());
>> - DeclPtr = Alloc;
>> + DeclPtr = CreateMemTemp(Ty, getContext().getDeclAlign(&D),
>> + D.getName() + ".addr");
>> DoStore = true;
>> }
>>
>> - LValue lv = MakeAddrLValue(DeclPtr, Ty, Align);
>> + llvm::Value *ArgVal = (DoStore ? Arg.getDirectValue() : nullptr);
>> +
>> + LValue lv = MakeAddrLValue(DeclPtr, Ty);
>> if (IsScalar) {
>> Qualifiers qs = Ty.getQualifiers();
>> if (Qualifiers::ObjCLifetime lt = qs.getObjCLifetime()) {
>> @@ -1755,26 +1747,26 @@ void CodeGenFunction::EmitParmDecl(const
>> // objc_storeStrong attempts to release its old value.
>> llvm::Value *Null = CGM.EmitNullConstant(D.getType());
>> EmitStoreOfScalar(Null, lv, /* isInitialization */ true);
>> - EmitARCStoreStrongCall(lv.getAddress(), Arg, true);
>> + EmitARCStoreStrongCall(lv.getAddress(), ArgVal, true);
>> DoStore = false;
>> }
>> else
>> // Don't use objc_retainBlock for block pointers, because we
>> // don't want to Block_copy something just because we got it
>> // as a parameter.
>> - Arg = EmitARCRetainNonBlock(Arg);
>> + ArgVal = EmitARCRetainNonBlock(ArgVal);
>> }
>> } else {
>> // Push the cleanup for a consumed parameter.
>> if (isConsumed) {
>> ARCPreciseLifetime_t precise = (D.hasAttr<ObjCPreciseLifetimeAttr>()
>> ? ARCPreciseLifetime : ARCImpreciseLifetime);
>> - EHStack.pushCleanup<ConsumeARCParameter>(getARCCleanupKind(), Arg,
>> + EHStack.pushCleanup<ConsumeARCParameter>(getARCCleanupKind(), ArgVal,
>> precise);
>> }
>>
>> if (lt == Qualifiers::OCL_Weak) {
>> - EmitARCInitWeak(DeclPtr, Arg);
>> + EmitARCInitWeak(DeclPtr, ArgVal);
>> DoStore = false; // The weak init is a store, no need to do two.
>> }
>> }
>> @@ -1786,20 +1778,18 @@ void CodeGenFunction::EmitParmDecl(const
>>
>> // Store the initial value into the alloca.
>> if (DoStore)
>> - EmitStoreOfScalar(Arg, lv, /* isInitialization */ true);
>> + EmitStoreOfScalar(ArgVal, lv, /* isInitialization */ true);
>>
>> - llvm::Value *&DMEntry = LocalDeclMap[&D];
>> - assert(!DMEntry && "Decl already exists in localdeclmap!");
>> - DMEntry = DeclPtr;
>> + setAddrOfLocalVar(&D, DeclPtr);
>>
>> // Emit debug info for param declaration.
>> if (CGDebugInfo *DI = getDebugInfo()) {
>> if (CGM.getCodeGenOpts().getDebugInfo()
>>> = CodeGenOptions::LimitedDebugInfo) {
>> - DI->EmitDeclareOfArgVariable(&D, DeclPtr, ArgNo, Builder);
>> + DI->EmitDeclareOfArgVariable(&D, DeclPtr.getPointer(), ArgNo, Builder);
>> }
>> }
>>
>> if (D.hasAttr<AnnotateAttr>())
>> - EmitVarAnnotations(&D, DeclPtr);
>> + EmitVarAnnotations(&D, DeclPtr.getPointer());
>> }
>>
>> Modified: cfe/trunk/lib/CodeGen/CGDeclCXX.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGDeclCXX.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGDeclCXX.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGDeclCXX.cpp Tue Sep 8 03:05:57 2015
>> @@ -24,16 +24,13 @@ using namespace clang;
>> using namespace CodeGen;
>>
>> static void EmitDeclInit(CodeGenFunction &CGF, const VarDecl &D,
>> - llvm::Constant *DeclPtr) {
>> + ConstantAddress DeclPtr) {
>> assert(D.hasGlobalStorage() && "VarDecl must have global storage!");
>> assert(!D.getType()->isReferenceType() &&
>> "Should not call EmitDeclInit on a reference!");
>>
>> - ASTContext &Context = CGF.getContext();
>> -
>> - CharUnits alignment = Context.getDeclAlign(&D);
>> QualType type = D.getType();
>> - LValue lv = CGF.MakeAddrLValue(DeclPtr, type, alignment);
>> + LValue lv = CGF.MakeAddrLValue(DeclPtr, type);
>>
>> const Expr *Init = D.getInit();
>> switch (CGF.getEvaluationKind(type)) {
>> @@ -64,7 +61,7 @@ static void EmitDeclInit(CodeGenFunction
>> /// Emit code to cause the destruction of the given variable with
>> /// static storage duration.
>> static void EmitDeclDestroy(CodeGenFunction &CGF, const VarDecl &D,
>> - llvm::Constant *addr) {
>> + ConstantAddress addr) {
>> CodeGenModule &CGM = CGF.CGM;
>>
>> // FIXME: __attribute__((cleanup)) ?
>> @@ -99,7 +96,7 @@ static void EmitDeclDestroy(CodeGenFunct
>>
>> function = CGM.getAddrOfCXXStructor(dtor, StructorType::Complete);
>> argument = llvm::ConstantExpr::getBitCast(
>> - addr, CGF.getTypes().ConvertType(type)->getPointerTo());
>> + addr.getPointer(), CGF.getTypes().ConvertType(type)->getPointerTo());
>>
>> // Otherwise, the standard logic requires a helper function.
>> } else {
>> @@ -162,25 +159,26 @@ void CodeGenFunction::EmitCXXGlobalVarDe
>> DeclPtr = llvm::ConstantExpr::getAddrSpaceCast(DeclPtr, PTy);
>> }
>>
>> + ConstantAddress DeclAddr(DeclPtr, getContext().getDeclAlign(&D));
>> +
>> if (!T->isReferenceType()) {
>> if (getLangOpts().OpenMP && D.hasAttr<OMPThreadPrivateDeclAttr>())
>> (void)CGM.getOpenMPRuntime().emitThreadPrivateVarDefinition(
>> - &D, DeclPtr, D.getAttr<OMPThreadPrivateDeclAttr>()->getLocation(),
>> + &D, DeclAddr, D.getAttr<OMPThreadPrivateDeclAttr>()->getLocation(),
>> PerformInit, this);
>> if (PerformInit)
>> - EmitDeclInit(*this, D, DeclPtr);
>> + EmitDeclInit(*this, D, DeclAddr);
>> if (CGM.isTypeConstant(D.getType(), true))
>> EmitDeclInvariant(*this, D, DeclPtr);
>> else
>> - EmitDeclDestroy(*this, D, DeclPtr);
>> + EmitDeclDestroy(*this, D, DeclAddr);
>> return;
>> }
>>
>> assert(PerformInit && "cannot have constant initializer which needs "
>> "destruction for reference");
>> - unsigned Alignment = getContext().getDeclAlign(&D).getQuantity();
>> RValue RV = EmitReferenceBindingToExpr(Init);
>> - EmitStoreOfScalar(RV.getScalarVal(), DeclPtr, false, Alignment, T);
>> + EmitStoreOfScalar(RV.getScalarVal(), DeclAddr, false, T);
>> }
>>
>> /// Create a stub function, suitable for being passed to atexit,
>> @@ -498,7 +496,7 @@ void CodeGenFunction::GenerateCXXGlobalV
>> void
>> CodeGenFunction::GenerateCXXGlobalInitFunc(llvm::Function *Fn,
>> ArrayRef<llvm::Function *> Decls,
>> - llvm::GlobalVariable *Guard) {
>> + Address Guard) {
>> {
>> auto NL = ApplyDebugLocation::CreateEmpty(*this);
>> StartFunction(GlobalDecl(), getContext().VoidTy, Fn,
>> @@ -507,7 +505,7 @@ CodeGenFunction::GenerateCXXGlobalInitFu
>> auto AL = ApplyDebugLocation::CreateArtificial(*this);
>>
>> llvm::BasicBlock *ExitBlock = nullptr;
>> - if (Guard) {
>> + if (Guard.isValid()) {
>> // If we have a guard variable, check whether we've already performed
>> // these initializations. This happens for TLS initialization functions.
>> llvm::Value *GuardVal = Builder.CreateLoad(Guard);
>> @@ -572,9 +570,10 @@ void CodeGenFunction::GenerateCXXGlobalD
>> }
>>
>> /// generateDestroyHelper - Generates a helper function which, when
>> -/// invoked, destroys the given object.
>> +/// invoked, destroys the given object. The address of the object
>> +/// should be in global memory.
>> llvm::Function *CodeGenFunction::generateDestroyHelper(
>> - llvm::Constant *addr, QualType type, Destroyer *destroyer,
>> + Address addr, QualType type, Destroyer *destroyer,
>> bool useEHCleanupForArray, const VarDecl *VD) {
>> FunctionArgList args;
>> ImplicitParamDecl dst(getContext(), nullptr, SourceLocation(), nullptr,
>>
>> Modified: cfe/trunk/lib/CodeGen/CGException.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGException.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGException.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGException.cpp Tue Sep 8 03:05:57 2015
>> @@ -340,16 +340,16 @@ namespace {
>> // differs from EmitAnyExprToMem only in that, if a final copy-ctor
>> // call is required, an exception within that copy ctor causes
>> // std::terminate to be invoked.
>> -void CodeGenFunction::EmitAnyExprToExn(const Expr *e, llvm::Value *addr) {
>> +void CodeGenFunction::EmitAnyExprToExn(const Expr *e, Address addr) {
>> // Make sure the exception object is cleaned up if there's an
>> // exception during initialization.
>> - pushFullExprCleanup<FreeException>(EHCleanup, addr);
>> + pushFullExprCleanup<FreeException>(EHCleanup, addr.getPointer());
>> EHScopeStack::stable_iterator cleanup = EHStack.stable_begin();
>>
>> // __cxa_allocate_exception returns a void*; we need to cast this
>> // to the appropriate type for the object.
>> llvm::Type *ty = ConvertTypeForMem(e->getType())->getPointerTo();
>> - llvm::Value *typedAddr = Builder.CreateBitCast(addr, ty);
>> + Address typedAddr = Builder.CreateBitCast(addr, ty);
>>
>> // FIXME: this isn't quite right! If there's a final unelided call
>> // to a copy constructor, then according to [except.terminate]p1 we
>> @@ -362,19 +362,20 @@ void CodeGenFunction::EmitAnyExprToExn(c
>> /*IsInit*/ true);
>>
>> // Deactivate the cleanup block.
>> - DeactivateCleanupBlock(cleanup, cast<llvm::Instruction>(typedAddr));
>> + DeactivateCleanupBlock(cleanup,
>> + cast<llvm::Instruction>(typedAddr.getPointer()));
>> }
>>
>> -llvm::Value *CodeGenFunction::getExceptionSlot() {
>> +Address CodeGenFunction::getExceptionSlot() {
>> if (!ExceptionSlot)
>> ExceptionSlot = CreateTempAlloca(Int8PtrTy, "exn.slot");
>> - return ExceptionSlot;
>> + return Address(ExceptionSlot, getPointerAlign());
>> }
>>
>> -llvm::Value *CodeGenFunction::getEHSelectorSlot() {
>> +Address CodeGenFunction::getEHSelectorSlot() {
>> if (!EHSelectorSlot)
>> EHSelectorSlot = CreateTempAlloca(Int32Ty, "ehselector.slot");
>> - return EHSelectorSlot;
>> + return Address(EHSelectorSlot, CharUnits::fromQuantity(4));
>> }
>>
>> llvm::Value *CodeGenFunction::getExceptionFromSlot() {
>> @@ -626,7 +627,7 @@ CodeGenFunction::getMSVCDispatchBlock(EH
>> DispatchBlock = getTerminateHandler();
>> else
>> DispatchBlock = createBasicBlock();
>> - CGBuilderTy Builder(DispatchBlock);
>> + CGBuilderTy Builder(*this, DispatchBlock);
>>
>> switch (EHS.getKind()) {
>> case EHScope::Catch:
>> @@ -879,7 +880,7 @@ static llvm::BasicBlock *emitMSVCCatchDi
>> // block is the block for the enclosing EH scope.
>> if (I + 1 == E) {
>> NextBlock = CGF.createBasicBlock("catchendblock");
>> - CGBuilderTy(NextBlock).CreateCatchEndPad(
>> + CGBuilderTy(CGF, NextBlock).CreateCatchEndPad(
>> CGF.getEHDispatchBlock(CatchScope.getEnclosingEHScope()));
>> } else {
>> NextBlock = CGF.createBasicBlock("catch.dispatch");
>> @@ -1098,7 +1099,7 @@ namespace {
>> CGF.createBasicBlock("finally.cleanup.cont");
>>
>> llvm::Value *ShouldEndCatch =
>> - CGF.Builder.CreateLoad(ForEHVar, "finally.endcatch");
>> + CGF.Builder.CreateFlagLoad(ForEHVar, "finally.endcatch");
>> CGF.Builder.CreateCondBr(ShouldEndCatch, EndCatchBB, CleanupContBB);
>> CGF.EmitBlock(EndCatchBB);
>> CGF.EmitRuntimeCallOrInvoke(EndCatchFn); // catch-all, so might throw
>> @@ -1141,13 +1142,13 @@ namespace {
>> llvm::BasicBlock *ContBB = CGF.createBasicBlock("finally.cont");
>>
>> llvm::Value *ShouldRethrow =
>> - CGF.Builder.CreateLoad(ForEHVar, "finally.shouldthrow");
>> + CGF.Builder.CreateFlagLoad(ForEHVar, "finally.shouldthrow");
>> CGF.Builder.CreateCondBr(ShouldRethrow, RethrowBB, ContBB);
>>
>> CGF.EmitBlock(RethrowBB);
>> if (SavedExnVar) {
>> CGF.EmitRuntimeCallOrInvoke(RethrowFn,
>> - CGF.Builder.CreateLoad(SavedExnVar));
>> + CGF.Builder.CreateAlignedLoad(SavedExnVar, CGF.getPointerAlign()));
>> } else {
>> CGF.EmitRuntimeCallOrInvoke(RethrowFn);
>> }
>> @@ -1222,7 +1223,7 @@ void CodeGenFunction::FinallyInfo::enter
>>
>> // Whether the finally block is being executed for EH purposes.
>> ForEHVar = CGF.CreateTempAlloca(CGF.Builder.getInt1Ty(), "finally.for-eh");
>> - CGF.Builder.CreateStore(CGF.Builder.getFalse(), ForEHVar);
>> + CGF.Builder.CreateFlagStore(false, ForEHVar);
>>
>> // Enter a normal cleanup which will perform the @finally block.
>> CGF.EHStack.pushCleanup<PerformFinally>(NormalCleanup, body,
>> @@ -1260,11 +1261,11 @@ void CodeGenFunction::FinallyInfo::exit(
>> // If we need to remember the exception pointer to rethrow later, do so.
>> if (SavedExnVar) {
>> if (!exn) exn = CGF.getExceptionFromSlot();
>> - CGF.Builder.CreateStore(exn, SavedExnVar);
>> + CGF.Builder.CreateAlignedStore(exn, SavedExnVar, CGF.getPointerAlign());
>> }
>>
>> // Tell the cleanups in the finally block that we're do this for EH.
>> - CGF.Builder.CreateStore(CGF.Builder.getTrue(), ForEHVar);
>> + CGF.Builder.CreateFlagStore(true, ForEHVar);
>>
>> // Thread a jump through the finally cleanup.
>> CGF.EmitBranchThroughCleanup(RethrowDest);
>> @@ -1433,13 +1434,13 @@ struct CaptureFinder : ConstStmtVisitor<
>> CodeGenFunction &ParentCGF;
>> const VarDecl *ParentThis;
>> SmallVector<const VarDecl *, 4> Captures;
>> - llvm::Value *SEHCodeSlot = nullptr;
>> + Address SEHCodeSlot = Address::invalid();
>> CaptureFinder(CodeGenFunction &ParentCGF, const VarDecl *ParentThis)
>> : ParentCGF(ParentCGF), ParentThis(ParentThis) {}
>>
>> // Return true if we need to do any capturing work.
>> bool foundCaptures() {
>> - return !Captures.empty() || SEHCodeSlot;
>> + return !Captures.empty() || SEHCodeSlot.isValid();
>> }
>>
>> void Visit(const Stmt *S) {
>> @@ -1478,7 +1479,7 @@ struct CaptureFinder : ConstStmtVisitor<
>> // This is the simple case where we are the outermost finally. All we
>> // have to do here is make sure we escape this and recover it in the
>> // outlined handler.
>> - if (!SEHCodeSlot)
>> + if (!SEHCodeSlot.isValid())
>> SEHCodeSlot = ParentCGF.SEHCodeSlotStack.back();
>> break;
>> }
>> @@ -1486,11 +1487,11 @@ struct CaptureFinder : ConstStmtVisitor<
>> };
>> }
>>
>> -llvm::Value *CodeGenFunction::recoverAddrOfEscapedLocal(
>> - CodeGenFunction &ParentCGF, llvm::Value *ParentVar, llvm::Value *ParentFP) {
>> +Address CodeGenFunction::recoverAddrOfEscapedLocal(
>> + CodeGenFunction &ParentCGF, Address ParentVar, llvm::Value *ParentFP) {
>> llvm::CallInst *RecoverCall = nullptr;
>> - CGBuilderTy Builder(AllocaInsertPt);
>> - if (auto *ParentAlloca = dyn_cast<llvm::AllocaInst>(ParentVar)) {
>> + CGBuilderTy Builder(*this, AllocaInsertPt);
>> + if (auto *ParentAlloca = dyn_cast<llvm::AllocaInst>(ParentVar.getPointer())) {
>> // Mark the variable escaped if nobody else referenced it and compute the
>> // localescape index.
>> auto InsertPair = ParentCGF.EscapedLocals.insert(
>> @@ -1510,7 +1511,7 @@ llvm::Value *CodeGenFunction::recoverAdd
>> // Just clone the existing localrecover call, but tweak the FP argument to
>> // use our FP value. All other arguments are constants.
>> auto *ParentRecover =
>> - cast<llvm::IntrinsicInst>(ParentVar->stripPointerCasts());
>> + cast<llvm::IntrinsicInst>(ParentVar.getPointer()->stripPointerCasts());
>> assert(ParentRecover->getIntrinsicID() == llvm::Intrinsic::localrecover &&
>> "expected alloca or localrecover in parent LocalDeclMap");
>> RecoverCall = cast<llvm::CallInst>(ParentRecover->clone());
>> @@ -1520,9 +1521,9 @@ llvm::Value *CodeGenFunction::recoverAdd
>>
>> // Bitcast the variable, rename it, and insert it in the local decl map.
>> llvm::Value *ChildVar =
>> - Builder.CreateBitCast(RecoverCall, ParentVar->getType());
>> - ChildVar->setName(ParentVar->getName());
>> - return ChildVar;
>> + Builder.CreateBitCast(RecoverCall, ParentVar.getType());
>> + ChildVar->setName(ParentVar.getName());
>> + return Address(ChildVar, ParentVar.getAlignment());
>> }
>>
>> void CodeGenFunction::EmitCapturedLocals(CodeGenFunction &ParentCGF,
>> @@ -1548,7 +1549,7 @@ void CodeGenFunction::EmitCapturedLocals
>> // EH registration is passed in as the EBP physical register. We can
>> // recover that with llvm.frameaddress(1), and adjust that to recover the
>> // parent's true frame pointer.
>> - CGBuilderTy Builder(AllocaInsertPt);
>> + CGBuilderTy Builder(CGM, AllocaInsertPt);
>> EntryEBP = Builder.CreateCall(
>> CGM.getIntrinsic(llvm::Intrinsic::frameaddress), {Builder.getInt32(1)});
>> llvm::Function *RecoverFPIntrin =
>> @@ -1583,13 +1584,13 @@ void CodeGenFunction::EmitCapturedLocals
>> auto I = ParentCGF.LocalDeclMap.find(VD);
>> if (I == ParentCGF.LocalDeclMap.end())
>> continue;
>> - llvm::Value *ParentVar = I->second;
>>
>> - LocalDeclMap[VD] =
>> - recoverAddrOfEscapedLocal(ParentCGF, ParentVar, ParentFP);
>> + Address ParentVar = I->second;
>> + setAddrOfLocalVar(VD,
>> + recoverAddrOfEscapedLocal(ParentCGF, ParentVar, ParentFP));
>> }
>>
>> - if (Finder.SEHCodeSlot) {
>> + if (Finder.SEHCodeSlot.isValid()) {
>> SEHCodeSlotStack.push_back(
>> recoverAddrOfEscapedLocal(ParentCGF, Finder.SEHCodeSlot, ParentFP));
>> }
>> @@ -1727,7 +1728,7 @@ void CodeGenFunction::EmitSEHExceptionCo
>> // load the pointer.
>> SEHInfo = Builder.CreateConstInBoundsGEP1_32(Int8Ty, EntryEBP, -20);
>> SEHInfo = Builder.CreateBitCast(SEHInfo, Int8PtrTy->getPointerTo());
>> - SEHInfo = Builder.CreateLoad(Int8PtrTy, SEHInfo);
>> + SEHInfo = Builder.CreateAlignedLoad(Int8PtrTy, SEHInfo, getPointerAlign());
>> SEHCodeSlotStack.push_back(recoverAddrOfEscapedLocal(
>> ParentCGF, ParentCGF.SEHCodeSlotStack.back(), ParentFP));
>> }
>> @@ -1743,8 +1744,8 @@ void CodeGenFunction::EmitSEHExceptionCo
>> llvm::Type *PtrsTy = llvm::StructType::get(RecordTy, CGM.VoidPtrTy, nullptr);
>> llvm::Value *Ptrs = Builder.CreateBitCast(SEHInfo, PtrsTy->getPointerTo());
>> llvm::Value *Rec = Builder.CreateStructGEP(PtrsTy, Ptrs, 0);
>> - Rec = Builder.CreateLoad(Rec);
>> - llvm::Value *Code = Builder.CreateLoad(Rec);
>> + Rec = Builder.CreateAlignedLoad(Rec, getPointerAlign());
>> + llvm::Value *Code = Builder.CreateAlignedLoad(Rec, getIntAlign());
>> assert(!SEHCodeSlotStack.empty() && "emitting EH code outside of __except");
>> Builder.CreateStore(Code, SEHCodeSlotStack.back());
>> }
>> @@ -1760,7 +1761,7 @@ llvm::Value *CodeGenFunction::EmitSEHExc
>>
>> llvm::Value *CodeGenFunction::EmitSEHExceptionCode() {
>> assert(!SEHCodeSlotStack.empty() && "emitting EH code outside of __except");
>> - return Builder.CreateLoad(Int32Ty, SEHCodeSlotStack.back());
>> + return Builder.CreateLoad(SEHCodeSlotStack.back());
>> }
>>
>> llvm::Value *CodeGenFunction::EmitSEHAbnormalTermination() {
>>
>> Modified: cfe/trunk/lib/CodeGen/CGExpr.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGExpr.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGExpr.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGExpr.cpp Tue Sep 8 03:05:57 2015
>> @@ -54,6 +54,15 @@ llvm::Value *CodeGenFunction::EmitCastTo
>>
>> /// CreateTempAlloca - This creates a alloca and inserts it into the entry
>> /// block.
>> +Address CodeGenFunction::CreateTempAlloca(llvm::Type *Ty, CharUnits Align,
>> + const Twine &Name) {
>> + auto Alloca = CreateTempAlloca(Ty, Name);
>> + Alloca->setAlignment(Align.getQuantity());
>> + return Address(Alloca, Align);
>> +}
>> +
>> +/// CreateTempAlloca - This creates a alloca and inserts it into the entry
>> +/// block.
>> llvm::AllocaInst *CodeGenFunction::CreateTempAlloca(llvm::Type *Ty,
>> const Twine &Name) {
>> if (!Builder.isNamePreserving())
>> @@ -61,29 +70,38 @@ llvm::AllocaInst *CodeGenFunction::Creat
>> return new llvm::AllocaInst(Ty, nullptr, Name, AllocaInsertPt);
>> }
>>
>> -void CodeGenFunction::InitTempAlloca(llvm::AllocaInst *Var,
>> - llvm::Value *Init) {
>> - auto *Store = new llvm::StoreInst(Init, Var);
>> +/// CreateDefaultAlignTempAlloca - This creates an alloca with the
>> +/// default alignment of the corresponding LLVM type, which is *not*
>> +/// guaranteed to be related in any way to the expected alignment of
>> +/// an AST type that might have been lowered to Ty.
>> +Address CodeGenFunction::CreateDefaultAlignTempAlloca(llvm::Type *Ty,
>> + const Twine &Name) {
>> + CharUnits Align =
>> + CharUnits::fromQuantity(CGM.getDataLayout().getABITypeAlignment(Ty));
>> + return CreateTempAlloca(Ty, Align, Name);
>> +}
>> +
>> +void CodeGenFunction::InitTempAlloca(Address Var, llvm::Value *Init) {
>> + assert(isa<llvm::AllocaInst>(Var.getPointer()));
>> + auto *Store = new llvm::StoreInst(Init, Var.getPointer());
>> + Store->setAlignment(Var.getAlignment().getQuantity());
>> llvm::BasicBlock *Block = AllocaInsertPt->getParent();
>> Block->getInstList().insertAfter(&*AllocaInsertPt, Store);
>> }
>>
>> -llvm::AllocaInst *CodeGenFunction::CreateIRTemp(QualType Ty,
>> - const Twine &Name) {
>> - llvm::AllocaInst *Alloc = CreateTempAlloca(ConvertType(Ty), Name);
>> - // FIXME: Should we prefer the preferred type alignment here?
>> +Address CodeGenFunction::CreateIRTemp(QualType Ty, const Twine &Name) {
>> CharUnits Align = getContext().getTypeAlignInChars(Ty);
>> - Alloc->setAlignment(Align.getQuantity());
>> - return Alloc;
>> + return CreateTempAlloca(ConvertType(Ty), Align, Name);
>> }
>>
>> -llvm::AllocaInst *CodeGenFunction::CreateMemTemp(QualType Ty,
>> - const Twine &Name) {
>> - llvm::AllocaInst *Alloc = CreateTempAlloca(ConvertTypeForMem(Ty), Name);
>> +Address CodeGenFunction::CreateMemTemp(QualType Ty, const Twine &Name) {
>> // FIXME: Should we prefer the preferred type alignment here?
>> - CharUnits Align = getContext().getTypeAlignInChars(Ty);
>> - Alloc->setAlignment(Align.getQuantity());
>> - return Alloc;
>> + return CreateMemTemp(Ty, getContext().getTypeAlignInChars(Ty), Name);
>> +}
>> +
>> +Address CodeGenFunction::CreateMemTemp(QualType Ty, CharUnits Align,
>> + const Twine &Name) {
>> + return CreateTempAlloca(ConvertTypeForMem(Ty), Align, Name);
>> }
>>
>> /// EvaluateExprAsBool - Perform the usual unary conversions on the specified
>> @@ -148,20 +166,18 @@ RValue CodeGenFunction::EmitAnyExprToTem
>> /// EmitAnyExprToMem - Evaluate an expression into a given memory
>> /// location.
>> void CodeGenFunction::EmitAnyExprToMem(const Expr *E,
>> - llvm::Value *Location,
>> + Address Location,
>> Qualifiers Quals,
>> bool IsInit) {
>> // FIXME: This function should take an LValue as an argument.
>> switch (getEvaluationKind(E->getType())) {
>> case TEK_Complex:
>> - EmitComplexExprIntoLValue(E,
>> - MakeNaturalAlignAddrLValue(Location, E->getType()),
>> + EmitComplexExprIntoLValue(E, MakeAddrLValue(Location, E->getType()),
>> /*isInit*/ false);
>> return;
>>
>> case TEK_Aggregate: {
>> - CharUnits Alignment = getContext().getTypeAlignInChars(E->getType());
>> - EmitAggExpr(E, AggValueSlot::forAddr(Location, Alignment, Quals,
>> + EmitAggExpr(E, AggValueSlot::forAddr(Location, Quals,
>> AggValueSlot::IsDestructed_t(IsInit),
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsAliased_t(!IsInit)));
>> @@ -180,7 +196,7 @@ void CodeGenFunction::EmitAnyExprToMem(c
>>
>> static void
>> pushTemporaryCleanup(CodeGenFunction &CGF, const MaterializeTemporaryExpr *M,
>> - const Expr *E, llvm::Value *ReferenceTemporary) {
>> + const Expr *E, Address ReferenceTemporary) {
>> // Objective-C++ ARC:
>> // If we are binding a reference to a temporary that has ownership, we
>> // need to perform retain/release operations on the temporary.
>> @@ -266,14 +282,14 @@ pushTemporaryCleanup(CodeGenFunction &CG
>> llvm::Constant *CleanupArg;
>> if (E->getType()->isArrayType()) {
>> CleanupFn = CodeGenFunction(CGF.CGM).generateDestroyHelper(
>> - cast<llvm::Constant>(ReferenceTemporary), E->getType(),
>> + ReferenceTemporary, E->getType(),
>> CodeGenFunction::destroyCXXObject, CGF.getLangOpts().Exceptions,
>> dyn_cast_or_null<VarDecl>(M->getExtendingDecl()));
>> CleanupArg = llvm::Constant::getNullValue(CGF.Int8PtrTy);
>> } else {
>> CleanupFn = CGF.CGM.getAddrOfCXXStructor(ReferenceTemporaryDtor,
>> StructorType::Complete);
>> - CleanupArg = cast<llvm::Constant>(ReferenceTemporary);
>> + CleanupArg = cast<llvm::Constant>(ReferenceTemporary.getPointer());
>> }
>> CGF.CGM.getCXXABI().registerGlobalDtor(
>> CGF, *cast<VarDecl>(M->getExtendingDecl()), CleanupFn, CleanupArg);
>> @@ -298,7 +314,7 @@ pushTemporaryCleanup(CodeGenFunction &CG
>> }
>> }
>>
>> -static llvm::Value *
>> +static Address
>> createReferenceTemporary(CodeGenFunction &CGF,
>> const MaterializeTemporaryExpr *M, const Expr *Inner) {
>> switch (M->getStorageDuration()) {
>> @@ -316,10 +332,10 @@ createReferenceTemporary(CodeGenFunction
>> auto *GV = new llvm::GlobalVariable(
>> CGF.CGM.getModule(), Init->getType(), /*isConstant=*/true,
>> llvm::GlobalValue::PrivateLinkage, Init, ".ref.tmp");
>> - GV->setAlignment(
>> - CGF.getContext().getTypeAlignInChars(Ty).getQuantity());
>> + CharUnits alignment = CGF.getContext().getTypeAlignInChars(Ty);
>> + GV->setAlignment(alignment.getQuantity());
>> // FIXME: Should we put the new global into a COMDAT?
>> - return GV;
>> + return Address(GV, alignment);
>> }
>> return CGF.CreateMemTemp(Ty, "ref.tmp");
>> }
>> @@ -343,16 +359,19 @@ EmitMaterializeTemporaryExpr(const Mater
>> M->getType()->isObjCLifetimeType() &&
>> M->getType().getObjCLifetime() != Qualifiers::OCL_None &&
>> M->getType().getObjCLifetime() != Qualifiers::OCL_ExplicitNone) {
>> - llvm::Value *Object = createReferenceTemporary(*this, M, E);
>> - if (auto *Var = dyn_cast<llvm::GlobalVariable>(Object)) {
>> - Object = llvm::ConstantExpr::getBitCast(
>> - Var, ConvertTypeForMem(E->getType())->getPointerTo());
>> + Address Object = createReferenceTemporary(*this, M, E);
>> + if (auto *Var = dyn_cast<llvm::GlobalVariable>(Object.getPointer())) {
>> + Object = Address(llvm::ConstantExpr::getBitCast(Var,
>> + ConvertTypeForMem(E->getType())
>> + ->getPointerTo(Object.getAddressSpace())),
>> + Object.getAlignment());
>> // We should not have emitted the initializer for this temporary as a
>> // constant.
>> assert(!Var->hasInitializer());
>> Var->setInitializer(CGM.EmitNullConstant(E->getType()));
>> }
>> - LValue RefTempDst = MakeAddrLValue(Object, M->getType());
>> + LValue RefTempDst = MakeAddrLValue(Object, M->getType(),
>> + AlignmentSource::Decl);
>>
>> switch (getEvaluationKind(E->getType())) {
>> default: llvm_unreachable("expected scalar or aggregate expression");
>> @@ -360,8 +379,7 @@ EmitMaterializeTemporaryExpr(const Mater
>> EmitScalarInit(E, M->getExtendingDecl(), RefTempDst, false);
>> break;
>> case TEK_Aggregate: {
>> - CharUnits Alignment = getContext().getTypeAlignInChars(E->getType());
>> - EmitAggExpr(E, AggValueSlot::forAddr(Object, Alignment,
>> + EmitAggExpr(E, AggValueSlot::forAddr(Object,
>> E->getType().getQualifiers(),
>> AggValueSlot::IsDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> @@ -389,10 +407,11 @@ EmitMaterializeTemporaryExpr(const Mater
>> }
>>
>> // Create and initialize the reference temporary.
>> - llvm::Value *Object = createReferenceTemporary(*this, M, E);
>> - if (auto *Var = dyn_cast<llvm::GlobalVariable>(Object)) {
>> - Object = llvm::ConstantExpr::getBitCast(
>> - Var, ConvertTypeForMem(E->getType())->getPointerTo());
>> + Address Object = createReferenceTemporary(*this, M, E);
>> + if (auto *Var = dyn_cast<llvm::GlobalVariable>(Object.getPointer())) {
>> + Object = Address(llvm::ConstantExpr::getBitCast(
>> + Var, ConvertTypeForMem(E->getType())->getPointerTo()),
>> + Object.getAlignment());
>> // If the temporary is a global and has a constant initializer or is a
>> // constant temporary that we promoted to a global, we may have already
>> // initialized it.
>> @@ -420,7 +439,8 @@ EmitMaterializeTemporaryExpr(const Mater
>> break;
>>
>> case SubobjectAdjustment::FieldAdjustment: {
>> - LValue LV = MakeAddrLValue(Object, E->getType());
>> + LValue LV = MakeAddrLValue(Object, E->getType(),
>> + AlignmentSource::Decl);
>> LV = EmitLValueForField(LV, Adjustment.Field);
>> assert(LV.isSimple() &&
>> "materialized temporary field is not a simple lvalue");
>> @@ -430,14 +450,14 @@ EmitMaterializeTemporaryExpr(const Mater
>>
>> case SubobjectAdjustment::MemberPointerAdjustment: {
>> llvm::Value *Ptr = EmitScalarExpr(Adjustment.Ptr.RHS);
>> - Object = CGM.getCXXABI().EmitMemberDataPointerAddress(
>> - *this, E, Object, Ptr, Adjustment.Ptr.MPT);
>> + Object = EmitCXXMemberDataPointerAddress(E, Object, Ptr,
>> + Adjustment.Ptr.MPT);
>> break;
>> }
>> }
>> }
>>
>> - return MakeAddrLValue(Object, M->getType());
>> + return MakeAddrLValue(Object, M->getType(), AlignmentSource::Decl);
>> }
>>
>> RValue
>> @@ -445,7 +465,7 @@ CodeGenFunction::EmitReferenceBindingToE
>> // Emit the expression as an lvalue.
>> LValue LV = EmitLValue(E);
>> assert(LV.isSimple());
>> - llvm::Value *Value = LV.getAddress();
>> + llvm::Value *Value = LV.getPointer();
>>
>> if (sanitizePerformTypeCheck() && !E->getType()->isFunctionType()) {
>> // C++11 [dcl.ref]p5 (as amended by core issue 453):
>> @@ -489,7 +509,7 @@ bool CodeGenFunction::sanitizePerformTyp
>> }
>>
>> void CodeGenFunction::EmitTypeCheck(TypeCheckKind TCK, SourceLocation Loc,
>> - llvm::Value *Address, QualType Ty,
>> + llvm::Value *Ptr, QualType Ty,
>> CharUnits Alignment, bool SkipNullCheck) {
>> if (!sanitizePerformTypeCheck())
>> return;
>> @@ -497,7 +517,7 @@ void CodeGenFunction::EmitTypeCheck(Type
>> // Don't check pointers outside the default address space. The null check
>> // isn't correct, the object-size check isn't supported by LLVM, and we can't
>> // communicate the addresses to the runtime handler for the vptr check.
>> - if (Address->getType()->getPointerAddressSpace())
>> + if (Ptr->getType()->getPointerAddressSpace())
>> return;
>>
>> SanitizerScope SanScope(this);
>> @@ -510,8 +530,7 @@ void CodeGenFunction::EmitTypeCheck(Type
>> if ((SanOpts.has(SanitizerKind::Null) || AllowNullPointers) &&
>> !SkipNullCheck) {
>> // The glvalue must not be an empty glvalue.
>> - llvm::Value *IsNonNull = Builder.CreateICmpNE(
>> - Address, llvm::Constant::getNullValue(Address->getType()));
>> + llvm::Value *IsNonNull = Builder.CreateIsNotNull(Ptr);
>>
>> if (AllowNullPointers) {
>> // When performing pointer casts, it's OK if the value is null.
>> @@ -535,7 +554,7 @@ void CodeGenFunction::EmitTypeCheck(Type
>> llvm::Type *Tys[2] = { IntPtrTy, Int8PtrTy };
>> llvm::Value *F = CGM.getIntrinsic(llvm::Intrinsic::objectsize, Tys);
>> llvm::Value *Min = Builder.getFalse();
>> - llvm::Value *CastAddr = Builder.CreateBitCast(Address, Int8PtrTy);
>> + llvm::Value *CastAddr = Builder.CreateBitCast(Ptr, Int8PtrTy);
>> llvm::Value *LargeEnough =
>> Builder.CreateICmpUGE(Builder.CreateCall(F, {CastAddr, Min}),
>> llvm::ConstantInt::get(IntPtrTy, Size));
>> @@ -552,7 +571,7 @@ void CodeGenFunction::EmitTypeCheck(Type
>> // The glvalue must be suitably aligned.
>> if (AlignVal) {
>> llvm::Value *Align =
>> - Builder.CreateAnd(Builder.CreatePtrToInt(Address, IntPtrTy),
>> + Builder.CreateAnd(Builder.CreatePtrToInt(Ptr, IntPtrTy),
>> llvm::ConstantInt::get(IntPtrTy, AlignVal - 1));
>> llvm::Value *Aligned =
>> Builder.CreateICmpEQ(Align, llvm::ConstantInt::get(IntPtrTy, 0));
>> @@ -567,7 +586,7 @@ void CodeGenFunction::EmitTypeCheck(Type
>> llvm::ConstantInt::get(SizeTy, AlignVal),
>> llvm::ConstantInt::get(Int8Ty, TCK)
>> };
>> - EmitCheck(Checks, "type_mismatch", StaticData, Address);
>> + EmitCheck(Checks, "type_mismatch", StaticData, Ptr);
>> }
>>
>> // If possible, check that the vptr indicates that there is a subobject of
>> @@ -602,7 +621,7 @@ void CodeGenFunction::EmitTypeCheck(Type
>> // Load the vptr, and compute hash_16_bytes(TypeHash, vptr).
>> llvm::Value *Low = llvm::ConstantInt::get(Int64Ty, TypeHash);
>> llvm::Type *VPtrTy = llvm::PointerType::get(IntPtrTy, 0);
>> - llvm::Value *VPtrAddr = Builder.CreateBitCast(Address, VPtrTy);
>> + Address VPtrAddr(Builder.CreateBitCast(Ptr, VPtrTy), getPointerAlign());
>> llvm::Value *VPtrVal = Builder.CreateLoad(VPtrAddr);
>> llvm::Value *High = Builder.CreateZExt(VPtrVal, Int64Ty);
>>
>> @@ -619,7 +638,8 @@ void CodeGenFunction::EmitTypeCheck(Type
>> CacheSize-1));
>> llvm::Value *Indices[] = { Builder.getInt32(0), Slot };
>> llvm::Value *CacheVal =
>> - Builder.CreateLoad(Builder.CreateInBoundsGEP(Cache, Indices));
>> + Builder.CreateAlignedLoad(Builder.CreateInBoundsGEP(Cache, Indices),
>> + getPointerAlign());
>>
>> // If the hash isn't in the cache, call a runtime handler to perform the
>> // hard work of checking whether the vptr is for an object of the right
>> @@ -632,7 +652,7 @@ void CodeGenFunction::EmitTypeCheck(Type
>> CGM.GetAddrOfRTTIDescriptor(Ty.getUnqualifiedType()),
>> llvm::ConstantInt::get(Int8Ty, TCK)
>> };
>> - llvm::Value *DynamicData[] = { Address, Hash };
>> + llvm::Value *DynamicData[] = { Ptr, Hash };
>> EmitCheck(std::make_pair(EqualHash, SanitizerKind::Vptr),
>> "dynamic_type_cache_miss", StaticData, DynamicData);
>> }
>> @@ -764,6 +784,84 @@ EmitComplexPrePostIncDec(const UnaryOper
>> // LValue Expression Emission
>> //===----------------------------------------------------------------------===//
>>
>> +/// EmitPointerWithAlignment - Given an expression of pointer type, try to
>> +/// derive a more accurate bound on the alignment of the pointer.
>> +Address CodeGenFunction::EmitPointerWithAlignment(const Expr *E,
>> + AlignmentSource *Source) {
>> + // We allow this with ObjC object pointers because of fragile ABIs.
>> + assert(E->getType()->isPointerType() ||
>> + E->getType()->isObjCObjectPointerType());
>> + E = E->IgnoreParens();
>> +
>> + // Casts:
>> + if (const CastExpr *CE = dyn_cast<CastExpr>(E)) {
>> + // Bind VLAs in the cast type.
>> + if (E->getType()->isVariablyModifiedType())
>> + EmitVariablyModifiedType(E->getType());
>> +
>> + switch (CE->getCastKind()) {
>> + // Non-converting casts (but not C's implicit conversion from void*).
>> + case CK_BitCast:
>> + case CK_NoOp:
>> + if (auto PtrTy = CE->getSubExpr()->getType()->getAs<PointerType>()) {
>> + if (PtrTy->getPointeeType()->isVoidType())
>> + break;
>> +
>> + AlignmentSource InnerSource;
>> + Address Addr = EmitPointerWithAlignment(CE->getSubExpr(), &InnerSource);
>> + if (Source) *Source = InnerSource;
>> +
>> + // If this is an explicit bitcast, and the source l-value is
>> + // opaque, honor the alignment of the casted-to type.
>> + if (isa<ExplicitCastExpr>(CE) &&
>> + CE->getCastKind() == CK_BitCast &&
>> + InnerSource != AlignmentSource::Decl) {
>> + Addr = Address(Addr.getPointer(),
>> + getNaturalPointeeTypeAlignment(E->getType(), Source));
>> + }
>> +
>> + return Builder.CreateBitCast(Addr, ConvertType(E->getType()));
>> + }
>> + break;
>> +
>> + // Array-to-pointer decay.
>> + case CK_ArrayToPointerDecay:
>> + return EmitArrayToPointerDecay(CE->getSubExpr(), Source);
>> +
>> + // Derived-to-base conversions.
>> + case CK_UncheckedDerivedToBase:
>> + case CK_DerivedToBase: {
>> + Address Addr = EmitPointerWithAlignment(CE->getSubExpr(), Source);
>> + auto Derived = CE->getSubExpr()->getType()->getPointeeCXXRecordDecl();
>> + return GetAddressOfBaseClass(Addr, Derived,
>> + CE->path_begin(), CE->path_end(),
>> + ShouldNullCheckClassCastValue(CE),
>> + CE->getExprLoc());
>> + }
>> +
>> + // TODO: Is there any reason to treat base-to-derived conversions
>> + // specially?
>> + default:
>> + break;
>> + }
>> + }
>> +
>> + // Unary &.
>> + if (const UnaryOperator *UO = dyn_cast<UnaryOperator>(E)) {
>> + if (UO->getOpcode() == UO_AddrOf) {
>> + LValue LV = EmitLValue(UO->getSubExpr());
>> + if (Source) *Source = LV.getAlignmentSource();
>> + return LV.getAddress();
>> + }
>> + }
>> +
>> + // TODO: conditional operators, comma.
>> +
>> + // Otherwise, use the alignment of the type.
>> + CharUnits Align = getNaturalPointeeTypeAlignment(E->getType(), Source);
>> + return Address(EmitScalarExpr(E), Align);
>> +}
>> +
>> RValue CodeGenFunction::GetUndefRValue(QualType Ty) {
>> if (Ty->isVoidType())
>> return RValue::get(nullptr);
>> @@ -780,7 +878,7 @@ RValue CodeGenFunction::GetUndefRValue(Q
>> // identifiable address. Just because the contents of the value are undefined
>> // doesn't mean that the address can't be taken and compared.
>> case TEK_Aggregate: {
>> - llvm::Value *DestPtr = CreateMemTemp(Ty, "undef.agg.tmp");
>> + Address DestPtr = CreateMemTemp(Ty, "undef.agg.tmp");
>> return RValue::getAggregate(DestPtr);
>> }
>>
>> @@ -800,7 +898,8 @@ LValue CodeGenFunction::EmitUnsupportedL
>> const char *Name) {
>> ErrorUnsupported(E, Name);
>> llvm::Type *Ty = llvm::PointerType::getUnqual(ConvertType(E->getType()));
>> - return MakeAddrLValue(llvm::UndefValue::get(Ty), E->getType());
>> + return MakeAddrLValue(Address(llvm::UndefValue::get(Ty), CharUnits::One()),
>> + E->getType());
>> }
>>
>> LValue CodeGenFunction::EmitCheckedLValue(const Expr *E, TypeCheckKind TCK) {
>> @@ -810,7 +909,7 @@ LValue CodeGenFunction::EmitCheckedLValu
>> else
>> LV = EmitLValue(E);
>> if (!isa<DeclRefExpr>(E) && !LV.isBitField() && LV.isSimple())
>> - EmitTypeCheck(TCK, E->getExprLoc(), LV.getAddress(),
>> + EmitTypeCheck(TCK, E->getExprLoc(), LV.getPointer(),
>> E->getType(), LV.getAlignment());
>> return LV;
>> }
>> @@ -1059,8 +1158,8 @@ CodeGenFunction::tryEmitAsConstant(DeclR
>> llvm::Value *CodeGenFunction::EmitLoadOfScalar(LValue lvalue,
>> SourceLocation Loc) {
>> return EmitLoadOfScalar(lvalue.getAddress(), lvalue.isVolatile(),
>> - lvalue.getAlignment().getQuantity(),
>> - lvalue.getType(), Loc, lvalue.getTBAAInfo(),
>> + lvalue.getType(), Loc, lvalue.getAlignmentSource(),
>> + lvalue.getTBAAInfo(),
>> lvalue.getTBAABaseType(), lvalue.getTBAAOffset());
>> }
>>
>> @@ -1121,37 +1220,31 @@ llvm::MDNode *CodeGenFunction::getRangeF
>> return MDHelper.createRange(Min, End);
>> }
>>
>> -llvm::Value *CodeGenFunction::EmitLoadOfScalar(llvm::Value *Addr, bool Volatile,
>> - unsigned Alignment, QualType Ty,
>> +llvm::Value *CodeGenFunction::EmitLoadOfScalar(Address Addr, bool Volatile,
>> + QualType Ty,
>> SourceLocation Loc,
>> + AlignmentSource AlignSource,
>> llvm::MDNode *TBAAInfo,
>> QualType TBAABaseType,
>> uint64_t TBAAOffset) {
>> // For better performance, handle vector loads differently.
>> if (Ty->isVectorType()) {
>> - llvm::Value *V;
>> - const llvm::Type *EltTy =
>> - cast<llvm::PointerType>(Addr->getType())->getElementType();
>> + const llvm::Type *EltTy = Addr.getElementType();
>>
>> const auto *VTy = cast<llvm::VectorType>(EltTy);
>>
>> - // Handle vectors of size 3, like size 4 for better performance.
>> + // Handle vectors of size 3 like size 4 for better performance.
>> if (VTy->getNumElements() == 3) {
>>
>> // Bitcast to vec4 type.
>> llvm::VectorType *vec4Ty = llvm::VectorType::get(VTy->getElementType(),
>> 4);
>> - llvm::PointerType *ptVec4Ty =
>> - llvm::PointerType::get(vec4Ty,
>> - (cast<llvm::PointerType>(
>> - Addr->getType()))->getAddressSpace());
>> - llvm::Value *Cast = Builder.CreateBitCast(Addr, ptVec4Ty,
>> - "castToVec4");
>> + Address Cast = Builder.CreateElementBitCast(Addr, vec4Ty, "castToVec4");
>> // Now load value.
>> - llvm::Value *LoadVal = Builder.CreateLoad(Cast, Volatile, "loadVec4");
>> + llvm::Value *V = Builder.CreateLoad(Cast, Volatile, "loadVec4");
>>
>> // Shuffle vector to get vec3.
>> - V = Builder.CreateShuffleVector(LoadVal, llvm::UndefValue::get(vec4Ty),
>> + V = Builder.CreateShuffleVector(V, llvm::UndefValue::get(vec4Ty),
>> {0, 1, 2}, "extractVec");
>> return EmitFromMemory(V, Ty);
>> }
>> @@ -1159,17 +1252,12 @@ llvm::Value *CodeGenFunction::EmitLoadOf
>>
>> // Atomic operations have to be done on integral types.
>> if (Ty->isAtomicType() || typeIsSuitableForInlineAtomic(Ty, Volatile)) {
>> - LValue lvalue = LValue::MakeAddr(Addr, Ty,
>> - CharUnits::fromQuantity(Alignment),
>> - getContext(), TBAAInfo);
>> + LValue lvalue =
>> + LValue::MakeAddr(Addr, Ty, getContext(), AlignSource, TBAAInfo);
>> return EmitAtomicLoad(lvalue, Loc).getScalarVal();
>> }
>>
>> - llvm::LoadInst *Load = Builder.CreateLoad(Addr);
>> - if (Volatile)
>> - Load->setVolatile(true);
>> - if (Alignment)
>> - Load->setAlignment(Alignment);
>> + llvm::LoadInst *Load = Builder.CreateLoad(Addr, Volatile);
>> if (TBAAInfo) {
>> llvm::MDNode *TBAAPath = CGM.getTBAAStructTagInfo(TBAABaseType, TBAAInfo,
>> TBAAOffset);
>> @@ -1237,9 +1325,10 @@ llvm::Value *CodeGenFunction::EmitFromMe
>> return Value;
>> }
>>
>> -void CodeGenFunction::EmitStoreOfScalar(llvm::Value *Value, llvm::Value *Addr,
>> - bool Volatile, unsigned Alignment,
>> - QualType Ty, llvm::MDNode *TBAAInfo,
>> +void CodeGenFunction::EmitStoreOfScalar(llvm::Value *Value, Address Addr,
>> + bool Volatile, QualType Ty,
>> + AlignmentSource AlignSource,
>> + llvm::MDNode *TBAAInfo,
>> bool isInit, QualType TBAABaseType,
>> uint64_t TBAAOffset) {
>>
>> @@ -1259,11 +1348,8 @@ void CodeGenFunction::EmitStoreOfScalar(
>> MaskV, "extractVec");
>> SrcTy = llvm::VectorType::get(VecTy->getElementType(), 4);
>> }
>> - auto *DstPtr = cast<llvm::PointerType>(Addr->getType());
>> - if (DstPtr->getElementType() != SrcTy) {
>> - llvm::Type *MemTy =
>> - llvm::PointerType::get(SrcTy, DstPtr->getAddressSpace());
>> - Addr = Builder.CreateBitCast(Addr, MemTy, "storetmp");
>> + if (Addr.getElementType() != SrcTy) {
>> + Addr = Builder.CreateElementBitCast(Addr, SrcTy, "storetmp");
>> }
>> }
>>
>> @@ -1272,16 +1358,13 @@ void CodeGenFunction::EmitStoreOfScalar(
>> if (Ty->isAtomicType() ||
>> (!isInit && typeIsSuitableForInlineAtomic(Ty, Volatile))) {
>> EmitAtomicStore(RValue::get(Value),
>> - LValue::MakeAddr(Addr, Ty,
>> - CharUnits::fromQuantity(Alignment),
>> - getContext(), TBAAInfo),
>> + LValue::MakeAddr(Addr, Ty, getContext(),
>> + AlignSource, TBAAInfo),
>> isInit);
>> return;
>> }
>>
>> llvm::StoreInst *Store = Builder.CreateStore(Value, Addr, Volatile);
>> - if (Alignment)
>> - Store->setAlignment(Alignment);
>> if (TBAAInfo) {
>> llvm::MDNode *TBAAPath = CGM.getTBAAStructTagInfo(TBAABaseType, TBAAInfo,
>> TBAAOffset);
>> @@ -1293,7 +1376,7 @@ void CodeGenFunction::EmitStoreOfScalar(
>> void CodeGenFunction::EmitStoreOfScalar(llvm::Value *value, LValue lvalue,
>> bool isInit) {
>> EmitStoreOfScalar(value, lvalue.getAddress(), lvalue.isVolatile(),
>> - lvalue.getAlignment().getQuantity(), lvalue.getType(),
>> + lvalue.getType(), lvalue.getAlignmentSource(),
>> lvalue.getTBAAInfo(), isInit, lvalue.getTBAABaseType(),
>> lvalue.getTBAAOffset());
>> }
>> @@ -1304,7 +1387,7 @@ void CodeGenFunction::EmitStoreOfScalar(
>> RValue CodeGenFunction::EmitLoadOfLValue(LValue LV, SourceLocation Loc) {
>> if (LV.isObjCWeak()) {
>> // load of a __weak object.
>> - llvm::Value *AddrWeakObj = LV.getAddress();
>> + Address AddrWeakObj = LV.getAddress();
>> return RValue::get(CGM.getObjCRuntime().EmitObjCWeakRead(*this,
>> AddrWeakObj));
>> }
>> @@ -1322,9 +1405,8 @@ RValue CodeGenFunction::EmitLoadOfLValue
>> }
>>
>> if (LV.isVectorElt()) {
>> - llvm::LoadInst *Load = Builder.CreateLoad(LV.getVectorAddr(),
>> + llvm::LoadInst *Load = Builder.CreateLoad(LV.getVectorAddress(),
>> LV.isVolatileQualified());
>> - Load->setAlignment(LV.getAlignment().getQuantity());
>> return RValue::get(Builder.CreateExtractElement(Load, LV.getVectorIdx(),
>> "vecext"));
>> }
>> @@ -1344,15 +1426,12 @@ RValue CodeGenFunction::EmitLoadOfLValue
>>
>> RValue CodeGenFunction::EmitLoadOfBitfieldLValue(LValue LV) {
>> const CGBitFieldInfo &Info = LV.getBitFieldInfo();
>> - CharUnits Align = LV.getAlignment().alignmentAtOffset(Info.StorageOffset);
>>
>> // Get the output type.
>> llvm::Type *ResLTy = ConvertType(LV.getType());
>>
>> - llvm::Value *Ptr = LV.getBitFieldAddr();
>> - llvm::Value *Val = Builder.CreateAlignedLoad(Ptr, Align.getQuantity(),
>> - LV.isVolatileQualified(),
>> - "bf.load");
>> + Address Ptr = LV.getBitFieldAddress();
>> + llvm::Value *Val = Builder.CreateLoad(Ptr, LV.isVolatileQualified(), "bf.load");
>>
>> if (Info.IsSigned) {
>> assert(static_cast<unsigned>(Info.Offset + Info.Size) <= Info.StorageSize);
>> @@ -1377,10 +1456,8 @@ RValue CodeGenFunction::EmitLoadOfBitfie
>> // If this is a reference to a subset of the elements of a vector, create an
>> // appropriate shufflevector.
>> RValue CodeGenFunction::EmitLoadOfExtVectorElementLValue(LValue LV) {
>> - llvm::LoadInst *Load = Builder.CreateLoad(LV.getExtVectorAddr(),
>> - LV.isVolatileQualified());
>> - Load->setAlignment(LV.getAlignment().getQuantity());
>> - llvm::Value *Vec = Load;
>> + llvm::Value *Vec = Builder.CreateLoad(LV.getExtVectorAddress(),
>> + LV.isVolatileQualified());
>>
>> const llvm::Constant *Elts = LV.getExtVectorElts();
>>
>> @@ -1407,24 +1484,24 @@ RValue CodeGenFunction::EmitLoadOfExtVec
>> }
>>
>> /// @brief Generates lvalue for partial ext_vector access.
>> -llvm::Value *CodeGenFunction::EmitExtVectorElementLValue(LValue LV) {
>> - llvm::Value *VectorAddress = LV.getExtVectorAddr();
>> +Address CodeGenFunction::EmitExtVectorElementLValue(LValue LV) {
>> + Address VectorAddress = LV.getExtVectorAddress();
>> const VectorType *ExprVT = LV.getType()->getAs<VectorType>();
>> QualType EQT = ExprVT->getElementType();
>> llvm::Type *VectorElementTy = CGM.getTypes().ConvertType(EQT);
>> - llvm::Type *VectorElementPtrToTy = VectorElementTy->getPointerTo();
>>
>> - llvm::Value *CastToPointerElement =
>> - Builder.CreateBitCast(VectorAddress,
>> - VectorElementPtrToTy, "conv.ptr.element");
>> + Address CastToPointerElement =
>> + Builder.CreateElementBitCast(VectorAddress, VectorElementTy,
>> + "conv.ptr.element");
>>
>> const llvm::Constant *Elts = LV.getExtVectorElts();
>> unsigned ix = getAccessedFieldNo(0, Elts);
>>
>> - llvm::Value *VectorBasePtrPlusIx =
>> - Builder.CreateInBoundsGEP(CastToPointerElement,
>> - llvm::ConstantInt::get(SizeTy, ix), "add.ptr");
>> -
>> + Address VectorBasePtrPlusIx =
>> + Builder.CreateConstInBoundsGEP(CastToPointerElement, ix,
>> + getContext().getTypeSizeInChars(EQT),
>> + "vector.elt");
>> +
>> return VectorBasePtrPlusIx;
>> }
>>
>> @@ -1459,15 +1536,12 @@ void CodeGenFunction::EmitStoreThroughLV
>> if (!Dst.isSimple()) {
>> if (Dst.isVectorElt()) {
>> // Read/modify/write the vector, inserting the new element.
>> - llvm::LoadInst *Load = Builder.CreateLoad(Dst.getVectorAddr(),
>> - Dst.isVolatileQualified());
>> - Load->setAlignment(Dst.getAlignment().getQuantity());
>> - llvm::Value *Vec = Load;
>> + llvm::Value *Vec = Builder.CreateLoad(Dst.getVectorAddress(),
>> + Dst.isVolatileQualified());
>> Vec = Builder.CreateInsertElement(Vec, Src.getScalarVal(),
>> Dst.getVectorIdx(), "vecins");
>> - llvm::StoreInst *Store = Builder.CreateStore(Vec, Dst.getVectorAddr(),
>> - Dst.isVolatileQualified());
>> - Store->setAlignment(Dst.getAlignment().getQuantity());
>> + Builder.CreateStore(Vec, Dst.getVectorAddress(),
>> + Dst.isVolatileQualified());
>> return;
>> }
>>
>> @@ -1511,7 +1585,7 @@ void CodeGenFunction::EmitStoreThroughLV
>>
>> if (Dst.isObjCWeak() && !Dst.isNonGC()) {
>> // load of a __weak object.
>> - llvm::Value *LvalueDst = Dst.getAddress();
>> + Address LvalueDst = Dst.getAddress();
>> llvm::Value *src = Src.getScalarVal();
>> CGM.getObjCRuntime().EmitObjCWeakAssign(*this, src, LvalueDst);
>> return;
>> @@ -1519,16 +1593,17 @@ void CodeGenFunction::EmitStoreThroughLV
>>
>> if (Dst.isObjCStrong() && !Dst.isNonGC()) {
>> // load of a __strong object.
>> - llvm::Value *LvalueDst = Dst.getAddress();
>> + Address LvalueDst = Dst.getAddress();
>> llvm::Value *src = Src.getScalarVal();
>> if (Dst.isObjCIvar()) {
>> assert(Dst.getBaseIvarExp() && "BaseIvarExp is NULL");
>> - llvm::Type *ResultType = ConvertType(getContext().LongTy);
>> - llvm::Value *RHS = EmitScalarExpr(Dst.getBaseIvarExp());
>> - llvm::Value *dst = RHS;
>> + llvm::Type *ResultType = IntPtrTy;
>> + Address dst = EmitPointerWithAlignment(Dst.getBaseIvarExp());
>> + llvm::Value *RHS = dst.getPointer();
>> RHS = Builder.CreatePtrToInt(RHS, ResultType, "sub.ptr.rhs.cast");
>> llvm::Value *LHS =
>> - Builder.CreatePtrToInt(LvalueDst, ResultType, "sub.ptr.lhs.cast");
>> + Builder.CreatePtrToInt(LvalueDst.getPointer(), ResultType,
>> + "sub.ptr.lhs.cast");
>> llvm::Value *BytesBetween = Builder.CreateSub(LHS, RHS, "ivar.offset");
>> CGM.getObjCRuntime().EmitObjCIvarAssign(*this, src, dst,
>> BytesBetween);
>> @@ -1548,16 +1623,14 @@ void CodeGenFunction::EmitStoreThroughLV
>> void CodeGenFunction::EmitStoreThroughBitfieldLValue(RValue Src, LValue Dst,
>> llvm::Value **Result) {
>> const CGBitFieldInfo &Info = Dst.getBitFieldInfo();
>> - CharUnits Align = Dst.getAlignment().alignmentAtOffset(Info.StorageOffset);
>> llvm::Type *ResLTy = ConvertTypeForMem(Dst.getType());
>> - llvm::Value *Ptr = Dst.getBitFieldAddr();
>> + Address Ptr = Dst.getBitFieldAddress();
>>
>> // Get the source value, truncated to the width of the bit-field.
>> llvm::Value *SrcVal = Src.getScalarVal();
>>
>> // Cast the source to the storage type and shift it into place.
>> - SrcVal = Builder.CreateIntCast(SrcVal,
>> - Ptr->getType()->getPointerElementType(),
>> + SrcVal = Builder.CreateIntCast(SrcVal, Ptr.getElementType(),
>> /*IsSigned=*/false);
>> llvm::Value *MaskedVal = SrcVal;
>>
>> @@ -1565,9 +1638,8 @@ void CodeGenFunction::EmitStoreThroughBi
>> // and mask together with source before storing.
>> if (Info.StorageSize != Info.Size) {
>> assert(Info.StorageSize > Info.Size && "Invalid bitfield size.");
>> - llvm::Value *Val = Builder.CreateAlignedLoad(Ptr, Align.getQuantity(),
>> - Dst.isVolatileQualified(),
>> - "bf.load");
>> + llvm::Value *Val =
>> + Builder.CreateLoad(Ptr, Dst.isVolatileQualified(), "bf.load");
>>
>> // Mask the source value as needed.
>> if (!hasBooleanRepresentation(Dst.getType()))
>> @@ -1593,8 +1665,7 @@ void CodeGenFunction::EmitStoreThroughBi
>> }
>>
>> // Write the new value back out.
>> - Builder.CreateAlignedStore(SrcVal, Ptr, Align.getQuantity(),
>> - Dst.isVolatileQualified());
>> + Builder.CreateStore(SrcVal, Ptr, Dst.isVolatileQualified());
>>
>> // Return the new value of the bit-field, if requested.
>> if (Result) {
>> @@ -1620,10 +1691,8 @@ void CodeGenFunction::EmitStoreThroughEx
>> LValue Dst) {
>> // This access turns into a read/modify/write of the vector. Load the input
>> // value now.
>> - llvm::LoadInst *Load = Builder.CreateLoad(Dst.getExtVectorAddr(),
>> - Dst.isVolatileQualified());
>> - Load->setAlignment(Dst.getAlignment().getQuantity());
>> - llvm::Value *Vec = Load;
>> + llvm::Value *Vec = Builder.CreateLoad(Dst.getExtVectorAddress(),
>> + Dst.isVolatileQualified());
>> const llvm::Constant *Elts = Dst.getExtVectorElts();
>>
>> llvm::Value *SrcVal = Src.getScalarVal();
>> @@ -1685,9 +1754,8 @@ void CodeGenFunction::EmitStoreThroughEx
>> Vec = Builder.CreateInsertElement(Vec, SrcVal, Elt);
>> }
>>
>> - llvm::StoreInst *Store = Builder.CreateStore(Vec, Dst.getExtVectorAddr(),
>> - Dst.isVolatileQualified());
>> - Store->setAlignment(Dst.getAlignment().getQuantity());
>> + Builder.CreateStore(Vec, Dst.getExtVectorAddress(),
>> + Dst.isVolatileQualified());
>> }
>>
>> /// @brief Store of global named registers are always calls to intrinsics.
>> @@ -1822,11 +1890,27 @@ EmitBitCastOfLValueToProperType(CodeGenF
>> }
>>
>> static LValue EmitThreadPrivateVarDeclLValue(
>> - CodeGenFunction &CGF, const VarDecl *VD, QualType T, llvm::Value *V,
>> - llvm::Type *RealVarTy, CharUnits Alignment, SourceLocation Loc) {
>> - V = CGF.CGM.getOpenMPRuntime().getAddrOfThreadPrivate(CGF, VD, V, Loc);
>> - V = EmitBitCastOfLValueToProperType(CGF, V, RealVarTy);
>> - return CGF.MakeAddrLValue(V, T, Alignment);
>> + CodeGenFunction &CGF, const VarDecl *VD, QualType T, Address Addr,
>> + llvm::Type *RealVarTy, SourceLocation Loc) {
>> + Addr = CGF.CGM.getOpenMPRuntime().getAddrOfThreadPrivate(CGF, VD, Addr, Loc);
>> + Addr = CGF.Builder.CreateElementBitCast(Addr, RealVarTy);
>> + return CGF.MakeAddrLValue(Addr, T, AlignmentSource::Decl);
>> +}
>> +
>> +Address CodeGenFunction::EmitLoadOfReference(Address Addr,
>> + const ReferenceType *RefTy,
>> + AlignmentSource *Source) {
>> + llvm::Value *Ptr = Builder.CreateLoad(Addr);
>> + return Address(Ptr, getNaturalTypeAlignment(RefTy->getPointeeType(),
>> + Source, /*forPointee*/ true));
>> +
>> +}
>> +
>> +LValue CodeGenFunction::EmitLoadOfReferenceLValue(Address RefAddr,
>> + const ReferenceType *RefTy) {
>> + AlignmentSource Source;
>> + Address Addr = EmitLoadOfReference(RefAddr, RefTy, &Source);
>> + return MakeAddrLValue(Addr, RefTy->getPointeeType(), Source);
>> }
>>
>> static LValue EmitGlobalVarDeclLValue(CodeGenFunction &CGF,
>> @@ -1842,19 +1926,17 @@ static LValue EmitGlobalVarDeclLValue(Co
>> llvm::Type *RealVarTy = CGF.getTypes().ConvertTypeForMem(VD->getType());
>> V = EmitBitCastOfLValueToProperType(CGF, V, RealVarTy);
>> CharUnits Alignment = CGF.getContext().getDeclAlign(VD);
>> + Address Addr(V, Alignment);
>> LValue LV;
>> // Emit reference to the private copy of the variable if it is an OpenMP
>> // threadprivate variable.
>> if (CGF.getLangOpts().OpenMP && VD->hasAttr<OMPThreadPrivateDeclAttr>())
>> - return EmitThreadPrivateVarDeclLValue(CGF, VD, T, V, RealVarTy, Alignment,
>> + return EmitThreadPrivateVarDeclLValue(CGF, VD, T, Addr, RealVarTy,
>> E->getExprLoc());
>> - if (VD->getType()->isReferenceType()) {
>> - llvm::LoadInst *LI = CGF.Builder.CreateLoad(V);
>> - LI->setAlignment(Alignment.getQuantity());
>> - V = LI;
>> - LV = CGF.MakeNaturalAlignAddrLValue(V, T);
>> + if (auto RefTy = VD->getType()->getAs<ReferenceType>()) {
>> + LV = CGF.EmitLoadOfReferenceLValue(Addr, RefTy);
>> } else {
>> - LV = CGF.MakeAddrLValue(V, T, Alignment);
>> + LV = CGF.MakeAddrLValue(Addr, T, AlignmentSource::Decl);
>> }
>> setObjCGCLValueClass(CGF.getContext(), E, LV);
>> return LV;
>> @@ -1876,7 +1958,7 @@ static LValue EmitFunctionDeclLValue(Cod
>> }
>> }
>> CharUnits Alignment = CGF.getContext().getDeclAlign(FD);
>> - return CGF.MakeAddrLValue(V, E->getType(), Alignment);
>> + return CGF.MakeAddrLValue(V, E->getType(), Alignment, AlignmentSource::Decl);
>> }
>>
>> static LValue EmitCapturedFieldLValue(CodeGenFunction &CGF, const FieldDecl *FD,
>> @@ -1892,9 +1974,7 @@ static LValue EmitCapturedFieldLValue(Co
>> /// So far, only the name is being passed down, but other options such as
>> /// register type, allocation type or even optimization options could be
>> /// passed down via the metadata node.
>> -static LValue EmitGlobalNamedRegister(const VarDecl *VD,
>> - CodeGenModule &CGM,
>> - CharUnits Alignment) {
>> +static LValue EmitGlobalNamedRegister(const VarDecl *VD, CodeGenModule &CGM) {
>> SmallString<64> Name("llvm.named.register.");
>> AsmLabelAttr *Asm = VD->getAttr<AsmLabelAttr>();
>> assert(Asm->getLabel().size() < 64-Name.size() &&
>> @@ -1908,21 +1988,23 @@ static LValue EmitGlobalNamedRegister(co
>> llvm::Metadata *Ops[] = {Str};
>> M->addOperand(llvm::MDNode::get(CGM.getLLVMContext(), Ops));
>> }
>> - return LValue::MakeGlobalReg(
>> - llvm::MetadataAsValue::get(CGM.getLLVMContext(), M->getOperand(0)),
>> - VD->getType(), Alignment);
>> +
>> + CharUnits Alignment = CGM.getContext().getDeclAlign(VD);
>> +
>> + llvm::Value *Ptr =
>> + llvm::MetadataAsValue::get(CGM.getLLVMContext(), M->getOperand(0));
>> + return LValue::MakeGlobalReg(Address(Ptr, Alignment), VD->getType());
>> }
>>
>> LValue CodeGenFunction::EmitDeclRefLValue(const DeclRefExpr *E) {
>> const NamedDecl *ND = E->getDecl();
>> - CharUnits Alignment = getContext().getDeclAlign(ND);
>> QualType T = E->getType();
>>
>> if (const auto *VD = dyn_cast<VarDecl>(ND)) {
>> // Global Named registers access via intrinsics only
>> if (VD->getStorageClass() == SC_Register &&
>> VD->hasAttr<AsmLabelAttr>() && !VD->isLocalVarDecl())
>> - return EmitGlobalNamedRegister(VD, CGM, Alignment);
>> + return EmitGlobalNamedRegister(VD, CGM);
>>
>> // A DeclRefExpr for a reference initialized by a constant expression can
>> // appear without being odr-used. Directly emit the constant initializer.
>> @@ -1934,7 +2016,12 @@ LValue CodeGenFunction::EmitDeclRefLValu
>> CGM.EmitConstantValue(*VD->evaluateValue(), VD->getType(), this);
>> assert(Val && "failed to emit reference constant expression");
>> // FIXME: Eventually we will want to emit vector element references.
>> - return MakeAddrLValue(Val, T, Alignment);
>> +
>> + // Should we be using the alignment of the constant pointer we emitted?
>> + CharUnits Alignment = getNaturalTypeAlignment(E->getType(), nullptr,
>> + /*pointee*/ true);
>> +
>> + return MakeAddrLValue(Address(Val, Alignment), T, AlignmentSource::Decl);
>> }
>>
>> // Check for captured variables.
>> @@ -1942,21 +2029,20 @@ LValue CodeGenFunction::EmitDeclRefLValu
>> if (auto *FD = LambdaCaptureFields.lookup(VD))
>> return EmitCapturedFieldLValue(*this, FD, CXXABIThisValue);
>> else if (CapturedStmtInfo) {
>> - if (auto *V = LocalDeclMap.lookup(VD)) {
>> - if (VD->getType()->isReferenceType()) {
>> - llvm::LoadInst *LI = Builder.CreateLoad(V);
>> - LI->setAlignment(Alignment.getQuantity());
>> - V = LI;
>> - return MakeNaturalAlignAddrLValue(V, T);
>> + auto it = LocalDeclMap.find(VD);
>> + if (it != LocalDeclMap.end()) {
>> + if (auto RefTy = VD->getType()->getAs<ReferenceType>()) {
>> + return EmitLoadOfReferenceLValue(it->second, RefTy);
>> }
>> - return MakeAddrLValue(V, T, Alignment);
>> + return MakeAddrLValue(it->second, T);
>> }
>> return EmitCapturedFieldLValue(*this, CapturedStmtInfo->lookup(VD),
>> CapturedStmtInfo->getContextValue());
>> }
>> +
>> assert(isa<BlockDecl>(CurCodeDecl));
>> - return MakeAddrLValue(GetAddrOfBlockDecl(VD, VD->hasAttr<BlocksAttr>()),
>> - T, Alignment);
>> + Address addr = GetAddrOfBlockDecl(VD, VD->hasAttr<BlocksAttr>());
>> + return MakeAddrLValue(addr, T, AlignmentSource::Decl);
>> }
>> }
>>
>> @@ -1969,8 +2055,8 @@ LValue CodeGenFunction::EmitDeclRefLValu
>>
>> if (ND->hasAttr<WeakRefAttr>()) {
>> const auto *VD = cast<ValueDecl>(ND);
>> - llvm::Constant *Aliasee = CGM.GetWeakRefReference(VD);
>> - return MakeAddrLValue(Aliasee, T, Alignment);
>> + ConstantAddress Aliasee = CGM.GetWeakRefReference(VD);
>> + return MakeAddrLValue(Aliasee, T, AlignmentSource::Decl);
>> }
>>
>> if (const auto *VD = dyn_cast<VarDecl>(ND)) {
>> @@ -1978,39 +2064,52 @@ LValue CodeGenFunction::EmitDeclRefLValu
>> if (VD->hasLinkage() || VD->isStaticDataMember())
>> return EmitGlobalVarDeclLValue(*this, E, VD);
>>
>> - bool isBlockVariable = VD->hasAttr<BlocksAttr>();
>> + Address addr = Address::invalid();
>>
>> - llvm::Value *V = LocalDeclMap.lookup(VD);
>> - if (!V && VD->isStaticLocal())
>> - V = CGM.getOrCreateStaticVarDecl(
>> - *VD, CGM.getLLVMLinkageVarDefinition(VD, /*isConstant=*/false));
>> + // The variable should generally be present in the local decl map.
>> + auto iter = LocalDeclMap.find(VD);
>> + if (iter != LocalDeclMap.end()) {
>> + addr = iter->second;
>> +
>> + // Otherwise, it might be static local we haven't emitted yet for
>> + // some reason; most likely, because it's in an outer function.
>> + } else if (VD->isStaticLocal()) {
>> + addr = Address(CGM.getOrCreateStaticVarDecl(
>> + *VD, CGM.getLLVMLinkageVarDefinition(VD, /*isConstant=*/false)),
>> + getContext().getDeclAlign(VD));
>>
>> - // Check if variable is threadprivate.
>> - if (V && getLangOpts().OpenMP && VD->hasAttr<OMPThreadPrivateDeclAttr>())
>> - return EmitThreadPrivateVarDeclLValue(
>> - *this, VD, T, V, getTypes().ConvertTypeForMem(VD->getType()),
>> - Alignment, E->getExprLoc());
>> + // No other cases for now.
>> + } else {
>> + llvm_unreachable("DeclRefExpr for Decl not entered in LocalDeclMap?");
>> + }
>>
>> - assert(V && "DeclRefExpr not entered in LocalDeclMap?");
>>
>> - if (isBlockVariable)
>> - V = BuildBlockByrefAddress(V, VD);
>> + // Check for OpenMP threadprivate variables.
>> + if (getLangOpts().OpenMP && VD->hasAttr<OMPThreadPrivateDeclAttr>()) {
>> + return EmitThreadPrivateVarDeclLValue(
>> + *this, VD, T, addr, getTypes().ConvertTypeForMem(VD->getType()),
>> + E->getExprLoc());
>> + }
>>
>> + // Drill into block byref variables.
>> + bool isBlockByref = VD->hasAttr<BlocksAttr>();
>> + if (isBlockByref) {
>> + addr = emitBlockByrefAddress(addr, VD);
>> + }
>> +
>> + // Drill into reference types.
>> LValue LV;
>> - if (VD->getType()->isReferenceType()) {
>> - llvm::LoadInst *LI = Builder.CreateLoad(V);
>> - LI->setAlignment(Alignment.getQuantity());
>> - V = LI;
>> - LV = MakeNaturalAlignAddrLValue(V, T);
>> + if (auto RefTy = VD->getType()->getAs<ReferenceType>()) {
>> + LV = EmitLoadOfReferenceLValue(addr, RefTy);
>> } else {
>> - LV = MakeAddrLValue(V, T, Alignment);
>> + LV = MakeAddrLValue(addr, T, AlignmentSource::Decl);
>> }
>>
>> bool isLocalStorage = VD->hasLocalStorage();
>>
>> bool NonGCable = isLocalStorage &&
>> !VD->getType()->isReferenceType() &&
>> - !isBlockVariable;
>> + !isBlockByref;
>> if (NonGCable) {
>> LV.getQuals().removeObjCGCAttr();
>> LV.setNonGC(true);
>> @@ -2042,7 +2141,9 @@ LValue CodeGenFunction::EmitUnaryOpLValu
>> QualType T = E->getSubExpr()->getType()->getPointeeType();
>> assert(!T.isNull() && "CodeGenFunction::EmitUnaryOpLValue: Illegal type");
>>
>> - LValue LV = MakeNaturalAlignAddrLValue(EmitScalarExpr(E->getSubExpr()), T);
>> + AlignmentSource AlignSource;
>> + Address Addr = EmitPointerWithAlignment(E->getSubExpr(), &AlignSource);
>> + LValue LV = MakeAddrLValue(Addr, T, AlignSource);
>> LV.getQuals().setAddressSpace(ExprTy.getAddressSpace());
>>
>> // We should not generate __weak write barrier on indirect reference
>> @@ -2059,22 +2160,22 @@ LValue CodeGenFunction::EmitUnaryOpLValu
>> case UO_Imag: {
>> LValue LV = EmitLValue(E->getSubExpr());
>> assert(LV.isSimple() && "real/imag on non-ordinary l-value");
>> - llvm::Value *Addr = LV.getAddress();
>>
>> // __real is valid on scalars. This is a faster way of testing that.
>> // __imag can only produce an rvalue on scalars.
>> if (E->getOpcode() == UO_Real &&
>> - !cast<llvm::PointerType>(Addr->getType())
>> - ->getElementType()->isStructTy()) {
>> + !LV.getAddress().getElementType()->isStructTy()) {
>> assert(E->getSubExpr()->getType()->isArithmeticType());
>> return LV;
>> }
>>
>> assert(E->getSubExpr()->getType()->isAnyComplexType());
>>
>> - unsigned Idx = E->getOpcode() == UO_Imag;
>> - return MakeAddrLValue(
>> - Builder.CreateStructGEP(nullptr, LV.getAddress(), Idx, "idx"), ExprTy);
>> + Address Component =
>> + (E->getOpcode() == UO_Real
>> + ? emitAddrOfRealComponent(LV.getAddress(), LV.getType())
>> + : emitAddrOfImagComponent(LV.getAddress(), LV.getType()));
>> + return MakeAddrLValue(Component, ExprTy, LV.getAlignmentSource());
>> }
>> case UO_PreInc:
>> case UO_PreDec: {
>> @@ -2092,12 +2193,12 @@ LValue CodeGenFunction::EmitUnaryOpLValu
>>
>> LValue CodeGenFunction::EmitStringLiteralLValue(const StringLiteral *E) {
>> return MakeAddrLValue(CGM.GetAddrOfConstantStringFromLiteral(E),
>> - E->getType());
>> + E->getType(), AlignmentSource::Decl);
>> }
>>
>> LValue CodeGenFunction::EmitObjCEncodeExprLValue(const ObjCEncodeExpr *E) {
>> return MakeAddrLValue(CGM.GetAddrOfConstantStringFromObjCEncode(E),
>> - E->getType());
>> + E->getType(), AlignmentSource::Decl);
>> }
>>
>> LValue CodeGenFunction::EmitPredefinedLValue(const PredefinedExpr *E) {
>> @@ -2110,11 +2211,11 @@ LValue CodeGenFunction::EmitPredefinedLV
>> PredefinedExpr::getIdentTypeName(E->getIdentType()), FnName};
>> std::string GVName = llvm::join(NameItems, NameItems + 2, ".");
>> if (CurCodeDecl && isa<BlockDecl>(CurCodeDecl)) {
>> - auto C = CGM.GetAddrOfConstantCString(FnName, GVName.c_str(), 1);
>> - return MakeAddrLValue(C, E->getType());
>> + auto C = CGM.GetAddrOfConstantCString(FnName, GVName.c_str());
>> + return MakeAddrLValue(C, E->getType(), AlignmentSource::Decl);
>> }
>> auto C = CGM.GetAddrOfConstantStringFromLiteral(SL, GVName);
>> - return MakeAddrLValue(C, E->getType());
>> + return MakeAddrLValue(C, E->getType(), AlignmentSource::Decl);
>> }
>>
>> /// Emit a type description suitable for use by a runtime sanitizer library. The
>> @@ -2188,9 +2289,9 @@ llvm::Value *CodeGenFunction::EmitCheckV
>>
>> // Pointers are passed directly, everything else is passed by address.
>> if (!V->getType()->isPointerTy()) {
>> - llvm::Value *Ptr = CreateTempAlloca(V->getType());
>> + Address Ptr = CreateDefaultAlignTempAlloca(V->getType());
>> Builder.CreateStore(V, Ptr);
>> - V = Ptr;
>> + V = Ptr.getPointer();
>> }
>> return Builder.CreatePtrToInt(V, TargetTy);
>> }
>> @@ -2211,8 +2312,9 @@ llvm::Constant *CodeGenFunction::EmitChe
>> PresumedLoc PLoc = getContext().getSourceManager().getPresumedLoc(Loc);
>> if (PLoc.isValid()) {
>> auto FilenameGV = CGM.GetAddrOfConstantCString(PLoc.getFilename(), ".src");
>> - CGM.getSanitizerMetadata()->disableSanitizerForGlobal(FilenameGV);
>> - Filename = FilenameGV;
>> + CGM.getSanitizerMetadata()->disableSanitizerForGlobal(
>> + cast<llvm::GlobalVariable>(FilenameGV.getPointer()));
>> + Filename = FilenameGV.getPointer();
>> Line = PLoc.getLine();
>> Column = PLoc.getColumn();
>> } else {
>> @@ -2420,6 +2522,33 @@ llvm::CallInst *CodeGenFunction::EmitTra
>> return TrapCall;
>> }
>>
>> +Address CodeGenFunction::EmitArrayToPointerDecay(const Expr *E,
>> + AlignmentSource *AlignSource) {
>> + assert(E->getType()->isArrayType() &&
>> + "Array to pointer decay must have array source type!");
>> +
>> + // Expressions of array type can't be bitfields or vector elements.
>> + LValue LV = EmitLValue(E);
>> + Address Addr = LV.getAddress();
>> + if (AlignSource) *AlignSource = LV.getAlignmentSource();
>> +
>> + // If the array type was an incomplete type, we need to make sure
>> + // the decay ends up being the right type.
>> + llvm::Type *NewTy = ConvertType(E->getType());
>> + Addr = Builder.CreateElementBitCast(Addr, NewTy);
>> +
>> + // Note that VLA pointers are always decayed, so we don't need to do
>> + // anything here.
>> + if (!E->getType()->isVariableArrayType()) {
>> + assert(isa<llvm::ArrayType>(Addr.getElementType()) &&
>> + "Expected pointer to array");
>> + Addr = Builder.CreateStructGEP(Addr, 0, CharUnits::Zero(), "arraydecay");
>> + }
>> +
>> + QualType EltType = E->getType()->castAsArrayTypeUnsafe()->getElementType();
>> + return Builder.CreateElementBitCast(Addr, ConvertTypeForMem(EltType));
>> +}
>> +
>> /// isSimpleArrayDecayOperand - If the specified expr is a simple decay from an
>> /// array to pointer, return the array subexpression.
>> static const Expr *isSimpleArrayDecayOperand(const Expr *E) {
>> @@ -2436,6 +2565,69 @@ static const Expr *isSimpleArrayDecayOpe
>> return SubExpr;
>> }
>>
>> +static llvm::Value *emitArraySubscriptGEP(CodeGenFunction &CGF,
>> + llvm::Value *ptr,
>> + ArrayRef<llvm::Value*> indices,
>> + bool inbounds,
>> + const llvm::Twine &name = "arrayidx") {
>> + if (inbounds) {
>> + return CGF.Builder.CreateInBoundsGEP(ptr, indices, name);
>> + } else {
>> + return CGF.Builder.CreateGEP(ptr, indices, name);
>> + }
>> +}
>> +
>> +static CharUnits getArrayElementAlign(CharUnits arrayAlign,
>> + llvm::Value *idx,
>> + CharUnits eltSize) {
>> + // If we have a constant index, we can use the exact offset of the
>> + // element we're accessing.
>> + if (auto constantIdx = dyn_cast<llvm::ConstantInt>(idx)) {
>> + CharUnits offset = constantIdx->getZExtValue() * eltSize;
>> + return arrayAlign.alignmentAtOffset(offset);
>> +
>> + // Otherwise, use the worst-case alignment for any element.
>> + } else {
>> + return arrayAlign.alignmentOfArrayElement(eltSize);
>> + }
>> +}
>> +
>> +static QualType getFixedSizeElementType(const ASTContext &ctx,
>> + const VariableArrayType *vla) {
>> + QualType eltType;
>> + do {
>> + eltType = vla->getElementType();
>> + } while ((vla = ctx.getAsVariableArrayType(eltType)));
>> + return eltType;
>> +}
>> +
>> +static Address emitArraySubscriptGEP(CodeGenFunction &CGF, Address addr,
>> + ArrayRef<llvm::Value*> indices,
>> + QualType eltType, bool inbounds,
>> + const llvm::Twine &name = "arrayidx") {
>> + // All the indices except that last must be zero.
>> +#ifndef NDEBUG
>> + for (auto idx : indices.drop_back())
>> + assert(isa<llvm::ConstantInt>(idx) &&
>> + cast<llvm::ConstantInt>(idx)->isZero());
>> +#endif
>> +
>> + // Determine the element size of the statically-sized base. This is
>> + // the thing that the indices are expressed in terms of.
>> + if (auto vla = CGF.getContext().getAsVariableArrayType(eltType)) {
>> + eltType = getFixedSizeElementType(CGF.getContext(), vla);
>> + }
>> +
>> + // We can use that to compute the best alignment of the element.
>> + CharUnits eltSize = CGF.getContext().getTypeSizeInChars(eltType);
>> + CharUnits eltAlign =
>> + getArrayElementAlign(addr.getAlignment(), indices.back(), eltSize);
>> +
>> + llvm::Value *eltPtr =
>> + emitArraySubscriptGEP(CGF, addr.getPointer(), indices, inbounds, name);
>> + return Address(eltPtr, eltAlign);
>> +}
>> +
>> LValue CodeGenFunction::EmitArraySubscriptExpr(const ArraySubscriptExpr *E,
>> bool Accessed) {
>> // The index must always be an integer, which is not an aggregate. Emit it.
>> @@ -2454,32 +2646,34 @@ LValue CodeGenFunction::EmitArraySubscri
>> LValue LHS = EmitLValue(E->getBase());
>> assert(LHS.isSimple() && "Can only subscript lvalue vectors here!");
>> return LValue::MakeVectorElt(LHS.getAddress(), Idx,
>> - E->getBase()->getType(), LHS.getAlignment());
>> + E->getBase()->getType(),
>> + LHS.getAlignmentSource());
>> }
>>
>> + // All the other cases basically behave like simple offsetting.
>> +
>> // Extend or truncate the index type to 32 or 64-bits.
>> if (Idx->getType() != IntPtrTy)
>> Idx = Builder.CreateIntCast(Idx, IntPtrTy, IdxSigned, "idxprom");
>>
>> - // We know that the pointer points to a type of the correct size, unless the
>> - // size is a VLA or Objective-C interface.
>> - llvm::Value *Address = nullptr;
>> - CharUnits ArrayAlignment;
>> + // Handle the extvector case we ignored above.
>> if (isa<ExtVectorElementExpr>(E->getBase())) {
>> LValue LV = EmitLValue(E->getBase());
>> - Address = EmitExtVectorElementLValue(LV);
>> - Address = Builder.CreateInBoundsGEP(Address, Idx, "arrayidx");
>> - const VectorType *ExprVT = LV.getType()->getAs<VectorType>();
>> - QualType EQT = ExprVT->getElementType();
>> - return MakeAddrLValue(Address, EQT,
>> - getContext().getTypeAlignInChars(EQT));
>> + Address Addr = EmitExtVectorElementLValue(LV);
>> +
>> + QualType EltType = LV.getType()->castAs<VectorType>()->getElementType();
>> + Addr = emitArraySubscriptGEP(*this, Addr, Idx, EltType, /*inbounds*/ true);
>> + return MakeAddrLValue(Addr, EltType, LV.getAlignmentSource());
>> }
>> - else if (const VariableArrayType *vla =
>> +
>> + AlignmentSource AlignSource;
>> + Address Addr = Address::invalid();
>> + if (const VariableArrayType *vla =
>> getContext().getAsVariableArrayType(E->getType())) {
>> // The base must be a pointer, which is not an aggregate. Emit
>> // it. It needs to be emitted first in case it's what captures
>> // the VLA bounds.
>> - Address = EmitScalarExpr(E->getBase());
>> + Addr = EmitPointerWithAlignment(E->getBase(), &AlignSource);
>>
>> // The element count here is the total number of non-VLA elements.
>> llvm::Value *numElements = getVLASize(vla).first;
>> @@ -2490,24 +2684,40 @@ LValue CodeGenFunction::EmitArraySubscri
>> // multiply. We suppress this if overflow is not undefined behavior.
>> if (getLangOpts().isSignedOverflowDefined()) {
>> Idx = Builder.CreateMul(Idx, numElements);
>> - Address = Builder.CreateGEP(Address, Idx, "arrayidx");
>> } else {
>> Idx = Builder.CreateNSWMul(Idx, numElements);
>> - Address = Builder.CreateInBoundsGEP(Address, Idx, "arrayidx");
>> }
>> +
>> + Addr = emitArraySubscriptGEP(*this, Addr, Idx, vla->getElementType(),
>> + !getLangOpts().isSignedOverflowDefined());
>> +
>> } else if (const ObjCObjectType *OIT = E->getType()->getAs<ObjCObjectType>()){
>> // Indexing over an interface, as in "NSString *P; P[4];"
>> - llvm::Value *InterfaceSize =
>> - llvm::ConstantInt::get(Idx->getType(),
>> - getContext().getTypeSizeInChars(OIT).getQuantity());
>> -
>> - Idx = Builder.CreateMul(Idx, InterfaceSize);
>> + CharUnits InterfaceSize = getContext().getTypeSizeInChars(OIT);
>> + llvm::Value *InterfaceSizeVal =
>> + llvm::ConstantInt::get(Idx->getType(), InterfaceSize.getQuantity());;
>> +
>> + llvm::Value *ScaledIdx = Builder.CreateMul(Idx, InterfaceSizeVal);
>> +
>> + // Emit the base pointer.
>> + Addr = EmitPointerWithAlignment(E->getBase(), &AlignSource);
>> +
>> + // We don't necessarily build correct LLVM struct types for ObjC
>> + // interfaces, so we can't rely on GEP to do this scaling
>> + // correctly, so we need to cast to i8*. FIXME: is this actually
>> + // true? A lot of other things in the fragile ABI would break...
>> + llvm::Type *OrigBaseTy = Addr.getType();
>> + Addr = Builder.CreateElementBitCast(Addr, Int8Ty);
>> +
>> + // Do the GEP.
>> + CharUnits EltAlign =
>> + getArrayElementAlign(Addr.getAlignment(), Idx, InterfaceSize);
>> + llvm::Value *EltPtr =
>> + emitArraySubscriptGEP(*this, Addr.getPointer(), ScaledIdx, false);
>> + Addr = Address(EltPtr, EltAlign);
>>
>> - // The base must be a pointer, which is not an aggregate. Emit it.
>> - llvm::Value *Base = EmitScalarExpr(E->getBase());
>> - Address = EmitCastToVoidPtr(Base);
>> - Address = Builder.CreateGEP(Address, Idx, "arrayidx");
>> - Address = Builder.CreateBitCast(Address, Base->getType());
>> + // Cast back.
>> + Addr = Builder.CreateBitCast(Addr, OrigBaseTy);
>> } else if (const Expr *Array = isSimpleArrayDecayOperand(E->getBase())) {
>> // If this is A[i] where A is an array, the frontend will have decayed the
>> // base to be a ArrayToPointerDecay implicit cast. While correct, it is
>> @@ -2522,42 +2732,23 @@ LValue CodeGenFunction::EmitArraySubscri
>> ArrayLV = EmitArraySubscriptExpr(ASE, /*Accessed*/ true);
>> else
>> ArrayLV = EmitLValue(Array);
>> - llvm::Value *ArrayPtr = ArrayLV.getAddress();
>> - llvm::Value *Zero = llvm::ConstantInt::get(Int32Ty, 0);
>> - llvm::Value *Args[] = { Zero, Idx };
>>
>> // Propagate the alignment from the array itself to the result.
>> - ArrayAlignment = ArrayLV.getAlignment();
>> -
>> - if (getLangOpts().isSignedOverflowDefined())
>> - Address = Builder.CreateGEP(ArrayPtr, Args, "arrayidx");
>> - else
>> - Address = Builder.CreateInBoundsGEP(ArrayPtr, Args, "arrayidx");
>> + Addr = emitArraySubscriptGEP(*this, ArrayLV.getAddress(),
>> + {CGM.getSize(CharUnits::Zero()), Idx},
>> + E->getType(),
>> + !getLangOpts().isSignedOverflowDefined());
>> + AlignSource = ArrayLV.getAlignmentSource();
>> } else {
>> - // The base must be a pointer, which is not an aggregate. Emit it.
>> - llvm::Value *Base = EmitScalarExpr(E->getBase());
>> - if (getLangOpts().isSignedOverflowDefined())
>> - Address = Builder.CreateGEP(Base, Idx, "arrayidx");
>> - else
>> - Address = Builder.CreateInBoundsGEP(Base, Idx, "arrayidx");
>> + // The base must be a pointer; emit it with an estimate of its alignment.
>> + Addr = EmitPointerWithAlignment(E->getBase(), &AlignSource);
>> + Addr = emitArraySubscriptGEP(*this, Addr, Idx, E->getType(),
>> + !getLangOpts().isSignedOverflowDefined());
>> }
>>
>> - QualType T = E->getBase()->getType()->getPointeeType();
>> - assert(!T.isNull() &&
>> - "CodeGenFunction::EmitArraySubscriptExpr(): Illegal base type");
>> + LValue LV = MakeAddrLValue(Addr, E->getType(), AlignSource);
>>
>> -
>> - // Limit the alignment to that of the result type.
>> - LValue LV;
>> - if (!ArrayAlignment.isZero()) {
>> - CharUnits Align = getContext().getTypeAlignInChars(T);
>> - ArrayAlignment = std::min(Align, ArrayAlignment);
>> - LV = MakeAddrLValue(Address, T, ArrayAlignment);
>> - } else {
>> - LV = MakeNaturalAlignAddrLValue(Address, T);
>> - }
>> -
>> - LV.getQuals().setAddressSpace(E->getBase()->getType().getAddressSpace());
>> + // TODO: Preserve/extend path TBAA metadata?
>>
>> if (getLangOpts().ObjC1 &&
>> getLangOpts().getGC() != LangOptions::NonGC) {
>> @@ -2665,11 +2856,12 @@ LValue CodeGenFunction::EmitOMPArraySect
>> }
>> assert(Idx);
>>
>> - llvm::Value *Address;
>> - CharUnits ArrayAlignment;
>> + llvm::Value *EltPtr;
>> + QualType FixedSizeEltType = ResultExprTy;
>> if (auto *VLA = getContext().getAsVariableArrayType(ResultExprTy)) {
>> // The element count here is the total number of non-VLA elements.
>> llvm::Value *numElements = getVLASize(VLA).first;
>> + FixedSizeEltType = getFixedSizeElementType(getContext(), VLA);
>>
>> // Effectively, the multiply by the VLA size is part of the GEP.
>> // GEP indexes are signed, and scaling an index isn't permitted to
>> @@ -2677,39 +2869,35 @@ LValue CodeGenFunction::EmitOMPArraySect
>> // multiply. We suppress this if overflow is not undefined behavior.
>> if (getLangOpts().isSignedOverflowDefined()) {
>> Idx = Builder.CreateMul(Idx, numElements);
>> - Address = Builder.CreateGEP(Base.getAddress(), Idx, "arrayidx");
>> + EltPtr = Builder.CreateGEP(Base.getPointer(), Idx, "arrayidx");
>> } else {
>> Idx = Builder.CreateNSWMul(Idx, numElements);
>> - Address = Builder.CreateInBoundsGEP(Base.getAddress(), Idx, "arrayidx");
>> + EltPtr = Builder.CreateInBoundsGEP(Base.getPointer(), Idx, "arrayidx");
>> }
>> } else if (BaseTy->isConstantArrayType()) {
>> - llvm::Value *ArrayPtr = Base.getAddress();
>> + llvm::Value *ArrayPtr = Base.getPointer();
>> llvm::Value *Zero = llvm::ConstantInt::getNullValue(IntPtrTy);
>> llvm::Value *Args[] = {Zero, Idx};
>>
>> - // Propagate the alignment from the array itself to the result.
>> - ArrayAlignment = Base.getAlignment();
>> -
>> if (getLangOpts().isSignedOverflowDefined())
>> - Address = Builder.CreateGEP(ArrayPtr, Args, "arrayidx");
>> + EltPtr = Builder.CreateGEP(ArrayPtr, Args, "arrayidx");
>> else
>> - Address = Builder.CreateInBoundsGEP(ArrayPtr, Args, "arrayidx");
>> + EltPtr = Builder.CreateInBoundsGEP(ArrayPtr, Args, "arrayidx");
>> } else {
>> // The base must be a pointer, which is not an aggregate. Emit it.
>> if (getLangOpts().isSignedOverflowDefined())
>> - Address = Builder.CreateGEP(Base.getAddress(), Idx, "arrayidx");
>> + EltPtr = Builder.CreateGEP(Base.getPointer(), Idx, "arrayidx");
>> else
>> - Address = Builder.CreateInBoundsGEP(Base.getAddress(), Idx, "arrayidx");
>> + EltPtr = Builder.CreateInBoundsGEP(Base.getPointer(), Idx, "arrayidx");
>> }
>>
>> + CharUnits EltAlign =
>> + Base.getAlignment().alignmentOfArrayElement(
>> + getContext().getTypeSizeInChars(FixedSizeEltType));
>> +
>> // Limit the alignment to that of the result type.
>> - LValue LV;
>> - if (!ArrayAlignment.isZero()) {
>> - CharUnits Align = getContext().getTypeAlignInChars(ResultExprTy);
>> - ArrayAlignment = std::min(Align, ArrayAlignment);
>> - LV = MakeAddrLValue(Address, ResultExprTy, ArrayAlignment);
>> - } else
>> - LV = MakeNaturalAlignAddrLValue(Address, ResultExprTy);
>> + LValue LV = MakeAddrLValue(Address(EltPtr, EltAlign), ResultExprTy,
>> + Base.getAlignmentSource());
>>
>> LV.getQuals().setAddressSpace(BaseTy.getAddressSpace());
>>
>> @@ -2725,9 +2913,10 @@ EmitExtVectorElementExpr(const ExtVector
>> if (E->isArrow()) {
>> // If it is a pointer to a vector, emit the address and form an lvalue with
>> // it.
>> - llvm::Value *Ptr = EmitScalarExpr(E->getBase());
>> + AlignmentSource AlignSource;
>> + Address Ptr = EmitPointerWithAlignment(E->getBase(), &AlignSource);
>> const PointerType *PT = E->getBase()->getType()->getAs<PointerType>();
>> - Base = MakeAddrLValue(Ptr, PT->getPointeeType());
>> + Base = MakeAddrLValue(Ptr, PT->getPointeeType(), AlignSource);
>> Base.getQuals().removeObjCGCAttr();
>> } else if (E->getBase()->isGLValue()) {
>> // Otherwise, if the base is an lvalue ( as in the case of foo.x.x),
>> @@ -2741,9 +2930,10 @@ EmitExtVectorElementExpr(const ExtVector
>> llvm::Value *Vec = EmitScalarExpr(E->getBase());
>>
>> // Store the vector to memory (because LValue wants an address).
>> - llvm::Value *VecMem = CreateMemTemp(E->getBase()->getType());
>> + Address VecMem = CreateMemTemp(E->getBase()->getType());
>> Builder.CreateStore(Vec, VecMem);
>> - Base = MakeAddrLValue(VecMem, E->getBase()->getType());
>> + Base = MakeAddrLValue(VecMem, E->getBase()->getType(),
>> + AlignmentSource::Decl);
>> }
>>
>> QualType type =
>> @@ -2757,7 +2947,7 @@ EmitExtVectorElementExpr(const ExtVector
>> llvm::Constant *CV =
>> llvm::ConstantDataVector::get(getLLVMContext(), Indices);
>> return LValue::MakeExtVectorElt(Base.getAddress(), CV, type,
>> - Base.getAlignment());
>> + Base.getAlignmentSource());
>> }
>> assert(Base.isExtVectorElt() && "Can only subscript lvalue vec elts here!");
>>
>> @@ -2767,8 +2957,8 @@ EmitExtVectorElementExpr(const ExtVector
>> for (unsigned i = 0, e = Indices.size(); i != e; ++i)
>> CElts.push_back(BaseElts->getAggregateElement(Indices[i]));
>> llvm::Constant *CV = llvm::ConstantVector::get(CElts);
>> - return LValue::MakeExtVectorElt(Base.getExtVectorAddr(), CV, type,
>> - Base.getAlignment());
>> + return LValue::MakeExtVectorElt(Base.getExtVectorAddress(), CV, type,
>> + Base.getAlignmentSource());
>> }
>>
>> LValue CodeGenFunction::EmitMemberExpr(const MemberExpr *E) {
>> @@ -2777,10 +2967,11 @@ LValue CodeGenFunction::EmitMemberExpr(c
>> // If this is s.x, emit s as an lvalue. If it is s->x, emit s as a scalar.
>> LValue BaseLV;
>> if (E->isArrow()) {
>> - llvm::Value *Ptr = EmitScalarExpr(BaseExpr);
>> + AlignmentSource AlignSource;
>> + Address Addr = EmitPointerWithAlignment(BaseExpr, &AlignSource);
>> QualType PtrTy = BaseExpr->getType()->getPointeeType();
>> - EmitTypeCheck(TCK_MemberAccess, E->getExprLoc(), Ptr, PtrTy);
>> - BaseLV = MakeNaturalAlignAddrLValue(Ptr, PtrTy);
>> + EmitTypeCheck(TCK_MemberAccess, E->getExprLoc(), Addr.getPointer(), PtrTy);
>> + BaseLV = MakeAddrLValue(Addr, PtrTy, AlignSource);
>> } else
>> BaseLV = EmitCheckedLValue(BaseExpr, TCK_MemberAccess);
>>
>> @@ -2811,41 +3002,65 @@ LValue CodeGenFunction::EmitLValueForLam
>> return EmitLValueForField(LambdaLV, Field);
>> }
>>
>> +/// Drill down to the storage of a field without walking into
>> +/// reference types.
>> +///
>> +/// The resulting address doesn't necessarily have the right type.
>> +static Address emitAddrOfFieldStorage(CodeGenFunction &CGF, Address base,
>> + const FieldDecl *field) {
>> + const RecordDecl *rec = field->getParent();
>> +
>> + unsigned idx =
>> + CGF.CGM.getTypes().getCGRecordLayout(rec).getLLVMFieldNo(field);
>> +
>> + CharUnits offset;
>> + // Adjust the alignment down to the given offset.
>> + // As a special case, if the LLVM field index is 0, we know that this
>> + // is zero.
>> + assert((idx != 0 || CGF.getContext().getASTRecordLayout(rec)
>> + .getFieldOffset(field->getFieldIndex()) == 0) &&
>> + "LLVM field at index zero had non-zero offset?");
>> + if (idx != 0) {
>> + auto &recLayout = CGF.getContext().getASTRecordLayout(rec);
>> + auto offsetInBits = recLayout.getFieldOffset(field->getFieldIndex());
>> + offset = CGF.getContext().toCharUnitsFromBits(offsetInBits);
>> + }
>> +
>> + return CGF.Builder.CreateStructGEP(base, idx, offset, field->getName());
>> +}
>> +
>> LValue CodeGenFunction::EmitLValueForField(LValue base,
>> const FieldDecl *field) {
>> + AlignmentSource fieldAlignSource =
>> + getFieldAlignmentSource(base.getAlignmentSource());
>> +
>> if (field->isBitField()) {
>> const CGRecordLayout &RL =
>> CGM.getTypes().getCGRecordLayout(field->getParent());
>> const CGBitFieldInfo &Info = RL.getBitFieldInfo(field);
>> - llvm::Value *Addr = base.getAddress();
>> + Address Addr = base.getAddress();
>> unsigned Idx = RL.getLLVMFieldNo(field);
>> if (Idx != 0)
>> // For structs, we GEP to the field that the record layout suggests.
>> - Addr = Builder.CreateStructGEP(nullptr, Addr, Idx, field->getName());
>> + Addr = Builder.CreateStructGEP(Addr, Idx, Info.StorageOffset,
>> + field->getName());
>> // Get the access type.
>> - llvm::Type *PtrTy = llvm::Type::getIntNPtrTy(
>> - getLLVMContext(), Info.StorageSize,
>> - CGM.getContext().getTargetAddressSpace(base.getType()));
>> - if (Addr->getType() != PtrTy)
>> - Addr = Builder.CreateBitCast(Addr, PtrTy);
>> + llvm::Type *FieldIntTy =
>> + llvm::Type::getIntNTy(getLLVMContext(), Info.StorageSize);
>> + if (Addr.getElementType() != FieldIntTy)
>> + Addr = Builder.CreateElementBitCast(Addr, FieldIntTy);
>>
>> QualType fieldType =
>> field->getType().withCVRQualifiers(base.getVRQualifiers());
>> - return LValue::MakeBitfield(Addr, Info, fieldType, base.getAlignment());
>> + return LValue::MakeBitfield(Addr, Info, fieldType, fieldAlignSource);
>> }
>>
>> const RecordDecl *rec = field->getParent();
>> QualType type = field->getType();
>> - CharUnits alignment = getContext().getDeclAlign(field);
>> -
>> - // FIXME: It should be impossible to have an LValue without alignment for a
>> - // complete type.
>> - if (!base.getAlignment().isZero())
>> - alignment = std::min(alignment, base.getAlignment());
>>
>> bool mayAlias = rec->hasAttr<MayAliasAttr>();
>>
>> - llvm::Value *addr = base.getAddress();
>> + Address addr = base.getAddress();
>> unsigned cvr = base.getVRQualifiers();
>> bool TBAAPath = CGM.getCodeGenOpts().StructPathTBAA;
>> if (rec->isUnion()) {
>> @@ -2855,14 +3070,12 @@ LValue CodeGenFunction::EmitLValueForFie
>> TBAAPath = false;
>> } else {
>> // For structs, we GEP to the field that the record layout suggests.
>> - unsigned idx = CGM.getTypes().getCGRecordLayout(rec).getLLVMFieldNo(field);
>> - addr = Builder.CreateStructGEP(nullptr, addr, idx, field->getName());
>> + addr = emitAddrOfFieldStorage(*this, addr, field);
>>
>> // If this is a reference field, load the reference right now.
>> if (const ReferenceType *refType = type->getAs<ReferenceType>()) {
>> llvm::LoadInst *load = Builder.CreateLoad(addr, "ref");
>> if (cvr & Qualifiers::Volatile) load->setVolatile(true);
>> - load->setAlignment(alignment.getQuantity());
>>
>> // Loading the reference will disable path-aware TBAA.
>> TBAAPath = false;
>> @@ -2876,14 +3089,17 @@ LValue CodeGenFunction::EmitLValueForFie
>> CGM.DecorateInstruction(load, tbaa);
>> }
>>
>> - addr = load;
>> mayAlias = false;
>> type = refType->getPointeeType();
>> - if (type->isIncompleteType())
>> - alignment = CharUnits();
>> - else
>> - alignment = getContext().getTypeAlignInChars(type);
>> - cvr = 0; // qualifiers don't recursively apply to referencee
>> +
>> + CharUnits alignment =
>> + getNaturalTypeAlignment(type, &fieldAlignSource, /*pointee*/ true);
>> + addr = Address(load, alignment);
>> +
>> + // Qualifiers on the struct don't apply to the referencee, and
>> + // we'll pick up CVR from the actual type later, so reset these
>> + // additional qualifiers now.
>> + cvr = 0;
>> }
>> }
>>
>> @@ -2891,14 +3107,14 @@ LValue CodeGenFunction::EmitLValueForFie
>> // for both unions and structs. A union needs a bitcast, a struct element
>> // will need a bitcast if the LLVM type laid out doesn't match the desired
>> // type.
>> - addr = EmitBitCastOfLValueToProperType(*this, addr,
>> - CGM.getTypes().ConvertTypeForMem(type),
>> - field->getName());
>> + addr = Builder.CreateElementBitCast(addr,
>> + CGM.getTypes().ConvertTypeForMem(type),
>> + field->getName());
>>
>> if (field->hasAttr<AnnotateAttr>())
>> addr = EmitFieldAnnotations(field, addr);
>>
>> - LValue LV = MakeAddrLValue(addr, type, alignment);
>> + LValue LV = MakeAddrLValue(addr, type, fieldAlignSource);
>> LV.getQuals().addCVRQualifiers(cvr);
>> if (TBAAPath) {
>> const ASTRecordLayout &Layout =
>> @@ -2932,41 +3148,29 @@ CodeGenFunction::EmitLValueForFieldIniti
>> if (!FieldType->isReferenceType())
>> return EmitLValueForField(Base, Field);
>>
>> - const CGRecordLayout &RL =
>> - CGM.getTypes().getCGRecordLayout(Field->getParent());
>> - unsigned idx = RL.getLLVMFieldNo(Field);
>> - llvm::Value *V = Builder.CreateStructGEP(nullptr, Base.getAddress(), idx);
>> - assert(!FieldType.getObjCGCAttr() && "fields cannot have GC attrs");
>> + Address V = emitAddrOfFieldStorage(*this, Base.getAddress(), Field);
>>
>> - // Make sure that the address is pointing to the right type. This is critical
>> - // for both unions and structs. A union needs a bitcast, a struct element
>> - // will need a bitcast if the LLVM type laid out doesn't match the desired
>> - // type.
>> + // Make sure that the address is pointing to the right type.
>> llvm::Type *llvmType = ConvertTypeForMem(FieldType);
>> - V = EmitBitCastOfLValueToProperType(*this, V, llvmType, Field->getName());
>> -
>> - CharUnits Alignment = getContext().getDeclAlign(Field);
>> -
>> - // FIXME: It should be impossible to have an LValue without alignment for a
>> - // complete type.
>> - if (!Base.getAlignment().isZero())
>> - Alignment = std::min(Alignment, Base.getAlignment());
>> + V = Builder.CreateElementBitCast(V, llvmType, Field->getName());
>>
>> - return MakeAddrLValue(V, FieldType, Alignment);
>> + // TODO: access-path TBAA?
>> + auto FieldAlignSource = getFieldAlignmentSource(Base.getAlignmentSource());
>> + return MakeAddrLValue(V, FieldType, FieldAlignSource);
>> }
>>
>> LValue CodeGenFunction::EmitCompoundLiteralLValue(const CompoundLiteralExpr *E){
>> if (E->isFileScope()) {
>> - llvm::Value *GlobalPtr = CGM.GetAddrOfConstantCompoundLiteral(E);
>> - return MakeAddrLValue(GlobalPtr, E->getType());
>> + ConstantAddress GlobalPtr = CGM.GetAddrOfConstantCompoundLiteral(E);
>> + return MakeAddrLValue(GlobalPtr, E->getType(), AlignmentSource::Decl);
>> }
>> if (E->getType()->isVariablyModifiedType())
>> // make sure to emit the VLA size.
>> EmitVariablyModifiedType(E->getType());
>>
>> - llvm::Value *DeclPtr = CreateMemTemp(E->getType(), ".compoundliteral");
>> + Address DeclPtr = CreateMemTemp(E->getType(), ".compoundliteral");
>> const Expr *InitExpr = E->getInitializer();
>> - LValue Result = MakeAddrLValue(DeclPtr, E->getType());
>> + LValue Result = MakeAddrLValue(DeclPtr, E->getType(), AlignmentSource::Decl);
>>
>> EmitAnyExprToMem(InitExpr, DeclPtr, E->getType().getQualifiers(),
>> /*Init*/ true);
>> @@ -3057,11 +3261,14 @@ EmitConditionalOperatorLValue(const Abst
>> EmitBlock(contBlock);
>>
>> if (lhs && rhs) {
>> - llvm::PHINode *phi = Builder.CreatePHI(lhs->getAddress()->getType(),
>> + llvm::PHINode *phi = Builder.CreatePHI(lhs->getPointer()->getType(),
>> 2, "cond-lvalue");
>> - phi->addIncoming(lhs->getAddress(), lhsBlock);
>> - phi->addIncoming(rhs->getAddress(), rhsBlock);
>> - return MakeAddrLValue(phi, expr->getType());
>> + phi->addIncoming(lhs->getPointer(), lhsBlock);
>> + phi->addIncoming(rhs->getPointer(), rhsBlock);
>> + Address result(phi, std::min(lhs->getAlignment(), rhs->getAlignment()));
>> + AlignmentSource alignSource =
>> + std::max(lhs->getAlignmentSource(), rhs->getAlignmentSource());
>> + return MakeAddrLValue(result, expr->getType(), alignSource);
>> } else {
>> assert((lhs || rhs) &&
>> "both operands of glvalue conditional are throw-expressions?");
>> @@ -3130,9 +3337,9 @@ LValue CodeGenFunction::EmitCastLValue(c
>>
>> case CK_Dynamic: {
>> LValue LV = EmitLValue(E->getSubExpr());
>> - llvm::Value *V = LV.getAddress();
>> + Address V = LV.getAddress();
>> const auto *DCE = cast<CXXDynamicCastExpr>(E);
>> - return MakeAddrLValue(EmitDynamicCast(V, DCE), E->getType());
>> + return MakeNaturalAlignAddrLValue(EmitDynamicCast(V, DCE), E->getType());
>> }
>>
>> case CK_ConstructorConversion:
>> @@ -3150,14 +3357,14 @@ LValue CodeGenFunction::EmitCastLValue(c
>> auto *DerivedClassDecl = cast<CXXRecordDecl>(DerivedClassTy->getDecl());
>>
>> LValue LV = EmitLValue(E->getSubExpr());
>> - llvm::Value *This = LV.getAddress();
>> + Address This = LV.getAddress();
>>
>> // Perform the derived-to-base conversion
>> - llvm::Value *Base = GetAddressOfBaseClass(
>> + Address Base = GetAddressOfBaseClass(
>> This, DerivedClassDecl, E->path_begin(), E->path_end(),
>> /*NullCheckValue=*/false, E->getExprLoc());
>>
>> - return MakeAddrLValue(Base, E->getType());
>> + return MakeAddrLValue(Base, E->getType(), LV.getAlignmentSource());
>> }
>> case CK_ToUnion:
>> return EmitAggExprToLValue(E);
>> @@ -3168,7 +3375,7 @@ LValue CodeGenFunction::EmitCastLValue(c
>> LValue LV = EmitLValue(E->getSubExpr());
>>
>> // Perform the base-to-derived conversion
>> - llvm::Value *Derived =
>> + Address Derived =
>> GetAddressOfDerivedClass(LV.getAddress(), DerivedClassDecl,
>> E->path_begin(), E->path_end(),
>> /*NullCheckValue=*/false);
>> @@ -3177,34 +3384,35 @@ LValue CodeGenFunction::EmitCastLValue(c
>> // performed and the object is not of the derived type.
>> if (sanitizePerformTypeCheck())
>> EmitTypeCheck(TCK_DowncastReference, E->getExprLoc(),
>> - Derived, E->getType());
>> + Derived.getPointer(), E->getType());
>>
>> if (SanOpts.has(SanitizerKind::CFIDerivedCast))
>> - EmitVTablePtrCheckForCast(E->getType(), Derived, /*MayBeNull=*/false,
>> + EmitVTablePtrCheckForCast(E->getType(), Derived.getPointer(),
>> + /*MayBeNull=*/false,
>> CFITCK_DerivedCast, E->getLocStart());
>>
>> - return MakeAddrLValue(Derived, E->getType());
>> + return MakeAddrLValue(Derived, E->getType(), LV.getAlignmentSource());
>> }
>> case CK_LValueBitCast: {
>> // This must be a reinterpret_cast (or c-style equivalent).
>> const auto *CE = cast<ExplicitCastExpr>(E);
>>
>> LValue LV = EmitLValue(E->getSubExpr());
>> - llvm::Value *V = Builder.CreateBitCast(LV.getAddress(),
>> - ConvertType(CE->getTypeAsWritten()));
>> + Address V = Builder.CreateBitCast(LV.getAddress(),
>> + ConvertType(CE->getTypeAsWritten()));
>>
>> if (SanOpts.has(SanitizerKind::CFIUnrelatedCast))
>> - EmitVTablePtrCheckForCast(E->getType(), V, /*MayBeNull=*/false,
>> + EmitVTablePtrCheckForCast(E->getType(), V.getPointer(),
>> + /*MayBeNull=*/false,
>> CFITCK_UnrelatedCast, E->getLocStart());
>>
>> - return MakeAddrLValue(V, E->getType());
>> + return MakeAddrLValue(V, E->getType(), LV.getAlignmentSource());
>> }
>> case CK_ObjCObjectLValueCast: {
>> LValue LV = EmitLValue(E->getSubExpr());
>> - QualType ToType = getContext().getLValueReferenceType(E->getType());
>> - llvm::Value *V = Builder.CreateBitCast(LV.getAddress(),
>> - ConvertType(ToType));
>> - return MakeAddrLValue(V, E->getType());
>> + Address V = Builder.CreateElementBitCast(LV.getAddress(),
>> + ConvertType(E->getType()));
>> + return MakeAddrLValue(V, E->getType(), LV.getAlignmentSource());
>> }
>> case CK_ZeroToOCLEvent:
>> llvm_unreachable("NULL to OpenCL event lvalue cast is not valid");
>> @@ -3271,12 +3479,12 @@ RValue CodeGenFunction::EmitCallExpr(con
>> // If the pseudo-expression names a retainable object with weak or
>> // strong lifetime, the object shall be released.
>> Expr *BaseExpr = PseudoDtor->getBase();
>> - llvm::Value *BaseValue = nullptr;
>> + Address BaseValue = Address::invalid();
>> Qualifiers BaseQuals;
>>
>> // If this is s.x, emit s as an lvalue. If it is s->x, emit s as a scalar.
>> if (PseudoDtor->isArrow()) {
>> - BaseValue = EmitScalarExpr(BaseExpr);
>> + BaseValue = EmitPointerWithAlignment(BaseExpr);
>> const PointerType *PTy = BaseExpr->getType()->getAs<PointerType>();
>> BaseQuals = PTy->getPointeeType().getQualifiers();
>> } else {
>> @@ -3371,13 +3579,14 @@ LValue CodeGenFunction::EmitCallExprLVal
>> RValue RV = EmitCallExpr(E);
>>
>> if (!RV.isScalar())
>> - return MakeAddrLValue(RV.getAggregateAddr(), E->getType());
>> + return MakeAddrLValue(RV.getAggregateAddress(), E->getType(),
>> + AlignmentSource::Decl);
>>
>> assert(E->getCallReturnType(getContext())->isReferenceType() &&
>> "Can't have a scalar return unless the return type is a "
>> "reference type!");
>>
>> - return MakeAddrLValue(RV.getScalarVal(), E->getType());
>> + return MakeNaturalAlignPointeeAddrLValue(RV.getScalarVal(), E->getType());
>> }
>>
>> LValue CodeGenFunction::EmitVAArgExprLValue(const VAArgExpr *E) {
>> @@ -3390,21 +3599,23 @@ LValue CodeGenFunction::EmitCXXConstruct
>> && "binding l-value to type which needs a temporary");
>> AggValueSlot Slot = CreateAggTemp(E->getType());
>> EmitCXXConstructExpr(E, Slot);
>> - return MakeAddrLValue(Slot.getAddr(), E->getType());
>> + return MakeAddrLValue(Slot.getAddress(), E->getType(),
>> + AlignmentSource::Decl);
>> }
>>
>> LValue
>> CodeGenFunction::EmitCXXTypeidLValue(const CXXTypeidExpr *E) {
>> - return MakeAddrLValue(EmitCXXTypeidExpr(E), E->getType());
>> + return MakeNaturalAlignAddrLValue(EmitCXXTypeidExpr(E), E->getType());
>> }
>>
>> -llvm::Value *CodeGenFunction::EmitCXXUuidofExpr(const CXXUuidofExpr *E) {
>> - return Builder.CreateBitCast(CGM.GetAddrOfUuidDescriptor(E),
>> - ConvertType(E->getType())->getPointerTo());
>> +Address CodeGenFunction::EmitCXXUuidofExpr(const CXXUuidofExpr *E) {
>> + return Builder.CreateElementBitCast(CGM.GetAddrOfUuidDescriptor(E),
>> + ConvertType(E->getType()));
>> }
>>
>> LValue CodeGenFunction::EmitCXXUuidofLValue(const CXXUuidofExpr *E) {
>> - return MakeAddrLValue(EmitCXXUuidofExpr(E), E->getType());
>> + return MakeAddrLValue(EmitCXXUuidofExpr(E), E->getType(),
>> + AlignmentSource::Decl);
>> }
>>
>> LValue
>> @@ -3412,34 +3623,37 @@ CodeGenFunction::EmitCXXBindTemporaryLVa
>> AggValueSlot Slot = CreateAggTemp(E->getType(), "temp.lvalue");
>> Slot.setExternallyDestructed();
>> EmitAggExpr(E->getSubExpr(), Slot);
>> - EmitCXXTemporary(E->getTemporary(), E->getType(), Slot.getAddr());
>> - return MakeAddrLValue(Slot.getAddr(), E->getType());
>> + EmitCXXTemporary(E->getTemporary(), E->getType(), Slot.getAddress());
>> + return MakeAddrLValue(Slot.getAddress(), E->getType(),
>> + AlignmentSource::Decl);
>> }
>>
>> LValue
>> CodeGenFunction::EmitLambdaLValue(const LambdaExpr *E) {
>> AggValueSlot Slot = CreateAggTemp(E->getType(), "temp.lvalue");
>> EmitLambdaExpr(E, Slot);
>> - return MakeAddrLValue(Slot.getAddr(), E->getType());
>> + return MakeAddrLValue(Slot.getAddress(), E->getType(),
>> + AlignmentSource::Decl);
>> }
>>
>> LValue CodeGenFunction::EmitObjCMessageExprLValue(const ObjCMessageExpr *E) {
>> RValue RV = EmitObjCMessageExpr(E);
>>
>> if (!RV.isScalar())
>> - return MakeAddrLValue(RV.getAggregateAddr(), E->getType());
>> + return MakeAddrLValue(RV.getAggregateAddress(), E->getType(),
>> + AlignmentSource::Decl);
>>
>> assert(E->getMethodDecl()->getReturnType()->isReferenceType() &&
>> "Can't have a scalar return unless the return type is a "
>> "reference type!");
>>
>> - return MakeAddrLValue(RV.getScalarVal(), E->getType());
>> + return MakeNaturalAlignPointeeAddrLValue(RV.getScalarVal(), E->getType());
>> }
>>
>> LValue CodeGenFunction::EmitObjCSelectorLValue(const ObjCSelectorExpr *E) {
>> - llvm::Value *V =
>> - CGM.getObjCRuntime().GetSelector(*this, E->getSelector(), true);
>> - return MakeAddrLValue(V, E->getType());
>> + Address V =
>> + CGM.getObjCRuntime().GetAddrOfSelector(*this, E->getSelector());
>> + return MakeAddrLValue(V, E->getType(), AlignmentSource::Decl);
>> }
>>
>> llvm::Value *CodeGenFunction::EmitIvarOffset(const ObjCInterfaceDecl *Interface,
>> @@ -3467,8 +3681,7 @@ LValue CodeGenFunction::EmitObjCIvarRefL
>> BaseQuals = ObjectTy.getQualifiers();
>> } else {
>> LValue BaseLV = EmitLValue(BaseExpr);
>> - // FIXME: this isn't right for bitfields.
>> - BaseValue = BaseLV.getAddress();
>> + BaseValue = BaseLV.getPointer();
>> ObjectTy = BaseExpr->getType();
>> BaseQuals = ObjectTy.getQualifiers();
>> }
>> @@ -3483,7 +3696,8 @@ LValue CodeGenFunction::EmitObjCIvarRefL
>> LValue CodeGenFunction::EmitStmtExprLValue(const StmtExpr *E) {
>> // Can only get l-value for message expression returning aggregate type
>> RValue RV = EmitAnyExprToTemp(E);
>> - return MakeAddrLValue(RV.getAggregateAddr(), E->getType());
>> + return MakeAddrLValue(RV.getAggregateAddress(), E->getType(),
>> + AlignmentSource::Decl);
>> }
>>
>> RValue CodeGenFunction::EmitCall(QualType CalleeType, llvm::Value *Callee,
>> @@ -3517,7 +3731,8 @@ RValue CodeGenFunction::EmitCall(QualTyp
>> Callee, llvm::PointerType::getUnqual(PrefixStructTy));
>> llvm::Value *CalleeSigPtr =
>> Builder.CreateConstGEP2_32(PrefixStructTy, CalleePrefixStruct, 0, 0);
>> - llvm::Value *CalleeSig = Builder.CreateLoad(CalleeSigPtr);
>> + llvm::Value *CalleeSig =
>> + Builder.CreateAlignedLoad(CalleeSigPtr, getIntAlign());
>> llvm::Value *CalleeSigMatch = Builder.CreateICmpEQ(CalleeSig, PrefixSig);
>>
>> llvm::BasicBlock *Cont = createBasicBlock("cont");
>> @@ -3527,7 +3742,8 @@ RValue CodeGenFunction::EmitCall(QualTyp
>> EmitBlock(TypeCheck);
>> llvm::Value *CalleeRTTIPtr =
>> Builder.CreateConstGEP2_32(PrefixStructTy, CalleePrefixStruct, 0, 1);
>> - llvm::Value *CalleeRTTI = Builder.CreateLoad(CalleeRTTIPtr);
>> + llvm::Value *CalleeRTTI =
>> + Builder.CreateAlignedLoad(CalleeRTTIPtr, getPointerAlign());
>> llvm::Value *CalleeRTTIMatch =
>> Builder.CreateICmpEQ(CalleeRTTI, FTRTTIConst);
>> llvm::Constant *StaticData[] = {
>> @@ -3583,29 +3799,32 @@ RValue CodeGenFunction::EmitCall(QualTyp
>>
>> LValue CodeGenFunction::
>> EmitPointerToDataMemberBinaryExpr(const BinaryOperator *E) {
>> - llvm::Value *BaseV;
>> - if (E->getOpcode() == BO_PtrMemI)
>> - BaseV = EmitScalarExpr(E->getLHS());
>> - else
>> - BaseV = EmitLValue(E->getLHS()).getAddress();
>> + Address BaseAddr = Address::invalid();
>> + if (E->getOpcode() == BO_PtrMemI) {
>> + BaseAddr = EmitPointerWithAlignment(E->getLHS());
>> + } else {
>> + BaseAddr = EmitLValue(E->getLHS()).getAddress();
>> + }
>>
>> llvm::Value *OffsetV = EmitScalarExpr(E->getRHS());
>>
>> const MemberPointerType *MPT
>> = E->getRHS()->getType()->getAs<MemberPointerType>();
>>
>> - llvm::Value *AddV = CGM.getCXXABI().EmitMemberDataPointerAddress(
>> - *this, E, BaseV, OffsetV, MPT);
>> + AlignmentSource AlignSource;
>> + Address MemberAddr =
>> + EmitCXXMemberDataPointerAddress(E, BaseAddr, OffsetV, MPT,
>> + &AlignSource);
>>
>> - return MakeAddrLValue(AddV, MPT->getPointeeType());
>> + return MakeAddrLValue(MemberAddr, MPT->getPointeeType(), AlignSource);
>> }
>>
>> /// Given the address of a temporary variable, produce an r-value of
>> /// its type.
>> -RValue CodeGenFunction::convertTempToRValue(llvm::Value *addr,
>> +RValue CodeGenFunction::convertTempToRValue(Address addr,
>> QualType type,
>> SourceLocation loc) {
>> - LValue lvalue = MakeNaturalAlignAddrLValue(addr, type);
>> + LValue lvalue = MakeAddrLValue(addr, type, AlignmentSource::Decl);
>> switch (getEvaluationKind(type)) {
>> case TEK_Complex:
>> return RValue::getComplex(EmitLoadOfComplex(lvalue, loc));
>> @@ -3661,7 +3880,8 @@ static LValueOrRValue emitPseudoObjectEx
>> CodeGenFunction::hasAggregateEvaluationKind(ov->getType())) {
>> CGF.EmitAggExpr(ov->getSourceExpr(), slot);
>>
>> - LValue LV = CGF.MakeAddrLValue(slot.getAddr(), ov->getType());
>> + LValue LV = CGF.MakeAddrLValue(slot.getAddress(), ov->getType(),
>> + AlignmentSource::Decl);
>> opaqueData = OVMA::bind(CGF, ov, LV);
>> result.RV = slot.asRValue();
>>
>>
>> Modified: cfe/trunk/lib/CodeGen/CGExprAgg.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGExprAgg.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGExprAgg.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGExprAgg.cpp Tue Sep 8 03:05:57 2015
>> @@ -49,7 +49,8 @@ class AggExprEmitter : public StmtVisito
>> if (!shouldUseDestForReturnSlot())
>> return ReturnValueSlot();
>>
>> - return ReturnValueSlot(Dest.getAddr(), Dest.isVolatile(), IsResultUnused);
>> + return ReturnValueSlot(Dest.getAddress(), Dest.isVolatile(),
>> + IsResultUnused);
>> }
>>
>> AggValueSlot EnsureSlot(QualType T) {
>> @@ -77,14 +78,13 @@ public:
>>
>> /// EmitFinalDestCopy - Perform the final copy to DestPtr, if desired.
>> void EmitFinalDestCopy(QualType type, const LValue &src);
>> - void EmitFinalDestCopy(QualType type, RValue src,
>> - CharUnits srcAlignment = CharUnits::Zero());
>> + void EmitFinalDestCopy(QualType type, RValue src);
>> void EmitCopy(QualType type, const AggValueSlot &dest,
>> const AggValueSlot &src);
>>
>> void EmitMoveFromReturnSlot(const Expr *E, RValue Src);
>>
>> - void EmitArrayInit(llvm::Value *DestPtr, llvm::ArrayType *AType,
>> + void EmitArrayInit(Address DestPtr, llvm::ArrayType *AType,
>> QualType elementType, InitListExpr *E);
>>
>> AggValueSlot::NeedsGCBarriers_t needsGC(QualType T) {
>> @@ -199,7 +199,7 @@ public:
>> // case Expr::ChooseExprClass:
>> void VisitCXXThrowExpr(const CXXThrowExpr *E) { CGF.EmitCXXThrowExpr(E); }
>> void VisitAtomicExpr(AtomicExpr *E) {
>> - CGF.EmitAtomicExpr(E, EnsureSlot(E->getType()).getAddr());
>> + CGF.EmitAtomicExpr(E, EnsureSlot(E->getType()).getAddress());
>> }
>> };
>> } // end anonymous namespace.
>> @@ -259,17 +259,14 @@ void AggExprEmitter::EmitMoveFromReturnS
>> }
>>
>> // Otherwise, copy from there to the destination.
>> - assert(Dest.getAddr() != src.getAggregateAddr());
>> - std::pair<CharUnits, CharUnits> typeInfo =
>> - CGF.getContext().getTypeInfoInChars(E->getType());
>> - EmitFinalDestCopy(E->getType(), src, typeInfo.second);
>> + assert(Dest.getPointer() != src.getAggregatePointer());
>> + EmitFinalDestCopy(E->getType(), src);
>> }
>>
>> /// EmitFinalDestCopy - Perform the final copy to DestPtr, if desired.
>> -void AggExprEmitter::EmitFinalDestCopy(QualType type, RValue src,
>> - CharUnits srcAlign) {
>> +void AggExprEmitter::EmitFinalDestCopy(QualType type, RValue src) {
>> assert(src.isAggregate() && "value must be aggregate value!");
>> - LValue srcLV = CGF.MakeAddrLValue(src.getAggregateAddr(), type, srcAlign);
>> + LValue srcLV = CGF.MakeAddrLValue(src.getAggregateAddress(), type);
>> EmitFinalDestCopy(type, srcLV);
>> }
>>
>> @@ -298,8 +295,8 @@ void AggExprEmitter::EmitCopy(QualType t
>> CharUnits sz = CGF.getContext().getTypeSizeInChars(type);
>> llvm::Value *size = llvm::ConstantInt::get(CGF.SizeTy, sz.getQuantity());
>> CGF.CGM.getObjCRuntime().EmitGCMemmoveCollectable(CGF,
>> - dest.getAddr(),
>> - src.getAddr(),
>> + dest.getAddress(),
>> + src.getAddress(),
>> size);
>> return;
>> }
>> @@ -307,9 +304,8 @@ void AggExprEmitter::EmitCopy(QualType t
>> // If the result of the assignment is used, copy the LHS there also.
>> // It's volatile if either side is. Use the minimum alignment of
>> // the two sides.
>> - CGF.EmitAggregateCopy(dest.getAddr(), src.getAddr(), type,
>> - dest.isVolatile() || src.isVolatile(),
>> - std::min(dest.getAlignment(), src.getAlignment()));
>> + CGF.EmitAggregateCopy(dest.getAddress(), src.getAddress(), type,
>> + dest.isVolatile() || src.isVolatile());
>> }
>>
>> /// \brief Emit the initializer for a std::initializer_list initialized with a
>> @@ -321,7 +317,7 @@ AggExprEmitter::VisitCXXStdInitializerLi
>> ASTContext &Ctx = CGF.getContext();
>> LValue Array = CGF.EmitLValue(E->getSubExpr());
>> assert(Array.isSimple() && "initializer_list array not a simple lvalue");
>> - llvm::Value *ArrayPtr = Array.getAddress();
>> + Address ArrayPtr = Array.getAddress();
>>
>> const ConstantArrayType *ArrayType =
>> Ctx.getAsConstantArrayType(E->getSubExpr()->getType());
>> @@ -344,13 +340,12 @@ AggExprEmitter::VisitCXXStdInitializerLi
>> }
>>
>> AggValueSlot Dest = EnsureSlot(E->getType());
>> - LValue DestLV = CGF.MakeAddrLValue(Dest.getAddr(), E->getType(),
>> - Dest.getAlignment());
>> + LValue DestLV = CGF.MakeAddrLValue(Dest.getAddress(), E->getType());
>> LValue Start = CGF.EmitLValueForFieldInitialization(DestLV, *Field);
>> llvm::Value *Zero = llvm::ConstantInt::get(CGF.PtrDiffTy, 0);
>> llvm::Value *IdxStart[] = { Zero, Zero };
>> llvm::Value *ArrayStart =
>> - Builder.CreateInBoundsGEP(ArrayPtr, IdxStart, "arraystart");
>> + Builder.CreateInBoundsGEP(ArrayPtr.getPointer(), IdxStart, "arraystart");
>> CGF.EmitStoreThroughLValue(RValue::get(ArrayStart), Start);
>> ++Field;
>>
>> @@ -367,7 +362,7 @@ AggExprEmitter::VisitCXXStdInitializerLi
>> // End pointer.
>> llvm::Value *IdxEnd[] = { Zero, Size };
>> llvm::Value *ArrayEnd =
>> - Builder.CreateInBoundsGEP(ArrayPtr, IdxEnd, "arrayend");
>> + Builder.CreateInBoundsGEP(ArrayPtr.getPointer(), IdxEnd, "arrayend");
>> CGF.EmitStoreThroughLValue(RValue::get(ArrayEnd), EndOrLength);
>> } else if (Ctx.hasSameType(Field->getType(), Ctx.getSizeType())) {
>> // Length.
>> @@ -402,7 +397,7 @@ static bool isTrivialFiller(Expr *E) {
>> }
>>
>> /// \brief Emit initialization of an array from an initializer list.
>> -void AggExprEmitter::EmitArrayInit(llvm::Value *DestPtr, llvm::ArrayType *AType,
>> +void AggExprEmitter::EmitArrayInit(Address DestPtr, llvm::ArrayType *AType,
>> QualType elementType, InitListExpr *E) {
>> uint64_t NumInitElements = E->getNumInits();
>>
>> @@ -414,13 +409,17 @@ void AggExprEmitter::EmitArrayInit(llvm:
>> llvm::Value *zero = llvm::ConstantInt::get(CGF.SizeTy, 0);
>> llvm::Value *indices[] = { zero, zero };
>> llvm::Value *begin =
>> - Builder.CreateInBoundsGEP(DestPtr, indices, "arrayinit.begin");
>> + Builder.CreateInBoundsGEP(DestPtr.getPointer(), indices, "arrayinit.begin");
>> +
>> + CharUnits elementSize = CGF.getContext().getTypeSizeInChars(elementType);
>> + CharUnits elementAlign =
>> + DestPtr.getAlignment().alignmentOfArrayElement(elementSize);
>>
>> // Exception safety requires us to destroy all the
>> // already-constructed members if an initializer throws.
>> // For that, we'll need an EH cleanup.
>> QualType::DestructionKind dtorKind = elementType.isDestructedType();
>> - llvm::AllocaInst *endOfInit = nullptr;
>> + Address endOfInit = Address::invalid();
>> EHScopeStack::stable_iterator cleanup;
>> llvm::Instruction *cleanupDominator = nullptr;
>> if (CGF.needsEHCleanup(dtorKind)) {
>> @@ -428,10 +427,11 @@ void AggExprEmitter::EmitArrayInit(llvm:
>> // directly, but the control flow can get so varied here that it
>> // would actually be quite complex. Therefore we go through an
>> // alloca.
>> - endOfInit = CGF.CreateTempAlloca(begin->getType(),
>> + endOfInit = CGF.CreateTempAlloca(begin->getType(), CGF.getPointerAlign(),
>> "arrayinit.endOfInit");
>> cleanupDominator = Builder.CreateStore(begin, endOfInit);
>> CGF.pushIrregularPartialArrayCleanup(begin, endOfInit, elementType,
>> + elementAlign,
>> CGF.getDestroyer(dtorKind));
>> cleanup = CGF.EHStack.stable_begin();
>>
>> @@ -458,10 +458,11 @@ void AggExprEmitter::EmitArrayInit(llvm:
>> // Tell the cleanup that it needs to destroy up to this
>> // element. TODO: some of these stores can be trivially
>> // observed to be unnecessary.
>> - if (endOfInit) Builder.CreateStore(element, endOfInit);
>> + if (endOfInit.isValid()) Builder.CreateStore(element, endOfInit);
>> }
>>
>> - LValue elementLV = CGF.MakeAddrLValue(element, elementType);
>> + LValue elementLV =
>> + CGF.MakeAddrLValue(Address(element, elementAlign), elementType);
>> EmitInitializationToLValue(E->getInit(i), elementLV);
>> }
>>
>> @@ -482,7 +483,7 @@ void AggExprEmitter::EmitArrayInit(llvm:
>> // Advance to the start of the rest of the array.
>> if (NumInitElements) {
>> element = Builder.CreateInBoundsGEP(element, one, "arrayinit.start");
>> - if (endOfInit) Builder.CreateStore(element, endOfInit);
>> + if (endOfInit.isValid()) Builder.CreateStore(element, endOfInit);
>> }
>>
>> // Compute the end of the array.
>> @@ -500,7 +501,8 @@ void AggExprEmitter::EmitArrayInit(llvm:
>> currentElement->addIncoming(element, entryBB);
>>
>> // Emit the actual filler expression.
>> - LValue elementLV = CGF.MakeAddrLValue(currentElement, elementType);
>> + LValue elementLV =
>> + CGF.MakeAddrLValue(Address(currentElement, elementAlign), elementType);
>> if (filler)
>> EmitInitializationToLValue(filler, elementLV);
>> else
>> @@ -511,7 +513,7 @@ void AggExprEmitter::EmitArrayInit(llvm:
>> Builder.CreateInBoundsGEP(currentElement, one, "arrayinit.next");
>>
>> // Tell the EH cleanup that we finished with the last element.
>> - if (endOfInit) Builder.CreateStore(nextElement, endOfInit);
>> + if (endOfInit.isValid()) Builder.CreateStore(nextElement, endOfInit);
>>
>> // Leave the loop if we're done.
>> llvm::Value *done = Builder.CreateICmpEQ(nextElement, end,
>> @@ -596,9 +598,8 @@ void AggExprEmitter::VisitCastExpr(CastE
>>
>> // GCC union extension
>> QualType Ty = E->getSubExpr()->getType();
>> - QualType PtrTy = CGF.getContext().getPointerType(Ty);
>> - llvm::Value *CastPtr = Builder.CreateBitCast(Dest.getAddr(),
>> - CGF.ConvertType(PtrTy));
>> + Address CastPtr =
>> + Builder.CreateElementBitCast(Dest.getAddress(), CGF.ConvertType(Ty));
>> EmitInitializationToLValue(E->getSubExpr(),
>> CGF.MakeAddrLValue(CastPtr, Ty));
>> break;
>> @@ -649,13 +650,13 @@ void AggExprEmitter::VisitCastExpr(CastE
>> // Zero-initialize. (Strictly speaking, we only need to intialize
>> // the padding at the end, but this is simpler.)
>> if (!Dest.isZeroed())
>> - CGF.EmitNullInitialization(Dest.getAddr(), atomicType);
>> + CGF.EmitNullInitialization(Dest.getAddress(), atomicType);
>>
>> // Build a GEP to refer to the subobject.
>> - llvm::Value *valueAddr =
>> - CGF.Builder.CreateStructGEP(nullptr, valueDest.getAddr(), 0);
>> + Address valueAddr =
>> + CGF.Builder.CreateStructGEP(valueDest.getAddress(), 0,
>> + CharUnits());
>> valueDest = AggValueSlot::forAddr(valueAddr,
>> - valueDest.getAlignment(),
>> valueDest.getQualifiers(),
>> valueDest.isExternallyDestructed(),
>> valueDest.requiresGCollection(),
>> @@ -673,8 +674,8 @@ void AggExprEmitter::VisitCastExpr(CastE
>> CGF.CreateAggTemp(atomicType, "atomic-to-nonatomic.temp");
>> CGF.EmitAggExpr(E->getSubExpr(), atomicSlot);
>>
>> - llvm::Value *valueAddr =
>> - Builder.CreateStructGEP(nullptr, atomicSlot.getAddr(), 0);
>> + Address valueAddr =
>> + Builder.CreateStructGEP(atomicSlot.getAddress(), 0, CharUnits());
>> RValue rvalue = RValue::getAggregate(valueAddr, atomicSlot.isVolatile());
>> return EmitFinalDestCopy(valueType, rvalue);
>> }
>> @@ -959,15 +960,15 @@ void AggExprEmitter::VisitChooseExpr(con
>> }
>>
>> void AggExprEmitter::VisitVAArgExpr(VAArgExpr *VE) {
>> - llvm::Value *ArgValue = CGF.EmitVAListRef(VE->getSubExpr());
>> - llvm::Value *ArgPtr = CGF.EmitVAArg(ArgValue, VE->getType());
>> + Address ArgValue = CGF.EmitVAListRef(VE->getSubExpr());
>> + Address ArgPtr = CGF.EmitVAArg(ArgValue, VE->getType());
>>
>> - if (!ArgPtr) {
>> + if (!ArgPtr.isValid()) {
>> // If EmitVAArg fails, we fall back to the LLVM instruction.
>> - llvm::Value *Val =
>> - Builder.CreateVAArg(ArgValue, CGF.ConvertType(VE->getType()));
>> + llvm::Value *Val = Builder.CreateVAArg(ArgValue.getPointer(),
>> + CGF.ConvertType(VE->getType()));
>> if (!Dest.isIgnored())
>> - Builder.CreateStore(Val, Dest.getAddr());
>> + Builder.CreateStore(Val, Dest.getAddress());
>> return;
>> }
>>
>> @@ -987,7 +988,7 @@ void AggExprEmitter::VisitCXXBindTempora
>>
>> // Push that destructor we promised.
>> if (!wasExternallyDestructed)
>> - CGF.EmitCXXTemporary(E->getTemporary(), E->getType(), Dest.getAddr());
>> + CGF.EmitCXXTemporary(E->getTemporary(), E->getType(), Dest.getAddress());
>> }
>>
>> void
>> @@ -1011,13 +1012,13 @@ void AggExprEmitter::VisitExprWithCleanu
>> void AggExprEmitter::VisitCXXScalarValueInitExpr(CXXScalarValueInitExpr *E) {
>> QualType T = E->getType();
>> AggValueSlot Slot = EnsureSlot(T);
>> - EmitNullInitializationToLValue(CGF.MakeAddrLValue(Slot.getAddr(), T));
>> + EmitNullInitializationToLValue(CGF.MakeAddrLValue(Slot.getAddress(), T));
>> }
>>
>> void AggExprEmitter::VisitImplicitValueInitExpr(ImplicitValueInitExpr *E) {
>> QualType T = E->getType();
>> AggValueSlot Slot = EnsureSlot(T);
>> - EmitNullInitializationToLValue(CGF.MakeAddrLValue(Slot.getAddr(), T));
>> + EmitNullInitializationToLValue(CGF.MakeAddrLValue(Slot.getAddress(), T));
>> }
>>
>> /// isSimpleZero - If emitting this value will obviously just cause a store of
>> @@ -1135,8 +1136,7 @@ void AggExprEmitter::VisitInitListExpr(I
>>
>> AggValueSlot Dest = EnsureSlot(E->getType());
>>
>> - LValue DestLV = CGF.MakeAddrLValue(Dest.getAddr(), E->getType(),
>> - Dest.getAlignment());
>> + LValue DestLV = CGF.MakeAddrLValue(Dest.getAddress(), E->getType());
>>
>> // Handle initialization of an array.
>> if (E->getType()->isArrayType()) {
>> @@ -1146,12 +1146,8 @@ void AggExprEmitter::VisitInitListExpr(I
>> QualType elementType =
>> CGF.getContext().getAsArrayType(E->getType())->getElementType();
>>
>> - llvm::PointerType *APType =
>> - cast<llvm::PointerType>(Dest.getAddr()->getType());
>> - llvm::ArrayType *AType =
>> - cast<llvm::ArrayType>(APType->getElementType());
>> -
>> - EmitArrayInit(Dest.getAddr(), AType, elementType, E);
>> + auto AType = cast<llvm::ArrayType>(Dest.getAddress().getElementType());
>> + EmitArrayInit(Dest.getAddress(), AType, elementType, E);
>> return;
>> }
>>
>> @@ -1175,7 +1171,7 @@ void AggExprEmitter::VisitInitListExpr(I
>> RecordDecl *record = E->getType()->castAs<RecordType>()->getDecl();
>>
>> // Prepare a 'this' for CXXDefaultInitExprs.
>> - CodeGenFunction::FieldConstructionScope FCS(CGF, Dest.getAddr());
>> + CodeGenFunction::FieldConstructionScope FCS(CGF, Dest.getAddress());
>>
>> if (record->isUnion()) {
>> // Only initialize one field of a union. The field itself is
>> @@ -1253,9 +1249,10 @@ void AggExprEmitter::VisitInitListExpr(I
>> assert(LV.isSimple());
>> if (CGF.needsEHCleanup(dtorKind)) {
>> if (!cleanupDominator)
>> - cleanupDominator = CGF.Builder.CreateLoad(
>> + cleanupDominator = CGF.Builder.CreateAlignedLoad(
>> CGF.Int8Ty,
>> - llvm::Constant::getNullValue(CGF.Int8PtrTy)); // placeholder
>> + llvm::Constant::getNullValue(CGF.Int8PtrTy),
>> + CharUnits::One()); // placeholder
>>
>> CGF.pushDestroy(EHCleanup, LV.getAddress(), field->getType(),
>> CGF.getDestroyer(dtorKind), false);
>> @@ -1268,7 +1265,7 @@ void AggExprEmitter::VisitInitListExpr(I
>> // else, clean it up for -O0 builds and general tidiness.
>> if (!pushedCleanup && LV.isSimple())
>> if (llvm::GetElementPtrInst *GEP =
>> - dyn_cast<llvm::GetElementPtrInst>(LV.getAddress()))
>> + dyn_cast<llvm::GetElementPtrInst>(LV.getPointer()))
>> if (GEP->use_empty())
>> GEP->eraseFromParent();
>> }
>> @@ -1286,8 +1283,7 @@ void AggExprEmitter::VisitInitListExpr(I
>> void AggExprEmitter::VisitDesignatedInitUpdateExpr(DesignatedInitUpdateExpr *E) {
>> AggValueSlot Dest = EnsureSlot(E->getType());
>>
>> - LValue DestLV = CGF.MakeAddrLValue(Dest.getAddr(), E->getType(),
>> - Dest.getAlignment());
>> + LValue DestLV = CGF.MakeAddrLValue(Dest.getAddress(), E->getType());
>> EmitInitializationToLValue(E->getBase(), DestLV);
>> VisitInitListExpr(E->getUpdater());
>> }
>> @@ -1357,7 +1353,7 @@ static void CheckAggExprForMemSetUse(Agg
>> CodeGenFunction &CGF) {
>> // If the slot is already known to be zeroed, nothing to do. Don't mess with
>> // volatile stores.
>> - if (Slot.isZeroed() || Slot.isVolatile() || Slot.getAddr() == nullptr)
>> + if (Slot.isZeroed() || Slot.isVolatile() || !Slot.getAddress().isValid())
>> return;
>>
>> // C++ objects with a user-declared constructor don't need zero'ing.
>> @@ -1370,26 +1366,22 @@ static void CheckAggExprForMemSetUse(Agg
>> }
>>
>> // If the type is 16-bytes or smaller, prefer individual stores over memset.
>> - std::pair<CharUnits, CharUnits> TypeInfo =
>> - CGF.getContext().getTypeInfoInChars(E->getType());
>> - if (TypeInfo.first <= CharUnits::fromQuantity(16))
>> + CharUnits Size = CGF.getContext().getTypeSizeInChars(E->getType());
>> + if (Size <= CharUnits::fromQuantity(16))
>> return;
>>
>> // Check to see if over 3/4 of the initializer are known to be zero. If so,
>> // we prefer to emit memset + individual stores for the rest.
>> CharUnits NumNonZeroBytes = GetNumNonZeroBytesInInit(E, CGF);
>> - if (NumNonZeroBytes*4 > TypeInfo.first)
>> + if (NumNonZeroBytes*4 > Size)
>> return;
>>
>> // Okay, it seems like a good idea to use an initial memset, emit the call.
>> - llvm::Constant *SizeVal = CGF.Builder.getInt64(TypeInfo.first.getQuantity());
>> - CharUnits Align = TypeInfo.second;
>> + llvm::Constant *SizeVal = CGF.Builder.getInt64(Size.getQuantity());
>>
>> - llvm::Value *Loc = Slot.getAddr();
>> -
>> - Loc = CGF.Builder.CreateBitCast(Loc, CGF.Int8PtrTy);
>> - CGF.Builder.CreateMemSet(Loc, CGF.Builder.getInt8(0), SizeVal,
>> - Align.getQuantity(), false);
>> + Address Loc = Slot.getAddress();
>> + Loc = CGF.Builder.CreateElementBitCast(Loc, CGF.Int8Ty);
>> + CGF.Builder.CreateMemSet(Loc, CGF.Builder.getInt8(0), SizeVal, false);
>>
>> // Tell the AggExprEmitter that the slot is known zero.
>> Slot.setZeroed();
>> @@ -1405,7 +1397,7 @@ static void CheckAggExprForMemSetUse(Agg
>> void CodeGenFunction::EmitAggExpr(const Expr *E, AggValueSlot Slot) {
>> assert(E && hasAggregateEvaluationKind(E->getType()) &&
>> "Invalid aggregate expression to emit");
>> - assert((Slot.getAddr() != nullptr || Slot.isIgnored()) &&
>> + assert((Slot.getAddress().isValid() || Slot.isIgnored()) &&
>> "slot has bits but no address");
>>
>> // Optimize the slot if possible.
>> @@ -1416,7 +1408,7 @@ void CodeGenFunction::EmitAggExpr(const
>>
>> LValue CodeGenFunction::EmitAggExprToLValue(const Expr *E) {
>> assert(hasAggregateEvaluationKind(E->getType()) && "Invalid argument!");
>> - llvm::Value *Temp = CreateMemTemp(E->getType());
>> + Address Temp = CreateMemTemp(E->getType());
>> LValue LV = MakeAddrLValue(Temp, E->getType());
>> EmitAggExpr(E, AggValueSlot::forLValue(LV, AggValueSlot::IsNotDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> @@ -1424,10 +1416,9 @@ LValue CodeGenFunction::EmitAggExprToLVa
>> return LV;
>> }
>>
>> -void CodeGenFunction::EmitAggregateCopy(llvm::Value *DestPtr,
>> - llvm::Value *SrcPtr, QualType Ty,
>> +void CodeGenFunction::EmitAggregateCopy(Address DestPtr,
>> + Address SrcPtr, QualType Ty,
>> bool isVolatile,
>> - CharUnits alignment,
>> bool isAssignment) {
>> assert(!Ty->isAnyComplexType() && "Shouldn't happen for complex");
>>
>> @@ -1458,17 +1449,16 @@ void CodeGenFunction::EmitAggregateCopy(
>> // implementation handles this case safely. If there is a libc that does not
>> // safely handle this, we can add a target hook.
>>
>> - // Get data size and alignment info for this aggregate. If this is an
>> - // assignment don't copy the tail padding. Otherwise copying it is fine.
>> + // Get data size info for this aggregate. If this is an assignment,
>> + // don't copy the tail padding, because we might be assigning into a
>> + // base subobject where the tail padding is claimed. Otherwise,
>> + // copying it is fine.
>> std::pair<CharUnits, CharUnits> TypeInfo;
>> if (isAssignment)
>> TypeInfo = getContext().getTypeInfoDataSizeInChars(Ty);
>> else
>> TypeInfo = getContext().getTypeInfoInChars(Ty);
>>
>> - if (alignment.isZero())
>> - alignment = TypeInfo.second;
>> -
>> llvm::Value *SizeVal = nullptr;
>> if (TypeInfo.first.isZero()) {
>> // But note that getTypeInfo returns 0 for a VLA.
>> @@ -1511,15 +1501,8 @@ void CodeGenFunction::EmitAggregateCopy(
>> // we need to use a different call here. We use isVolatile to indicate when
>> // either the source or the destination is volatile.
>>
>> - llvm::PointerType *DPT = cast<llvm::PointerType>(DestPtr->getType());
>> - llvm::Type *DBP =
>> - llvm::Type::getInt8PtrTy(getLLVMContext(), DPT->getAddressSpace());
>> - DestPtr = Builder.CreateBitCast(DestPtr, DBP);
>> -
>> - llvm::PointerType *SPT = cast<llvm::PointerType>(SrcPtr->getType());
>> - llvm::Type *SBP =
>> - llvm::Type::getInt8PtrTy(getLLVMContext(), SPT->getAddressSpace());
>> - SrcPtr = Builder.CreateBitCast(SrcPtr, SBP);
>> + DestPtr = Builder.CreateElementBitCast(DestPtr, Int8Ty);
>> + SrcPtr = Builder.CreateElementBitCast(SrcPtr, Int8Ty);
>>
>> // Don't do any of the memmove_collectable tests if GC isn't set.
>> if (CGM.getLangOpts().getGC() == LangOptions::NonGC) {
>> @@ -1542,11 +1525,11 @@ void CodeGenFunction::EmitAggregateCopy(
>> }
>> }
>>
>> + auto Inst = Builder.CreateMemCpy(DestPtr, SrcPtr, SizeVal, isVolatile);
>> +
>> // Determine the metadata to describe the position of any padding in this
>> // memcpy, as well as the TBAA tags for the members of the struct, in case
>> // the optimizer wishes to expand it in to scalar memory operations.
>> - llvm::MDNode *TBAAStructTag = CGM.getTBAAStructInfo(Ty);
>> -
>> - Builder.CreateMemCpy(DestPtr, SrcPtr, SizeVal, alignment.getQuantity(),
>> - isVolatile, /*TBAATag=*/nullptr, TBAAStructTag);
>> + if (llvm::MDNode *TBAAStructTag = CGM.getTBAAStructInfo(Ty))
>> + Inst->setMetadata(llvm::LLVMContext::MD_tbaa_struct, TBAAStructTag);
>> }
>>
>> Modified: cfe/trunk/lib/CodeGen/CGExprCXX.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGExprCXX.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGExprCXX.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGExprCXX.cpp Tue Sep 8 03:05:57 2015
>> @@ -166,9 +166,9 @@ RValue CodeGenFunction::EmitCXXMemberOrO
>> }
>> }
>>
>> - llvm::Value *This;
>> + Address This = Address::invalid();
>> if (IsArrow)
>> - This = EmitScalarExpr(Base);
>> + This = EmitPointerWithAlignment(Base);
>> else
>> This = EmitLValue(Base).getAddress();
>>
>> @@ -185,19 +185,18 @@ RValue CodeGenFunction::EmitCXXMemberOrO
>> // when it isn't necessary; just produce the proper effect here.
>> // Special case: skip first argument of CXXOperatorCall (it is "this").
>> unsigned ArgsToSkip = isa<CXXOperatorCallExpr>(CE) ? 1 : 0;
>> - llvm::Value *RHS =
>> - EmitLValue(*(CE->arg_begin() + ArgsToSkip)).getAddress();
>> + Address RHS = EmitLValue(*(CE->arg_begin() + ArgsToSkip)).getAddress();
>> EmitAggregateAssign(This, RHS, CE->getType());
>> - return RValue::get(This);
>> + return RValue::get(This.getPointer());
>> }
>>
>> if (isa<CXXConstructorDecl>(MD) &&
>> cast<CXXConstructorDecl>(MD)->isCopyOrMoveConstructor()) {
>> // Trivial move and copy ctor are the same.
>> assert(CE->getNumArgs() == 1 && "unexpected argcount for trivial ctor");
>> - llvm::Value *RHS = EmitLValue(*CE->arg_begin()).getAddress();
>> + Address RHS = EmitLValue(*CE->arg_begin()).getAddress();
>> EmitAggregateCopy(This, RHS, (*CE->arg_begin())->getType());
>> - return RValue::get(This);
>> + return RValue::get(This.getPointer());
>> }
>> llvm_unreachable("unknown trivial member function");
>> }
>> @@ -245,7 +244,7 @@ RValue CodeGenFunction::EmitCXXMemberOrO
>> cast<CXXDestructorDecl>(DevirtualizedMethod);
>> Callee = CGM.GetAddrOfFunction(GlobalDecl(DDtor, Dtor_Complete), Ty);
>> }
>> - EmitCXXMemberOrOperatorCall(MD, Callee, ReturnValue, This,
>> + EmitCXXMemberOrOperatorCall(MD, Callee, ReturnValue, This.getPointer(),
>> /*ImplicitParam=*/nullptr, QualType(), CE);
>> }
>> return RValue::get(nullptr);
>> @@ -277,7 +276,7 @@ RValue CodeGenFunction::EmitCXXMemberOrO
>> *this, MD, This, UseVirtualCall);
>> }
>>
>> - return EmitCXXMemberOrOperatorCall(MD, Callee, ReturnValue, This,
>> + return EmitCXXMemberOrOperatorCall(MD, Callee, ReturnValue, This.getPointer(),
>> /*ImplicitParam=*/nullptr, QualType(), CE);
>> }
>>
>> @@ -301,19 +300,20 @@ CodeGenFunction::EmitCXXMemberPointerCal
>> llvm::Value *MemFnPtr = EmitScalarExpr(MemFnExpr);
>>
>> // Emit the 'this' pointer.
>> - llvm::Value *This;
>> -
>> + Address This = Address::invalid();
>> if (BO->getOpcode() == BO_PtrMemI)
>> - This = EmitScalarExpr(BaseExpr);
>> + This = EmitPointerWithAlignment(BaseExpr);
>> else
>> This = EmitLValue(BaseExpr).getAddress();
>>
>> - EmitTypeCheck(TCK_MemberCall, E->getExprLoc(), This,
>> + EmitTypeCheck(TCK_MemberCall, E->getExprLoc(), This.getPointer(),
>> QualType(MPT->getClass(), 0));
>>
>> // Ask the ABI to load the callee. Note that This is modified.
>> + llvm::Value *ThisPtrForCall = nullptr;
>> llvm::Value *Callee =
>> - CGM.getCXXABI().EmitLoadOfMemberFunctionPointer(*this, BO, This, MemFnPtr, MPT);
>> + CGM.getCXXABI().EmitLoadOfMemberFunctionPointer(*this, BO, This,
>> + ThisPtrForCall, MemFnPtr, MPT);
>>
>> CallArgList Args;
>>
>> @@ -321,7 +321,7 @@ CodeGenFunction::EmitCXXMemberPointerCal
>> getContext().getPointerType(getContext().getTagDeclType(RD));
>>
>> // Push the this ptr.
>> - Args.add(RValue::get(This), ThisType);
>> + Args.add(RValue::get(ThisPtrForCall), ThisType);
>>
>> RequiredArgs required = RequiredArgs::forPrototypePlus(FPT, 1);
>>
>> @@ -348,18 +348,15 @@ RValue CodeGenFunction::EmitCUDAKernelCa
>> }
>>
>> static void EmitNullBaseClassInitialization(CodeGenFunction &CGF,
>> - llvm::Value *DestPtr,
>> + Address DestPtr,
>> const CXXRecordDecl *Base) {
>> if (Base->isEmpty())
>> return;
>>
>> - DestPtr = CGF.EmitCastToVoidPtr(DestPtr);
>> + DestPtr = CGF.Builder.CreateElementBitCast(DestPtr, CGF.Int8Ty);
>>
>> const ASTRecordLayout &Layout = CGF.getContext().getASTRecordLayout(Base);
>> - CharUnits Size = Layout.getNonVirtualSize();
>> - CharUnits Align = Layout.getNonVirtualAlignment();
>> -
>> - llvm::Value *SizeVal = CGF.CGM.getSize(Size);
>> + llvm::Value *SizeVal = CGF.CGM.getSize(Layout.getNonVirtualSize());
>>
>> // If the type contains a pointer to data member we can't memset it to zero.
>> // Instead, create a null constant and copy it to the destination.
>> @@ -375,19 +372,22 @@ static void EmitNullBaseClassInitializat
>> /*isConstant=*/true,
>> llvm::GlobalVariable::PrivateLinkage,
>> NullConstant, Twine());
>> +
>> + CharUnits Align = std::max(Layout.getNonVirtualAlignment(),
>> + DestPtr.getAlignment());
>> NullVariable->setAlignment(Align.getQuantity());
>> - llvm::Value *SrcPtr = CGF.EmitCastToVoidPtr(NullVariable);
>> +
>> + Address SrcPtr = Address(CGF.EmitCastToVoidPtr(NullVariable), Align);
>>
>> // Get and call the appropriate llvm.memcpy overload.
>> - CGF.Builder.CreateMemCpy(DestPtr, SrcPtr, SizeVal, Align.getQuantity());
>> + CGF.Builder.CreateMemCpy(DestPtr, SrcPtr, SizeVal);
>> return;
>> }
>>
>> // Otherwise, just memset the whole thing to zero. This is legal
>> // because in LLVM, all default initializers (other than the ones we just
>> // handled above) are guaranteed to have a bit pattern of all zeros.
>> - CGF.Builder.CreateMemSet(DestPtr, CGF.Builder.getInt8(0), SizeVal,
>> - Align.getQuantity());
>> + CGF.Builder.CreateMemSet(DestPtr, CGF.Builder.getInt8(0), SizeVal);
>> }
>>
>> void
>> @@ -404,11 +404,12 @@ CodeGenFunction::EmitCXXConstructExpr(co
>> switch (E->getConstructionKind()) {
>> case CXXConstructExpr::CK_Delegating:
>> case CXXConstructExpr::CK_Complete:
>> - EmitNullInitialization(Dest.getAddr(), E->getType());
>> + EmitNullInitialization(Dest.getAddress(), E->getType());
>> break;
>> case CXXConstructExpr::CK_VirtualBase:
>> case CXXConstructExpr::CK_NonVirtualBase:
>> - EmitNullBaseClassInitialization(*this, Dest.getAddr(), CD->getParent());
>> + EmitNullBaseClassInitialization(*this, Dest.getAddress(),
>> + CD->getParent());
>> break;
>> }
>> }
>> @@ -431,7 +432,7 @@ CodeGenFunction::EmitCXXConstructExpr(co
>>
>> if (const ConstantArrayType *arrayType
>> = getContext().getAsConstantArrayType(E->getType())) {
>> - EmitCXXAggrConstructorCall(CD, arrayType, Dest.getAddr(), E);
>> + EmitCXXAggrConstructorCall(CD, arrayType, Dest.getAddress(), E);
>> } else {
>> CXXCtorType Type = Ctor_Complete;
>> bool ForVirtualBase = false;
>> @@ -457,15 +458,13 @@ CodeGenFunction::EmitCXXConstructExpr(co
>> }
>>
>> // Call the constructor.
>> - EmitCXXConstructorCall(CD, Type, ForVirtualBase, Delegating, Dest.getAddr(),
>> - E);
>> + EmitCXXConstructorCall(CD, Type, ForVirtualBase, Delegating,
>> + Dest.getAddress(), E);
>> }
>> }
>>
>> -void
>> -CodeGenFunction::EmitSynthesizedCXXCopyCtor(llvm::Value *Dest,
>> - llvm::Value *Src,
>> - const Expr *Exp) {
>> +void CodeGenFunction::EmitSynthesizedCXXCopyCtor(Address Dest, Address Src,
>> + const Expr *Exp) {
>> if (const ExprWithCleanups *E = dyn_cast<ExprWithCleanups>(Exp))
>> Exp = E->getSubExpr();
>> assert(isa<CXXConstructExpr>(Exp) &&
>> @@ -759,22 +758,20 @@ static llvm::Value *EmitCXXNewAllocSize(
>> }
>>
>> static void StoreAnyExprIntoOneUnit(CodeGenFunction &CGF, const Expr *Init,
>> - QualType AllocType, llvm::Value *NewPtr) {
>> + QualType AllocType, Address NewPtr) {
>> // FIXME: Refactor with EmitExprAsInit.
>> - CharUnits Alignment = CGF.getContext().getTypeAlignInChars(AllocType);
>> switch (CGF.getEvaluationKind(AllocType)) {
>> case TEK_Scalar:
>> CGF.EmitScalarInit(Init, nullptr,
>> - CGF.MakeAddrLValue(NewPtr, AllocType, Alignment), false);
>> + CGF.MakeAddrLValue(NewPtr, AllocType), false);
>> return;
>> case TEK_Complex:
>> - CGF.EmitComplexExprIntoLValue(Init, CGF.MakeAddrLValue(NewPtr, AllocType,
>> - Alignment),
>> + CGF.EmitComplexExprIntoLValue(Init, CGF.MakeAddrLValue(NewPtr, AllocType),
>> /*isInit*/ true);
>> return;
>> case TEK_Aggregate: {
>> AggValueSlot Slot
>> - = AggValueSlot::forAddr(NewPtr, Alignment, AllocType.getQualifiers(),
>> + = AggValueSlot::forAddr(NewPtr, AllocType.getQualifiers(),
>> AggValueSlot::IsDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsNotAliased);
>> @@ -787,23 +784,27 @@ static void StoreAnyExprIntoOneUnit(Code
>>
>> void CodeGenFunction::EmitNewArrayInitializer(
>> const CXXNewExpr *E, QualType ElementType, llvm::Type *ElementTy,
>> - llvm::Value *BeginPtr, llvm::Value *NumElements,
>> + Address BeginPtr, llvm::Value *NumElements,
>> llvm::Value *AllocSizeWithoutCookie) {
>> // If we have a type with trivial initialization and no initializer,
>> // there's nothing to do.
>> if (!E->hasInitializer())
>> return;
>>
>> - llvm::Value *CurPtr = BeginPtr;
>> + Address CurPtr = BeginPtr;
>>
>> unsigned InitListElements = 0;
>>
>> const Expr *Init = E->getInitializer();
>> - llvm::AllocaInst *EndOfInit = nullptr;
>> + Address EndOfInit = Address::invalid();
>> QualType::DestructionKind DtorKind = ElementType.isDestructedType();
>> EHScopeStack::stable_iterator Cleanup;
>> llvm::Instruction *CleanupDominator = nullptr;
>>
>> + CharUnits ElementSize = getContext().getTypeSizeInChars(ElementType);
>> + CharUnits ElementAlign =
>> + BeginPtr.getAlignment().alignmentOfArrayElement(ElementSize);
>> +
>> // If the initializer is an initializer list, first do the explicit elements.
>> if (const InitListExpr *ILE = dyn_cast<InitListExpr>(Init)) {
>> InitListElements = ILE->getNumInits();
>> @@ -813,10 +814,8 @@ void CodeGenFunction::EmitNewArrayInitia
>> QualType AllocType = E->getAllocatedType();
>> if (const ConstantArrayType *CAT = dyn_cast_or_null<ConstantArrayType>(
>> AllocType->getAsArrayTypeUnsafe())) {
>> - unsigned AS = CurPtr->getType()->getPointerAddressSpace();
>> ElementTy = ConvertTypeForMem(AllocType);
>> - llvm::Type *AllocPtrTy = ElementTy->getPointerTo(AS);
>> - CurPtr = Builder.CreateBitCast(CurPtr, AllocPtrTy);
>> + CurPtr = Builder.CreateElementBitCast(CurPtr, ElementTy);
>> InitListElements *= getContext().getConstantArrayElementCount(CAT);
>> }
>>
>> @@ -826,27 +825,34 @@ void CodeGenFunction::EmitNewArrayInitia
>> // directly, but the control flow can get so varied here that it
>> // would actually be quite complex. Therefore we go through an
>> // alloca.
>> - EndOfInit = CreateTempAlloca(BeginPtr->getType(), "array.init.end");
>> - CleanupDominator = Builder.CreateStore(BeginPtr, EndOfInit);
>> - pushIrregularPartialArrayCleanup(BeginPtr, EndOfInit, ElementType,
>> + EndOfInit = CreateTempAlloca(BeginPtr.getType(), getPointerAlign(),
>> + "array.init.end");
>> + CleanupDominator = Builder.CreateStore(BeginPtr.getPointer(), EndOfInit);
>> + pushIrregularPartialArrayCleanup(BeginPtr.getPointer(), EndOfInit,
>> + ElementType, ElementAlign,
>> getDestroyer(DtorKind));
>> Cleanup = EHStack.stable_begin();
>> }
>>
>> + CharUnits StartAlign = CurPtr.getAlignment();
>> for (unsigned i = 0, e = ILE->getNumInits(); i != e; ++i) {
>> // Tell the cleanup that it needs to destroy up to this
>> // element. TODO: some of these stores can be trivially
>> // observed to be unnecessary.
>> - if (EndOfInit)
>> - Builder.CreateStore(Builder.CreateBitCast(CurPtr, BeginPtr->getType()),
>> - EndOfInit);
>> + if (EndOfInit.isValid()) {
>> + auto FinishedPtr =
>> + Builder.CreateBitCast(CurPtr.getPointer(), BeginPtr.getType());
>> + Builder.CreateStore(FinishedPtr, EndOfInit);
>> + }
>> // FIXME: If the last initializer is an incomplete initializer list for
>> // an array, and we have an array filler, we can fold together the two
>> // initialization loops.
>> StoreAnyExprIntoOneUnit(*this, ILE->getInit(i),
>> ILE->getInit(i)->getType(), CurPtr);
>> - CurPtr = Builder.CreateConstInBoundsGEP1_32(ElementTy, CurPtr, 1,
>> - "array.exp.next");
>> + CurPtr = Address(Builder.CreateInBoundsGEP(CurPtr.getPointer(),
>> + Builder.getSize(1),
>> + "array.exp.next"),
>> + StartAlign.alignmentAtOffset((i + 1) * ElementSize));
>> }
>>
>> // The remaining elements are filled with the array filler expression.
>> @@ -864,7 +870,7 @@ void CodeGenFunction::EmitNewArrayInitia
>> }
>>
>> // Switch back to initializing one base element at a time.
>> - CurPtr = Builder.CreateBitCast(CurPtr, BeginPtr->getType());
>> + CurPtr = Builder.CreateBitCast(CurPtr, BeginPtr.getType());
>> }
>>
>> // Attempt to perform zero-initialization using memset.
>> @@ -889,9 +895,7 @@ void CodeGenFunction::EmitNewArrayInitia
>> }
>>
>> // Create the memset.
>> - CharUnits Alignment = getContext().getTypeAlignInChars(ElementType);
>> - Builder.CreateMemSet(CurPtr, Builder.getInt8(0), RemainingSize,
>> - Alignment.getQuantity(), false);
>> + Builder.CreateMemSet(CurPtr, Builder.getInt8(0), RemainingSize, false);
>> return true;
>> };
>>
>> @@ -925,7 +929,8 @@ void CodeGenFunction::EmitNewArrayInitia
>> //
>> // FIXME: Share this cleanup with the constructor call emission rather than
>> // having it create a cleanup of its own.
>> - if (EndOfInit) Builder.CreateStore(CurPtr, EndOfInit);
>> + if (EndOfInit.isValid())
>> + Builder.CreateStore(CurPtr.getPointer(), EndOfInit);
>>
>> // Emit a constructor call loop to initialize the remaining elements.
>> if (InitListElements)
>> @@ -985,13 +990,13 @@ void CodeGenFunction::EmitNewArrayInitia
>>
>> // Find the end of the array, hoisted out of the loop.
>> llvm::Value *EndPtr =
>> - Builder.CreateInBoundsGEP(BeginPtr, NumElements, "array.end");
>> + Builder.CreateInBoundsGEP(BeginPtr.getPointer(), NumElements, "array.end");
>>
>> // If the number of elements isn't constant, we have to now check if there is
>> // anything left to initialize.
>> if (!ConstNum) {
>> - llvm::Value *IsEmpty = Builder.CreateICmpEQ(CurPtr, EndPtr,
>> - "array.isempty");
>> + llvm::Value *IsEmpty =
>> + Builder.CreateICmpEQ(CurPtr.getPointer(), EndPtr, "array.isempty");
>> Builder.CreateCondBr(IsEmpty, ContBB, LoopBB);
>> }
>>
>> @@ -1000,16 +1005,19 @@ void CodeGenFunction::EmitNewArrayInitia
>>
>> // Set up the current-element phi.
>> llvm::PHINode *CurPtrPhi =
>> - Builder.CreatePHI(CurPtr->getType(), 2, "array.cur");
>> - CurPtrPhi->addIncoming(CurPtr, EntryBB);
>> - CurPtr = CurPtrPhi;
>> + Builder.CreatePHI(CurPtr.getType(), 2, "array.cur");
>> + CurPtrPhi->addIncoming(CurPtr.getPointer(), EntryBB);
>> +
>> + CurPtr = Address(CurPtrPhi, ElementAlign);
>>
>> // Store the new Cleanup position for irregular Cleanups.
>> - if (EndOfInit) Builder.CreateStore(CurPtr, EndOfInit);
>> + if (EndOfInit.isValid())
>> + Builder.CreateStore(CurPtr.getPointer(), EndOfInit);
>>
>> // Enter a partial-destruction Cleanup if necessary.
>> if (!CleanupDominator && needsEHCleanup(DtorKind)) {
>> - pushRegularPartialArrayCleanup(BeginPtr, CurPtr, ElementType,
>> + pushRegularPartialArrayCleanup(BeginPtr.getPointer(), CurPtr.getPointer(),
>> + ElementType, ElementAlign,
>> getDestroyer(DtorKind));
>> Cleanup = EHStack.stable_begin();
>> CleanupDominator = Builder.CreateUnreachable();
>> @@ -1026,7 +1034,8 @@ void CodeGenFunction::EmitNewArrayInitia
>>
>> // Advance to the next element by adjusting the pointer type as necessary.
>> llvm::Value *NextPtr =
>> - Builder.CreateConstInBoundsGEP1_32(ElementTy, CurPtr, 1, "array.next");
>> + Builder.CreateConstInBoundsGEP1_32(ElementTy, CurPtr.getPointer(), 1,
>> + "array.next");
>>
>> // Check whether we've gotten to the end of the array and, if so,
>> // exit the loop.
>> @@ -1039,7 +1048,7 @@ void CodeGenFunction::EmitNewArrayInitia
>>
>> static void EmitNewInitializer(CodeGenFunction &CGF, const CXXNewExpr *E,
>> QualType ElementType, llvm::Type *ElementTy,
>> - llvm::Value *NewPtr, llvm::Value *NumElements,
>> + Address NewPtr, llvm::Value *NumElements,
>> llvm::Value *AllocSizeWithoutCookie) {
>> ApplyDebugLocation DL(CGF, E);
>> if (E->isArray())
>> @@ -1218,7 +1227,7 @@ namespace {
>> /// new-expression throws.
>> static void EnterNewDeleteCleanup(CodeGenFunction &CGF,
>> const CXXNewExpr *E,
>> - llvm::Value *NewPtr,
>> + Address NewPtr,
>> llvm::Value *AllocSize,
>> const CallArgList &NewArgs) {
>> // If we're not inside a conditional branch, then the cleanup will
>> @@ -1228,7 +1237,8 @@ static void EnterNewDeleteCleanup(CodeGe
>> .pushCleanupWithExtra<CallDeleteDuringNew>(EHCleanup,
>> E->getNumPlacementArgs(),
>> E->getOperatorDelete(),
>> - NewPtr, AllocSize);
>> + NewPtr.getPointer(),
>> + AllocSize);
>> for (unsigned I = 0, N = E->getNumPlacementArgs(); I != N; ++I)
>> Cleanup->setPlacementArg(I, NewArgs[I+1].RV);
>>
>> @@ -1237,7 +1247,7 @@ static void EnterNewDeleteCleanup(CodeGe
>>
>> // Otherwise, we need to save all this stuff.
>> DominatingValue<RValue>::saved_type SavedNewPtr =
>> - DominatingValue<RValue>::save(CGF, RValue::get(NewPtr));
>> + DominatingValue<RValue>::save(CGF, RValue::get(NewPtr.getPointer()));
>> DominatingValue<RValue>::saved_type SavedAllocSize =
>> DominatingValue<RValue>::save(CGF, RValue::get(AllocSize));
>>
>> @@ -1260,13 +1270,6 @@ llvm::Value *CodeGenFunction::EmitCXXNew
>>
>> // 1. Build a call to the allocation function.
>> FunctionDecl *allocator = E->getOperatorNew();
>> - const FunctionProtoType *allocatorType =
>> - allocator->getType()->castAs<FunctionProtoType>();
>> -
>> - CallArgList allocatorArgs;
>> -
>> - // The allocation size is the first argument.
>> - QualType sizeType = getContext().getSizeType();
>>
>> // If there is a brace-initializer, cannot allocate fewer elements than inits.
>> unsigned minElements = 0;
>> @@ -1281,24 +1284,51 @@ llvm::Value *CodeGenFunction::EmitCXXNew
>> EmitCXXNewAllocSize(*this, E, minElements, numElements,
>> allocSizeWithoutCookie);
>>
>> - allocatorArgs.add(RValue::get(allocSize), sizeType);
>> -
>> - // We start at 1 here because the first argument (the allocation size)
>> - // has already been emitted.
>> - EmitCallArgs(allocatorArgs, allocatorType, E->placement_arguments(),
>> - /* CalleeDecl */ nullptr,
>> - /*ParamsToSkip*/ 1);
>> -
>> // Emit the allocation call. If the allocator is a global placement
>> // operator, just "inline" it directly.
>> - RValue RV;
>> + Address allocation = Address::invalid();
>> + CallArgList allocatorArgs;
>> if (allocator->isReservedGlobalPlacementOperator()) {
>> - assert(allocatorArgs.size() == 2);
>> - RV = allocatorArgs[1].RV;
>> - // TODO: kill any unnecessary computations done for the size
>> - // argument.
>> + AlignmentSource alignSource;
>> + allocation = EmitPointerWithAlignment(*E->placement_arguments().begin(),
>> + &alignSource);
>> +
>> + // The pointer expression will, in many cases, be an opaque void*.
>> + // In these cases, discard the computed alignment and use the
>> + // formal alignment of the allocated type.
>> + if (alignSource != AlignmentSource::Decl) {
>> + allocation = Address(allocation.getPointer(),
>> + getContext().getTypeAlignInChars(allocType));
>> + }
>> +
>> } else {
>> - RV = EmitNewDeleteCall(*this, allocator, allocatorType, allocatorArgs);
>> + const FunctionProtoType *allocatorType =
>> + allocator->getType()->castAs<FunctionProtoType>();
>> +
>> + // The allocation size is the first argument.
>> + QualType sizeType = getContext().getSizeType();
>> + allocatorArgs.add(RValue::get(allocSize), sizeType);
>> +
>> + // We start at 1 here because the first argument (the allocation size)
>> + // has already been emitted.
>> + EmitCallArgs(allocatorArgs, allocatorType, E->placement_arguments(),
>> + /* CalleeDecl */ nullptr,
>> + /*ParamsToSkip*/ 1);
>> +
>> + RValue RV =
>> + EmitNewDeleteCall(*this, allocator, allocatorType, allocatorArgs);
>> +
>> + // For now, only assume that the allocation function returns
>> + // something satisfactorily aligned for the element type, plus
>> + // the cookie if we have one.
>> + CharUnits allocationAlign =
>> + getContext().getTypeAlignInChars(allocType);
>> + if (allocSize != allocSizeWithoutCookie) {
>> + CharUnits cookieAlign = getSizeAlign(); // FIXME?
>> + allocationAlign = std::max(allocationAlign, cookieAlign);
>> + }
>> +
>> + allocation = Address(RV.getScalarVal(), allocationAlign);
>> }
>>
>> // Emit a null check on the allocation result if the allocation
>> @@ -1311,9 +1341,6 @@ llvm::Value *CodeGenFunction::EmitCXXNew
>> llvm::BasicBlock *nullCheckBB = nullptr;
>> llvm::BasicBlock *contBB = nullptr;
>>
>> - llvm::Value *allocation = RV.getScalarVal();
>> - unsigned AS = allocation->getType()->getPointerAddressSpace();
>> -
>> // The null-check means that the initializer is conditionally
>> // evaluated.
>> ConditionalEvaluation conditional(*this);
>> @@ -1325,7 +1352,8 @@ llvm::Value *CodeGenFunction::EmitCXXNew
>> llvm::BasicBlock *notNullBB = createBasicBlock("new.notnull");
>> contBB = createBasicBlock("new.cont");
>>
>> - llvm::Value *isNull = Builder.CreateIsNull(allocation, "new.isnull");
>> + llvm::Value *isNull =
>> + Builder.CreateIsNull(allocation.getPointer(), "new.isnull");
>> Builder.CreateCondBr(isNull, contBB, notNullBB);
>> EmitBlock(notNullBB);
>> }
>> @@ -1351,8 +1379,7 @@ llvm::Value *CodeGenFunction::EmitCXXNew
>> }
>>
>> llvm::Type *elementTy = ConvertTypeForMem(allocType);
>> - llvm::Type *elementPtrTy = elementTy->getPointerTo(AS);
>> - llvm::Value *result = Builder.CreateBitCast(allocation, elementPtrTy);
>> + Address result = Builder.CreateElementBitCast(allocation, elementTy);
>>
>> EmitNewInitializer(*this, E, allocType, elementTy, result, numElements,
>> allocSizeWithoutCookie);
>> @@ -1361,7 +1388,7 @@ llvm::Value *CodeGenFunction::EmitCXXNew
>> // allocating an array of arrays, we'll need to cast back to the
>> // array pointer type.
>> llvm::Type *resultType = ConvertTypeForMem(E->getType());
>> - if (result->getType() != resultType)
>> + if (result.getType() != resultType)
>> result = Builder.CreateBitCast(result, resultType);
>> }
>>
>> @@ -1372,21 +1399,22 @@ llvm::Value *CodeGenFunction::EmitCXXNew
>> cleanupDominator->eraseFromParent();
>> }
>>
>> + llvm::Value *resultPtr = result.getPointer();
>> if (nullCheck) {
>> conditional.end(*this);
>>
>> llvm::BasicBlock *notNullBB = Builder.GetInsertBlock();
>> EmitBlock(contBB);
>>
>> - llvm::PHINode *PHI = Builder.CreatePHI(result->getType(), 2);
>> - PHI->addIncoming(result, notNullBB);
>> - PHI->addIncoming(llvm::Constant::getNullValue(result->getType()),
>> + llvm::PHINode *PHI = Builder.CreatePHI(resultPtr->getType(), 2);
>> + PHI->addIncoming(resultPtr, notNullBB);
>> + PHI->addIncoming(llvm::Constant::getNullValue(resultPtr->getType()),
>> nullCheckBB);
>>
>> - result = PHI;
>> + resultPtr = PHI;
>> }
>>
>> - return result;
>> + return resultPtr;
>> }
>>
>> void CodeGenFunction::EmitDeleteCall(const FunctionDecl *DeleteFD,
>> @@ -1449,7 +1477,7 @@ CodeGenFunction::pushCallObjectDeleteCle
>> /// Emit the code for deleting a single object.
>> static void EmitObjectDelete(CodeGenFunction &CGF,
>> const CXXDeleteExpr *DE,
>> - llvm::Value *Ptr,
>> + Address Ptr,
>> QualType ElementType) {
>> // Find the destructor for the type, if applicable. If the
>> // destructor is virtual, we'll just emit the vcall and return.
>> @@ -1472,7 +1500,8 @@ static void EmitObjectDelete(CodeGenFunc
>> // to pop it off in a second.
>> const FunctionDecl *OperatorDelete = DE->getOperatorDelete();
>> CGF.EHStack.pushCleanup<CallObjectDelete>(NormalAndEHCleanup,
>> - Ptr, OperatorDelete, ElementType);
>> + Ptr.getPointer(),
>> + OperatorDelete, ElementType);
>>
>> if (Dtor)
>> CGF.EmitCXXDestructorCall(Dtor, Dtor_Complete,
>> @@ -1487,14 +1516,9 @@ static void EmitObjectDelete(CodeGenFunc
>> case Qualifiers::OCL_Autoreleasing:
>> break;
>>
>> - case Qualifiers::OCL_Strong: {
>> - // Load the pointer value.
>> - llvm::Value *PtrValue = CGF.Builder.CreateLoad(Ptr,
>> - ElementType.isVolatileQualified());
>> -
>> - CGF.EmitARCRelease(PtrValue, ARCPreciseLifetime);
>> + case Qualifiers::OCL_Strong:
>> + CGF.EmitARCDestroyStrong(Ptr, ARCPreciseLifetime);
>> break;
>> - }
>>
>> case Qualifiers::OCL_Weak:
>> CGF.EmitARCDestroyWeak(Ptr);
>> @@ -1569,7 +1593,7 @@ namespace {
>> /// Emit the code for deleting an array of objects.
>> static void EmitArrayDelete(CodeGenFunction &CGF,
>> const CXXDeleteExpr *E,
>> - llvm::Value *deletedPtr,
>> + Address deletedPtr,
>> QualType elementType) {
>> llvm::Value *numElements = nullptr;
>> llvm::Value *allocatedPtr = nullptr;
>> @@ -1590,13 +1614,18 @@ static void EmitArrayDelete(CodeGenFunct
>> if (QualType::DestructionKind dtorKind = elementType.isDestructedType()) {
>> assert(numElements && "no element count for a type with a destructor!");
>>
>> + CharUnits elementSize = CGF.getContext().getTypeSizeInChars(elementType);
>> + CharUnits elementAlign =
>> + deletedPtr.getAlignment().alignmentOfArrayElement(elementSize);
>> +
>> + llvm::Value *arrayBegin = deletedPtr.getPointer();
>> llvm::Value *arrayEnd =
>> - CGF.Builder.CreateInBoundsGEP(deletedPtr, numElements, "delete.end");
>> + CGF.Builder.CreateInBoundsGEP(arrayBegin, numElements, "delete.end");
>>
>> // Note that it is legal to allocate a zero-length array, and we
>> // can never fold the check away because the length should always
>> // come from a cookie.
>> - CGF.emitArrayDestroy(deletedPtr, arrayEnd, elementType,
>> + CGF.emitArrayDestroy(arrayBegin, arrayEnd, elementType, elementAlign,
>> CGF.getDestroyer(dtorKind),
>> /*checkZeroLength*/ true,
>> CGF.needsEHCleanup(dtorKind));
>> @@ -1608,13 +1637,13 @@ static void EmitArrayDelete(CodeGenFunct
>>
>> void CodeGenFunction::EmitCXXDeleteExpr(const CXXDeleteExpr *E) {
>> const Expr *Arg = E->getArgument();
>> - llvm::Value *Ptr = EmitScalarExpr(Arg);
>> + Address Ptr = EmitPointerWithAlignment(Arg);
>>
>> // Null check the pointer.
>> llvm::BasicBlock *DeleteNotNull = createBasicBlock("delete.notnull");
>> llvm::BasicBlock *DeleteEnd = createBasicBlock("delete.end");
>>
>> - llvm::Value *IsNull = Builder.CreateIsNull(Ptr, "isnull");
>> + llvm::Value *IsNull = Builder.CreateIsNull(Ptr.getPointer(), "isnull");
>>
>> Builder.CreateCondBr(IsNull, DeleteEnd, DeleteNotNull);
>> EmitBlock(DeleteNotNull);
>> @@ -1639,11 +1668,11 @@ void CodeGenFunction::EmitCXXDeleteExpr(
>> GEP.push_back(Zero);
>> }
>>
>> - Ptr = Builder.CreateInBoundsGEP(Ptr, GEP, "del.first");
>> + Ptr = Address(Builder.CreateInBoundsGEP(Ptr.getPointer(), GEP, "del.first"),
>> + Ptr.getAlignment());
>> }
>>
>> - assert(ConvertTypeForMem(DeleteTy) ==
>> - cast<llvm::PointerType>(Ptr->getType())->getElementType());
>> + assert(ConvertTypeForMem(DeleteTy) == Ptr.getElementType());
>>
>> if (E->isArrayForm()) {
>> EmitArrayDelete(*this, E, Ptr, DeleteTy);
>> @@ -1689,7 +1718,7 @@ static bool isGLValueFromPointerDeref(co
>> static llvm::Value *EmitTypeidFromVTable(CodeGenFunction &CGF, const Expr *E,
>> llvm::Type *StdTypeInfoPtrTy) {
>> // Get the vtable pointer.
>> - llvm::Value *ThisPtr = CGF.EmitLValue(E).getAddress();
>> + Address ThisPtr = CGF.EmitLValue(E).getAddress();
>>
>> // C++ [expr.typeid]p2:
>> // If the glvalue expression is obtained by applying the unary * operator to
>> @@ -1706,7 +1735,7 @@ static llvm::Value *EmitTypeidFromVTable
>> CGF.createBasicBlock("typeid.bad_typeid");
>> llvm::BasicBlock *EndBlock = CGF.createBasicBlock("typeid.end");
>>
>> - llvm::Value *IsNull = CGF.Builder.CreateIsNull(ThisPtr);
>> + llvm::Value *IsNull = CGF.Builder.CreateIsNull(ThisPtr.getPointer());
>> CGF.Builder.CreateCondBr(IsNull, BadTypeidBlock, EndBlock);
>>
>> CGF.EmitBlock(BadTypeidBlock);
>> @@ -1757,7 +1786,7 @@ static llvm::Value *EmitDynamicCastToNul
>> return llvm::UndefValue::get(DestLTy);
>> }
>>
>> -llvm::Value *CodeGenFunction::EmitDynamicCast(llvm::Value *Value,
>> +llvm::Value *CodeGenFunction::EmitDynamicCast(Address ThisAddr,
>> const CXXDynamicCastExpr *DCE) {
>> QualType DestTy = DCE->getTypeAsWritten();
>>
>> @@ -1802,18 +1831,19 @@ llvm::Value *CodeGenFunction::EmitDynami
>> CastNull = createBasicBlock("dynamic_cast.null");
>> CastNotNull = createBasicBlock("dynamic_cast.notnull");
>>
>> - llvm::Value *IsNull = Builder.CreateIsNull(Value);
>> + llvm::Value *IsNull = Builder.CreateIsNull(ThisAddr.getPointer());
>> Builder.CreateCondBr(IsNull, CastNull, CastNotNull);
>> EmitBlock(CastNotNull);
>> }
>>
>> + llvm::Value *Value;
>> if (isDynamicCastToVoid) {
>> - Value = CGM.getCXXABI().EmitDynamicCastToVoid(*this, Value, SrcRecordTy,
>> + Value = CGM.getCXXABI().EmitDynamicCastToVoid(*this, ThisAddr, SrcRecordTy,
>> DestTy);
>> } else {
>> assert(DestRecordTy->isRecordType() &&
>> "destination type must be a record type!");
>> - Value = CGM.getCXXABI().EmitDynamicCastCall(*this, Value, SrcRecordTy,
>> + Value = CGM.getCXXABI().EmitDynamicCastCall(*this, ThisAddr, SrcRecordTy,
>> DestTy, DestRecordTy, CastEnd);
>> }
>>
>> @@ -1839,8 +1869,7 @@ llvm::Value *CodeGenFunction::EmitDynami
>>
>> void CodeGenFunction::EmitLambdaExpr(const LambdaExpr *E, AggValueSlot Slot) {
>> RunCleanupsScope Scope(*this);
>> - LValue SlotLV =
>> - MakeAddrLValue(Slot.getAddr(), E->getType(), Slot.getAlignment());
>> + LValue SlotLV = MakeAddrLValue(Slot.getAddress(), E->getType());
>>
>> CXXRecordDecl::field_iterator CurField = E->getLambdaClass()->field_begin();
>> for (LambdaExpr::const_capture_init_iterator i = E->capture_init_begin(),
>>
>> Modified: cfe/trunk/lib/CodeGen/CGExprComplex.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGExprComplex.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGExprComplex.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGExprComplex.cpp Tue Sep 8 03:05:57 2015
>> @@ -298,6 +298,19 @@ public:
>> // Utilities
>> //===----------------------------------------------------------------------===//
>>
>> +Address CodeGenFunction::emitAddrOfRealComponent(Address addr,
>> + QualType complexType) {
>> + CharUnits offset = CharUnits::Zero();
>> + return Builder.CreateStructGEP(addr, 0, offset, addr.getName() + ".realp");
>> +}
>> +
>> +Address CodeGenFunction::emitAddrOfImagComponent(Address addr,
>> + QualType complexType) {
>> + QualType eltType = complexType->castAs<ComplexType>()->getElementType();
>> + CharUnits offset = getContext().getTypeSizeInChars(eltType);
>> + return Builder.CreateStructGEP(addr, 1, offset, addr.getName() + ".imagp");
>> +}
>> +
>> /// EmitLoadOfLValue - Given an RValue reference for a complex, emit code to
>> /// load the real and imaginary pieces, returning them as Real/Imag.
>> ComplexPairTy ComplexExprEmitter::EmitLoadOfLValue(LValue lvalue,
>> @@ -306,29 +319,21 @@ ComplexPairTy ComplexExprEmitter::EmitLo
>> if (lvalue.getType()->isAtomicType())
>> return CGF.EmitAtomicLoad(lvalue, loc).getComplexVal();
>>
>> - llvm::Value *SrcPtr = lvalue.getAddress();
>> + Address SrcPtr = lvalue.getAddress();
>> bool isVolatile = lvalue.isVolatileQualified();
>> - unsigned AlignR = lvalue.getAlignment().getQuantity();
>> - ASTContext &C = CGF.getContext();
>> - QualType ComplexTy = lvalue.getType();
>> - unsigned ComplexAlign = C.getTypeAlignInChars(ComplexTy).getQuantity();
>> - unsigned AlignI = std::min(AlignR, ComplexAlign);
>>
>> - llvm::Value *Real=nullptr, *Imag=nullptr;
>> + llvm::Value *Real = nullptr, *Imag = nullptr;
>>
>> if (!IgnoreReal || isVolatile) {
>> - llvm::Value *RealP = Builder.CreateStructGEP(nullptr, SrcPtr, 0,
>> - SrcPtr->getName() + ".realp");
>> - Real = Builder.CreateAlignedLoad(RealP, AlignR, isVolatile,
>> - SrcPtr->getName() + ".real");
>> + Address RealP = CGF.emitAddrOfRealComponent(SrcPtr, lvalue.getType());
>> + Real = Builder.CreateLoad(RealP, isVolatile, SrcPtr.getName() + ".real");
>> }
>>
>> if (!IgnoreImag || isVolatile) {
>> - llvm::Value *ImagP = Builder.CreateStructGEP(nullptr, SrcPtr, 1,
>> - SrcPtr->getName() + ".imagp");
>> - Imag = Builder.CreateAlignedLoad(ImagP, AlignI, isVolatile,
>> - SrcPtr->getName() + ".imag");
>> + Address ImagP = CGF.emitAddrOfImagComponent(SrcPtr, lvalue.getType());
>> + Imag = Builder.CreateLoad(ImagP, isVolatile, SrcPtr.getName() + ".imag");
>> }
>> +
>> return ComplexPairTy(Real, Imag);
>> }
>>
>> @@ -340,19 +345,12 @@ void ComplexExprEmitter::EmitStoreOfComp
>> (!isInit && CGF.LValueIsSuitableForInlineAtomic(lvalue)))
>> return CGF.EmitAtomicStore(RValue::getComplex(Val), lvalue, isInit);
>>
>> - llvm::Value *Ptr = lvalue.getAddress();
>> - llvm::Value *RealPtr = Builder.CreateStructGEP(nullptr, Ptr, 0, "real");
>> - llvm::Value *ImagPtr = Builder.CreateStructGEP(nullptr, Ptr, 1, "imag");
>> - unsigned AlignR = lvalue.getAlignment().getQuantity();
>> - ASTContext &C = CGF.getContext();
>> - QualType ComplexTy = lvalue.getType();
>> - unsigned ComplexAlign = C.getTypeAlignInChars(ComplexTy).getQuantity();
>> - unsigned AlignI = std::min(AlignR, ComplexAlign);
>> -
>> - Builder.CreateAlignedStore(Val.first, RealPtr, AlignR,
>> - lvalue.isVolatileQualified());
>> - Builder.CreateAlignedStore(Val.second, ImagPtr, AlignI,
>> - lvalue.isVolatileQualified());
>> + Address Ptr = lvalue.getAddress();
>> + Address RealPtr = CGF.emitAddrOfRealComponent(Ptr, lvalue.getType());
>> + Address ImagPtr = CGF.emitAddrOfImagComponent(Ptr, lvalue.getType());
>> +
>> + Builder.CreateStore(Val.first, RealPtr, lvalue.isVolatileQualified());
>> + Builder.CreateStore(Val.second, ImagPtr, lvalue.isVolatileQualified());
>> }
>>
>>
>> @@ -385,8 +383,8 @@ ComplexPairTy ComplexExprEmitter::VisitC
>>
>> ComplexPairTy ComplexExprEmitter::VisitStmtExpr(const StmtExpr *E) {
>> CodeGenFunction::StmtExprEvaluation eval(CGF);
>> - llvm::Value *RetAlloca = CGF.EmitCompoundStmt(*E->getSubStmt(), true);
>> - assert(RetAlloca && "Expected complex return value");
>> + Address RetAlloca = CGF.EmitCompoundStmt(*E->getSubStmt(), true);
>> + assert(RetAlloca.isValid() && "Expected complex return value");
>> return EmitLoadOfLValue(CGF.MakeAddrLValue(RetAlloca, E->getType()),
>> E->getExprLoc());
>> }
>> @@ -436,12 +434,9 @@ ComplexPairTy ComplexExprEmitter::EmitCa
>>
>> case CK_LValueBitCast: {
>> LValue origLV = CGF.EmitLValue(Op);
>> - llvm::Value *V = origLV.getAddress();
>> - V = Builder.CreateBitCast(V,
>> - CGF.ConvertType(CGF.getContext().getPointerType(DestTy)));
>> - return EmitLoadOfLValue(CGF.MakeAddrLValue(V, DestTy,
>> - origLV.getAlignment()),
>> - Op->getExprLoc());
>> + Address V = origLV.getAddress();
>> + V = Builder.CreateElementBitCast(V, CGF.ConvertType(DestTy));
>> + return EmitLoadOfLValue(CGF.MakeAddrLValue(V, DestTy), Op->getExprLoc());
>> }
>>
>> case CK_BitCast:
>> @@ -1016,10 +1011,10 @@ ComplexPairTy ComplexExprEmitter::VisitI
>> }
>>
>> ComplexPairTy ComplexExprEmitter::VisitVAArgExpr(VAArgExpr *E) {
>> - llvm::Value *ArgValue = CGF.EmitVAListRef(E->getSubExpr());
>> - llvm::Value *ArgPtr = CGF.EmitVAArg(ArgValue, E->getType());
>> + Address ArgValue = CGF.EmitVAListRef(E->getSubExpr());
>> + Address ArgPtr = CGF.EmitVAArg(ArgValue, E->getType());
>>
>> - if (!ArgPtr) {
>> + if (!ArgPtr.isValid()) {
>> CGF.ErrorUnsupported(E, "complex va_arg expression");
>> llvm::Type *EltTy =
>> CGF.ConvertType(E->getType()->castAs<ComplexType>()->getElementType());
>> @@ -1027,7 +1022,7 @@ ComplexPairTy ComplexExprEmitter::VisitV
>> return ComplexPairTy(U, U);
>> }
>>
>> - return EmitLoadOfLValue(CGF.MakeNaturalAlignAddrLValue(ArgPtr, E->getType()),
>> + return EmitLoadOfLValue(CGF.MakeAddrLValue(ArgPtr, E->getType()),
>> E->getExprLoc());
>> }
>>
>>
>> Modified: cfe/trunk/lib/CodeGen/CGExprConstant.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGExprConstant.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGExprConstant.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGExprConstant.cpp Tue Sep 8 03:05:57 2015
>> @@ -977,23 +977,26 @@ public:
>> }
>>
>> public:
>> - llvm::Constant *EmitLValue(APValue::LValueBase LVBase) {
>> + ConstantAddress EmitLValue(APValue::LValueBase LVBase) {
>> if (const ValueDecl *Decl = LVBase.dyn_cast<const ValueDecl*>()) {
>> if (Decl->hasAttr<WeakRefAttr>())
>> return CGM.GetWeakRefReference(Decl);
>> if (const FunctionDecl *FD = dyn_cast<FunctionDecl>(Decl))
>> - return CGM.GetAddrOfFunction(FD);
>> + return ConstantAddress(CGM.GetAddrOfFunction(FD), CharUnits::One());
>> if (const VarDecl* VD = dyn_cast<VarDecl>(Decl)) {
>> // We can never refer to a variable with local storage.
>> if (!VD->hasLocalStorage()) {
>> + CharUnits Align = CGM.getContext().getDeclAlign(VD);
>> if (VD->isFileVarDecl() || VD->hasExternalStorage())
>> - return CGM.GetAddrOfGlobalVar(VD);
>> - else if (VD->isLocalVarDecl())
>> - return CGM.getOrCreateStaticVarDecl(
>> + return ConstantAddress(CGM.GetAddrOfGlobalVar(VD), Align);
>> + else if (VD->isLocalVarDecl()) {
>> + auto Ptr = CGM.getOrCreateStaticVarDecl(
>> *VD, CGM.getLLVMLinkageVarDefinition(VD, /*isConstant=*/false));
>> + return ConstantAddress(Ptr, Align);
>> + }
>> }
>> }
>> - return nullptr;
>> + return ConstantAddress::invalid();
>> }
>>
>> Expr *E = const_cast<Expr*>(LVBase.get<const Expr*>());
>> @@ -1006,14 +1009,18 @@ public:
>> llvm::Constant* C = CGM.EmitConstantExpr(CLE->getInitializer(),
>> CLE->getType(), CGF);
>> // FIXME: "Leaked" on failure.
>> - if (C)
>> - C = new llvm::GlobalVariable(CGM.getModule(), C->getType(),
>> + if (!C) return ConstantAddress::invalid();
>> +
>> + CharUnits Align = CGM.getContext().getTypeAlignInChars(E->getType());
>> +
>> + auto GV = new llvm::GlobalVariable(CGM.getModule(), C->getType(),
>> E->getType().isConstant(CGM.getContext()),
>> llvm::GlobalValue::InternalLinkage,
>> C, ".compoundliteral", nullptr,
>> llvm::GlobalVariable::NotThreadLocal,
>> CGM.getContext().getTargetAddressSpace(E->getType()));
>> - return C;
>> + GV->setAlignment(Align.getQuantity());
>> + return ConstantAddress(GV, Align);
>> }
>> case Expr::StringLiteralClass:
>> return CGM.GetAddrOfConstantStringFromLiteral(cast<StringLiteral>(E));
>> @@ -1021,15 +1028,15 @@ public:
>> return CGM.GetAddrOfConstantStringFromObjCEncode(cast<ObjCEncodeExpr>(E));
>> case Expr::ObjCStringLiteralClass: {
>> ObjCStringLiteral* SL = cast<ObjCStringLiteral>(E);
>> - llvm::Constant *C =
>> + ConstantAddress C =
>> CGM.getObjCRuntime().GenerateConstantString(SL->getString());
>> - return llvm::ConstantExpr::getBitCast(C, ConvertType(E->getType()));
>> + return C.getElementBitCast(ConvertType(E->getType()));
>> }
>> case Expr::PredefinedExprClass: {
>> unsigned Type = cast<PredefinedExpr>(E)->getIdentType();
>> if (CGF) {
>> LValue Res = CGF->EmitPredefinedLValue(cast<PredefinedExpr>(E));
>> - return cast<llvm::Constant>(Res.getAddress());
>> + return cast<ConstantAddress>(Res.getAddress());
>> } else if (Type == PredefinedExpr::PrettyFunction) {
>> return CGM.GetAddrOfConstantCString("top level", ".tmp");
>> }
>> @@ -1040,7 +1047,8 @@ public:
>> assert(CGF && "Invalid address of label expression outside function.");
>> llvm::Constant *Ptr =
>> CGF->GetAddrOfLabel(cast<AddrLabelExpr>(E)->getLabel());
>> - return llvm::ConstantExpr::getBitCast(Ptr, ConvertType(E->getType()));
>> + Ptr = llvm::ConstantExpr::getBitCast(Ptr, ConvertType(E->getType()));
>> + return ConstantAddress(Ptr, CharUnits::One());
>> }
>> case Expr::CallExprClass: {
>> CallExpr* CE = cast<CallExpr>(E);
>> @@ -1066,7 +1074,10 @@ public:
>> else
>> FunctionName = "global";
>>
>> - return CGM.GetAddrOfGlobalBlock(cast<BlockExpr>(E), FunctionName.c_str());
>> + // This is not really an l-value.
>> + llvm::Constant *Ptr =
>> + CGM.GetAddrOfGlobalBlock(cast<BlockExpr>(E), FunctionName.c_str());
>> + return ConstantAddress(Ptr, CGM.getPointerAlign());
>> }
>> case Expr::CXXTypeidExprClass: {
>> CXXTypeidExpr *Typeid = cast<CXXTypeidExpr>(E);
>> @@ -1075,7 +1086,8 @@ public:
>> T = Typeid->getTypeOperand(CGM.getContext());
>> else
>> T = Typeid->getExprOperand()->getType();
>> - return CGM.GetAddrOfRTTIDescriptor(T);
>> + return ConstantAddress(CGM.GetAddrOfRTTIDescriptor(T),
>> + CGM.getPointerAlign());
>> }
>> case Expr::CXXUuidofExprClass: {
>> return CGM.GetAddrOfUuidDescriptor(cast<CXXUuidofExpr>(E));
>> @@ -1091,7 +1103,7 @@ public:
>> }
>> }
>>
>> - return nullptr;
>> + return ConstantAddress::invalid();
>> }
>> };
>>
>> @@ -1255,7 +1267,7 @@ llvm::Constant *CodeGenModule::EmitConst
>> llvm::Constant *Offset =
>> llvm::ConstantInt::get(Int64Ty, Value.getLValueOffset().getQuantity());
>>
>> - llvm::Constant *C;
>> + llvm::Constant *C = nullptr;
>> if (APValue::LValueBase LVBase = Value.getLValueBase()) {
>> // An array can be represented as an lvalue referring to the base.
>> if (isa<llvm::ArrayType>(DestTy)) {
>> @@ -1264,7 +1276,7 @@ llvm::Constant *CodeGenModule::EmitConst
>> const_cast<Expr*>(LVBase.get<const Expr*>()));
>> }
>>
>> - C = ConstExprEmitter(*this, CGF).EmitLValue(LVBase);
>> + C = ConstExprEmitter(*this, CGF).EmitLValue(LVBase).getPointer();
>>
>> // Apply offset if necessary.
>> if (!Offset->isNullValue()) {
>> @@ -1438,7 +1450,7 @@ CodeGenModule::EmitConstantValueForMemor
>> return C;
>> }
>>
>> -llvm::Constant *
>> +ConstantAddress
>> CodeGenModule::GetAddrOfConstantCompoundLiteral(const CompoundLiteralExpr *E) {
>> assert(E->isFileScope() && "not a file-scope compound literal expr");
>> return ConstExprEmitter(*this, nullptr).EmitLValue(E);
>>
>> Modified: cfe/trunk/lib/CodeGen/CGExprScalar.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGExprScalar.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGExprScalar.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGExprScalar.cpp Tue Sep 8 03:05:57 2015
>> @@ -363,7 +363,7 @@ public:
>> if (isa<MemberPointerType>(E->getType())) // never sugared
>> return CGF.CGM.getMemberPointerConstant(E);
>>
>> - return EmitLValue(E->getSubExpr()).getAddress();
>> + return EmitLValue(E->getSubExpr()).getPointer();
>> }
>> Value *VisitUnaryDeref(const UnaryOperator *E) {
>> if (E->getType()->isVoidType())
>> @@ -1327,13 +1327,13 @@ Value *ScalarExprEmitter::VisitInitListE
>> return V;
>> }
>>
>> -static bool ShouldNullCheckClassCastValue(const CastExpr *CE) {
>> +bool CodeGenFunction::ShouldNullCheckClassCastValue(const CastExpr *CE) {
>> const Expr *E = CE->getSubExpr();
>>
>> if (CE->getCastKind() == CK_UncheckedDerivedToBase)
>> return false;
>>
>> - if (isa<CXXThisExpr>(E)) {
>> + if (isa<CXXThisExpr>(E->IgnoreParens())) {
>> // We always assume that 'this' is never null.
>> return false;
>> }
>> @@ -1368,11 +1368,10 @@ Value *ScalarExprEmitter::VisitCastExpr(
>>
>> case CK_LValueBitCast:
>> case CK_ObjCObjectLValueCast: {
>> - Value *V = EmitLValue(E).getAddress();
>> - V = Builder.CreateBitCast(V,
>> - ConvertType(CGF.getContext().getPointerType(DestTy)));
>> - return EmitLoadOfLValue(CGF.MakeNaturalAlignAddrLValue(V, DestTy),
>> - CE->getExprLoc());
>> + Address Addr = EmitLValue(E).getAddress();
>> + Addr = Builder.CreateElementBitCast(Addr, ConvertType(DestTy));
>> + LValue LV = CGF.MakeAddrLValue(Addr, DestTy);
>> + return EmitLoadOfLValue(LV, CE->getExprLoc());
>> }
>>
>> case CK_CPointerToObjCPointerCast:
>> @@ -1412,68 +1411,44 @@ Value *ScalarExprEmitter::VisitCastExpr(
>> const CXXRecordDecl *DerivedClassDecl = DestTy->getPointeeCXXRecordDecl();
>> assert(DerivedClassDecl && "BaseToDerived arg isn't a C++ object pointer!");
>>
>> - llvm::Value *V = Visit(E);
>> -
>> - llvm::Value *Derived =
>> - CGF.GetAddressOfDerivedClass(V, DerivedClassDecl,
>> + Address Base = CGF.EmitPointerWithAlignment(E);
>> + Address Derived =
>> + CGF.GetAddressOfDerivedClass(Base, DerivedClassDecl,
>> CE->path_begin(), CE->path_end(),
>> - ShouldNullCheckClassCastValue(CE));
>> + CGF.ShouldNullCheckClassCastValue(CE));
>>
>> // C++11 [expr.static.cast]p11: Behavior is undefined if a downcast is
>> // performed and the object is not of the derived type.
>> if (CGF.sanitizePerformTypeCheck())
>> CGF.EmitTypeCheck(CodeGenFunction::TCK_DowncastPointer, CE->getExprLoc(),
>> - Derived, DestTy->getPointeeType());
>> + Derived.getPointer(), DestTy->getPointeeType());
>>
>> if (CGF.SanOpts.has(SanitizerKind::CFIDerivedCast))
>> - CGF.EmitVTablePtrCheckForCast(DestTy->getPointeeType(), Derived,
>> + CGF.EmitVTablePtrCheckForCast(DestTy->getPointeeType(),
>> + Derived.getPointer(),
>> /*MayBeNull=*/true,
>> CodeGenFunction::CFITCK_DerivedCast,
>> CE->getLocStart());
>>
>> - return Derived;
>> + return Derived.getPointer();
>> }
>> case CK_UncheckedDerivedToBase:
>> case CK_DerivedToBase: {
>> - const CXXRecordDecl *DerivedClassDecl =
>> - E->getType()->getPointeeCXXRecordDecl();
>> - assert(DerivedClassDecl && "DerivedToBase arg isn't a C++ object pointer!");
>> -
>> - return CGF.GetAddressOfBaseClass(
>> - Visit(E), DerivedClassDecl, CE->path_begin(), CE->path_end(),
>> - ShouldNullCheckClassCastValue(CE), CE->getExprLoc());
>> + // The EmitPointerWithAlignment path does this fine; just discard
>> + // the alignment.
>> + return CGF.EmitPointerWithAlignment(CE).getPointer();
>> }
>> +
>> case CK_Dynamic: {
>> - Value *V = Visit(const_cast<Expr*>(E));
>> + Address V = CGF.EmitPointerWithAlignment(E);
>> const CXXDynamicCastExpr *DCE = cast<CXXDynamicCastExpr>(CE);
>> return CGF.EmitDynamicCast(V, DCE);
>> }
>>
>> - case CK_ArrayToPointerDecay: {
>> - assert(E->getType()->isArrayType() &&
>> - "Array to pointer decay must have array source type!");
>> -
>> - Value *V = EmitLValue(E).getAddress(); // Bitfields can't be arrays.
>> -
>> - // Note that VLA pointers are always decayed, so we don't need to do
>> - // anything here.
>> - if (!E->getType()->isVariableArrayType()) {
>> - assert(isa<llvm::PointerType>(V->getType()) && "Expected pointer");
>> - llvm::Type *NewTy = ConvertType(E->getType());
>> - V = CGF.Builder.CreatePointerCast(
>> - V, NewTy->getPointerTo(V->getType()->getPointerAddressSpace()));
>> -
>> - assert(isa<llvm::ArrayType>(V->getType()->getPointerElementType()) &&
>> - "Expected pointer to array");
>> - V = Builder.CreateStructGEP(NewTy, V, 0, "arraydecay");
>> - }
>> -
>> - // Make sure the array decay ends up being the right type. This matters if
>> - // the array type was of an incomplete type.
>> - return CGF.Builder.CreatePointerCast(V, ConvertType(CE->getType()));
>> - }
>> + case CK_ArrayToPointerDecay:
>> + return CGF.EmitArrayToPointerDecay(E).getPointer();
>> case CK_FunctionToPointerDecay:
>> - return EmitLValue(E).getAddress();
>> + return EmitLValue(E).getPointer();
>>
>> case CK_NullToPointer:
>> if (MustVisitNullValue(E))
>> @@ -1609,9 +1584,9 @@ Value *ScalarExprEmitter::VisitCastExpr(
>>
>> Value *ScalarExprEmitter::VisitStmtExpr(const StmtExpr *E) {
>> CodeGenFunction::StmtExprEvaluation eval(CGF);
>> - llvm::Value *RetAlloca = CGF.EmitCompoundStmt(*E->getSubStmt(),
>> - !E->getType()->isVoidType());
>> - if (!RetAlloca)
>> + Address RetAlloca = CGF.EmitCompoundStmt(*E->getSubStmt(),
>> + !E->getType()->isVoidType());
>> + if (!RetAlloca.isValid())
>> return nullptr;
>> return CGF.EmitLoadOfScalar(CGF.MakeAddrLValue(RetAlloca, E->getType()),
>> E->getExprLoc());
>> @@ -1667,16 +1642,14 @@ ScalarExprEmitter::EmitScalarPrePostIncD
>> if (isInc && type->isBooleanType()) {
>> llvm::Value *True = CGF.EmitToMemory(Builder.getTrue(), type);
>> if (isPre) {
>> - Builder.Insert(new llvm::StoreInst(True,
>> - LV.getAddress(), LV.isVolatileQualified(),
>> - LV.getAlignment().getQuantity(),
>> - llvm::SequentiallyConsistent));
>> + Builder.CreateStore(True, LV.getAddress(), LV.isVolatileQualified())
>> + ->setAtomic(llvm::SequentiallyConsistent);
>> return Builder.getTrue();
>> }
>> // For atomic bool increment, we just store true and return it for
>> // preincrement, do an atomic swap with true for postincrement
>> return Builder.CreateAtomicRMW(llvm::AtomicRMWInst::Xchg,
>> - LV.getAddress(), True, llvm::SequentiallyConsistent);
>> + LV.getPointer(), True, llvm::SequentiallyConsistent);
>> }
>> // Special case for atomic increment / decrement on integers, emit
>> // atomicrmw instructions. We skip this if we want to be doing overflow
>> @@ -1693,7 +1666,7 @@ ScalarExprEmitter::EmitScalarPrePostIncD
>> llvm::Value *amt = CGF.EmitToMemory(
>> llvm::ConstantInt::get(ConvertType(type), 1, true), type);
>> llvm::Value *old = Builder.CreateAtomicRMW(aop,
>> - LV.getAddress(), amt, llvm::SequentiallyConsistent);
>> + LV.getPointer(), amt, llvm::SequentiallyConsistent);
>> return isPre ? Builder.CreateBinOp(op, old, amt) : old;
>> }
>> value = EmitLoadOfLValue(LV, E->getExprLoc());
>> @@ -2174,7 +2147,7 @@ LValue ScalarExprEmitter::EmitCompoundAs
>> EmitScalarConversion(OpInfo.RHS, E->getRHS()->getType(), LHSTy,
>> E->getExprLoc()),
>> LHSTy);
>> - Builder.CreateAtomicRMW(aop, LHSLV.getAddress(), amt,
>> + Builder.CreateAtomicRMW(aop, LHSLV.getPointer(), amt,
>> llvm::SequentiallyConsistent);
>> return LHSLV;
>> }
>> @@ -3384,13 +3357,13 @@ Value *ScalarExprEmitter::VisitVAArgExpr
>> if (Ty->isVariablyModifiedType())
>> CGF.EmitVariablyModifiedType(Ty);
>>
>> - llvm::Value *ArgValue = CGF.EmitVAListRef(VE->getSubExpr());
>> - llvm::Value *ArgPtr = CGF.EmitVAArg(ArgValue, VE->getType());
>> + Address ArgValue = CGF.EmitVAListRef(VE->getSubExpr());
>> + Address ArgPtr = CGF.EmitVAArg(ArgValue, VE->getType());
>> llvm::Type *ArgTy = ConvertType(VE->getType());
>>
>> // If EmitVAArg fails, we fall back to the LLVM instruction.
>> - if (!ArgPtr)
>> - return Builder.CreateVAArg(ArgValue, ArgTy);
>> + if (!ArgPtr.isValid())
>> + return Builder.CreateVAArg(ArgValue.getPointer(), ArgTy);
>>
>> // FIXME Volatility.
>> llvm::Value *Val = Builder.CreateLoad(ArgPtr);
>> @@ -3507,30 +3480,20 @@ EmitScalarPrePostIncDec(const UnaryOpera
>> }
>>
>> LValue CodeGenFunction::EmitObjCIsaExpr(const ObjCIsaExpr *E) {
>> - llvm::Value *V;
>> // object->isa or (*object).isa
>> // Generate code as for: *(Class*)object
>> - // build Class* type
>> - llvm::Type *ClassPtrTy = ConvertType(E->getType());
>>
>> Expr *BaseExpr = E->getBase();
>> + Address Addr = Address::invalid();
>> if (BaseExpr->isRValue()) {
>> - V = CreateMemTemp(E->getType(), "resval");
>> - llvm::Value *Src = EmitScalarExpr(BaseExpr);
>> - Builder.CreateStore(Src, V);
>> - V = ScalarExprEmitter(*this).EmitLoadOfLValue(
>> - MakeNaturalAlignAddrLValue(V, E->getType()), E->getExprLoc());
>> + Addr = Address(EmitScalarExpr(BaseExpr), getPointerAlign());
>> } else {
>> - if (E->isArrow())
>> - V = ScalarExprEmitter(*this).EmitLoadOfLValue(BaseExpr);
>> - else
>> - V = EmitLValue(BaseExpr).getAddress();
>> + Addr = EmitLValue(BaseExpr).getAddress();
>> }
>>
>> - // build Class* type
>> - ClassPtrTy = ClassPtrTy->getPointerTo();
>> - V = Builder.CreateBitCast(V, ClassPtrTy);
>> - return MakeNaturalAlignAddrLValue(V, E->getType());
>> + // Cast the address to Class*.
>> + Addr = Builder.CreateElementBitCast(Addr, ConvertType(E->getType()));
>> + return MakeAddrLValue(Addr, E->getType());
>> }
>>
>>
>>
>> Modified: cfe/trunk/lib/CodeGen/CGObjC.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGObjC.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGObjC.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGObjC.cpp Tue Sep 8 03:05:57 2015
>> @@ -37,9 +37,8 @@ static RValue AdjustObjCObjectType(CodeG
>>
>> /// Given the address of a variable of pointer type, find the correct
>> /// null to store into it.
>> -static llvm::Constant *getNullForVariable(llvm::Value *addr) {
>> - llvm::Type *type =
>> - cast<llvm::PointerType>(addr->getType())->getElementType();
>> +static llvm::Constant *getNullForVariable(Address addr) {
>> + llvm::Type *type = addr.getElementType();
>> return llvm::ConstantPointerNull::get(cast<llvm::PointerType>(type));
>> }
>>
>> @@ -47,7 +46,7 @@ static llvm::Constant *getNullForVariabl
>> llvm::Value *CodeGenFunction::EmitObjCStringLiteral(const ObjCStringLiteral *E)
>> {
>> llvm::Constant *C =
>> - CGM.getObjCRuntime().GenerateConstantString(E->getString());
>> + CGM.getObjCRuntime().GenerateConstantString(E->getString()).getPointer();
>> // FIXME: This bitcast should just be made an invariant on the Runtime.
>> return llvm::ConstantExpr::getBitCast(C, ConvertType(E->getType()));
>> }
>> @@ -84,16 +83,15 @@ CodeGenFunction::EmitObjCBoxedExpr(const
>> if (ValueType->isObjCBoxableRecordType()) {
>> // Emit CodeGen for first parameter
>> // and cast value to correct type
>> - llvm::Value *Temporary = CreateMemTemp(SubExpr->getType());
>> + Address Temporary = CreateMemTemp(SubExpr->getType());
>> EmitAnyExprToMem(SubExpr, Temporary, Qualifiers(), /*isInit*/ true);
>> - llvm::Value *BitCast = Builder.CreateBitCast(Temporary,
>> - ConvertType(ArgQT));
>> - Args.add(RValue::get(BitCast), ArgQT);
>> + Address BitCast = Builder.CreateBitCast(Temporary, ConvertType(ArgQT));
>> + Args.add(RValue::get(BitCast.getPointer()), ArgQT);
>>
>> // Create char array to store type encoding
>> std::string Str;
>> getContext().getObjCEncodingForType(ValueType, Str);
>> - llvm::GlobalVariable *GV = CGM.GetAddrOfConstantCString(Str);
>> + llvm::Constant *GV = CGM.GetAddrOfConstantCString(Str).getPointer();
>>
>> // Cast type encoding to correct type
>> const ParmVarDecl *EncodingDecl = BoxingMethod->parameters()[1];
>> @@ -131,8 +129,8 @@ llvm::Value *CodeGenFunction::EmitObjCCo
>> ArrayType::Normal, /*IndexTypeQuals=*/0);
>>
>> // Allocate the temporary array(s).
>> - llvm::AllocaInst *Objects = CreateMemTemp(ElementArrayType, "objects");
>> - llvm::AllocaInst *Keys = nullptr;
>> + Address Objects = CreateMemTemp(ElementArrayType, "objects");
>> + Address Keys = Address::invalid();
>> if (DLE)
>> Keys = CreateMemTemp(ElementArrayType, "keys");
>>
>> @@ -148,9 +146,9 @@ llvm::Value *CodeGenFunction::EmitObjCCo
>> if (ALE) {
>> // Emit the element and store it to the appropriate array slot.
>> const Expr *Rhs = ALE->getElement(i);
>> - LValue LV = LValue::MakeAddr(
>> - Builder.CreateStructGEP(Objects->getAllocatedType(), Objects, i),
>> - ElementType, Context.getTypeAlignInChars(Rhs->getType()), Context);
>> + LValue LV = MakeAddrLValue(
>> + Builder.CreateConstArrayGEP(Objects, i, getPointerSize()),
>> + ElementType, AlignmentSource::Decl);
>>
>> llvm::Value *value = EmitScalarExpr(Rhs);
>> EmitStoreThroughLValue(RValue::get(value), LV, true);
>> @@ -160,17 +158,17 @@ llvm::Value *CodeGenFunction::EmitObjCCo
>> } else {
>> // Emit the key and store it to the appropriate array slot.
>> const Expr *Key = DLE->getKeyValueElement(i).Key;
>> - LValue KeyLV = LValue::MakeAddr(
>> - Builder.CreateStructGEP(Keys->getAllocatedType(), Keys, i),
>> - ElementType, Context.getTypeAlignInChars(Key->getType()), Context);
>> + LValue KeyLV = MakeAddrLValue(
>> + Builder.CreateConstArrayGEP(Keys, i, getPointerSize()),
>> + ElementType, AlignmentSource::Decl);
>> llvm::Value *keyValue = EmitScalarExpr(Key);
>> EmitStoreThroughLValue(RValue::get(keyValue), KeyLV, /*isInit=*/true);
>>
>> // Emit the value and store it to the appropriate array slot.
>> const Expr *Value = DLE->getKeyValueElement(i).Value;
>> - LValue ValueLV = LValue::MakeAddr(
>> - Builder.CreateStructGEP(Objects->getAllocatedType(), Objects, i),
>> - ElementType, Context.getTypeAlignInChars(Value->getType()), Context);
>> + LValue ValueLV = MakeAddrLValue(
>> + Builder.CreateConstArrayGEP(Objects, i, getPointerSize()),
>> + ElementType, AlignmentSource::Decl);
>> llvm::Value *valueValue = EmitScalarExpr(Value);
>> EmitStoreThroughLValue(RValue::get(valueValue), ValueLV, /*isInit=*/true);
>> if (TrackNeededObjects) {
>> @@ -185,11 +183,11 @@ llvm::Value *CodeGenFunction::EmitObjCCo
>> ObjCMethodDecl::param_const_iterator PI = MethodWithObjects->param_begin();
>> const ParmVarDecl *argDecl = *PI++;
>> QualType ArgQT = argDecl->getType().getUnqualifiedType();
>> - Args.add(RValue::get(Objects), ArgQT);
>> + Args.add(RValue::get(Objects.getPointer()), ArgQT);
>> if (DLE) {
>> argDecl = *PI++;
>> ArgQT = argDecl->getType().getUnqualifiedType();
>> - Args.add(RValue::get(Keys), ArgQT);
>> + Args.add(RValue::get(Keys.getPointer()), ArgQT);
>> }
>> argDecl = *PI;
>> ArgQT = argDecl->getType().getUnqualifiedType();
>> @@ -404,10 +402,8 @@ RValue CodeGenFunction::EmitObjCMessageE
>> "delegate init calls should only be marked in ARC");
>>
>> // Do an unsafe store of null into self.
>> - llvm::Value *selfAddr =
>> - LocalDeclMap[cast<ObjCMethodDecl>(CurCodeDecl)->getSelfDecl()];
>> - assert(selfAddr && "no self entry for a delegate init call?");
>> -
>> + Address selfAddr =
>> + GetAddrOfLocalVar(cast<ObjCMethodDecl>(CurCodeDecl)->getSelfDecl());
>> Builder.CreateStore(getNullForVariable(selfAddr), selfAddr);
>> }
>>
>> @@ -434,14 +430,13 @@ RValue CodeGenFunction::EmitObjCMessageE
>> // For delegate init calls in ARC, implicitly store the result of
>> // the call back into self. This takes ownership of the value.
>> if (isDelegateInit) {
>> - llvm::Value *selfAddr =
>> - LocalDeclMap[cast<ObjCMethodDecl>(CurCodeDecl)->getSelfDecl()];
>> + Address selfAddr =
>> + GetAddrOfLocalVar(cast<ObjCMethodDecl>(CurCodeDecl)->getSelfDecl());
>> llvm::Value *newSelf = result.getScalarVal();
>>
>> // The delegate return type isn't necessarily a matching type; in
>> // fact, it's quite likely to be 'id'.
>> - llvm::Type *selfTy =
>> - cast<llvm::PointerType>(selfAddr->getType())->getElementType();
>> + llvm::Type *selfTy = selfAddr.getElementType();
>> newSelf = Builder.CreateBitCast(newSelf, selfTy);
>>
>> Builder.CreateStore(newSelf, selfAddr);
>> @@ -536,19 +531,19 @@ static void emitStructGetterCall(CodeGen
>> bool isAtomic, bool hasStrong) {
>> ASTContext &Context = CGF.getContext();
>>
>> - llvm::Value *src =
>> - CGF.EmitLValueForIvar(CGF.TypeOfSelfObject(), CGF.LoadObjCSelf(),
>> - ivar, 0).getAddress();
>> + Address src =
>> + CGF.EmitLValueForIvar(CGF.TypeOfSelfObject(), CGF.LoadObjCSelf(), ivar, 0)
>> + .getAddress();
>>
>> // objc_copyStruct (ReturnValue, &structIvar,
>> // sizeof (Type of Ivar), isAtomic, false);
>> CallArgList args;
>>
>> - llvm::Value *dest = CGF.Builder.CreateBitCast(CGF.ReturnValue, CGF.VoidPtrTy);
>> - args.add(RValue::get(dest), Context.VoidPtrTy);
>> + Address dest = CGF.Builder.CreateBitCast(CGF.ReturnValue, CGF.VoidPtrTy);
>> + args.add(RValue::get(dest.getPointer()), Context.VoidPtrTy);
>>
>> src = CGF.Builder.CreateBitCast(src, CGF.VoidPtrTy);
>> - args.add(RValue::get(src), Context.VoidPtrTy);
>> + args.add(RValue::get(src.getPointer()), Context.VoidPtrTy);
>>
>> CharUnits size = CGF.getContext().getTypeSizeInChars(ivar->getType());
>> args.add(RValue::get(CGF.CGM.getSize(size)), Context.getSizeType());
>> @@ -812,8 +807,8 @@ static void emitCPPObjectAtomicGetterCal
>>
>> // The 2nd argument is the address of the ivar.
>> llvm::Value *ivarAddr =
>> - CGF.EmitLValueForIvar(CGF.TypeOfSelfObject(),
>> - CGF.LoadObjCSelf(), ivar, 0).getAddress();
>> + CGF.EmitLValueForIvar(CGF.TypeOfSelfObject(),
>> + CGF.LoadObjCSelf(), ivar, 0).getPointer();
>> ivarAddr = CGF.Builder.CreateBitCast(ivarAddr, CGF.Int8PtrTy);
>> args.add(RValue::get(ivarAddr), CGF.getContext().VoidPtrTy);
>>
>> @@ -843,7 +838,7 @@ CodeGenFunction::generateObjCGetterBody(
>> }
>> else {
>> ObjCIvarDecl *ivar = propImpl->getPropertyIvarDecl();
>> - emitCPPObjectAtomicGetterCall(*this, ReturnValue,
>> + emitCPPObjectAtomicGetterCall(*this, ReturnValue.getPointer(),
>> ivar, AtomicHelperFn);
>> }
>> return;
>> @@ -873,10 +868,9 @@ CodeGenFunction::generateObjCGetterBody(
>> bitcastType = bitcastType->getPointerTo(); // addrspace 0 okay
>>
>> // Perform an atomic load. This does not impose ordering constraints.
>> - llvm::Value *ivarAddr = LV.getAddress();
>> + Address ivarAddr = LV.getAddress();
>> ivarAddr = Builder.CreateBitCast(ivarAddr, bitcastType);
>> llvm::LoadInst *load = Builder.CreateLoad(ivarAddr, "load");
>> - load->setAlignment(strategy.getIvarAlignment().getQuantity());
>> load->setAtomic(llvm::Unordered);
>>
>> // Store that value into the return address. Doing this with a
>> @@ -901,7 +895,7 @@ CodeGenFunction::generateObjCGetterBody(
>> // FIXME: Can't this be simpler? This might even be worse than the
>> // corresponding gcc code.
>> llvm::Value *cmd =
>> - Builder.CreateLoad(LocalDeclMap[getterMethod->getCmdDecl()], "cmd");
>> + Builder.CreateLoad(GetAddrOfLocalVar(getterMethod->getCmdDecl()), "cmd");
>> llvm::Value *self = Builder.CreateBitCast(LoadObjCSelf(), VoidPtrTy);
>> llvm::Value *ivarOffset =
>> EmitIvarOffset(classImpl->getClassInterface(), ivar);
>> @@ -952,8 +946,7 @@ CodeGenFunction::generateObjCGetterBody(
>> switch (getEvaluationKind(ivarType)) {
>> case TEK_Complex: {
>> ComplexPairTy pair = EmitLoadOfComplex(LV, SourceLocation());
>> - EmitStoreOfComplex(pair,
>> - MakeNaturalAlignAddrLValue(ReturnValue, ivarType),
>> + EmitStoreOfComplex(pair, MakeAddrLValue(ReturnValue, ivarType),
>> /*init*/ true);
>> return;
>> }
>> @@ -966,7 +959,7 @@ CodeGenFunction::generateObjCGetterBody(
>> case TEK_Scalar: {
>> llvm::Value *value;
>> if (propType->isReferenceType()) {
>> - value = LV.getAddress();
>> + value = LV.getAddress().getPointer();
>> } else {
>> // We want to load and autoreleaseReturnValue ARC __weak ivars.
>> if (LV.getQuals().getObjCLifetime() == Qualifiers::OCL_Weak) {
>> @@ -1006,7 +999,7 @@ static void emitStructSetterCall(CodeGen
>> // The first argument is the address of the ivar.
>> llvm::Value *ivarAddr = CGF.EmitLValueForIvar(CGF.TypeOfSelfObject(),
>> CGF.LoadObjCSelf(), ivar, 0)
>> - .getAddress();
>> + .getPointer();
>> ivarAddr = CGF.Builder.CreateBitCast(ivarAddr, CGF.Int8PtrTy);
>> args.add(RValue::get(ivarAddr), CGF.getContext().VoidPtrTy);
>>
>> @@ -1014,7 +1007,7 @@ static void emitStructSetterCall(CodeGen
>> ParmVarDecl *argVar = *OMD->param_begin();
>> DeclRefExpr argRef(argVar, false, argVar->getType().getNonReferenceType(),
>> VK_LValue, SourceLocation());
>> - llvm::Value *argAddr = CGF.EmitLValue(&argRef).getAddress();
>> + llvm::Value *argAddr = CGF.EmitLValue(&argRef).getPointer();
>> argAddr = CGF.Builder.CreateBitCast(argAddr, CGF.Int8PtrTy);
>> args.add(RValue::get(argAddr), CGF.getContext().VoidPtrTy);
>>
>> @@ -1052,7 +1045,7 @@ static void emitCPPObjectAtomicSetterCal
>> // The first argument is the address of the ivar.
>> llvm::Value *ivarAddr =
>> CGF.EmitLValueForIvar(CGF.TypeOfSelfObject(),
>> - CGF.LoadObjCSelf(), ivar, 0).getAddress();
>> + CGF.LoadObjCSelf(), ivar, 0).getPointer();
>> ivarAddr = CGF.Builder.CreateBitCast(ivarAddr, CGF.Int8PtrTy);
>> args.add(RValue::get(ivarAddr), CGF.getContext().VoidPtrTy);
>>
>> @@ -1060,7 +1053,7 @@ static void emitCPPObjectAtomicSetterCal
>> ParmVarDecl *argVar = *OMD->param_begin();
>> DeclRefExpr argRef(argVar, false, argVar->getType().getNonReferenceType(),
>> VK_LValue, SourceLocation());
>> - llvm::Value *argAddr = CGF.EmitLValue(&argRef).getAddress();
>> + llvm::Value *argAddr = CGF.EmitLValue(&argRef).getPointer();
>> argAddr = CGF.Builder.CreateBitCast(argAddr, CGF.Int8PtrTy);
>> args.add(RValue::get(argAddr), CGF.getContext().VoidPtrTy);
>>
>> @@ -1135,29 +1128,27 @@ CodeGenFunction::generateObjCSetterBody(
>> if (strategy.getIvarSize().isZero())
>> return;
>>
>> - llvm::Value *argAddr = LocalDeclMap[*setterMethod->param_begin()];
>> + Address argAddr = GetAddrOfLocalVar(*setterMethod->param_begin());
>>
>> LValue ivarLValue =
>> EmitLValueForIvar(TypeOfSelfObject(), LoadObjCSelf(), ivar, /*quals*/ 0);
>> - llvm::Value *ivarAddr = ivarLValue.getAddress();
>> + Address ivarAddr = ivarLValue.getAddress();
>>
>> // Currently, all atomic accesses have to be through integer
>> // types, so there's no point in trying to pick a prettier type.
>> llvm::Type *bitcastType =
>> llvm::Type::getIntNTy(getLLVMContext(),
>> getContext().toBits(strategy.getIvarSize()));
>> - bitcastType = bitcastType->getPointerTo(); // addrspace 0 okay
>>
>> // Cast both arguments to the chosen operation type.
>> - argAddr = Builder.CreateBitCast(argAddr, bitcastType);
>> - ivarAddr = Builder.CreateBitCast(ivarAddr, bitcastType);
>> + argAddr = Builder.CreateElementBitCast(argAddr, bitcastType);
>> + ivarAddr = Builder.CreateElementBitCast(ivarAddr, bitcastType);
>>
>> // This bitcast load is likely to cause some nasty IR.
>> llvm::Value *load = Builder.CreateLoad(argAddr);
>>
>> // Perform an atomic store. There are no memory ordering requirements.
>> llvm::StoreInst *store = Builder.CreateStore(load, ivarAddr);
>> - store->setAlignment(strategy.getIvarAlignment().getQuantity());
>> store->setAtomic(llvm::Unordered);
>> return;
>> }
>> @@ -1189,13 +1180,14 @@ CodeGenFunction::generateObjCSetterBody(
>> // Emit objc_setProperty((id) self, _cmd, offset, arg,
>> // <is-atomic>, <is-copy>).
>> llvm::Value *cmd =
>> - Builder.CreateLoad(LocalDeclMap[setterMethod->getCmdDecl()]);
>> + Builder.CreateLoad(GetAddrOfLocalVar(setterMethod->getCmdDecl()));
>> llvm::Value *self =
>> Builder.CreateBitCast(LoadObjCSelf(), VoidPtrTy);
>> llvm::Value *ivarOffset =
>> EmitIvarOffset(classImpl->getClassInterface(), ivar);
>> - llvm::Value *arg = LocalDeclMap[*setterMethod->param_begin()];
>> - arg = Builder.CreateBitCast(Builder.CreateLoad(arg, "arg"), VoidPtrTy);
>> + Address argAddr = GetAddrOfLocalVar(*setterMethod->param_begin());
>> + llvm::Value *arg = Builder.CreateLoad(argAddr, "arg");
>> + arg = Builder.CreateBitCast(arg, VoidPtrTy);
>>
>> CallArgList args;
>> args.add(RValue::get(self), getContext().getObjCIdType());
>> @@ -1328,7 +1320,7 @@ namespace {
>>
>> /// Like CodeGenFunction::destroyARCStrong, but do it with a call.
>> static void destroyARCStrongWithStore(CodeGenFunction &CGF,
>> - llvm::Value *addr,
>> + Address addr,
>> QualType type) {
>> llvm::Value *null = getNullForVariable(addr);
>> CGF.EmitARCStoreStrongCall(addr, null, /*ignored*/ true);
>> @@ -1458,7 +1450,7 @@ void CodeGenFunction::EmitObjCForCollect
>>
>> // Fast enumeration state.
>> QualType StateTy = CGM.getObjCFastEnumerationStateType();
>> - llvm::AllocaInst *StatePtr = CreateMemTemp(StateTy, "state.ptr");
>> + Address StatePtr = CreateMemTemp(StateTy, "state.ptr");
>> EmitNullInitialization(StatePtr, StateTy);
>>
>> // Number of elements in the items array.
>> @@ -1477,7 +1469,7 @@ void CodeGenFunction::EmitObjCForCollect
>> getContext().getConstantArrayType(getContext().getObjCIdType(),
>> llvm::APInt(32, NumItems),
>> ArrayType::Normal, 0);
>> - llvm::Value *ItemsPtr = CreateMemTemp(ItemsTy, "items.ptr");
>> + Address ItemsPtr = CreateMemTemp(ItemsTy, "items.ptr");
>>
>> // Emit the collection pointer. In ARC, we do a retain.
>> llvm::Value *Collection;
>> @@ -1498,14 +1490,16 @@ void CodeGenFunction::EmitObjCForCollect
>> CallArgList Args;
>>
>> // The first argument is a temporary of the enumeration-state type.
>> - Args.add(RValue::get(StatePtr), getContext().getPointerType(StateTy));
>> + Args.add(RValue::get(StatePtr.getPointer()),
>> + getContext().getPointerType(StateTy));
>>
>> // The second argument is a temporary array with space for NumItems
>> // pointers. We'll actually be loading elements from the array
>> // pointer written into the control state; this buffer is so that
>> // collections that *aren't* backed by arrays can still queue up
>> // batches of elements.
>> - Args.add(RValue::get(ItemsPtr), getContext().getPointerType(ItemsTy));
>> + Args.add(RValue::get(ItemsPtr.getPointer()),
>> + getContext().getPointerType(ItemsTy));
>>
>> // The third argument is the capacity of that temporary array.
>> llvm::Type *UnsignedLongLTy = ConvertType(getContext().UnsignedLongTy);
>> @@ -1542,13 +1536,14 @@ void CodeGenFunction::EmitObjCForCollect
>> // Save the initial mutations value. This is the value at an
>> // address that was written into the state object by
>> // countByEnumeratingWithState:objects:count:.
>> - llvm::Value *StateMutationsPtrPtr = Builder.CreateStructGEP(
>> - StatePtr->getAllocatedType(), StatePtr, 2, "mutationsptr.ptr");
>> - llvm::Value *StateMutationsPtr = Builder.CreateLoad(StateMutationsPtrPtr,
>> - "mutationsptr");
>> + Address StateMutationsPtrPtr = Builder.CreateStructGEP(
>> + StatePtr, 2, 2 * getPointerSize(), "mutationsptr.ptr");
>> + llvm::Value *StateMutationsPtr
>> + = Builder.CreateLoad(StateMutationsPtrPtr, "mutationsptr");
>>
>> llvm::Value *initialMutations =
>> - Builder.CreateLoad(StateMutationsPtr, "forcoll.initial-mutations");
>> + Builder.CreateAlignedLoad(StateMutationsPtr, getPointerAlign(),
>> + "forcoll.initial-mutations");
>>
>> // Start looping. This is the point we return to whenever we have a
>> // fresh, non-empty batch of objects.
>> @@ -1570,7 +1565,8 @@ void CodeGenFunction::EmitObjCForCollect
>> // refreshes.
>> StateMutationsPtr = Builder.CreateLoad(StateMutationsPtrPtr, "mutationsptr");
>> llvm::Value *currentMutations
>> - = Builder.CreateLoad(StateMutationsPtr, "statemutations");
>> + = Builder.CreateAlignedLoad(StateMutationsPtr, getPointerAlign(),
>> + "statemutations");
>>
>> llvm::BasicBlock *WasMutatedBB = createBasicBlock("forcoll.mutated");
>> llvm::BasicBlock *WasNotMutatedBB = createBasicBlock("forcoll.notmutated");
>> @@ -1623,15 +1619,16 @@ void CodeGenFunction::EmitObjCForCollect
>> // Fetch the buffer out of the enumeration state.
>> // TODO: this pointer should actually be invariant between
>> // refreshes, which would help us do certain loop optimizations.
>> - llvm::Value *StateItemsPtr = Builder.CreateStructGEP(
>> - StatePtr->getAllocatedType(), StatePtr, 1, "stateitems.ptr");
>> + Address StateItemsPtr = Builder.CreateStructGEP(
>> + StatePtr, 1, getPointerSize(), "stateitems.ptr");
>> llvm::Value *EnumStateItems =
>> Builder.CreateLoad(StateItemsPtr, "stateitems");
>>
>> // Fetch the value at the current index from the buffer.
>> llvm::Value *CurrentItemPtr =
>> Builder.CreateGEP(EnumStateItems, index, "currentitem.ptr");
>> - llvm::Value *CurrentItem = Builder.CreateLoad(CurrentItemPtr);
>> + llvm::Value *CurrentItem =
>> + Builder.CreateAlignedLoad(CurrentItemPtr, getPointerAlign());
>>
>> // Cast that value to the right type.
>> CurrentItem = Builder.CreateBitCast(CurrentItem, convertedElementType,
>> @@ -1838,7 +1835,7 @@ static llvm::Value *emitARCValueOperatio
>> /// Perform an operation having the following signature:
>> /// i8* (i8**)
>> static llvm::Value *emitARCLoadOperation(CodeGenFunction &CGF,
>> - llvm::Value *addr,
>> + Address addr,
>> llvm::Constant *&fn,
>> StringRef fnName) {
>> if (!fn) {
>> @@ -1848,16 +1845,15 @@ static llvm::Value *emitARCLoadOperation
>> }
>>
>> // Cast the argument to 'id*'.
>> - llvm::Type *origType = addr->getType();
>> + llvm::Type *origType = addr.getElementType();
>> addr = CGF.Builder.CreateBitCast(addr, CGF.Int8PtrPtrTy);
>>
>> // Call the function.
>> - llvm::Value *result = CGF.EmitNounwindRuntimeCall(fn, addr);
>> + llvm::Value *result = CGF.EmitNounwindRuntimeCall(fn, addr.getPointer());
>>
>> // Cast the result back to a dereference of the original type.
>> - if (origType != CGF.Int8PtrPtrTy)
>> - result = CGF.Builder.CreateBitCast(result,
>> - cast<llvm::PointerType>(origType)->getElementType());
>> + if (origType != CGF.Int8PtrTy)
>> + result = CGF.Builder.CreateBitCast(result, origType);
>>
>> return result;
>> }
>> @@ -1865,13 +1861,12 @@ static llvm::Value *emitARCLoadOperation
>> /// Perform an operation having the following signature:
>> /// i8* (i8**, i8*)
>> static llvm::Value *emitARCStoreOperation(CodeGenFunction &CGF,
>> - llvm::Value *addr,
>> + Address addr,
>> llvm::Value *value,
>> llvm::Constant *&fn,
>> StringRef fnName,
>> bool ignored) {
>> - assert(cast<llvm::PointerType>(addr->getType())->getElementType()
>> - == value->getType());
>> + assert(addr.getElementType() == value->getType());
>>
>> if (!fn) {
>> llvm::Type *argTypes[] = { CGF.Int8PtrPtrTy, CGF.Int8PtrTy };
>> @@ -1884,7 +1879,7 @@ static llvm::Value *emitARCStoreOperatio
>> llvm::Type *origType = value->getType();
>>
>> llvm::Value *args[] = {
>> - CGF.Builder.CreateBitCast(addr, CGF.Int8PtrPtrTy),
>> + CGF.Builder.CreateBitCast(addr.getPointer(), CGF.Int8PtrPtrTy),
>> CGF.Builder.CreateBitCast(value, CGF.Int8PtrTy)
>> };
>> llvm::CallInst *result = CGF.EmitNounwindRuntimeCall(fn, args);
>> @@ -1897,11 +1892,11 @@ static llvm::Value *emitARCStoreOperatio
>> /// Perform an operation having the following signature:
>> /// void (i8**, i8**)
>> static void emitARCCopyOperation(CodeGenFunction &CGF,
>> - llvm::Value *dst,
>> - llvm::Value *src,
>> + Address dst,
>> + Address src,
>> llvm::Constant *&fn,
>> StringRef fnName) {
>> - assert(dst->getType() == src->getType());
>> + assert(dst.getType() == src.getType());
>>
>> if (!fn) {
>> llvm::Type *argTypes[] = { CGF.Int8PtrPtrTy, CGF.Int8PtrPtrTy };
>> @@ -1912,8 +1907,8 @@ static void emitARCCopyOperation(CodeGen
>> }
>>
>> llvm::Value *args[] = {
>> - CGF.Builder.CreateBitCast(dst, CGF.Int8PtrPtrTy),
>> - CGF.Builder.CreateBitCast(src, CGF.Int8PtrPtrTy)
>> + CGF.Builder.CreateBitCast(dst.getPointer(), CGF.Int8PtrPtrTy),
>> + CGF.Builder.CreateBitCast(src.getPointer(), CGF.Int8PtrPtrTy)
>> };
>> CGF.EmitNounwindRuntimeCall(fn, args);
>> }
>> @@ -2050,12 +2045,10 @@ void CodeGenFunction::EmitARCRelease(llv
>> /// At -O1 and above, just load and call objc_release.
>> ///
>> /// call void \@objc_storeStrong(i8** %addr, i8* null)
>> -void CodeGenFunction::EmitARCDestroyStrong(llvm::Value *addr,
>> +void CodeGenFunction::EmitARCDestroyStrong(Address addr,
>> ARCPreciseLifetime_t precise) {
>> if (CGM.getCodeGenOpts().OptimizationLevel == 0) {
>> - llvm::PointerType *addrTy = cast<llvm::PointerType>(addr->getType());
>> - llvm::Value *null = llvm::ConstantPointerNull::get(
>> - cast<llvm::PointerType>(addrTy->getElementType()));
>> + llvm::Value *null = getNullForVariable(addr);
>> EmitARCStoreStrongCall(addr, null, /*ignored*/ true);
>> return;
>> }
>> @@ -2066,11 +2059,10 @@ void CodeGenFunction::EmitARCDestroyStro
>>
>> /// Store into a strong object. Always calls this:
>> /// call void \@objc_storeStrong(i8** %addr, i8* %value)
>> -llvm::Value *CodeGenFunction::EmitARCStoreStrongCall(llvm::Value *addr,
>> +llvm::Value *CodeGenFunction::EmitARCStoreStrongCall(Address addr,
>> llvm::Value *value,
>> bool ignored) {
>> - assert(cast<llvm::PointerType>(addr->getType())->getElementType()
>> - == value->getType());
>> + assert(addr.getElementType() == value->getType());
>>
>> llvm::Constant *&fn = CGM.getARCEntrypoints().objc_storeStrong;
>> if (!fn) {
>> @@ -2081,7 +2073,7 @@ llvm::Value *CodeGenFunction::EmitARCSto
>> }
>>
>> llvm::Value *args[] = {
>> - Builder.CreateBitCast(addr, Int8PtrPtrTy),
>> + Builder.CreateBitCast(addr.getPointer(), Int8PtrPtrTy),
>> Builder.CreateBitCast(value, Int8PtrTy)
>> };
>> EmitNounwindRuntimeCall(fn, args);
>> @@ -2184,14 +2176,14 @@ CodeGenFunction::EmitARCRetainAutoreleas
>>
>> /// i8* \@objc_loadWeak(i8** %addr)
>> /// Essentially objc_autorelease(objc_loadWeakRetained(addr)).
>> -llvm::Value *CodeGenFunction::EmitARCLoadWeak(llvm::Value *addr) {
>> +llvm::Value *CodeGenFunction::EmitARCLoadWeak(Address addr) {
>> return emitARCLoadOperation(*this, addr,
>> CGM.getARCEntrypoints().objc_loadWeak,
>> "objc_loadWeak");
>> }
>>
>> /// i8* \@objc_loadWeakRetained(i8** %addr)
>> -llvm::Value *CodeGenFunction::EmitARCLoadWeakRetained(llvm::Value *addr) {
>> +llvm::Value *CodeGenFunction::EmitARCLoadWeakRetained(Address addr) {
>> return emitARCLoadOperation(*this, addr,
>> CGM.getARCEntrypoints().objc_loadWeakRetained,
>> "objc_loadWeakRetained");
>> @@ -2199,7 +2191,7 @@ llvm::Value *CodeGenFunction::EmitARCLoa
>>
>> /// i8* \@objc_storeWeak(i8** %addr, i8* %value)
>> /// Returns %value.
>> -llvm::Value *CodeGenFunction::EmitARCStoreWeak(llvm::Value *addr,
>> +llvm::Value *CodeGenFunction::EmitARCStoreWeak(Address addr,
>> llvm::Value *value,
>> bool ignored) {
>> return emitARCStoreOperation(*this, addr, value,
>> @@ -2211,7 +2203,7 @@ llvm::Value *CodeGenFunction::EmitARCSto
>> /// Returns %value. %addr is known to not have a current weak entry.
>> /// Essentially equivalent to:
>> /// *addr = nil; objc_storeWeak(addr, value);
>> -void CodeGenFunction::EmitARCInitWeak(llvm::Value *addr, llvm::Value *value) {
>> +void CodeGenFunction::EmitARCInitWeak(Address addr, llvm::Value *value) {
>> // If we're initializing to null, just write null to memory; no need
>> // to get the runtime involved. But don't do this if optimization
>> // is enabled, because accounting for this would make the optimizer
>> @@ -2229,7 +2221,7 @@ void CodeGenFunction::EmitARCInitWeak(ll
>>
>> /// void \@objc_destroyWeak(i8** %addr)
>> /// Essentially objc_storeWeak(addr, nil).
>> -void CodeGenFunction::EmitARCDestroyWeak(llvm::Value *addr) {
>> +void CodeGenFunction::EmitARCDestroyWeak(Address addr) {
>> llvm::Constant *&fn = CGM.getARCEntrypoints().objc_destroyWeak;
>> if (!fn) {
>> llvm::FunctionType *fnType =
>> @@ -2240,13 +2232,13 @@ void CodeGenFunction::EmitARCDestroyWeak
>> // Cast the argument to 'id*'.
>> addr = Builder.CreateBitCast(addr, Int8PtrPtrTy);
>>
>> - EmitNounwindRuntimeCall(fn, addr);
>> + EmitNounwindRuntimeCall(fn, addr.getPointer());
>> }
>>
>> /// void \@objc_moveWeak(i8** %dest, i8** %src)
>> /// Disregards the current value in %dest. Leaves %src pointing to nothing.
>> /// Essentially (objc_copyWeak(dest, src), objc_destroyWeak(src)).
>> -void CodeGenFunction::EmitARCMoveWeak(llvm::Value *dst, llvm::Value *src) {
>> +void CodeGenFunction::EmitARCMoveWeak(Address dst, Address src) {
>> emitARCCopyOperation(*this, dst, src,
>> CGM.getARCEntrypoints().objc_moveWeak,
>> "objc_moveWeak");
>> @@ -2255,7 +2247,7 @@ void CodeGenFunction::EmitARCMoveWeak(ll
>> /// void \@objc_copyWeak(i8** %dest, i8** %src)
>> /// Disregards the current value in %dest. Essentially
>> /// objc_release(objc_initWeak(dest, objc_readWeakRetained(src)))
>> -void CodeGenFunction::EmitARCCopyWeak(llvm::Value *dst, llvm::Value *src) {
>> +void CodeGenFunction::EmitARCCopyWeak(Address dst, Address src) {
>> emitARCCopyOperation(*this, dst, src,
>> CGM.getARCEntrypoints().objc_copyWeak,
>> "objc_copyWeak");
>> @@ -2332,19 +2324,19 @@ void CodeGenFunction::EmitObjCMRRAutorel
>> }
>>
>> void CodeGenFunction::destroyARCStrongPrecise(CodeGenFunction &CGF,
>> - llvm::Value *addr,
>> + Address addr,
>> QualType type) {
>> CGF.EmitARCDestroyStrong(addr, ARCPreciseLifetime);
>> }
>>
>> void CodeGenFunction::destroyARCStrongImprecise(CodeGenFunction &CGF,
>> - llvm::Value *addr,
>> + Address addr,
>> QualType type) {
>> CGF.EmitARCDestroyStrong(addr, ARCImpreciseLifetime);
>> }
>>
>> void CodeGenFunction::destroyARCWeak(CodeGenFunction &CGF,
>> - llvm::Value *addr,
>> + Address addr,
>> QualType type) {
>> CGF.EmitARCDestroyWeak(addr);
>> }
>> @@ -3046,7 +3038,8 @@ CodeGenFunction::GenerateObjCAtomicGette
>> CharUnits Alignment
>> = getContext().getTypeAlignInChars(TheCXXConstructExpr->getType());
>> EmitAggExpr(TheCXXConstructExpr,
>> - AggValueSlot::forAddr(DV.getScalarVal(), Alignment, Qualifiers(),
>> + AggValueSlot::forAddr(Address(DV.getScalarVal(), Alignment),
>> + Qualifiers(),
>> AggValueSlot::IsDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsNotAliased));
>>
>> Modified: cfe/trunk/lib/CodeGen/CGObjCGNU.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGObjCGNU.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGObjCGNU.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGObjCGNU.cpp Tue Sep 8 03:05:57 2015
>> @@ -166,9 +166,9 @@ protected:
>> /// where the C code specifies const char*.
>> llvm::Constant *MakeConstantString(const std::string &Str,
>> const std::string &Name="") {
>> - auto *ConstStr = CGM.GetAddrOfConstantCString(Str, Name.c_str());
>> - return llvm::ConstantExpr::getGetElementPtr(ConstStr->getValueType(),
>> - ConstStr, Zeros);
>> + ConstantAddress Array = CGM.GetAddrOfConstantCString(Str, Name.c_str());
>> + return llvm::ConstantExpr::getGetElementPtr(Array.getElementType(),
>> + Array.getPointer(), Zeros);
>> }
>> /// Emits a linkonce_odr string, whose name is the prefix followed by the
>> /// string value. This allows the linker to combine the strings between
>> @@ -191,34 +191,41 @@ protected:
>> /// first argument.
>> llvm::GlobalVariable *MakeGlobal(llvm::StructType *Ty,
>> ArrayRef<llvm::Constant *> V,
>> + CharUnits Align,
>> StringRef Name="",
>> llvm::GlobalValue::LinkageTypes linkage
>> =llvm::GlobalValue::InternalLinkage) {
>> llvm::Constant *C = llvm::ConstantStruct::get(Ty, V);
>> - return new llvm::GlobalVariable(TheModule, Ty, false,
>> - linkage, C, Name);
>> + auto GV = new llvm::GlobalVariable(TheModule, Ty, false,
>> + linkage, C, Name);
>> + GV->setAlignment(Align.getQuantity());
>> + return GV;
>> }
>> /// Generates a global array. The vector must contain the same number of
>> /// elements that the array type declares, of the type specified as the array
>> /// element type.
>> llvm::GlobalVariable *MakeGlobal(llvm::ArrayType *Ty,
>> ArrayRef<llvm::Constant *> V,
>> + CharUnits Align,
>> StringRef Name="",
>> llvm::GlobalValue::LinkageTypes linkage
>> =llvm::GlobalValue::InternalLinkage) {
>> llvm::Constant *C = llvm::ConstantArray::get(Ty, V);
>> - return new llvm::GlobalVariable(TheModule, Ty, false,
>> - linkage, C, Name);
>> + auto GV = new llvm::GlobalVariable(TheModule, Ty, false,
>> + linkage, C, Name);
>> + GV->setAlignment(Align.getQuantity());
>> + return GV;
>> }
>> /// Generates a global array, inferring the array type from the specified
>> /// element type and the size of the initialiser.
>> llvm::GlobalVariable *MakeGlobalArray(llvm::Type *Ty,
>> ArrayRef<llvm::Constant *> V,
>> + CharUnits Align,
>> StringRef Name="",
>> llvm::GlobalValue::LinkageTypes linkage
>> =llvm::GlobalValue::InternalLinkage) {
>> llvm::ArrayType *ArrayTy = llvm::ArrayType::get(Ty, V.size());
>> - return MakeGlobal(ArrayTy, V, Name, linkage);
>> + return MakeGlobal(ArrayTy, V, Align, Name, linkage);
>> }
>> /// Returns a property name and encoding string.
>> llvm::Constant *MakePropertyEncodingString(const ObjCPropertyDecl *PD,
>> @@ -234,9 +241,7 @@ protected:
>> NameAndAttributes += TypeStr;
>> NameAndAttributes += '\0';
>> NameAndAttributes += PD->getNameAsString();
>> - auto *ConstStr = CGM.GetAddrOfConstantCString(NameAndAttributes);
>> - return llvm::ConstantExpr::getGetElementPtr(ConstStr->getValueType(),
>> - ConstStr, Zeros);
>> + return MakeConstantString(NameAndAttributes);
>> }
>> return MakeConstantString(PD->getNameAsString());
>> }
>> @@ -275,6 +280,10 @@ protected:
>> if (V->getType() == Ty) return V;
>> return B.CreateBitCast(V, Ty);
>> }
>> + Address EnforceType(CGBuilderTy &B, Address V, llvm::Type *Ty) {
>> + if (V.getType() == Ty) return V;
>> + return B.CreateBitCast(V, Ty);
>> + }
>> // Some zeros used for GEPs in lots of places.
>> llvm::Constant *Zeros[2];
>> /// Null pointer value. Mainly used as a terminator in various arrays.
>> @@ -435,7 +444,7 @@ private:
>> /// Returns a selector with the specified type encoding. An empty string is
>> /// used to return an untyped selector (with the types field set to NULL).
>> llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel,
>> - const std::string &TypeEncoding, bool lval);
>> + const std::string &TypeEncoding);
>> /// Returns the variable used to store the offset of an instance variable.
>> llvm::GlobalVariable *ObjCIvarOffsetVariable(const ObjCInterfaceDecl *ID,
>> const ObjCIvarDecl *Ivar);
>> @@ -458,7 +467,7 @@ protected:
>> /// mechanism differs between the GCC and GNU runtimes, so this method must
>> /// be overridden in subclasses.
>> virtual llvm::Value *LookupIMPSuper(CodeGenFunction &CGF,
>> - llvm::Value *ObjCSuper,
>> + Address ObjCSuper,
>> llvm::Value *cmd,
>> MessageSendInfo &MSI) = 0;
>> /// Libobjc2 uses a bitfield representation where small(ish) bitfields are
>> @@ -477,7 +486,7 @@ public:
>> CGObjCGNU(CodeGenModule &cgm, unsigned runtimeABIVersion,
>> unsigned protocolClassVersion);
>>
>> - llvm::Constant *GenerateConstantString(const StringLiteral *) override;
>> + ConstantAddress GenerateConstantString(const StringLiteral *) override;
>>
>> RValue
>> GenerateMessageSend(CodeGenFunction &CGF, ReturnValueSlot Return,
>> @@ -494,8 +503,8 @@ public:
>> const ObjCMethodDecl *Method) override;
>> llvm::Value *GetClass(CodeGenFunction &CGF,
>> const ObjCInterfaceDecl *OID) override;
>> - llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lval = false) override;
>> + llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel) override;
>> + Address GetAddrOfSelector(CodeGenFunction &CGF, Selector Sel) override;
>> llvm::Value *GetSelector(CodeGenFunction &CGF,
>> const ObjCMethodDecl *Method) override;
>> llvm::Constant *GetEHType(QualType T) override;
>> @@ -527,18 +536,18 @@ public:
>> const ObjCAtThrowStmt &S,
>> bool ClearInsertionPoint=true) override;
>> llvm::Value * EmitObjCWeakRead(CodeGenFunction &CGF,
>> - llvm::Value *AddrWeakObj) override;
>> + Address AddrWeakObj) override;
>> void EmitObjCWeakAssign(CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) override;
>> + llvm::Value *src, Address dst) override;
>> void EmitObjCGlobalAssign(CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest,
>> + llvm::Value *src, Address dest,
>> bool threadlocal=false) override;
>> void EmitObjCIvarAssign(CodeGenFunction &CGF, llvm::Value *src,
>> - llvm::Value *dest, llvm::Value *ivarOffset) override;
>> + Address dest, llvm::Value *ivarOffset) override;
>> void EmitObjCStrongCastAssign(CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest) override;
>> - void EmitGCMemmoveCollectable(CodeGenFunction &CGF, llvm::Value *DestPtr,
>> - llvm::Value *SrcPtr,
>> + llvm::Value *src, Address dest) override;
>> + void EmitGCMemmoveCollectable(CodeGenFunction &CGF, Address DestPtr,
>> + Address SrcPtr,
>> llvm::Value *Size) override;
>> LValue EmitObjCValueForIvar(CodeGenFunction &CGF, QualType ObjectTy,
>> llvm::Value *BaseValue, const ObjCIvarDecl *Ivar,
>> @@ -593,11 +602,11 @@ protected:
>> imp->setMetadata(msgSendMDKind, node);
>> return imp.getInstruction();
>> }
>> - llvm::Value *LookupIMPSuper(CodeGenFunction &CGF, llvm::Value *ObjCSuper,
>> + llvm::Value *LookupIMPSuper(CodeGenFunction &CGF, Address ObjCSuper,
>> llvm::Value *cmd, MessageSendInfo &MSI) override {
>> CGBuilderTy &Builder = CGF.Builder;
>> llvm::Value *lookupArgs[] = {EnforceType(Builder, ObjCSuper,
>> - PtrToObjCSuperTy), cmd};
>> + PtrToObjCSuperTy).getPointer(), cmd};
>> return CGF.EmitNounwindRuntimeCall(MsgLookupSuperFn, lookupArgs);
>> }
>> public:
>> @@ -647,7 +656,8 @@ class CGObjCGNUstep : public CGObjCGNU {
>> llvm::Function *LookupFn = SlotLookupFn;
>>
>> // Store the receiver on the stack so that we can reload it later
>> - llvm::Value *ReceiverPtr = CGF.CreateTempAlloca(Receiver->getType());
>> + Address ReceiverPtr =
>> + CGF.CreateTempAlloca(Receiver->getType(), CGF.getPointerAlign());
>> Builder.CreateStore(Receiver, ReceiverPtr);
>>
>> llvm::Value *self;
>> @@ -662,7 +672,7 @@ class CGObjCGNUstep : public CGObjCGNU {
>> LookupFn->setDoesNotCapture(1);
>>
>> llvm::Value *args[] = {
>> - EnforceType(Builder, ReceiverPtr, PtrToIdTy),
>> + EnforceType(Builder, ReceiverPtr.getPointer(), PtrToIdTy),
>> EnforceType(Builder, cmd, SelectorTy),
>> EnforceType(Builder, self, IdTy) };
>> llvm::CallSite slot = CGF.EmitRuntimeCallOrInvoke(LookupFn, args);
>> @@ -670,25 +680,27 @@ class CGObjCGNUstep : public CGObjCGNU {
>> slot->setMetadata(msgSendMDKind, node);
>>
>> // Load the imp from the slot
>> - llvm::Value *imp = Builder.CreateLoad(
>> - Builder.CreateStructGEP(nullptr, slot.getInstruction(), 4));
>> + llvm::Value *imp = Builder.CreateAlignedLoad(
>> + Builder.CreateStructGEP(nullptr, slot.getInstruction(), 4),
>> + CGF.getPointerAlign());
>>
>> // The lookup function may have changed the receiver, so make sure we use
>> // the new one.
>> Receiver = Builder.CreateLoad(ReceiverPtr, true);
>> return imp;
>> }
>> - llvm::Value *LookupIMPSuper(CodeGenFunction &CGF, llvm::Value *ObjCSuper,
>> + llvm::Value *LookupIMPSuper(CodeGenFunction &CGF, Address ObjCSuper,
>> llvm::Value *cmd,
>> MessageSendInfo &MSI) override {
>> CGBuilderTy &Builder = CGF.Builder;
>> - llvm::Value *lookupArgs[] = {ObjCSuper, cmd};
>> + llvm::Value *lookupArgs[] = {ObjCSuper.getPointer(), cmd};
>>
>> llvm::CallInst *slot =
>> CGF.EmitNounwindRuntimeCall(SlotLookupSuperFn, lookupArgs);
>> slot->setOnlyReadsMemory();
>>
>> - return Builder.CreateLoad(Builder.CreateStructGEP(nullptr, slot, 4));
>> + return Builder.CreateAlignedLoad(Builder.CreateStructGEP(nullptr, slot, 4),
>> + CGF.getPointerAlign());
>> }
>> public:
>> CGObjCGNUstep(CodeGenModule &Mod) : CGObjCGNU(Mod, 9, 3) {
>> @@ -807,10 +819,10 @@ protected:
>> return imp.getInstruction();
>> }
>>
>> - llvm::Value *LookupIMPSuper(CodeGenFunction &CGF, llvm::Value *ObjCSuper,
>> + llvm::Value *LookupIMPSuper(CodeGenFunction &CGF, Address ObjCSuper,
>> llvm::Value *cmd, MessageSendInfo &MSI) override {
>> CGBuilderTy &Builder = CGF.Builder;
>> - llvm::Value *lookupArgs[] = {EnforceType(Builder, ObjCSuper,
>> + llvm::Value *lookupArgs[] = {EnforceType(Builder, ObjCSuper.getPointer(),
>> PtrToObjCSuperTy), cmd};
>>
>> if (CGM.ReturnTypeUsesSRet(MSI.CallInfo))
>> @@ -1011,7 +1023,7 @@ CGObjCGNU::CGObjCGNU(CodeGenModule &cgm,
>> llvm::Value *CGObjCGNU::GetClassNamed(CodeGenFunction &CGF,
>> const std::string &Name,
>> bool isWeak) {
>> - llvm::GlobalVariable *ClassNameGV = CGM.GetAddrOfConstantCString(Name);
>> + llvm::Constant *ClassName = MakeConstantString(Name);
>> // With the incompatible ABI, this will need to be replaced with a direct
>> // reference to the class symbol. For the compatible nonfragile ABI we are
>> // still performing this lookup at run time but emitting the symbol for the
>> @@ -1021,8 +1033,6 @@ llvm::Value *CGObjCGNU::GetClassNamed(Co
>> // with memoized versions or with static references if it's safe to do so.
>> if (!isWeak)
>> EmitClassRef(Name);
>> - llvm::Value *ClassName =
>> - CGF.Builder.CreateStructGEP(ClassNameGV->getValueType(), ClassNameGV, 0);
>>
>> llvm::Constant *ClassLookupFn =
>> CGM.CreateRuntimeFunction(llvm::FunctionType::get(IdTy, PtrToInt8Ty, true),
>> @@ -1041,7 +1051,7 @@ llvm::Value *CGObjCGNU::EmitNSAutoreleas
>> }
>>
>> llvm::Value *CGObjCGNU::GetSelector(CodeGenFunction &CGF, Selector Sel,
>> - const std::string &TypeEncoding, bool lval) {
>> + const std::string &TypeEncoding) {
>>
>> SmallVectorImpl<TypedSelector> &Types = SelectorTable[Sel];
>> llvm::GlobalAlias *SelValue = nullptr;
>> @@ -1060,24 +1070,29 @@ llvm::Value *CGObjCGNU::GetSelector(Code
>> Types.emplace_back(TypeEncoding, SelValue);
>> }
>>
>> - if (lval) {
>> - llvm::Value *tmp = CGF.CreateTempAlloca(SelValue->getType());
>> - CGF.Builder.CreateStore(SelValue, tmp);
>> - return tmp;
>> - }
>> return SelValue;
>> }
>>
>> -llvm::Value *CGObjCGNU::GetSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lval) {
>> - return GetSelector(CGF, Sel, std::string(), lval);
>> +Address CGObjCGNU::GetAddrOfSelector(CodeGenFunction &CGF, Selector Sel) {
>> + llvm::Value *SelValue = GetSelector(CGF, Sel);
>> +
>> + // Store it to a temporary. Does this satisfy the semantics of
>> + // GetAddrOfSelector? Hopefully.
>> + Address tmp = CGF.CreateTempAlloca(SelValue->getType(),
>> + CGF.getPointerAlign());
>> + CGF.Builder.CreateStore(SelValue, tmp);
>> + return tmp;
>> +}
>> +
>> +llvm::Value *CGObjCGNU::GetSelector(CodeGenFunction &CGF, Selector Sel) {
>> + return GetSelector(CGF, Sel, std::string());
>> }
>>
>> llvm::Value *CGObjCGNU::GetSelector(CodeGenFunction &CGF,
>> const ObjCMethodDecl *Method) {
>> std::string SelTypes;
>> CGM.getContext().getObjCEncodingForMethodDecl(Method, SelTypes);
>> - return GetSelector(CGF, Method->getSelector(), SelTypes, false);
>> + return GetSelector(CGF, Method->getSelector(), SelTypes);
>> }
>>
>> llvm::Constant *CGObjCGNU::GetEHType(QualType T) {
>> @@ -1160,21 +1175,23 @@ llvm::Constant *CGObjCGNUstep::GetEHType
>> fields.push_back(BVtable);
>> fields.push_back(typeName);
>> llvm::Constant *TI =
>> - MakeGlobal(llvm::StructType::get(PtrToInt8Ty, PtrToInt8Ty,
>> - nullptr), fields, "__objc_eh_typeinfo_" + className,
>> + MakeGlobal(llvm::StructType::get(PtrToInt8Ty, PtrToInt8Ty, nullptr),
>> + fields, CGM.getPointerAlign(),
>> + "__objc_eh_typeinfo_" + className,
>> llvm::GlobalValue::LinkOnceODRLinkage);
>> return llvm::ConstantExpr::getBitCast(TI, PtrToInt8Ty);
>> }
>>
>> /// Generate an NSConstantString object.
>> -llvm::Constant *CGObjCGNU::GenerateConstantString(const StringLiteral *SL) {
>> +ConstantAddress CGObjCGNU::GenerateConstantString(const StringLiteral *SL) {
>>
>> std::string Str = SL->getString().str();
>> + CharUnits Align = CGM.getPointerAlign();
>>
>> // Look for an existing one
>> llvm::StringMap<llvm::Constant*>::iterator old = ObjCStrings.find(Str);
>> if (old != ObjCStrings.end())
>> - return old->getValue();
>> + return ConstantAddress(old->getValue(), Align);
>>
>> StringRef StringClass = CGM.getLangOpts().ObjCConstantStringClass;
>>
>> @@ -1197,11 +1214,11 @@ llvm::Constant *CGObjCGNU::GenerateConst
>> Ivars.push_back(llvm::ConstantInt::get(IntTy, Str.size()));
>> llvm::Constant *ObjCStr = MakeGlobal(
>> llvm::StructType::get(PtrToIdTy, PtrToInt8Ty, IntTy, nullptr),
>> - Ivars, ".objc_str");
>> + Ivars, Align, ".objc_str");
>> ObjCStr = llvm::ConstantExpr::getBitCast(ObjCStr, PtrToInt8Ty);
>> ObjCStrings[Str] = ObjCStr;
>> ConstantStrings.push_back(ObjCStr);
>> - return ObjCStr;
>> + return ConstantAddress(ObjCStr, Align);
>> }
>>
>> ///Generates a message send where the super is the receiver. This is a message
>> @@ -1281,16 +1298,20 @@ CGObjCGNU::GenerateMessageSendSuper(Code
>> // Get the superclass pointer
>> ReceiverClass = Builder.CreateStructGEP(CastTy, ReceiverClass, 1);
>> // Load the superclass pointer
>> - ReceiverClass = Builder.CreateLoad(ReceiverClass);
>> + ReceiverClass =
>> + Builder.CreateAlignedLoad(ReceiverClass, CGF.getPointerAlign());
>> // Construct the structure used to look up the IMP
>> llvm::StructType *ObjCSuperTy = llvm::StructType::get(
>> Receiver->getType(), IdTy, nullptr);
>> - llvm::Value *ObjCSuper = Builder.CreateAlloca(ObjCSuperTy);
>> +
>> + // FIXME: Is this really supposed to be a dynamic alloca?
>> + Address ObjCSuper = Address(Builder.CreateAlloca(ObjCSuperTy),
>> + CGF.getPointerAlign());
>>
>> Builder.CreateStore(Receiver,
>> - Builder.CreateStructGEP(ObjCSuperTy, ObjCSuper, 0));
>> + Builder.CreateStructGEP(ObjCSuper, 0, CharUnits::Zero()));
>> Builder.CreateStore(ReceiverClass,
>> - Builder.CreateStructGEP(ObjCSuperTy, ObjCSuper, 1));
>> + Builder.CreateStructGEP(ObjCSuper, 1, CGF.getPointerSize()));
>>
>> ObjCSuper = EnforceType(Builder, ObjCSuper, PtrToObjCSuperTy);
>>
>> @@ -1435,16 +1456,14 @@ CGObjCGNU::GenerateMessageSend(CodeGenFu
>> phi->addIncoming(llvm::Constant::getNullValue(v->getType()), startBB);
>> msgRet = RValue::get(phi);
>> } else if (msgRet.isAggregate()) {
>> - llvm::Value *v = msgRet.getAggregateAddr();
>> - llvm::PHINode *phi = Builder.CreatePHI(v->getType(), 2);
>> - llvm::PointerType *RetTy = cast<llvm::PointerType>(v->getType());
>> - llvm::AllocaInst *NullVal =
>> - CGF.CreateTempAlloca(RetTy->getElementType(), "null");
>> - CGF.InitTempAlloca(NullVal,
>> - llvm::Constant::getNullValue(RetTy->getElementType()));
>> - phi->addIncoming(v, messageBB);
>> - phi->addIncoming(NullVal, startBB);
>> - msgRet = RValue::getAggregate(phi);
>> + Address v = msgRet.getAggregateAddress();
>> + llvm::PHINode *phi = Builder.CreatePHI(v.getType(), 2);
>> + llvm::Type *RetTy = v.getElementType();
>> + Address NullVal = CGF.CreateTempAlloca(RetTy, v.getAlignment(), "null");
>> + CGF.InitTempAlloca(NullVal, llvm::Constant::getNullValue(RetTy));
>> + phi->addIncoming(v.getPointer(), messageBB);
>> + phi->addIncoming(NullVal.getPointer(), startBB);
>> + msgRet = RValue::getAggregate(Address(phi, v.getAlignment()));
>> } else /* isComplex() */ {
>> std::pair<llvm::Value*,llvm::Value*> v = msgRet.getComplexVal();
>> llvm::PHINode *phi = Builder.CreatePHI(v.first->getType(), 2);
>> @@ -1517,7 +1536,8 @@ GenerateMethodList(StringRef ClassName,
>> Methods.push_back(MethodArray);
>>
>> // Create an instance of the structure
>> - return MakeGlobal(ObjCMethodListTy, Methods, ".objc_method_list");
>> + return MakeGlobal(ObjCMethodListTy, Methods, CGM.getPointerAlign(),
>> + ".objc_method_list");
>> }
>>
>> /// Generates an IvarList. Used in construction of a objc_class.
>> @@ -1557,7 +1577,8 @@ GenerateIvarList(ArrayRef<llvm::Constant
>> nullptr);
>>
>> // Create an instance of the structure
>> - return MakeGlobal(ObjCIvarListTy, Elements, ".objc_ivar_list");
>> + return MakeGlobal(ObjCIvarListTy, Elements, CGM.getPointerAlign(),
>> + ".objc_ivar_list");
>> }
>>
>> /// Generate a class structure
>> @@ -1640,8 +1661,9 @@ llvm::Constant *CGObjCGNU::GenerateClass
>> std::string ClassSym((isMeta ? "_OBJC_METACLASS_": "_OBJC_CLASS_") +
>> std::string(Name));
>> llvm::GlobalVariable *ClassRef = TheModule.getNamedGlobal(ClassSym);
>> - llvm::Constant *Class = MakeGlobal(ClassTy, Elements, ClassSym,
>> - llvm::GlobalValue::ExternalLinkage);
>> + llvm::Constant *Class =
>> + MakeGlobal(ClassTy, Elements, CGM.getPointerAlign(), ClassSym,
>> + llvm::GlobalValue::ExternalLinkage);
>> if (ClassRef) {
>> ClassRef->replaceAllUsesWith(llvm::ConstantExpr::getBitCast(Class,
>> ClassRef->getType()));
>> @@ -1676,7 +1698,8 @@ GenerateProtocolMethodList(ArrayRef<llvm
>> Methods.clear();
>> Methods.push_back(llvm::ConstantInt::get(IntTy, MethodNames.size()));
>> Methods.push_back(Array);
>> - return MakeGlobal(ObjCMethodDescListTy, Methods, ".objc_method_list");
>> + return MakeGlobal(ObjCMethodDescListTy, Methods, CGM.getPointerAlign(),
>> + ".objc_method_list");
>> }
>>
>> // Create the protocol list structure used in classes, categories and so on
>> @@ -1709,7 +1732,8 @@ llvm::Constant *CGObjCGNU::GenerateProto
>> Elements.push_back(NULLPtr);
>> Elements.push_back(llvm::ConstantInt::get(LongTy, Protocols.size()));
>> Elements.push_back(ProtocolArray);
>> - return MakeGlobal(ProtocolListTy, Elements, ".objc_protocol_list");
>> + return MakeGlobal(ProtocolListTy, Elements, CGM.getPointerAlign(),
>> + ".objc_protocol_list");
>> }
>>
>> llvm::Value *CGObjCGNU::GenerateProtocolRef(CodeGenFunction &CGF,
>> @@ -1749,7 +1773,8 @@ llvm::Constant *CGObjCGNU::GenerateEmpty
>> Elements.push_back(MethodList);
>> Elements.push_back(MethodList);
>> Elements.push_back(MethodList);
>> - return MakeGlobal(ProtocolTy, Elements, ".objc_protocol");
>> + return MakeGlobal(ProtocolTy, Elements, CGM.getPointerAlign(),
>> + ".objc_protocol");
>> }
>>
>> void CGObjCGNU::GenerateProtocol(const ObjCProtocolDecl *PD) {
>> @@ -1910,7 +1935,7 @@ void CGObjCGNU::GenerateProtocol(const O
>> Elements.push_back(OptionalPropertyList);
>> ExistingProtocols[ProtocolName] =
>> llvm::ConstantExpr::getBitCast(MakeGlobal(ProtocolTy, Elements,
>> - ".objc_protocol"), IdTy);
>> + CGM.getPointerAlign(), ".objc_protocol"), IdTy);
>> }
>> void CGObjCGNU::GenerateProtocolHolderCategory() {
>> // Collect information about instance methods
>> @@ -1952,10 +1977,12 @@ void CGObjCGNU::GenerateProtocolHolderCa
>> ExistingProtocols.size()));
>> ProtocolElements.push_back(ProtocolArray);
>> Elements.push_back(llvm::ConstantExpr::getBitCast(MakeGlobal(ProtocolListTy,
>> - ProtocolElements, ".objc_protocol_list"), PtrTy));
>> + ProtocolElements, CGM.getPointerAlign(),
>> + ".objc_protocol_list"), PtrTy));
>> Categories.push_back(llvm::ConstantExpr::getBitCast(
>> MakeGlobal(llvm::StructType::get(PtrToInt8Ty, PtrToInt8Ty,
>> - PtrTy, PtrTy, PtrTy, nullptr), Elements), PtrTy));
>> + PtrTy, PtrTy, PtrTy, nullptr), Elements, CGM.getPointerAlign()),
>> + PtrTy));
>> }
>>
>> /// Libobjc2 uses a bitfield representation where small(ish) bitfields are
>> @@ -1995,7 +2022,7 @@ llvm::Constant *CGObjCGNU::MakeBitField(
>> llvm::ConstantInt::get(Int32Ty, values.size()),
>> array };
>> llvm::Constant *GS = MakeGlobal(llvm::StructType::get(Int32Ty, arrayTy,
>> - nullptr), fields);
>> + nullptr), fields, CharUnits::fromQuantity(4));
>> llvm::Constant *ptr = llvm::ConstantExpr::getPtrToInt(GS, IntPtrTy);
>> return ptr;
>> }
>> @@ -2047,7 +2074,8 @@ void CGObjCGNU::GenerateCategory(const O
>> GenerateProtocolList(Protocols), PtrTy));
>> Categories.push_back(llvm::ConstantExpr::getBitCast(
>> MakeGlobal(llvm::StructType::get(PtrToInt8Ty, PtrToInt8Ty,
>> - PtrTy, PtrTy, PtrTy, nullptr), Elements), PtrTy));
>> + PtrTy, PtrTy, PtrTy, nullptr), Elements, CGM.getPointerAlign()),
>> + PtrTy));
>> }
>>
>> llvm::Constant *CGObjCGNU::GeneratePropertyList(const ObjCImplementationDecl *OID,
>> @@ -2225,7 +2253,8 @@ void CGObjCGNU::GenerateClass(const ObjC
>> llvm::Constant *StrongIvarBitmap = MakeBitField(StrongIvars);
>> llvm::Constant *WeakIvarBitmap = MakeBitField(WeakIvars);
>> llvm::GlobalVariable *IvarOffsetArray =
>> - MakeGlobalArray(PtrToIntTy, IvarOffsetValues, ".ivar.offsets");
>> + MakeGlobalArray(PtrToIntTy, IvarOffsetValues, CGM.getPointerAlign(),
>> + ".ivar.offsets");
>>
>>
>> // Collect information about instance methods
>> @@ -2385,13 +2414,15 @@ llvm::Function *CGObjCGNU::ModuleInitFun
>> llvm::StructType::get(PtrToInt8Ty, StaticsArrayTy, nullptr);
>> llvm::Type *StaticsListPtrTy =
>> llvm::PointerType::getUnqual(StaticsListTy);
>> - Statics = MakeGlobal(StaticsListTy, Elements, ".objc_statics");
>> + Statics = MakeGlobal(StaticsListTy, Elements, CGM.getPointerAlign(),
>> + ".objc_statics");
>> llvm::ArrayType *StaticsListArrayTy =
>> llvm::ArrayType::get(StaticsListPtrTy, 2);
>> Elements.clear();
>> Elements.push_back(Statics);
>> Elements.push_back(llvm::Constant::getNullValue(StaticsListPtrTy));
>> - Statics = MakeGlobal(StaticsListArrayTy, Elements, ".objc_statics_ptr");
>> + Statics = MakeGlobal(StaticsListArrayTy, Elements,
>> + CGM.getPointerAlign(), ".objc_statics_ptr");
>> Statics = llvm::ConstantExpr::getBitCast(Statics, PtrTy);
>> }
>> // Array of classes, categories, and constant objects
>> @@ -2442,7 +2473,8 @@ llvm::Function *CGObjCGNU::ModuleInitFun
>> // Number of static selectors
>> Elements.push_back(llvm::ConstantInt::get(LongTy, SelectorCount));
>> llvm::GlobalVariable *SelectorList =
>> - MakeGlobalArray(SelStructTy, Selectors, ".objc_selector_list");
>> + MakeGlobalArray(SelStructTy, Selectors, CGM.getPointerAlign(),
>> + ".objc_selector_list");
>> Elements.push_back(llvm::ConstantExpr::getBitCast(SelectorList,
>> SelStructPtrTy));
>>
>> @@ -2475,7 +2507,8 @@ llvm::Function *CGObjCGNU::ModuleInitFun
>> llvm::Constant *ClassList = llvm::ConstantArray::get(ClassListTy, Classes);
>> Elements.push_back(ClassList);
>> // Construct the symbol table
>> - llvm::Constant *SymTab= MakeGlobal(SymTabTy, Elements);
>> + llvm::Constant *SymTab =
>> + MakeGlobal(SymTabTy, Elements, CGM.getPointerAlign());
>>
>> // The symbol table is contained in a module which has some version-checking
>> // constants
>> @@ -2516,7 +2549,7 @@ llvm::Function *CGObjCGNU::ModuleInitFun
>> break;
>> }
>>
>> - llvm::Value *Module = MakeGlobal(ModuleTy, Elements);
>> + llvm::Value *Module = MakeGlobal(ModuleTy, Elements, CGM.getPointerAlign());
>>
>> // Create the load function calling the runtime entry point with the module
>> // structure
>> @@ -2526,7 +2559,7 @@ llvm::Function *CGObjCGNU::ModuleInitFun
>> &TheModule);
>> llvm::BasicBlock *EntryBB =
>> llvm::BasicBlock::Create(VMContext, "entry", LoadFunction);
>> - CGBuilderTy Builder(VMContext);
>> + CGBuilderTy Builder(CGM, VMContext);
>> Builder.SetInsertPoint(EntryBB);
>>
>> llvm::FunctionType *FT =
>> @@ -2678,57 +2711,63 @@ void CGObjCGNU::EmitThrowStmt(CodeGenFun
>> }
>>
>> llvm::Value * CGObjCGNU::EmitObjCWeakRead(CodeGenFunction &CGF,
>> - llvm::Value *AddrWeakObj) {
>> + Address AddrWeakObj) {
>> CGBuilderTy &B = CGF.Builder;
>> AddrWeakObj = EnforceType(B, AddrWeakObj, PtrToIdTy);
>> - return B.CreateCall(WeakReadFn.getType(), WeakReadFn, AddrWeakObj);
>> + return B.CreateCall(WeakReadFn.getType(), WeakReadFn,
>> + AddrWeakObj.getPointer());
>> }
>>
>> void CGObjCGNU::EmitObjCWeakAssign(CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) {
>> + llvm::Value *src, Address dst) {
>> CGBuilderTy &B = CGF.Builder;
>> src = EnforceType(B, src, IdTy);
>> dst = EnforceType(B, dst, PtrToIdTy);
>> - B.CreateCall(WeakAssignFn.getType(), WeakAssignFn, {src, dst});
>> + B.CreateCall(WeakAssignFn.getType(), WeakAssignFn,
>> + {src, dst.getPointer()});
>> }
>>
>> void CGObjCGNU::EmitObjCGlobalAssign(CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst,
>> + llvm::Value *src, Address dst,
>> bool threadlocal) {
>> CGBuilderTy &B = CGF.Builder;
>> src = EnforceType(B, src, IdTy);
>> dst = EnforceType(B, dst, PtrToIdTy);
>> // FIXME. Add threadloca assign API
>> assert(!threadlocal && "EmitObjCGlobalAssign - Threal Local API NYI");
>> - B.CreateCall(GlobalAssignFn.getType(), GlobalAssignFn, {src, dst});
>> + B.CreateCall(GlobalAssignFn.getType(), GlobalAssignFn,
>> + {src, dst.getPointer()});
>> }
>>
>> void CGObjCGNU::EmitObjCIvarAssign(CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst,
>> + llvm::Value *src, Address dst,
>> llvm::Value *ivarOffset) {
>> CGBuilderTy &B = CGF.Builder;
>> src = EnforceType(B, src, IdTy);
>> dst = EnforceType(B, dst, IdTy);
>> - B.CreateCall(IvarAssignFn.getType(), IvarAssignFn, {src, dst, ivarOffset});
>> + B.CreateCall(IvarAssignFn.getType(), IvarAssignFn,
>> + {src, dst.getPointer(), ivarOffset});
>> }
>>
>> void CGObjCGNU::EmitObjCStrongCastAssign(CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) {
>> + llvm::Value *src, Address dst) {
>> CGBuilderTy &B = CGF.Builder;
>> src = EnforceType(B, src, IdTy);
>> dst = EnforceType(B, dst, PtrToIdTy);
>> - B.CreateCall(StrongCastAssignFn.getType(), StrongCastAssignFn, {src, dst});
>> + B.CreateCall(StrongCastAssignFn.getType(), StrongCastAssignFn,
>> + {src, dst.getPointer()});
>> }
>>
>> void CGObjCGNU::EmitGCMemmoveCollectable(CodeGenFunction &CGF,
>> - llvm::Value *DestPtr,
>> - llvm::Value *SrcPtr,
>> + Address DestPtr,
>> + Address SrcPtr,
>> llvm::Value *Size) {
>> CGBuilderTy &B = CGF.Builder;
>> DestPtr = EnforceType(B, DestPtr, PtrTy);
>> SrcPtr = EnforceType(B, SrcPtr, PtrTy);
>>
>> - B.CreateCall(MemMoveFn.getType(), MemMoveFn, {DestPtr, SrcPtr, Size});
>> + B.CreateCall(MemMoveFn.getType(), MemMoveFn,
>> + {DestPtr.getPointer(), SrcPtr.getPointer(), Size});
>> }
>>
>> llvm::GlobalVariable *CGObjCGNU::ObjCIvarOffsetVariable(
>> @@ -2811,17 +2850,22 @@ llvm::Value *CGObjCGNU::EmitIvarOffset(C
>> Interface = FindIvarInterface(CGM.getContext(), Interface, Ivar);
>> if (RuntimeVersion < 10)
>> return CGF.Builder.CreateZExtOrBitCast(
>> - CGF.Builder.CreateLoad(CGF.Builder.CreateLoad(
>> - ObjCIvarOffsetVariable(Interface, Ivar), false, "ivar")),
>> + CGF.Builder.CreateDefaultAlignedLoad(CGF.Builder.CreateAlignedLoad(
>> + ObjCIvarOffsetVariable(Interface, Ivar),
>> + CGF.getPointerAlign(), "ivar")),
>> PtrDiffTy);
>> std::string name = "__objc_ivar_offset_value_" +
>> Interface->getNameAsString() +"." + Ivar->getNameAsString();
>> + CharUnits Align = CGM.getIntAlign();
>> llvm::Value *Offset = TheModule.getGlobalVariable(name);
>> - if (!Offset)
>> - Offset = new llvm::GlobalVariable(TheModule, IntTy,
>> + if (!Offset) {
>> + auto GV = new llvm::GlobalVariable(TheModule, IntTy,
>> false, llvm::GlobalValue::LinkOnceAnyLinkage,
>> llvm::Constant::getNullValue(IntTy), name);
>> - Offset = CGF.Builder.CreateLoad(Offset);
>> + GV->setAlignment(Align.getQuantity());
>> + Offset = GV;
>> + }
>> + Offset = CGF.Builder.CreateAlignedLoad(Offset, Align);
>> if (Offset->getType() != PtrDiffTy)
>> Offset = CGF.Builder.CreateZExtOrBitCast(Offset, PtrDiffTy);
>> return Offset;
>>
>> Modified: cfe/trunk/lib/CodeGen/CGObjCMac.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGObjCMac.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGObjCMac.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGObjCMac.cpp Tue Sep 8 03:05:57 2015
>> @@ -1017,7 +1017,7 @@ protected:
>> /// \param AddToUsed - Whether the variable should be added to
>> /// "llvm.used".
>> llvm::GlobalVariable *CreateMetadataVar(Twine Name, llvm::Constant *Init,
>> - StringRef Section, unsigned Align,
>> + StringRef Section, CharUnits Align,
>> bool AddToUsed);
>>
>> CodeGen::RValue EmitMessageSend(CodeGen::CodeGenFunction &CGF,
>> @@ -1039,7 +1039,7 @@ public:
>> CGObjCCommonMac(CodeGen::CodeGenModule &cgm) :
>> CGObjCRuntime(cgm), VMContext(cgm.getLLVMContext()) { }
>>
>> - llvm::Constant *GenerateConstantString(const StringLiteral *SL) override;
>> + ConstantAddress GenerateConstantString(const StringLiteral *SL) override;
>>
>> llvm::Function *GenerateMethod(const ObjCMethodDecl *OMD,
>> const ObjCContainerDecl *CD=nullptr) override;
>> @@ -1172,8 +1172,8 @@ private:
>>
>> /// EmitSelector - Return a Value*, of type ObjCTypes.SelectorPtrTy,
>> /// for the given selector.
>> - llvm::Value *EmitSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lval=false);
>> + llvm::Value *EmitSelector(CodeGenFunction &CGF, Selector Sel);
>> + Address EmitSelectorAddr(CodeGenFunction &CGF, Selector Sel);
>>
>> public:
>> CGObjCMac(CodeGen::CodeGenModule &cgm);
>> @@ -1199,8 +1199,8 @@ public:
>> llvm::Value *GetClass(CodeGenFunction &CGF,
>> const ObjCInterfaceDecl *ID) override;
>>
>> - llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lval = false) override;
>> + llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel) override;
>> + Address GetAddrOfSelector(CodeGenFunction &CGF, Selector Sel) override;
>>
>> /// The NeXT/Apple runtimes do not support typed selectors; just emit an
>> /// untyped one.
>> @@ -1236,19 +1236,19 @@ public:
>> void EmitThrowStmt(CodeGen::CodeGenFunction &CGF, const ObjCAtThrowStmt &S,
>> bool ClearInsertionPoint=true) override;
>> llvm::Value * EmitObjCWeakRead(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *AddrWeakObj) override;
>> + Address AddrWeakObj) override;
>> void EmitObjCWeakAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) override;
>> + llvm::Value *src, Address dst) override;
>> void EmitObjCGlobalAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest,
>> + llvm::Value *src, Address dest,
>> bool threadlocal = false) override;
>> void EmitObjCIvarAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest,
>> + llvm::Value *src, Address dest,
>> llvm::Value *ivarOffset) override;
>> void EmitObjCStrongCastAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest) override;
>> + llvm::Value *src, Address dest) override;
>> void EmitGCMemmoveCollectable(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *dest, llvm::Value *src,
>> + Address dest, Address src,
>> llvm::Value *size) override;
>>
>> LValue EmitObjCValueForIvar(CodeGen::CodeGenFunction &CGF, QualType ObjectTy,
>> @@ -1395,8 +1395,8 @@ private:
>>
>> /// EmitSelector - Return a Value*, of type ObjCTypes.SelectorPtrTy,
>> /// for the given selector.
>> - llvm::Value *EmitSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lval=false);
>> + llvm::Value *EmitSelector(CodeGenFunction &CGF, Selector Sel);
>> + Address EmitSelectorAddr(CodeGenFunction &CGF, Selector Sel);
>>
>> /// GetInterfaceEHType - Get the cached ehtype for the given Objective-C
>> /// interface. The return value has type EHTypePtrTy.
>> @@ -1474,9 +1474,10 @@ public:
>> llvm::Value *GetClass(CodeGenFunction &CGF,
>> const ObjCInterfaceDecl *ID) override;
>>
>> - llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lvalue = false) override
>> - { return EmitSelector(CGF, Sel, lvalue); }
>> + llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel) override
>> + { return EmitSelector(CGF, Sel); }
>> + Address GetAddrOfSelector(CodeGenFunction &CGF, Selector Sel) override
>> + { return EmitSelectorAddr(CGF, Sel); }
>>
>> /// The NeXT/Apple runtimes do not support typed selectors; just emit an
>> /// untyped one.
>> @@ -1531,19 +1532,19 @@ public:
>> void EmitThrowStmt(CodeGen::CodeGenFunction &CGF, const ObjCAtThrowStmt &S,
>> bool ClearInsertionPoint=true) override;
>> llvm::Value * EmitObjCWeakRead(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *AddrWeakObj) override;
>> + Address AddrWeakObj) override;
>> void EmitObjCWeakAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) override;
>> + llvm::Value *src, Address edst) override;
>> void EmitObjCGlobalAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest,
>> + llvm::Value *src, Address dest,
>> bool threadlocal = false) override;
>> void EmitObjCIvarAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest,
>> + llvm::Value *src, Address dest,
>> llvm::Value *ivarOffset) override;
>> void EmitObjCStrongCastAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest) override;
>> + llvm::Value *src, Address dest) override;
>> void EmitGCMemmoveCollectable(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *dest, llvm::Value *src,
>> + Address dest, Address src,
>> llvm::Value *size) override;
>> LValue EmitObjCValueForIvar(CodeGen::CodeGenFunction &CGF, QualType ObjectTy,
>> llvm::Value *BaseValue, const ObjCIvarDecl *Ivar,
>> @@ -1645,7 +1646,7 @@ struct NullReturnState {
>> // memory or (2) agg values in registers.
>> if (result.isAggregate()) {
>> assert(result.isAggregate() && "null init of non-aggregate result?");
>> - CGF.EmitNullInitialization(result.getAggregateAddr(), resultType);
>> + CGF.EmitNullInitialization(result.getAggregateAddress(), resultType);
>> if (contBB) CGF.EmitBlock(contBB);
>> return result;
>> }
>> @@ -1711,9 +1712,11 @@ llvm::Value *CGObjCMac::GetClass(CodeGen
>> }
>>
>> /// GetSelector - Return the pointer to the unique'd string for this selector.
>> -llvm::Value *CGObjCMac::GetSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lval) {
>> - return EmitSelector(CGF, Sel, lval);
>> +llvm::Value *CGObjCMac::GetSelector(CodeGenFunction &CGF, Selector Sel) {
>> + return EmitSelector(CGF, Sel);
>> +}
>> +Address CGObjCMac::GetAddrOfSelector(CodeGenFunction &CGF, Selector Sel) {
>> + return EmitSelectorAddr(CGF, Sel);
>> }
>> llvm::Value *CGObjCMac::GetSelector(CodeGenFunction &CGF, const ObjCMethodDecl
>> *Method) {
>> @@ -1756,7 +1759,7 @@ llvm::Constant *CGObjCMac::GetEHType(Qua
>> };
>> */
>>
>> -llvm::Constant *CGObjCCommonMac::GenerateConstantString(
>> +ConstantAddress CGObjCCommonMac::GenerateConstantString(
>> const StringLiteral *SL) {
>> return (CGM.getLangOpts().NoConstantCFStrings == 0 ?
>> CGM.GetAddrOfConstantCFString(SL) :
>> @@ -1783,13 +1786,14 @@ CGObjCMac::GenerateMessageSendSuper(Code
>> const ObjCMethodDecl *Method) {
>> // Create and init a super structure; this is a (receiver, class)
>> // pair we will pass to objc_msgSendSuper.
>> - llvm::Value *ObjCSuper =
>> - CGF.CreateTempAlloca(ObjCTypes.SuperTy, "objc_super");
>> + Address ObjCSuper =
>> + CGF.CreateTempAlloca(ObjCTypes.SuperTy, CGF.getPointerAlign(),
>> + "objc_super");
>> llvm::Value *ReceiverAsObject =
>> CGF.Builder.CreateBitCast(Receiver, ObjCTypes.ObjectPtrTy);
>> CGF.Builder.CreateStore(
>> ReceiverAsObject,
>> - CGF.Builder.CreateStructGEP(ObjCTypes.SuperTy, ObjCSuper, 0));
>> + CGF.Builder.CreateStructGEP(ObjCSuper, 0, CharUnits::Zero()));
>>
>> // If this is a class message the metaclass is passed as the target.
>> llvm::Value *Target;
>> @@ -1803,12 +1807,13 @@ CGObjCMac::GenerateMessageSendSuper(Code
>> // isa" is the first ivar in a class (which it must be).
>> Target = EmitClassRef(CGF, Class->getSuperClass());
>> Target = CGF.Builder.CreateStructGEP(ObjCTypes.ClassTy, Target, 0);
>> - Target = CGF.Builder.CreateLoad(Target);
>> + Target = CGF.Builder.CreateAlignedLoad(Target, CGF.getPointerAlign());
>> } else {
>> llvm::Constant *MetaClassPtr = EmitMetaClassRef(Class);
>> llvm::Value *SuperPtr =
>> CGF.Builder.CreateStructGEP(ObjCTypes.ClassTy, MetaClassPtr, 1);
>> - llvm::Value *Super = CGF.Builder.CreateLoad(SuperPtr);
>> + llvm::Value *Super =
>> + CGF.Builder.CreateAlignedLoad(SuperPtr, CGF.getPointerAlign());
>> Target = Super;
>> }
>> } else if (isCategoryImpl)
>> @@ -1816,18 +1821,18 @@ CGObjCMac::GenerateMessageSendSuper(Code
>> else {
>> llvm::Value *ClassPtr = EmitSuperClassRef(Class);
>> ClassPtr = CGF.Builder.CreateStructGEP(ObjCTypes.ClassTy, ClassPtr, 1);
>> - Target = CGF.Builder.CreateLoad(ClassPtr);
>> + Target = CGF.Builder.CreateAlignedLoad(ClassPtr, CGF.getPointerAlign());
>> }
>> // FIXME: We shouldn't need to do this cast, rectify the ASTContext and
>> // ObjCTypes types.
>> llvm::Type *ClassTy =
>> CGM.getTypes().ConvertType(CGF.getContext().getObjCClassType());
>> Target = CGF.Builder.CreateBitCast(Target, ClassTy);
>> - CGF.Builder.CreateStore(
>> - Target, CGF.Builder.CreateStructGEP(ObjCTypes.SuperTy, ObjCSuper, 1));
>> + CGF.Builder.CreateStore(Target,
>> + CGF.Builder.CreateStructGEP(ObjCSuper, 1, CGF.getPointerSize()));
>> return EmitMessageSend(CGF, Return, ResultType,
>> EmitSelector(CGF, Sel),
>> - ObjCSuper, ObjCTypes.SuperPtrCTy,
>> + ObjCSuper.getPointer(), ObjCTypes.SuperPtrCTy,
>> true, CallArgs, Method, ObjCTypes);
>> }
>>
>> @@ -2361,9 +2366,8 @@ llvm::Constant *CGObjCCommonMac::getBitm
>> }
>> }
>>
>> - int e = Layout.size()-1;
>> - while (e >= 0) {
>> - unsigned char inst = Layout[e--];
>> + while (!Layout.empty()) {
>> + unsigned char inst = Layout.back();
>> enum BLOCK_LAYOUT_OPCODE opcode = (enum BLOCK_LAYOUT_OPCODE) (inst >> 4);
>> if (opcode == BLOCK_LAYOUT_NON_OBJECT_BYTES || opcode == BLOCK_LAYOUT_NON_OBJECT_WORDS)
>> Layout.pop_back();
>> @@ -2376,19 +2380,19 @@ llvm::Constant *CGObjCCommonMac::getBitm
>> // Block variable layout instruction has been inlined.
>> if (CGM.getLangOpts().ObjCGCBitmapPrint) {
>> if (ComputeByrefLayout)
>> - printf("\n Inline instruction for BYREF variable layout: ");
>> + printf("\n Inline BYREF variable layout: ");
>> else
>> - printf("\n Inline instruction for block variable layout: ");
>> - printf("0x0%" PRIx64 "\n", Result);
>> - }
>> - if (WordSizeInBytes == 8) {
>> - const llvm::APInt Instruction(64, Result);
>> - return llvm::Constant::getIntegerValue(CGM.Int64Ty, Instruction);
>> - }
>> - else {
>> - const llvm::APInt Instruction(32, Result);
>> - return llvm::Constant::getIntegerValue(CGM.Int32Ty, Instruction);
>> + printf("\n Inline block variable layout: ");
>> + printf("0x0%" PRIx64 "", Result);
>> + if (auto numStrong = (Result & 0xF00) >> 8)
>> + printf(", BL_STRONG:%d", (int) numStrong);
>> + if (auto numByref = (Result & 0x0F0) >> 4)
>> + printf(", BL_BYREF:%d", (int) numByref);
>> + if (auto numWeak = (Result & 0x00F) >> 0)
>> + printf(", BL_WEAK:%d", (int) numWeak);
>> + printf(", BL_OPERATOR:0\n");
>> }
>> + return llvm::ConstantInt::get(CGM.IntPtrTy, Result);
>> }
>>
>> unsigned char inst = (BLOCK_LAYOUT_OPERATOR << 4) | 0;
>> @@ -2399,9 +2403,9 @@ llvm::Constant *CGObjCCommonMac::getBitm
>>
>> if (CGM.getLangOpts().ObjCGCBitmapPrint) {
>> if (ComputeByrefLayout)
>> - printf("\n BYREF variable layout: ");
>> + printf("\n Byref variable layout: ");
>> else
>> - printf("\n block variable layout: ");
>> + printf("\n Block variable layout: ");
>> for (unsigned i = 0, e = BitMap.size(); i != e; i++) {
>> unsigned char inst = BitMap[i];
>> enum BLOCK_LAYOUT_OPCODE opcode = (enum BLOCK_LAYOUT_OPCODE) (inst >> 4);
>> @@ -2443,7 +2447,7 @@ llvm::Constant *CGObjCCommonMac::getBitm
>> llvm::GlobalVariable *Entry = CreateMetadataVar(
>> "OBJC_CLASS_NAME_",
>> llvm::ConstantDataArray::getString(VMContext, BitMap, false),
>> - "__TEXT,__objc_classname,cstring_literals", 1, true);
>> + "__TEXT,__objc_classname,cstring_literals", CharUnits::One(), true);
>> return getConstantGEP(VMContext, Entry, 0, 0);
>> }
>>
>> @@ -2699,7 +2703,7 @@ CGObjCMac::EmitProtocolExtension(const O
>>
>> // No special section, but goes in llvm.used
>> return CreateMetadataVar("\01l_OBJC_PROTOCOLEXT_" + PD->getName(), Init,
>> - StringRef(), 0, true);
>> + StringRef(), CGM.getPointerAlign(), true);
>> }
>>
>> /*
>> @@ -2738,7 +2742,7 @@ CGObjCMac::EmitProtocolList(Twine Name,
>> llvm::Constant *Init = llvm::ConstantStruct::getAnon(Values);
>> llvm::GlobalVariable *GV =
>> CreateMetadataVar(Name, Init, "__OBJC,__cat_cls_meth,regular,no_dead_strip",
>> - 4, false);
>> + CGM.getPointerAlign(), false);
>> return llvm::ConstantExpr::getBitCast(GV, ObjCTypes.ProtocolListPtrTy);
>> }
>>
>> @@ -2815,7 +2819,7 @@ llvm::Constant *CGObjCCommonMac::EmitPro
>> CreateMetadataVar(Name, Init,
>> (ObjCABI == 2) ? "__DATA, __objc_const" :
>> "__OBJC,__property,regular,no_dead_strip",
>> - (ObjCABI == 2) ? 8 : 4,
>> + CGM.getPointerAlign(),
>> true);
>> return llvm::ConstantExpr::getBitCast(GV, ObjCTypes.PropertyListPtrTy);
>> }
>> @@ -2834,7 +2838,7 @@ CGObjCCommonMac::EmitProtocolMethodTypes
>>
>> llvm::GlobalVariable *GV = CreateMetadataVar(
>> Name, Init, (ObjCABI == 2) ? "__DATA, __objc_const" : StringRef(),
>> - (ObjCABI == 2) ? 8 : 4, true);
>> + CGM.getPointerAlign(), true);
>> return llvm::ConstantExpr::getBitCast(GV, ObjCTypes.Int8PtrPtrTy);
>> }
>>
>> @@ -2872,7 +2876,8 @@ CGObjCMac::EmitMethodDescList(Twine Name
>> Values[1] = llvm::ConstantArray::get(AT, Methods);
>> llvm::Constant *Init = llvm::ConstantStruct::getAnon(Values);
>>
>> - llvm::GlobalVariable *GV = CreateMetadataVar(Name, Init, Section, 4, true);
>> + llvm::GlobalVariable *GV =
>> + CreateMetadataVar(Name, Init, Section, CGM.getPointerAlign(), true);
>> return llvm::ConstantExpr::getBitCast(GV,
>> ObjCTypes.MethodDescriptionListPtrTy);
>> }
>> @@ -2944,7 +2949,8 @@ void CGObjCMac::GenerateCategory(const O
>>
>> llvm::GlobalVariable *GV =
>> CreateMetadataVar("OBJC_CATEGORY_" + ExtName.str(), Init,
>> - "__OBJC,__category,regular,no_dead_strip", 4, true);
>> + "__OBJC,__category,regular,no_dead_strip",
>> + CGM.getPointerAlign(), true);
>> DefinedCategories.push_back(GV);
>> DefinedCategoryNames.insert(ExtName.str());
>> // method definition entries must be clear for next implementation.
>> @@ -3084,10 +3090,10 @@ void CGObjCMac::GenerateClass(const ObjC
>> "Forward metaclass reference has incorrect type.");
>> GV->setInitializer(Init);
>> GV->setSection(Section);
>> - GV->setAlignment(4);
>> + GV->setAlignment(CGM.getPointerAlign().getQuantity());
>> CGM.addCompilerUsedGlobal(GV);
>> } else
>> - GV = CreateMetadataVar(Name, Init, Section, 4, true);
>> + GV = CreateMetadataVar(Name, Init, Section, CGM.getPointerAlign(), true);
>> DefinedClasses.push_back(GV);
>> ImplementedClasses.push_back(Interface);
>> // method definition entries must be clear for next implementation.
>> @@ -3222,7 +3228,8 @@ CGObjCMac::EmitClassExtension(const ObjC
>> llvm::Constant *Init =
>> llvm::ConstantStruct::get(ObjCTypes.ClassExtensionTy, Values);
>> return CreateMetadataVar("OBJC_CLASSEXT_" + ID->getName(), Init,
>> - "__OBJC,__class_ext,regular,no_dead_strip", 4, true);
>> + "__OBJC,__class_ext,regular,no_dead_strip",
>> + CGM.getPointerAlign(), true);
>> }
>>
>> /*
>> @@ -3280,11 +3287,12 @@ llvm::Constant *CGObjCMac::EmitIvarList(
>> if (ForClass)
>> GV =
>> CreateMetadataVar("OBJC_CLASS_VARIABLES_" + ID->getName(), Init,
>> - "__OBJC,__class_vars,regular,no_dead_strip", 4, true);
>> + "__OBJC,__class_vars,regular,no_dead_strip",
>> + CGM.getPointerAlign(), true);
>> else
>> GV = CreateMetadataVar("OBJC_INSTANCE_VARIABLES_" + ID->getName(), Init,
>> - "__OBJC,__instance_vars,regular,no_dead_strip", 4,
>> - true);
>> + "__OBJC,__instance_vars,regular,no_dead_strip",
>> + CGM.getPointerAlign(), true);
>> return llvm::ConstantExpr::getBitCast(GV, ObjCTypes.IvarListPtrTy);
>> }
>>
>> @@ -3334,7 +3342,8 @@ llvm::Constant *CGObjCMac::EmitMethodLis
>> Values[2] = llvm::ConstantArray::get(AT, Methods);
>> llvm::Constant *Init = llvm::ConstantStruct::getAnon(Values);
>>
>> - llvm::GlobalVariable *GV = CreateMetadataVar(Name, Init, Section, 4, true);
>> + llvm::GlobalVariable *GV =
>> + CreateMetadataVar(Name, Init, Section, CGM.getPointerAlign(), true);
>> return llvm::ConstantExpr::getBitCast(GV, ObjCTypes.MethodListPtrTy);
>> }
>>
>> @@ -3359,7 +3368,7 @@ llvm::Function *CGObjCCommonMac::Generat
>> llvm::GlobalVariable *CGObjCCommonMac::CreateMetadataVar(Twine Name,
>> llvm::Constant *Init,
>> StringRef Section,
>> - unsigned Align,
>> + CharUnits Align,
>> bool AddToUsed) {
>> llvm::Type *Ty = Init->getType();
>> llvm::GlobalVariable *GV =
>> @@ -3367,8 +3376,7 @@ llvm::GlobalVariable *CGObjCCommonMac::C
>> llvm::GlobalValue::PrivateLinkage, Init, Name);
>> if (!Section.empty())
>> GV->setSection(Section);
>> - if (Align)
>> - GV->setAlignment(Align);
>> + GV->setAlignment(Align.getQuantity());
>> if (AddToUsed)
>> CGM.addCompilerUsedGlobal(GV);
>> return GV;
>> @@ -3423,14 +3431,14 @@ void CGObjCMac::EmitSynchronizedStmt(Cod
>> namespace {
>> struct PerformFragileFinally final : EHScopeStack::Cleanup {
>> const Stmt &S;
>> - llvm::Value *SyncArgSlot;
>> - llvm::Value *CallTryExitVar;
>> - llvm::Value *ExceptionData;
>> + Address SyncArgSlot;
>> + Address CallTryExitVar;
>> + Address ExceptionData;
>> ObjCTypesHelper &ObjCTypes;
>> PerformFragileFinally(const Stmt *S,
>> - llvm::Value *SyncArgSlot,
>> - llvm::Value *CallTryExitVar,
>> - llvm::Value *ExceptionData,
>> + Address SyncArgSlot,
>> + Address CallTryExitVar,
>> + Address ExceptionData,
>> ObjCTypesHelper *ObjCTypes)
>> : S(*S), SyncArgSlot(SyncArgSlot), CallTryExitVar(CallTryExitVar),
>> ExceptionData(ExceptionData), ObjCTypes(*ObjCTypes) {}
>> @@ -3447,7 +3455,7 @@ namespace {
>>
>> CGF.EmitBlock(FinallyCallExit);
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getExceptionTryExitFn(),
>> - ExceptionData);
>> + ExceptionData.getPointer());
>>
>> CGF.EmitBlock(FinallyNoCallExit);
>>
>> @@ -3568,7 +3576,7 @@ void FragileHazards::emitReadHazard(CGBu
>> void FragileHazards::emitHazardsInNewBlocks() {
>> if (Locals.empty()) return;
>>
>> - CGBuilderTy Builder(CGF.getLLVMContext());
>> + CGBuilderTy Builder(CGF, CGF.getLLVMContext());
>>
>> // Iterate through all blocks, skipping those prior to the try.
>> for (llvm::Function::iterator
>> @@ -3607,6 +3615,10 @@ static void addIfPresent(llvm::DenseSet<
>> if (V) S.insert(V);
>> }
>>
>> +static void addIfPresent(llvm::DenseSet<llvm::Value*> &S, Address V) {
>> + if (V.isValid()) S.insert(V.getPointer());
>> +}
>> +
>> void FragileHazards::collectLocals() {
>> // Compute a set of allocas to ignore.
>> llvm::DenseSet<llvm::Value*> AllocasToIgnore;
>> @@ -3760,21 +3772,23 @@ void CGObjCMac::EmitTryOrSynchronizedStm
>> // @synchronized. We can't avoid a temp here because we need the
>> // value to be preserved. If the backend ever does liveness
>> // correctly after setjmp, this will be unnecessary.
>> - llvm::Value *SyncArgSlot = nullptr;
>> + Address SyncArgSlot = Address::invalid();
>> if (!isTry) {
>> llvm::Value *SyncArg =
>> CGF.EmitScalarExpr(cast<ObjCAtSynchronizedStmt>(S).getSynchExpr());
>> SyncArg = CGF.Builder.CreateBitCast(SyncArg, ObjCTypes.ObjectPtrTy);
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getSyncEnterFn(), SyncArg);
>>
>> - SyncArgSlot = CGF.CreateTempAlloca(SyncArg->getType(), "sync.arg");
>> + SyncArgSlot = CGF.CreateTempAlloca(SyncArg->getType(),
>> + CGF.getPointerAlign(), "sync.arg");
>> CGF.Builder.CreateStore(SyncArg, SyncArgSlot);
>> }
>>
>> // Allocate memory for the setjmp buffer. This needs to be kept
>> // live throughout the try and catch blocks.
>> - llvm::Value *ExceptionData = CGF.CreateTempAlloca(ObjCTypes.ExceptionDataTy,
>> - "exceptiondata.ptr");
>> + Address ExceptionData = CGF.CreateTempAlloca(ObjCTypes.ExceptionDataTy,
>> + CGF.getPointerAlign(),
>> + "exceptiondata.ptr");
>>
>> // Create the fragile hazards. Note that this will not capture any
>> // of the allocas required for exception processing, but will
>> @@ -3790,12 +3804,13 @@ void CGObjCMac::EmitTryOrSynchronizedStm
>> // The setjmp-safety rule here is that we should always store to this
>> // variable in a place that dominates the branch through the cleanup
>> // without passing through any setjmps.
>> - llvm::Value *CallTryExitVar = CGF.CreateTempAlloca(CGF.Builder.getInt1Ty(),
>> - "_call_try_exit");
>> + Address CallTryExitVar = CGF.CreateTempAlloca(CGF.Builder.getInt1Ty(),
>> + CharUnits::One(),
>> + "_call_try_exit");
>>
>> // A slot containing the exception to rethrow. Only needed when we
>> // have both a @catch and a @finally.
>> - llvm::Value *PropagatingExnVar = nullptr;
>> + Address PropagatingExnVar = Address::invalid();
>>
>> // Push a normal cleanup to leave the try scope.
>> CGF.EHStack.pushCleanup<PerformFragileFinally>(NormalAndEHCleanup, &S,
>> @@ -3808,13 +3823,14 @@ void CGObjCMac::EmitTryOrSynchronizedStm
>> // - Call objc_exception_try_enter to push ExceptionData on top of
>> // the EH stack.
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getExceptionTryEnterFn(),
>> - ExceptionData);
>> + ExceptionData.getPointer());
>>
>> // - Call setjmp on the exception data buffer.
>> llvm::Constant *Zero = llvm::ConstantInt::get(CGF.Builder.getInt32Ty(), 0);
>> llvm::Value *GEPIndexes[] = { Zero, Zero, Zero };
>> llvm::Value *SetJmpBuffer = CGF.Builder.CreateGEP(
>> - ObjCTypes.ExceptionDataTy, ExceptionData, GEPIndexes, "setjmp_buffer");
>> + ObjCTypes.ExceptionDataTy, ExceptionData.getPointer(), GEPIndexes,
>> + "setjmp_buffer");
>> llvm::CallInst *SetJmpResult = CGF.EmitNounwindRuntimeCall(
>> ObjCTypes.getSetJmpFn(), SetJmpBuffer, "setjmp_result");
>> SetJmpResult->setCanReturnTwice();
>> @@ -3854,7 +3870,7 @@ void CGObjCMac::EmitTryOrSynchronizedStm
>> // nothing can cross this so the value is already in SSA form.
>> llvm::CallInst *Caught =
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getExceptionExtractFn(),
>> - ExceptionData, "caught");
>> + ExceptionData.getPointer(), "caught");
>>
>> // Push the exception to rethrow onto the EH value stack for the
>> // benefit of any @throws in the handlers.
>> @@ -3870,13 +3886,14 @@ void CGObjCMac::EmitTryOrSynchronizedStm
>> // Save the currently-propagating exception before
>> // objc_exception_try_enter clears the exception slot.
>> PropagatingExnVar = CGF.CreateTempAlloca(Caught->getType(),
>> + CGF.getPointerAlign(),
>> "propagating_exception");
>> CGF.Builder.CreateStore(Caught, PropagatingExnVar);
>>
>> // Enter a new exception try block (in case a @catch block
>> // throws an exception).
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getExceptionTryEnterFn(),
>> - ExceptionData);
>> + ExceptionData.getPointer());
>>
>> llvm::CallInst *SetJmpResult =
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getSetJmpFn(),
>> @@ -4008,10 +4025,10 @@ void CGObjCMac::EmitTryOrSynchronizedStm
>>
>> // Extract the new exception and save it to the
>> // propagating-exception slot.
>> - assert(PropagatingExnVar);
>> + assert(PropagatingExnVar.isValid());
>> llvm::CallInst *NewCaught =
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getExceptionExtractFn(),
>> - ExceptionData, "caught");
>> + ExceptionData.getPointer(), "caught");
>> CGF.Builder.CreateStore(NewCaught, PropagatingExnVar);
>>
>> // Don't pop the catch handler; the throw already did.
>> @@ -4036,14 +4053,14 @@ void CGObjCMac::EmitTryOrSynchronizedStm
>> if (CGF.HaveInsertPoint()) {
>> // If we have a propagating-exception variable, check it.
>> llvm::Value *PropagatingExn;
>> - if (PropagatingExnVar) {
>> + if (PropagatingExnVar.isValid()) {
>> PropagatingExn = CGF.Builder.CreateLoad(PropagatingExnVar);
>>
>> // Otherwise, just look in the buffer for the exception to throw.
>> } else {
>> llvm::CallInst *Caught =
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getExceptionExtractFn(),
>> - ExceptionData);
>> + ExceptionData.getPointer());
>> PropagatingExn = Caught;
>> }
>>
>> @@ -4083,14 +4100,13 @@ void CGObjCMac::EmitThrowStmt(CodeGen::C
>> /// object: objc_read_weak (id *src)
>> ///
>> llvm::Value * CGObjCMac::EmitObjCWeakRead(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *AddrWeakObj) {
>> - llvm::Type* DestTy =
>> - cast<llvm::PointerType>(AddrWeakObj->getType())->getElementType();
>> + Address AddrWeakObj) {
>> + llvm::Type* DestTy = AddrWeakObj.getElementType();
>> AddrWeakObj = CGF.Builder.CreateBitCast(AddrWeakObj,
>> ObjCTypes.PtrObjectPtrTy);
>> llvm::Value *read_weak =
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcReadWeakFn(),
>> - AddrWeakObj, "weakread");
>> + AddrWeakObj.getPointer(), "weakread");
>> read_weak = CGF.Builder.CreateBitCast(read_weak, DestTy);
>> return read_weak;
>> }
>> @@ -4099,7 +4115,7 @@ llvm::Value * CGObjCMac::EmitObjCWeakRea
>> /// objc_assign_weak (id src, id *dst)
>> ///
>> void CGObjCMac::EmitObjCWeakAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) {
>> + llvm::Value *src, Address dst) {
>> llvm::Type * SrcTy = src->getType();
>> if (!isa<llvm::PointerType>(SrcTy)) {
>> unsigned Size = CGM.getDataLayout().getTypeAllocSize(SrcTy);
>> @@ -4110,7 +4126,7 @@ void CGObjCMac::EmitObjCWeakAssign(CodeG
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst };
>> + llvm::Value *args[] = { src, dst.getPointer() };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignWeakFn(),
>> args, "weakassign");
>> return;
>> @@ -4120,7 +4136,7 @@ void CGObjCMac::EmitObjCWeakAssign(CodeG
>> /// objc_assign_global (id src, id *dst)
>> ///
>> void CGObjCMac::EmitObjCGlobalAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst,
>> + llvm::Value *src, Address dst,
>> bool threadlocal) {
>> llvm::Type * SrcTy = src->getType();
>> if (!isa<llvm::PointerType>(SrcTy)) {
>> @@ -4132,7 +4148,7 @@ void CGObjCMac::EmitObjCGlobalAssign(Cod
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst };
>> + llvm::Value *args[] = { src, dst.getPointer() };
>> if (!threadlocal)
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignGlobalFn(),
>> args, "globalassign");
>> @@ -4146,7 +4162,7 @@ void CGObjCMac::EmitObjCGlobalAssign(Cod
>> /// objc_assign_ivar (id src, id *dst, ptrdiff_t ivaroffset)
>> ///
>> void CGObjCMac::EmitObjCIvarAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst,
>> + llvm::Value *src, Address dst,
>> llvm::Value *ivarOffset) {
>> assert(ivarOffset && "EmitObjCIvarAssign - ivarOffset is NULL");
>> llvm::Type * SrcTy = src->getType();
>> @@ -4159,7 +4175,7 @@ void CGObjCMac::EmitObjCIvarAssign(CodeG
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst, ivarOffset };
>> + llvm::Value *args[] = { src, dst.getPointer(), ivarOffset };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignIvarFn(), args);
>> return;
>> }
>> @@ -4168,7 +4184,7 @@ void CGObjCMac::EmitObjCIvarAssign(CodeG
>> /// objc_assign_strongCast (id src, id *dst)
>> ///
>> void CGObjCMac::EmitObjCStrongCastAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) {
>> + llvm::Value *src, Address dst) {
>> llvm::Type * SrcTy = src->getType();
>> if (!isa<llvm::PointerType>(SrcTy)) {
>> unsigned Size = CGM.getDataLayout().getTypeAllocSize(SrcTy);
>> @@ -4179,19 +4195,19 @@ void CGObjCMac::EmitObjCStrongCastAssign
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst };
>> + llvm::Value *args[] = { src, dst.getPointer() };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignStrongCastFn(),
>> - args, "weakassign");
>> + args, "strongassign");
>> return;
>> }
>>
>> void CGObjCMac::EmitGCMemmoveCollectable(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *DestPtr,
>> - llvm::Value *SrcPtr,
>> + Address DestPtr,
>> + Address SrcPtr,
>> llvm::Value *size) {
>> SrcPtr = CGF.Builder.CreateBitCast(SrcPtr, ObjCTypes.Int8PtrTy);
>> DestPtr = CGF.Builder.CreateBitCast(DestPtr, ObjCTypes.Int8PtrTy);
>> - llvm::Value *args[] = { DestPtr, SrcPtr, size };
>> + llvm::Value *args[] = { DestPtr.getPointer(), SrcPtr.getPointer(), size };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.GcMemmoveCollectableFn(), args);
>> }
>>
>> @@ -4312,7 +4328,8 @@ void CGObjCMac::EmitModuleInfo() {
>> };
>> CreateMetadataVar("OBJC_MODULES",
>> llvm::ConstantStruct::get(ObjCTypes.ModuleTy, Values),
>> - "__OBJC,__module_info,regular,no_dead_strip", 4, true);
>> + "__OBJC,__module_info,regular,no_dead_strip",
>> + CGM.getPointerAlign(), true);
>> }
>>
>> llvm::Constant *CGObjCMac::EmitModuleSymbols() {
>> @@ -4356,7 +4373,8 @@ llvm::Constant *CGObjCMac::EmitModuleSym
>> llvm::Constant *Init = llvm::ConstantStruct::getAnon(Values);
>>
>> llvm::GlobalVariable *GV = CreateMetadataVar(
>> - "OBJC_SYMBOLS", Init, "__OBJC,__symbols,regular,no_dead_strip", 4, true);
>> + "OBJC_SYMBOLS", Init, "__OBJC,__symbols,regular,no_dead_strip",
>> + CGM.getPointerAlign(), true);
>> return llvm::ConstantExpr::getBitCast(GV, ObjCTypes.SymtabPtrTy);
>> }
>>
>> @@ -4372,10 +4390,11 @@ llvm::Value *CGObjCMac::EmitClassRefFrom
>> ObjCTypes.ClassPtrTy);
>> Entry = CreateMetadataVar(
>> "OBJC_CLASS_REFERENCES_", Casted,
>> - "__OBJC,__cls_refs,literal_pointers,no_dead_strip", 4, true);
>> + "__OBJC,__cls_refs,literal_pointers,no_dead_strip",
>> + CGM.getPointerAlign(), true);
>> }
>>
>> - return CGF.Builder.CreateLoad(Entry);
>> + return CGF.Builder.CreateAlignedLoad(Entry, CGF.getPointerAlign());
>> }
>>
>> llvm::Value *CGObjCMac::EmitClassRef(CodeGenFunction &CGF,
>> @@ -4388,23 +4407,25 @@ llvm::Value *CGObjCMac::EmitNSAutoreleas
>> return EmitClassRefFromId(CGF, II);
>> }
>>
>> -llvm::Value *CGObjCMac::EmitSelector(CodeGenFunction &CGF, Selector Sel,
>> - bool lvalue) {
>> - llvm::GlobalVariable *&Entry = SelectorReferences[Sel];
>> +llvm::Value *CGObjCMac::EmitSelector(CodeGenFunction &CGF, Selector Sel) {
>> + return CGF.Builder.CreateLoad(EmitSelectorAddr(CGF, Sel));
>> +}
>>
>> +Address CGObjCMac::EmitSelectorAddr(CodeGenFunction &CGF, Selector Sel) {
>> + CharUnits Align = CGF.getPointerAlign();
>> +
>> + llvm::GlobalVariable *&Entry = SelectorReferences[Sel];
>> if (!Entry) {
>> llvm::Constant *Casted =
>> llvm::ConstantExpr::getBitCast(GetMethodVarName(Sel),
>> ObjCTypes.SelectorPtrTy);
>> Entry = CreateMetadataVar(
>> "OBJC_SELECTOR_REFERENCES_", Casted,
>> - "__OBJC,__message_refs,literal_pointers,no_dead_strip", 4, true);
>> + "__OBJC,__message_refs,literal_pointers,no_dead_strip", Align, true);
>> Entry->setExternallyInitialized(true);
>> }
>>
>> - if (lvalue)
>> - return Entry;
>> - return CGF.Builder.CreateLoad(Entry);
>> + return Address(Entry, Align);
>> }
>>
>> llvm::Constant *CGObjCCommonMac::GetClassName(StringRef RuntimeName) {
>> @@ -4415,7 +4436,7 @@ llvm::Constant *CGObjCCommonMac::GetClas
>> llvm::ConstantDataArray::getString(VMContext, RuntimeName),
>> ((ObjCABI == 2) ? "__TEXT,__objc_classname,cstring_literals"
>> : "__TEXT,__cstring,cstring_literals"),
>> - 1, true);
>> + CharUnits::One(), true);
>> return getConstantGEP(VMContext, Entry, 0, 0);
>> }
>>
>> @@ -4735,7 +4756,7 @@ llvm::Constant *CGObjCCommonMac::BuildIv
>> llvm::ConstantDataArray::getString(VMContext, BitMap, false),
>> ((ObjCABI == 2) ? "__TEXT,__objc_classname,cstring_literals"
>> : "__TEXT,__cstring,cstring_literals"),
>> - 1, true);
>> + CharUnits::One(), true);
>> return getConstantGEP(VMContext, Entry, 0, 0);
>> }
>>
>> @@ -4825,7 +4846,7 @@ llvm::Constant *CGObjCCommonMac::GetMeth
>> llvm::ConstantDataArray::getString(VMContext, Sel.getAsString()),
>> ((ObjCABI == 2) ? "__TEXT,__objc_methname,cstring_literals"
>> : "__TEXT,__cstring,cstring_literals"),
>> - 1, true);
>> + CharUnits::One(), true);
>>
>> return getConstantGEP(VMContext, Entry, 0, 0);
>> }
>> @@ -4847,7 +4868,7 @@ llvm::Constant *CGObjCCommonMac::GetMeth
>> llvm::ConstantDataArray::getString(VMContext, TypeStr),
>> ((ObjCABI == 2) ? "__TEXT,__objc_methtype,cstring_literals"
>> : "__TEXT,__cstring,cstring_literals"),
>> - 1, true);
>> + CharUnits::One(), true);
>>
>> return getConstantGEP(VMContext, Entry, 0, 0);
>> }
>> @@ -4866,7 +4887,7 @@ llvm::Constant *CGObjCCommonMac::GetMeth
>> llvm::ConstantDataArray::getString(VMContext, TypeStr),
>> ((ObjCABI == 2) ? "__TEXT,__objc_methtype,cstring_literals"
>> : "__TEXT,__cstring,cstring_literals"),
>> - 1, true);
>> + CharUnits::One(), true);
>>
>> return getConstantGEP(VMContext, Entry, 0, 0);
>> }
>> @@ -4879,7 +4900,7 @@ llvm::Constant *CGObjCCommonMac::GetProp
>> Entry = CreateMetadataVar(
>> "OBJC_PROP_NAME_ATTR_",
>> llvm::ConstantDataArray::getString(VMContext, Ident->getName()),
>> - "__TEXT,__cstring,cstring_literals", 1, true);
>> + "__TEXT,__cstring,cstring_literals", CharUnits::One(), true);
>>
>> return getConstantGEP(VMContext, Entry, 0, 0);
>> }
>> @@ -5887,9 +5908,11 @@ llvm::Value *CGObjCNonFragileABIMac::Gen
>> std::string ProtocolName("\01l_OBJC_PROTOCOL_REFERENCE_$_");
>> ProtocolName += PD->getObjCRuntimeNameAsString();
>>
>> + CharUnits Align = CGF.getPointerAlign();
>> +
>> llvm::GlobalVariable *PTGV = CGM.getModule().getGlobalVariable(ProtocolName);
>> if (PTGV)
>> - return CGF.Builder.CreateLoad(PTGV);
>> + return CGF.Builder.CreateAlignedLoad(PTGV, Align);
>> PTGV = new llvm::GlobalVariable(
>> CGM.getModule(),
>> Init->getType(), false,
>> @@ -5898,8 +5921,9 @@ llvm::Value *CGObjCNonFragileABIMac::Gen
>> ProtocolName);
>> PTGV->setSection("__DATA, __objc_protorefs, coalesced, no_dead_strip");
>> PTGV->setVisibility(llvm::GlobalValue::HiddenVisibility);
>> + PTGV->setAlignment(Align.getQuantity());
>> CGM.addCompilerUsedGlobal(PTGV);
>> - return CGF.Builder.CreateLoad(PTGV);
>> + return CGF.Builder.CreateAlignedLoad(PTGV, Align);
>> }
>>
>> /// GenerateCategory - Build metadata for a category implementation.
>> @@ -6428,7 +6452,8 @@ llvm::Value *CGObjCNonFragileABIMac::Emi
>> const ObjCInterfaceDecl *Interface,
>> const ObjCIvarDecl *Ivar) {
>> llvm::Value *IvarOffsetValue = ObjCIvarOffsetVariable(Interface, Ivar);
>> - IvarOffsetValue = CGF.Builder.CreateLoad(IvarOffsetValue, "ivar");
>> + IvarOffsetValue = CGF.Builder.CreateAlignedLoad(IvarOffsetValue,
>> + CGF.getSizeAlign(), "ivar");
>> if (IsIvarOffsetKnownIdempotent(CGF, Ivar))
>> cast<llvm::LoadInst>(IvarOffsetValue)
>> ->setMetadata(CGM.getModule().getMDKindID("invariant.load"),
>> @@ -6559,16 +6584,17 @@ CGObjCNonFragileABIMac::EmitVTableMessag
>> }
>> }
>>
>> - llvm::Value *mref =
>> - CGF.Builder.CreateBitCast(messageRef, ObjCTypes.MessageRefPtrTy);
>> + Address mref =
>> + Address(CGF.Builder.CreateBitCast(messageRef, ObjCTypes.MessageRefPtrTy),
>> + CGF.getPointerAlign());
>>
>> // Update the message ref argument.
>> - args[1].RV = RValue::get(mref);
>> + args[1].RV = RValue::get(mref.getPointer());
>>
>> // Load the function to call from the message ref table.
>> - llvm::Value *callee =
>> - CGF.Builder.CreateStructGEP(ObjCTypes.MessageRefTy, mref, 0);
>> - callee = CGF.Builder.CreateLoad(callee, "msgSend_fn");
>> + Address calleeAddr =
>> + CGF.Builder.CreateStructGEP(mref, 0, CharUnits::Zero());
>> + llvm::Value *callee = CGF.Builder.CreateLoad(calleeAddr, "msgSend_fn");
>>
>> callee = CGF.Builder.CreateBitCast(callee, MSI.MessengerType);
>>
>> @@ -6617,6 +6643,7 @@ llvm::Value *CGObjCNonFragileABIMac::Emi
>> IdentifierInfo *II,
>> bool Weak,
>> const ObjCInterfaceDecl *ID) {
>> + CharUnits Align = CGF.getPointerAlign();
>> llvm::GlobalVariable *&Entry = ClassReferences[II];
>>
>> if (!Entry) {
>> @@ -6627,13 +6654,11 @@ llvm::Value *CGObjCNonFragileABIMac::Emi
>> Entry = new llvm::GlobalVariable(CGM.getModule(), ObjCTypes.ClassnfABIPtrTy,
>> false, llvm::GlobalValue::PrivateLinkage,
>> ClassGV, "OBJC_CLASSLIST_REFERENCES_$_");
>> - Entry->setAlignment(
>> - CGM.getDataLayout().getABITypeAlignment(
>> - ObjCTypes.ClassnfABIPtrTy));
>> + Entry->setAlignment(Align.getQuantity());
>> Entry->setSection("__DATA, __objc_classrefs, regular, no_dead_strip");
>> CGM.addCompilerUsedGlobal(Entry);
>> }
>> - return CGF.Builder.CreateLoad(Entry);
>> + return CGF.Builder.CreateAlignedLoad(Entry, Align);
>> }
>>
>> llvm::Value *CGObjCNonFragileABIMac::EmitClassRef(CodeGenFunction &CGF,
>> @@ -6650,6 +6675,7 @@ llvm::Value *CGObjCNonFragileABIMac::Emi
>> llvm::Value *
>> CGObjCNonFragileABIMac::EmitSuperClassRef(CodeGenFunction &CGF,
>> const ObjCInterfaceDecl *ID) {
>> + CharUnits Align = CGF.getPointerAlign();
>> llvm::GlobalVariable *&Entry = SuperClassReferences[ID->getIdentifier()];
>>
>> if (!Entry) {
>> @@ -6660,13 +6686,11 @@ CGObjCNonFragileABIMac::EmitSuperClassRe
>> Entry = new llvm::GlobalVariable(CGM.getModule(), ObjCTypes.ClassnfABIPtrTy,
>> false, llvm::GlobalValue::PrivateLinkage,
>> ClassGV, "OBJC_CLASSLIST_SUP_REFS_$_");
>> - Entry->setAlignment(
>> - CGM.getDataLayout().getABITypeAlignment(
>> - ObjCTypes.ClassnfABIPtrTy));
>> + Entry->setAlignment(Align.getQuantity());
>> Entry->setSection("__DATA, __objc_superrefs, regular, no_dead_strip");
>> CGM.addCompilerUsedGlobal(Entry);
>> }
>> - return CGF.Builder.CreateLoad(Entry);
>> + return CGF.Builder.CreateAlignedLoad(Entry, Align);
>> }
>>
>> /// EmitMetaClassRef - Return a Value * of the address of _class_t
>> @@ -6675,6 +6699,7 @@ CGObjCNonFragileABIMac::EmitSuperClassRe
>> llvm::Value *CGObjCNonFragileABIMac::EmitMetaClassRef(CodeGenFunction &CGF,
>> const ObjCInterfaceDecl *ID,
>> bool Weak) {
>> + CharUnits Align = CGF.getPointerAlign();
>> llvm::GlobalVariable * &Entry = MetaClassReferences[ID->getIdentifier()];
>> if (!Entry) {
>> llvm::SmallString<64> MetaClassName(getMetaclassSymbolPrefix());
>> @@ -6685,14 +6710,13 @@ llvm::Value *CGObjCNonFragileABIMac::Emi
>> Entry = new llvm::GlobalVariable(CGM.getModule(), ObjCTypes.ClassnfABIPtrTy,
>> false, llvm::GlobalValue::PrivateLinkage,
>> MetaClassGV, "OBJC_CLASSLIST_SUP_REFS_$_");
>> - Entry->setAlignment(
>> - CGM.getDataLayout().getABITypeAlignment(ObjCTypes.ClassnfABIPtrTy));
>> + Entry->setAlignment(Align.getQuantity());
>>
>> Entry->setSection("__DATA, __objc_superrefs, regular, no_dead_strip");
>> CGM.addCompilerUsedGlobal(Entry);
>> }
>>
>> - return CGF.Builder.CreateLoad(Entry);
>> + return CGF.Builder.CreateAlignedLoad(Entry, Align);
>> }
>>
>> /// GetClass - Return a reference to the class for the given interface
>> @@ -6727,14 +6751,15 @@ CGObjCNonFragileABIMac::GenerateMessageS
>> // ...
>> // Create and init a super structure; this is a (receiver, class)
>> // pair we will pass to objc_msgSendSuper.
>> - llvm::Value *ObjCSuper =
>> - CGF.CreateTempAlloca(ObjCTypes.SuperTy, "objc_super");
>> + Address ObjCSuper =
>> + CGF.CreateTempAlloca(ObjCTypes.SuperTy, CGF.getPointerAlign(),
>> + "objc_super");
>>
>> llvm::Value *ReceiverAsObject =
>> CGF.Builder.CreateBitCast(Receiver, ObjCTypes.ObjectPtrTy);
>> CGF.Builder.CreateStore(
>> ReceiverAsObject,
>> - CGF.Builder.CreateStructGEP(ObjCTypes.SuperTy, ObjCSuper, 0));
>> + CGF.Builder.CreateStructGEP(ObjCSuper, 0, CharUnits::Zero()));
>>
>> // If this is a class message the metaclass is passed as the target.
>> llvm::Value *Target;
>> @@ -6749,22 +6774,33 @@ CGObjCNonFragileABIMac::GenerateMessageS
>> CGM.getTypes().ConvertType(CGF.getContext().getObjCClassType());
>> Target = CGF.Builder.CreateBitCast(Target, ClassTy);
>> CGF.Builder.CreateStore(
>> - Target, CGF.Builder.CreateStructGEP(ObjCTypes.SuperTy, ObjCSuper, 1));
>> + Target, CGF.Builder.CreateStructGEP(ObjCSuper, 1, CGF.getPointerSize()));
>>
>> return (isVTableDispatchedSelector(Sel))
>> ? EmitVTableMessageSend(CGF, Return, ResultType, Sel,
>> - ObjCSuper, ObjCTypes.SuperPtrCTy,
>> + ObjCSuper.getPointer(), ObjCTypes.SuperPtrCTy,
>> true, CallArgs, Method)
>> : EmitMessageSend(CGF, Return, ResultType,
>> EmitSelector(CGF, Sel),
>> - ObjCSuper, ObjCTypes.SuperPtrCTy,
>> + ObjCSuper.getPointer(), ObjCTypes.SuperPtrCTy,
>> true, CallArgs, Method, ObjCTypes);
>> }
>>
>> llvm::Value *CGObjCNonFragileABIMac::EmitSelector(CodeGenFunction &CGF,
>> - Selector Sel, bool lval) {
>> + Selector Sel) {
>> + Address Addr = EmitSelectorAddr(CGF, Sel);
>> +
>> + llvm::LoadInst* LI = CGF.Builder.CreateLoad(Addr);
>> + LI->setMetadata(CGM.getModule().getMDKindID("invariant.load"),
>> + llvm::MDNode::get(VMContext, None));
>> + return LI;
>> +}
>> +
>> +Address CGObjCNonFragileABIMac::EmitSelectorAddr(CodeGenFunction &CGF,
>> + Selector Sel) {
>> llvm::GlobalVariable *&Entry = SelectorReferences[Sel];
>>
>> + CharUnits Align = CGF.getPointerAlign();
>> if (!Entry) {
>> llvm::Constant *Casted =
>> llvm::ConstantExpr::getBitCast(GetMethodVarName(Sel),
>> @@ -6774,23 +6810,19 @@ llvm::Value *CGObjCNonFragileABIMac::Emi
>> Casted, "OBJC_SELECTOR_REFERENCES_");
>> Entry->setExternallyInitialized(true);
>> Entry->setSection("__DATA, __objc_selrefs, literal_pointers, no_dead_strip");
>> + Entry->setAlignment(Align.getQuantity());
>> CGM.addCompilerUsedGlobal(Entry);
>> }
>>
>> - if (lval)
>> - return Entry;
>> - llvm::LoadInst* LI = CGF.Builder.CreateLoad(Entry);
>> -
>> - LI->setMetadata(CGM.getModule().getMDKindID("invariant.load"),
>> - llvm::MDNode::get(VMContext, None));
>> - return LI;
>> + return Address(Entry, Align);
>> }
>> +
>> /// EmitObjCIvarAssign - Code gen for assigning to a __strong object.
>> /// objc_assign_ivar (id src, id *dst, ptrdiff_t)
>> ///
>> void CGObjCNonFragileABIMac::EmitObjCIvarAssign(CodeGen::CodeGenFunction &CGF,
>> llvm::Value *src,
>> - llvm::Value *dst,
>> + Address dst,
>> llvm::Value *ivarOffset) {
>> llvm::Type * SrcTy = src->getType();
>> if (!isa<llvm::PointerType>(SrcTy)) {
>> @@ -6802,7 +6834,7 @@ void CGObjCNonFragileABIMac::EmitObjCIva
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst, ivarOffset };
>> + llvm::Value *args[] = { src, dst.getPointer(), ivarOffset };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignIvarFn(), args);
>> }
>>
>> @@ -6811,7 +6843,7 @@ void CGObjCNonFragileABIMac::EmitObjCIva
>> ///
>> void CGObjCNonFragileABIMac::EmitObjCStrongCastAssign(
>> CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) {
>> + llvm::Value *src, Address dst) {
>> llvm::Type * SrcTy = src->getType();
>> if (!isa<llvm::PointerType>(SrcTy)) {
>> unsigned Size = CGM.getDataLayout().getTypeAllocSize(SrcTy);
>> @@ -6822,19 +6854,19 @@ void CGObjCNonFragileABIMac::EmitObjCStr
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst };
>> + llvm::Value *args[] = { src, dst.getPointer() };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignStrongCastFn(),
>> args, "weakassign");
>> }
>>
>> void CGObjCNonFragileABIMac::EmitGCMemmoveCollectable(
>> CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *DestPtr,
>> - llvm::Value *SrcPtr,
>> + Address DestPtr,
>> + Address SrcPtr,
>> llvm::Value *Size) {
>> SrcPtr = CGF.Builder.CreateBitCast(SrcPtr, ObjCTypes.Int8PtrTy);
>> DestPtr = CGF.Builder.CreateBitCast(DestPtr, ObjCTypes.Int8PtrTy);
>> - llvm::Value *args[] = { DestPtr, SrcPtr, Size };
>> + llvm::Value *args[] = { DestPtr.getPointer(), SrcPtr.getPointer(), Size };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.GcMemmoveCollectableFn(), args);
>> }
>>
>> @@ -6843,13 +6875,12 @@ void CGObjCNonFragileABIMac::EmitGCMemmo
>> ///
>> llvm::Value * CGObjCNonFragileABIMac::EmitObjCWeakRead(
>> CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *AddrWeakObj) {
>> - llvm::Type* DestTy =
>> - cast<llvm::PointerType>(AddrWeakObj->getType())->getElementType();
>> + Address AddrWeakObj) {
>> + llvm::Type *DestTy = AddrWeakObj.getElementType();
>> AddrWeakObj = CGF.Builder.CreateBitCast(AddrWeakObj, ObjCTypes.PtrObjectPtrTy);
>> llvm::Value *read_weak =
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcReadWeakFn(),
>> - AddrWeakObj, "weakread");
>> + AddrWeakObj.getPointer(), "weakread");
>> read_weak = CGF.Builder.CreateBitCast(read_weak, DestTy);
>> return read_weak;
>> }
>> @@ -6858,7 +6889,7 @@ llvm::Value * CGObjCNonFragileABIMac::Em
>> /// objc_assign_weak (id src, id *dst)
>> ///
>> void CGObjCNonFragileABIMac::EmitObjCWeakAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst) {
>> + llvm::Value *src, Address dst) {
>> llvm::Type * SrcTy = src->getType();
>> if (!isa<llvm::PointerType>(SrcTy)) {
>> unsigned Size = CGM.getDataLayout().getTypeAllocSize(SrcTy);
>> @@ -6869,7 +6900,7 @@ void CGObjCNonFragileABIMac::EmitObjCWea
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst };
>> + llvm::Value *args[] = { src, dst.getPointer() };
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignWeakFn(),
>> args, "weakassign");
>> }
>> @@ -6878,7 +6909,7 @@ void CGObjCNonFragileABIMac::EmitObjCWea
>> /// objc_assign_global (id src, id *dst)
>> ///
>> void CGObjCNonFragileABIMac::EmitObjCGlobalAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dst,
>> + llvm::Value *src, Address dst,
>> bool threadlocal) {
>> llvm::Type * SrcTy = src->getType();
>> if (!isa<llvm::PointerType>(SrcTy)) {
>> @@ -6890,7 +6921,7 @@ void CGObjCNonFragileABIMac::EmitObjCGlo
>> }
>> src = CGF.Builder.CreateBitCast(src, ObjCTypes.ObjectPtrTy);
>> dst = CGF.Builder.CreateBitCast(dst, ObjCTypes.PtrObjectPtrTy);
>> - llvm::Value *args[] = { src, dst };
>> + llvm::Value *args[] = { src, dst.getPointer() };
>> if (!threadlocal)
>> CGF.EmitNounwindRuntimeCall(ObjCTypes.getGcAssignGlobalFn(),
>> args, "globalassign");
>>
>> Modified: cfe/trunk/lib/CodeGen/CGObjCRuntime.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGObjCRuntime.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGObjCRuntime.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGObjCRuntime.cpp Tue Sep 8 03:05:57 2015
>> @@ -136,12 +136,13 @@ LValue CGObjCRuntime::EmitValueForIvarAt
>> CGF.CGM.getContext().toBits(StorageSize),
>> CharUnits::fromQuantity(0)));
>>
>> - V = CGF.Builder.CreateBitCast(V,
>> - llvm::Type::getIntNPtrTy(CGF.getLLVMContext(),
>> + Address Addr(V, Alignment);
>> + Addr = CGF.Builder.CreateElementBitCast(Addr,
>> + llvm::Type::getIntNTy(CGF.getLLVMContext(),
>> Info->StorageSize));
>> - return LValue::MakeBitfield(V, *Info,
>> + return LValue::MakeBitfield(Addr, *Info,
>> IvarTy.withCVRQualifiers(CVRQualifiers),
>> - Alignment);
>> + AlignmentSource::Decl);
>> }
>>
>> namespace {
>> @@ -256,7 +257,7 @@ void CGObjCRuntime::EmitTryCatchStmt(Cod
>>
>> CGF.EmitAutoVarDecl(*CatchParam);
>>
>> - llvm::Value *CatchParamAddr = CGF.GetAddrOfLocalVar(CatchParam);
>> + Address CatchParamAddr = CGF.GetAddrOfLocalVar(CatchParam);
>>
>> switch (CatchParam->getType().getQualifiers().getObjCLifetime()) {
>> case Qualifiers::OCL_Strong:
>>
>> Modified: cfe/trunk/lib/CodeGen/CGObjCRuntime.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGObjCRuntime.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGObjCRuntime.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGObjCRuntime.h Tue Sep 8 03:05:57 2015
>> @@ -116,11 +116,16 @@ public:
>> /// this compilation unit with the runtime library.
>> virtual llvm::Function *ModuleInitFunction() = 0;
>>
>> - /// Get a selector for the specified name and type values. The
>> - /// return value should have the LLVM type for pointer-to
>> + /// Get a selector for the specified name and type values.
>> + /// The result should have the LLVM type for ASTContext::getObjCSelType().
>> + virtual llvm::Value *GetSelector(CodeGenFunction &CGF, Selector Sel) = 0;
>> +
>> + /// Get the address of a selector for the specified name and type values.
>> + /// This is a rarely-used language extension, but sadly it exists.
>> + ///
>> + /// The result should have the LLVM type for a pointer to
>> /// ASTContext::getObjCSelType().
>> - virtual llvm::Value *GetSelector(CodeGenFunction &CGF,
>> - Selector Sel, bool lval=false) = 0;
>> + virtual Address GetAddrOfSelector(CodeGenFunction &CGF, Selector Sel) = 0;
>>
>> /// Get a typed selector.
>> virtual llvm::Value *GetSelector(CodeGenFunction &CGF,
>> @@ -133,7 +138,7 @@ public:
>> virtual llvm::Constant *GetEHType(QualType T) = 0;
>>
>> /// Generate a constant string object.
>> - virtual llvm::Constant *GenerateConstantString(const StringLiteral *) = 0;
>> + virtual ConstantAddress GenerateConstantString(const StringLiteral *) = 0;
>>
>> /// Generate a category. A category contains a list of methods (and
>> /// accompanying metadata) and a list of protocols.
>> @@ -238,17 +243,17 @@ public:
>> const ObjCAtThrowStmt &S,
>> bool ClearInsertionPoint=true) = 0;
>> virtual llvm::Value *EmitObjCWeakRead(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *AddrWeakObj) = 0;
>> + Address AddrWeakObj) = 0;
>> virtual void EmitObjCWeakAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest) = 0;
>> + llvm::Value *src, Address dest) = 0;
>> virtual void EmitObjCGlobalAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest,
>> + llvm::Value *src, Address dest,
>> bool threadlocal=false) = 0;
>> virtual void EmitObjCIvarAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest,
>> + llvm::Value *src, Address dest,
>> llvm::Value *ivarOffset) = 0;
>> virtual void EmitObjCStrongCastAssign(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *src, llvm::Value *dest) = 0;
>> + llvm::Value *src, Address dest) = 0;
>>
>> virtual LValue EmitObjCValueForIvar(CodeGen::CodeGenFunction &CGF,
>> QualType ObjectTy,
>> @@ -259,8 +264,8 @@ public:
>> const ObjCInterfaceDecl *Interface,
>> const ObjCIvarDecl *Ivar) = 0;
>> virtual void EmitGCMemmoveCollectable(CodeGen::CodeGenFunction &CGF,
>> - llvm::Value *DestPtr,
>> - llvm::Value *SrcPtr,
>> + Address DestPtr,
>> + Address SrcPtr,
>> llvm::Value *Size) = 0;
>> virtual llvm::Constant *BuildGCBlockLayout(CodeGen::CodeGenModule &CGM,
>> const CodeGen::CGBlockInfo &blockInfo) = 0;
>>
>> Modified: cfe/trunk/lib/CodeGen/CGOpenMPRuntime.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGOpenMPRuntime.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGOpenMPRuntime.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGOpenMPRuntime.cpp Tue Sep 8 03:05:57 2015
>> @@ -233,9 +233,8 @@ public:
>>
>> LValue CGOpenMPRegionInfo::getThreadIDVariableLValue(CodeGenFunction &CGF) {
>> return CGF.MakeNaturalAlignAddrLValue(
>> - CGF.Builder.CreateAlignedLoad(
>> - CGF.GetAddrOfLocalVar(getThreadIDVariable()),
>> - CGF.PointerAlignInBytes),
>> + CGF.Builder.CreateLoad(
>> + CGF.GetAddrOfLocalVar(getThreadIDVariable())),
>> getThreadIDVariable()
>> ->getType()
>> ->castAs<PointerType>()
>> @@ -258,7 +257,7 @@ void CGOpenMPRegionInfo::EmitBody(CodeGe
>>
>> LValue CGOpenMPTaskOutlinedRegionInfo::getThreadIDVariableLValue(
>> CodeGenFunction &CGF) {
>> - return CGF.MakeNaturalAlignAddrLValue(
>> + return CGF.MakeAddrLValue(
>> CGF.GetAddrOfLocalVar(getThreadIDVariable()),
>> getThreadIDVariable()->getType());
>> }
>> @@ -280,6 +279,25 @@ void CGOpenMPRuntime::clear() {
>> InternalVars.clear();
>> }
>>
>> +// Layout information for ident_t.
>> +static CharUnits getIdentAlign(CodeGenModule &CGM) {
>> + return CGM.getPointerAlign();
>> +}
>> +static CharUnits getIdentSize(CodeGenModule &CGM) {
>> + assert((4 * CGM.getPointerSize()).isMultipleOf(CGM.getPointerAlign()));
>> + return CharUnits::fromQuantity(16) + CGM.getPointerSize();
>> +}
>> +static CharUnits getOffsetOfIdentField(CGOpenMPRuntime::IdentFieldIndex Field) {
>> + // All the fields except the last are i32, so this works beautifully.
>> + return unsigned(Field) * CharUnits::fromQuantity(4);
>> +}
>> +static Address createIdentFieldGEP(CodeGenFunction &CGF, Address Addr,
>> + CGOpenMPRuntime::IdentFieldIndex Field,
>> + const llvm::Twine &Name = "") {
>> + auto Offset = getOffsetOfIdentField(Field);
>> + return CGF.Builder.CreateStructGEP(Addr, Field, Offset, Name);
>> +}
>> +
>> llvm::Value *CGOpenMPRuntime::emitParallelOutlinedFunction(
>> const OMPExecutableDirective &D, const VarDecl *ThreadIDVar,
>> OpenMPDirectiveKind InnermostKind, const RegionCodeGenTy &CodeGen) {
>> @@ -305,8 +323,8 @@ llvm::Value *CGOpenMPRuntime::emitTaskOu
>> return CGF.GenerateCapturedStmtFunction(*CS);
>> }
>>
>> -llvm::Value *
>> -CGOpenMPRuntime::getOrCreateDefaultLocation(OpenMPLocationFlags Flags) {
>> +Address CGOpenMPRuntime::getOrCreateDefaultLocation(OpenMPLocationFlags Flags) {
>> + CharUnits Align = getIdentAlign(CGM);
>> llvm::Value *Entry = OpenMPDefaultLocMap.lookup(Flags);
>> if (!Entry) {
>> if (!DefaultOpenMPPSource) {
>> @@ -315,7 +333,7 @@ CGOpenMPRuntime::getOrCreateDefaultLocat
>> // Taken from
>> // http://llvm.org/svn/llvm-project/openmp/trunk/runtime/src/kmp_str.c
>> DefaultOpenMPPSource =
>> - CGM.GetAddrOfConstantCString(";unknown;unknown;0;0;;");
>> + CGM.GetAddrOfConstantCString(";unknown;unknown;0;0;;").getPointer();
>> DefaultOpenMPPSource =
>> llvm::ConstantExpr::getBitCast(DefaultOpenMPPSource, CGM.Int8PtrTy);
>> }
>> @@ -323,6 +341,7 @@ CGOpenMPRuntime::getOrCreateDefaultLocat
>> CGM.getModule(), IdentTy, /*isConstant*/ true,
>> llvm::GlobalValue::PrivateLinkage, /*Initializer*/ nullptr);
>> DefaultOpenMPLocation->setUnnamedAddr(true);
>> + DefaultOpenMPLocation->setAlignment(Align.getQuantity());
>>
>> llvm::Constant *Zero = llvm::ConstantInt::get(CGM.Int32Ty, 0, true);
>> llvm::Constant *Values[] = {Zero,
>> @@ -330,10 +349,9 @@ CGOpenMPRuntime::getOrCreateDefaultLocat
>> Zero, Zero, DefaultOpenMPPSource};
>> llvm::Constant *Init = llvm::ConstantStruct::get(IdentTy, Values);
>> DefaultOpenMPLocation->setInitializer(Init);
>> - OpenMPDefaultLocMap[Flags] = DefaultOpenMPLocation;
>> - return DefaultOpenMPLocation;
>> + OpenMPDefaultLocMap[Flags] = Entry = DefaultOpenMPLocation;
>> }
>> - return Entry;
>> + return Address(Entry, Align);
>> }
>>
>> llvm::Value *CGOpenMPRuntime::emitUpdateLocation(CodeGenFunction &CGF,
>> @@ -342,34 +360,33 @@ llvm::Value *CGOpenMPRuntime::emitUpdate
>> // If no debug info is generated - return global default location.
>> if (CGM.getCodeGenOpts().getDebugInfo() == CodeGenOptions::NoDebugInfo ||
>> Loc.isInvalid())
>> - return getOrCreateDefaultLocation(Flags);
>> + return getOrCreateDefaultLocation(Flags).getPointer();
>>
>> assert(CGF.CurFn && "No function in current CodeGenFunction.");
>>
>> - llvm::Value *LocValue = nullptr;
>> + Address LocValue = Address::invalid();
>> auto I = OpenMPLocThreadIDMap.find(CGF.CurFn);
>> if (I != OpenMPLocThreadIDMap.end())
>> - LocValue = I->second.DebugLoc;
>> + LocValue = Address(I->second.DebugLoc, getIdentAlign(CGF.CGM));
>> +
>> // OpenMPLocThreadIDMap may have null DebugLoc and non-null ThreadID, if
>> // GetOpenMPThreadID was called before this routine.
>> - if (LocValue == nullptr) {
>> + if (!LocValue.isValid()) {
>> // Generate "ident_t .kmpc_loc.addr;"
>> - llvm::AllocaInst *AI = CGF.CreateTempAlloca(IdentTy, ".kmpc_loc.addr");
>> - AI->setAlignment(CGM.getDataLayout().getPrefTypeAlignment(IdentTy));
>> + Address AI = CGF.CreateTempAlloca(IdentTy, getIdentAlign(CGF.CGM),
>> + ".kmpc_loc.addr");
>> auto &Elem = OpenMPLocThreadIDMap.FindAndConstruct(CGF.CurFn);
>> - Elem.second.DebugLoc = AI;
>> + Elem.second.DebugLoc = AI.getPointer();
>> LocValue = AI;
>>
>> CGBuilderTy::InsertPointGuard IPG(CGF.Builder);
>> CGF.Builder.SetInsertPoint(CGF.AllocaInsertPt);
>> CGF.Builder.CreateMemCpy(LocValue, getOrCreateDefaultLocation(Flags),
>> - llvm::ConstantExpr::getSizeOf(IdentTy),
>> - CGM.PointerAlignInBytes);
>> + CGM.getSize(getIdentSize(CGF.CGM)));
>> }
>>
>> // char **psource = &.kmpc_loc_<flags>.addr.psource;
>> - auto *PSource = CGF.Builder.CreateConstInBoundsGEP2_32(IdentTy, LocValue, 0,
>> - IdentField_PSource);
>> + Address PSource = createIdentFieldGEP(CGF, LocValue, IdentField_PSource);
>>
>> auto OMPDebugLoc = OpenMPDebugLocMap.lookup(Loc.getRawEncoding());
>> if (OMPDebugLoc == nullptr) {
>> @@ -389,7 +406,9 @@ llvm::Value *CGOpenMPRuntime::emitUpdate
>> // *psource = ";<File>;<Function>;<Line>;<Column>;;";
>> CGF.Builder.CreateStore(OMPDebugLoc, PSource);
>>
>> - return LocValue;
>> + // Our callers always pass this to a runtime function, so for
>> + // convenience, go ahead and return a naked pointer.
>> + return LocValue.getPointer();
>> }
>>
>> llvm::Value *CGOpenMPRuntime::getThreadID(CodeGenFunction &CGF,
>> @@ -939,25 +958,27 @@ CGOpenMPRuntime::getOrCreateThreadPrivat
>> Twine(CGM.getMangledName(VD)) + ".cache.");
>> }
>>
>> -llvm::Value *CGOpenMPRuntime::getAddrOfThreadPrivate(CodeGenFunction &CGF,
>> - const VarDecl *VD,
>> - llvm::Value *VDAddr,
>> - SourceLocation Loc) {
>> +Address CGOpenMPRuntime::getAddrOfThreadPrivate(CodeGenFunction &CGF,
>> + const VarDecl *VD,
>> + Address VDAddr,
>> + SourceLocation Loc) {
>> if (CGM.getLangOpts().OpenMPUseTLS &&
>> CGM.getContext().getTargetInfo().isTLSSupported())
>> return VDAddr;
>>
>> - auto VarTy = VDAddr->getType()->getPointerElementType();
>> + auto VarTy = VDAddr.getElementType();
>> llvm::Value *Args[] = {emitUpdateLocation(CGF, Loc), getThreadID(CGF, Loc),
>> - CGF.Builder.CreatePointerCast(VDAddr, CGM.Int8PtrTy),
>> + CGF.Builder.CreatePointerCast(VDAddr.getPointer(),
>> + CGM.Int8PtrTy),
>> CGM.getSize(CGM.GetTargetTypeStoreSize(VarTy)),
>> getOrCreateThreadPrivateCache(VD)};
>> - return CGF.EmitRuntimeCall(
>> - createRuntimeFunction(OMPRTL__kmpc_threadprivate_cached), Args);
>> + return Address(CGF.EmitRuntimeCall(
>> + createRuntimeFunction(OMPRTL__kmpc_threadprivate_cached), Args),
>> + VDAddr.getAlignment());
>> }
>>
>> void CGOpenMPRuntime::emitThreadPrivateVarInit(
>> - CodeGenFunction &CGF, llvm::Value *VDAddr, llvm::Value *Ctor,
>> + CodeGenFunction &CGF, Address VDAddr, llvm::Value *Ctor,
>> llvm::Value *CopyCtor, llvm::Value *Dtor, SourceLocation Loc) {
>> // Call kmp_int32 __kmpc_global_thread_num(&loc) to init OpenMP runtime
>> // library.
>> @@ -967,14 +988,15 @@ void CGOpenMPRuntime::emitThreadPrivateV
>> // Call __kmpc_threadprivate_register(&loc, &var, ctor, cctor/*NULL*/, dtor)
>> // to register constructor/destructor for variable.
>> llvm::Value *Args[] = {OMPLoc,
>> - CGF.Builder.CreatePointerCast(VDAddr, CGM.VoidPtrTy),
>> + CGF.Builder.CreatePointerCast(VDAddr.getPointer(),
>> + CGM.VoidPtrTy),
>> Ctor, CopyCtor, Dtor};
>> CGF.EmitRuntimeCall(
>> createRuntimeFunction(OMPRTL__kmpc_threadprivate_register), Args);
>> }
>>
>> llvm::Function *CGOpenMPRuntime::emitThreadPrivateVarDefinition(
>> - const VarDecl *VD, llvm::Value *VDAddr, SourceLocation Loc,
>> + const VarDecl *VD, Address VDAddr, SourceLocation Loc,
>> bool PerformInit, CodeGenFunction *CGF) {
>> if (CGM.getLangOpts().OpenMPUseTLS &&
>> CGM.getContext().getTargetInfo().isTLSSupported())
>> @@ -1005,17 +1027,15 @@ llvm::Function *CGOpenMPRuntime::emitThr
>> CtorCGF.StartFunction(GlobalDecl(), CGM.getContext().VoidPtrTy, Fn, FI,
>> Args, SourceLocation());
>> auto ArgVal = CtorCGF.EmitLoadOfScalar(
>> - CtorCGF.GetAddrOfLocalVar(&Dst),
>> - /*Volatile=*/false, CGM.PointerAlignInBytes,
>> + CtorCGF.GetAddrOfLocalVar(&Dst), /*Volatile=*/false,
>> CGM.getContext().VoidPtrTy, Dst.getLocation());
>> - auto Arg = CtorCGF.Builder.CreatePointerCast(
>> - ArgVal,
>> - CtorCGF.ConvertTypeForMem(CGM.getContext().getPointerType(ASTTy)));
>> + Address Arg = Address(ArgVal, VDAddr.getAlignment());
>> + Arg = CtorCGF.Builder.CreateElementBitCast(Arg,
>> + CtorCGF.ConvertTypeForMem(ASTTy));
>> CtorCGF.EmitAnyExprToMem(Init, Arg, Init->getType().getQualifiers(),
>> /*IsInitializer=*/true);
>> ArgVal = CtorCGF.EmitLoadOfScalar(
>> - CtorCGF.GetAddrOfLocalVar(&Dst),
>> - /*Volatile=*/false, CGM.PointerAlignInBytes,
>> + CtorCGF.GetAddrOfLocalVar(&Dst), /*Volatile=*/false,
>> CGM.getContext().VoidPtrTy, Dst.getLocation());
>> CtorCGF.Builder.CreateStore(ArgVal, CtorCGF.ReturnValue);
>> CtorCGF.FinishFunction();
>> @@ -1040,9 +1060,8 @@ llvm::Function *CGOpenMPRuntime::emitThr
>> SourceLocation());
>> auto ArgVal = DtorCGF.EmitLoadOfScalar(
>> DtorCGF.GetAddrOfLocalVar(&Dst),
>> - /*Volatile=*/false, CGM.PointerAlignInBytes,
>> - CGM.getContext().VoidPtrTy, Dst.getLocation());
>> - DtorCGF.emitDestroy(ArgVal, ASTTy,
>> + /*Volatile=*/false, CGM.getContext().VoidPtrTy, Dst.getLocation());
>> + DtorCGF.emitDestroy(Address(ArgVal, VDAddr.getAlignment()), ASTTy,
>> DtorCGF.getDestroyer(ASTTy.isDestructedType()),
>> DtorCGF.needsEHCleanup(ASTTy.isDestructedType()));
>> DtorCGF.FinishFunction();
>> @@ -1149,7 +1168,7 @@ static void emitOMPIfClause(CodeGenFunct
>>
>> void CGOpenMPRuntime::emitParallelCall(CodeGenFunction &CGF, SourceLocation Loc,
>> llvm::Value *OutlinedFn,
>> - llvm::Value *CapturedStruct,
>> + Address CapturedStruct,
>> const Expr *IfCond) {
>> auto *RTLoc = emitUpdateLocation(CGF, Loc);
>> auto &&ThenGen =
>> @@ -1162,7 +1181,7 @@ void CGOpenMPRuntime::emitParallelCall(C
>> 1), // Number of arguments after 'microtask' argument
>> // (there is only one additional argument - 'context')
>> CGF.Builder.CreateBitCast(OutlinedFn, getKmpc_MicroPointerTy()),
>> - CGF.EmitCastToVoidPtr(CapturedStruct)};
>> + CGF.EmitCastToVoidPtr(CapturedStruct.getPointer())};
>> auto RTLFn = createRuntimeFunction(OMPRTL__kmpc_fork_call);
>> CGF.EmitRuntimeCall(RTLFn, Args);
>> };
>> @@ -1177,11 +1196,15 @@ void CGOpenMPRuntime::emitParallelCall(C
>>
>> // OutlinedFn(>id, &zero, CapturedStruct);
>> auto ThreadIDAddr = emitThreadIDAddress(CGF, Loc);
>> - auto Int32Ty = CGF.getContext().getIntTypeForBitwidth(/*DestWidth*/ 32,
>> - /*Signed*/ true);
>> - auto ZeroAddr = CGF.CreateMemTemp(Int32Ty, /*Name*/ ".zero.addr");
>> + Address ZeroAddr =
>> + CGF.CreateTempAlloca(CGF.Int32Ty, CharUnits::fromQuantity(4),
>> + /*Name*/ ".zero.addr");
>> CGF.InitTempAlloca(ZeroAddr, CGF.Builder.getInt32(/*C*/ 0));
>> - llvm::Value *OutlinedFnArgs[] = {ThreadIDAddr, ZeroAddr, CapturedStruct};
>> + llvm::Value *OutlinedFnArgs[] = {
>> + ThreadIDAddr.getPointer(),
>> + ZeroAddr.getPointer(),
>> + CapturedStruct.getPointer()
>> + };
>> CGF.EmitCallOrInvoke(OutlinedFn, OutlinedFnArgs);
>>
>> // __kmpc_end_serialized_parallel(&Loc, GTid);
>> @@ -1203,8 +1226,8 @@ void CGOpenMPRuntime::emitParallelCall(C
>> // regular serial code region, get thread ID by calling kmp_int32
>> // kmpc_global_thread_num(ident_t *loc), stash this thread ID in a temporary and
>> // return the address of that temp.
>> -llvm::Value *CGOpenMPRuntime::emitThreadIDAddress(CodeGenFunction &CGF,
>> - SourceLocation Loc) {
>> +Address CGOpenMPRuntime::emitThreadIDAddress(CodeGenFunction &CGF,
>> + SourceLocation Loc) {
>> if (auto OMPRegionInfo =
>> dyn_cast_or_null<CGOpenMPRegionInfo>(CGF.CapturedStmtInfo))
>> if (OMPRegionInfo->getThreadIDVariable())
>> @@ -1215,7 +1238,7 @@ llvm::Value *CGOpenMPRuntime::emitThread
>> CGF.getContext().getIntTypeForBitwidth(/*DestWidth*/ 32, /*Signed*/ true);
>> auto ThreadIDTemp = CGF.CreateMemTemp(Int32Ty, /*Name*/ ".threadid_temp.");
>> CGF.EmitStoreOfScalar(ThreadID,
>> - CGF.MakeNaturalAlignAddrLValue(ThreadIDTemp, Int32Ty));
>> + CGF.MakeAddrLValue(ThreadIDTemp, Int32Ty));
>>
>> return ThreadIDTemp;
>> }
>> @@ -1353,6 +1376,22 @@ void CGOpenMPRuntime::emitTaskgroupRegio
>> }
>> }
>>
>> +/// Given an array of pointers to variables, project the address of a
>> +/// given variable.
>> +static Address emitAddrOfVarFromArray(CodeGenFunction &CGF,
>> + Address Array, unsigned Index,
>> + const VarDecl *Var) {
>> + // Pull out the pointer to the variable.
>> + Address PtrAddr =
>> + CGF.Builder.CreateConstArrayGEP(Array, Index, CGF.getPointerSize());
>> + llvm::Value *Ptr = CGF.Builder.CreateLoad(PtrAddr);
>> +
>> + Address Addr = Address(Ptr, CGF.getContext().getDeclAlign(Var));
>> + Addr = CGF.Builder.CreateElementBitCast(Addr,
>> + CGF.ConvertTypeForMem(Var->getType()));
>> + return Addr;
>> +}
>> +
>> static llvm::Value *emitCopyprivateCopyFunction(
>> CodeGenModule &CGM, llvm::Type *ArgsType,
>> ArrayRef<const Expr *> CopyprivateVars, ArrayRef<const Expr *> DestExprs,
>> @@ -1377,35 +1416,26 @@ static llvm::Value *emitCopyprivateCopyF
>> CGF.StartFunction(GlobalDecl(), C.VoidTy, Fn, CGFI, Args);
>> // Dest = (void*[n])(LHSArg);
>> // Src = (void*[n])(RHSArg);
>> - auto *LHS = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(CGF.GetAddrOfLocalVar(&LHSArg),
>> - CGF.PointerAlignInBytes),
>> - ArgsType);
>> - auto *RHS = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(CGF.GetAddrOfLocalVar(&RHSArg),
>> - CGF.PointerAlignInBytes),
>> - ArgsType);
>> + Address LHS(CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> + CGF.Builder.CreateLoad(CGF.GetAddrOfLocalVar(&LHSArg)),
>> + ArgsType), CGF.getPointerAlign());
>> + Address RHS(CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> + CGF.Builder.CreateLoad(CGF.GetAddrOfLocalVar(&RHSArg)),
>> + ArgsType), CGF.getPointerAlign());
>> // *(Type0*)Dst[0] = *(Type0*)Src[0];
>> // *(Type1*)Dst[1] = *(Type1*)Src[1];
>> // ...
>> // *(Typen*)Dst[n] = *(Typen*)Src[n];
>> for (unsigned I = 0, E = AssignmentOps.size(); I < E; ++I) {
>> - auto *DestAddr = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(
>> - CGF.Builder.CreateStructGEP(nullptr, LHS, I),
>> - CGM.PointerAlignInBytes),
>> - CGF.ConvertTypeForMem(C.getPointerType(SrcExprs[I]->getType())));
>> - auto *SrcAddr = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(
>> - CGF.Builder.CreateStructGEP(nullptr, RHS, I),
>> - CGM.PointerAlignInBytes),
>> - CGF.ConvertTypeForMem(C.getPointerType(SrcExprs[I]->getType())));
>> + auto DestVar = cast<VarDecl>(cast<DeclRefExpr>(DestExprs[I])->getDecl());
>> + Address DestAddr = emitAddrOfVarFromArray(CGF, LHS, I, DestVar);
>> +
>> + auto SrcVar = cast<VarDecl>(cast<DeclRefExpr>(SrcExprs[I])->getDecl());
>> + Address SrcAddr = emitAddrOfVarFromArray(CGF, RHS, I, SrcVar);
>> +
>> auto *VD = cast<DeclRefExpr>(CopyprivateVars[I])->getDecl();
>> QualType Type = VD->getType();
>> - CGF.EmitOMPCopy(CGF, Type, DestAddr, SrcAddr,
>> - cast<VarDecl>(cast<DeclRefExpr>(DestExprs[I])->getDecl()),
>> - cast<VarDecl>(cast<DeclRefExpr>(SrcExprs[I])->getDecl()),
>> - AssignmentOps[I]);
>> + CGF.EmitOMPCopy(Type, DestAddr, SrcAddr, DestVar, SrcVar, AssignmentOps[I]);
>> }
>> CGF.FinishFunction();
>> return Fn;
>> @@ -1431,13 +1461,12 @@ void CGOpenMPRuntime::emitSingleRegion(C
>> // call __kmpc_copyprivate(ident_t *, gtid, <buf_size>, <copyprivate list>,
>> // <copy_func>, did_it);
>>
>> - llvm::AllocaInst *DidIt = nullptr;
>> + Address DidIt = Address::invalid();
>> if (!CopyprivateVars.empty()) {
>> // int32 did_it = 0;
>> auto KmpInt32Ty = C.getIntTypeForBitwidth(/*DestWidth=*/32, /*Signed=*/1);
>> DidIt = CGF.CreateMemTemp(KmpInt32Ty, ".omp.copyprivate.did_it");
>> - CGF.Builder.CreateAlignedStore(CGF.Builder.getInt32(0), DidIt,
>> - DidIt->getAlignment());
>> + CGF.Builder.CreateStore(CGF.Builder.getInt32(0), DidIt);
>> }
>> // Prepare arguments and build a call to __kmpc_single
>> llvm::Value *Args[] = {emitUpdateLocation(CGF, Loc), getThreadID(CGF, Loc)};
>> @@ -1452,29 +1481,28 @@ void CGOpenMPRuntime::emitSingleRegion(C
>> NormalAndEHCleanup, createRuntimeFunction(OMPRTL__kmpc_end_single),
>> llvm::makeArrayRef(Args));
>> SingleOpGen(CGF);
>> - if (DidIt) {
>> + if (DidIt.isValid()) {
>> // did_it = 1;
>> - CGF.Builder.CreateAlignedStore(CGF.Builder.getInt32(1), DidIt,
>> - DidIt->getAlignment());
>> + CGF.Builder.CreateStore(CGF.Builder.getInt32(1), DidIt);
>> }
>> });
>> // call __kmpc_copyprivate(ident_t *, gtid, <buf_size>, <copyprivate list>,
>> // <copy_func>, did_it);
>> - if (DidIt) {
>> + if (DidIt.isValid()) {
>> llvm::APInt ArraySize(/*unsigned int numBits=*/32, CopyprivateVars.size());
>> auto CopyprivateArrayTy =
>> C.getConstantArrayType(C.VoidPtrTy, ArraySize, ArrayType::Normal,
>> /*IndexTypeQuals=*/0);
>> // Create a list of all private variables for copyprivate.
>> - auto *CopyprivateList =
>> + Address CopyprivateList =
>> CGF.CreateMemTemp(CopyprivateArrayTy, ".omp.copyprivate.cpr_list");
>> for (unsigned I = 0, E = CopyprivateVars.size(); I < E; ++I) {
>> - auto *Elem = CGF.Builder.CreateStructGEP(
>> - CopyprivateList->getAllocatedType(), CopyprivateList, I);
>> - CGF.Builder.CreateAlignedStore(
>> + Address Elem = CGF.Builder.CreateConstArrayGEP(
>> + CopyprivateList, I, CGF.getPointerSize());
>> + CGF.Builder.CreateStore(
>> CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.EmitLValue(CopyprivateVars[I]).getAddress(), CGF.VoidPtrTy),
>> - Elem, CGM.PointerAlignInBytes);
>> + CGF.EmitLValue(CopyprivateVars[I]).getPointer(), CGF.VoidPtrTy),
>> + Elem);
>> }
>> // Build function that copies private values from single region to all other
>> // threads in the corresponding parallel region.
>> @@ -1483,15 +1511,15 @@ void CGOpenMPRuntime::emitSingleRegion(C
>> CopyprivateVars, SrcExprs, DstExprs, AssignmentOps);
>> auto *BufSize = llvm::ConstantInt::get(
>> CGM.SizeTy, C.getTypeSizeInChars(CopyprivateArrayTy).getQuantity());
>> - auto *CL = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(CopyprivateList,
>> - CGF.VoidPtrTy);
>> - auto *DidItVal =
>> - CGF.Builder.CreateAlignedLoad(DidIt, CGF.PointerAlignInBytes);
>> + Address CL =
>> + CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(CopyprivateList,
>> + CGF.VoidPtrTy);
>> + auto *DidItVal = CGF.Builder.CreateLoad(DidIt);
>> llvm::Value *Args[] = {
>> emitUpdateLocation(CGF, Loc), // ident_t *<loc>
>> getThreadID(CGF, Loc), // i32 <gtid>
>> BufSize, // size_t <buf_size>
>> - CL, // void *<copyprivate list>
>> + CL.getPointer(), // void *<copyprivate list>
>> CpyFn, // void (*) (void *, void *) <copy_func>
>> DidItVal // i32 did_it
>> };
>> @@ -1625,61 +1653,77 @@ bool CGOpenMPRuntime::isDynamic(OpenMPSc
>> return Schedule != OMP_sch_static;
>> }
>>
>> -void CGOpenMPRuntime::emitForInit(CodeGenFunction &CGF, SourceLocation Loc,
>> - OpenMPScheduleClauseKind ScheduleKind,
>> - unsigned IVSize, bool IVSigned, bool Ordered,
>> - llvm::Value *IL, llvm::Value *LB,
>> - llvm::Value *UB, llvm::Value *ST,
>> - llvm::Value *Chunk) {
>> +void CGOpenMPRuntime::emitForDispatchInit(CodeGenFunction &CGF,
>> + SourceLocation Loc,
>> + OpenMPScheduleClauseKind ScheduleKind,
>> + unsigned IVSize, bool IVSigned,
>> + bool Ordered, llvm::Value *UB,
>> + llvm::Value *Chunk) {
>> OpenMPSchedType Schedule =
>> getRuntimeSchedule(ScheduleKind, Chunk != nullptr, Ordered);
>> - if (Ordered ||
>> - (Schedule != OMP_sch_static && Schedule != OMP_sch_static_chunked &&
>> - Schedule != OMP_ord_static && Schedule != OMP_ord_static_chunked)) {
>> - // Call __kmpc_dispatch_init(
>> - // ident_t *loc, kmp_int32 tid, kmp_int32 schedule,
>> - // kmp_int[32|64] lower, kmp_int[32|64] upper,
>> - // kmp_int[32|64] stride, kmp_int[32|64] chunk);
>> + assert(Ordered ||
>> + (Schedule != OMP_sch_static && Schedule != OMP_sch_static_chunked &&
>> + Schedule != OMP_ord_static && Schedule != OMP_ord_static_chunked));
>> + // Call __kmpc_dispatch_init(
>> + // ident_t *loc, kmp_int32 tid, kmp_int32 schedule,
>> + // kmp_int[32|64] lower, kmp_int[32|64] upper,
>> + // kmp_int[32|64] stride, kmp_int[32|64] chunk);
>> +
>> + // If the Chunk was not specified in the clause - use default value 1.
>> + if (Chunk == nullptr)
>> + Chunk = CGF.Builder.getIntN(IVSize, 1);
>> + llvm::Value *Args[] = {
>> + emitUpdateLocation(CGF, Loc, OMP_IDENT_KMPC),
>> + getThreadID(CGF, Loc),
>> + CGF.Builder.getInt32(Schedule), // Schedule type
>> + CGF.Builder.getIntN(IVSize, 0), // Lower
>> + UB, // Upper
>> + CGF.Builder.getIntN(IVSize, 1), // Stride
>> + Chunk // Chunk
>> + };
>> + CGF.EmitRuntimeCall(createDispatchInitFunction(IVSize, IVSigned), Args);
>> +}
>>
>> +void CGOpenMPRuntime::emitForStaticInit(CodeGenFunction &CGF,
>> + SourceLocation Loc,
>> + OpenMPScheduleClauseKind ScheduleKind,
>> + unsigned IVSize, bool IVSigned,
>> + bool Ordered, Address IL, Address LB,
>> + Address UB, Address ST,
>> + llvm::Value *Chunk) {
>> + OpenMPSchedType Schedule =
>> + getRuntimeSchedule(ScheduleKind, Chunk != nullptr, Ordered);
>> + assert(!Ordered);
>> + assert(Schedule == OMP_sch_static || Schedule == OMP_sch_static_chunked ||
>> + Schedule == OMP_ord_static || Schedule == OMP_ord_static_chunked);
>> +
>> + // Call __kmpc_for_static_init(
>> + // ident_t *loc, kmp_int32 tid, kmp_int32 schedtype,
>> + // kmp_int32 *p_lastiter, kmp_int[32|64] *p_lower,
>> + // kmp_int[32|64] *p_upper, kmp_int[32|64] *p_stride,
>> + // kmp_int[32|64] incr, kmp_int[32|64] chunk);
>> + if (Chunk == nullptr) {
>> + assert((Schedule == OMP_sch_static || Schedule == OMP_ord_static) &&
>> + "expected static non-chunked schedule");
>> // If the Chunk was not specified in the clause - use default value 1.
>> - if (Chunk == nullptr)
>> Chunk = CGF.Builder.getIntN(IVSize, 1);
>> - llvm::Value *Args[] = { emitUpdateLocation(CGF, Loc, OMP_IDENT_KMPC),
>> - getThreadID(CGF, Loc),
>> - CGF.Builder.getInt32(Schedule), // Schedule type
>> - CGF.Builder.getIntN(IVSize, 0), // Lower
>> - UB, // Upper
>> - CGF.Builder.getIntN(IVSize, 1), // Stride
>> - Chunk // Chunk
>> - };
>> - CGF.EmitRuntimeCall(createDispatchInitFunction(IVSize, IVSigned), Args);
>> } else {
>> - // Call __kmpc_for_static_init(
>> - // ident_t *loc, kmp_int32 tid, kmp_int32 schedtype,
>> - // kmp_int32 *p_lastiter, kmp_int[32|64] *p_lower,
>> - // kmp_int[32|64] *p_upper, kmp_int[32|64] *p_stride,
>> - // kmp_int[32|64] incr, kmp_int[32|64] chunk);
>> - if (Chunk == nullptr) {
>> - assert((Schedule == OMP_sch_static || Schedule == OMP_ord_static) &&
>> - "expected static non-chunked schedule");
>> - // If the Chunk was not specified in the clause - use default value 1.
>> - Chunk = CGF.Builder.getIntN(IVSize, 1);
>> - } else
>> - assert((Schedule == OMP_sch_static_chunked ||
>> - Schedule == OMP_ord_static_chunked) &&
>> - "expected static chunked schedule");
>> - llvm::Value *Args[] = { emitUpdateLocation(CGF, Loc, OMP_IDENT_KMPC),
>> - getThreadID(CGF, Loc),
>> - CGF.Builder.getInt32(Schedule), // Schedule type
>> - IL, // &isLastIter
>> - LB, // &LB
>> - UB, // &UB
>> - ST, // &Stride
>> - CGF.Builder.getIntN(IVSize, 1), // Incr
>> - Chunk // Chunk
>> - };
>> - CGF.EmitRuntimeCall(createForStaticInitFunction(IVSize, IVSigned), Args);
>> + assert((Schedule == OMP_sch_static_chunked ||
>> + Schedule == OMP_ord_static_chunked) &&
>> + "expected static chunked schedule");
>> }
>> + llvm::Value *Args[] = {
>> + emitUpdateLocation(CGF, Loc, OMP_IDENT_KMPC),
>> + getThreadID(CGF, Loc),
>> + CGF.Builder.getInt32(Schedule), // Schedule type
>> + IL.getPointer(), // &isLastIter
>> + LB.getPointer(), // &LB
>> + UB.getPointer(), // &UB
>> + ST.getPointer(), // &Stride
>> + CGF.Builder.getIntN(IVSize, 1), // Incr
>> + Chunk // Chunk
>> + };
>> + CGF.EmitRuntimeCall(createForStaticInitFunction(IVSize, IVSigned), Args);
>> }
>>
>> void CGOpenMPRuntime::emitForStaticFinish(CodeGenFunction &CGF,
>> @@ -1703,19 +1747,19 @@ void CGOpenMPRuntime::emitForOrderedIter
>>
>> llvm::Value *CGOpenMPRuntime::emitForNext(CodeGenFunction &CGF,
>> SourceLocation Loc, unsigned IVSize,
>> - bool IVSigned, llvm::Value *IL,
>> - llvm::Value *LB, llvm::Value *UB,
>> - llvm::Value *ST) {
>> + bool IVSigned, Address IL,
>> + Address LB, Address UB,
>> + Address ST) {
>> // Call __kmpc_dispatch_next(
>> // ident_t *loc, kmp_int32 tid, kmp_int32 *p_lastiter,
>> // kmp_int[32|64] *p_lower, kmp_int[32|64] *p_upper,
>> // kmp_int[32|64] *p_stride);
>> llvm::Value *Args[] = {
>> emitUpdateLocation(CGF, Loc, OMP_IDENT_KMPC), getThreadID(CGF, Loc),
>> - IL, // &isLastIter
>> - LB, // &Lower
>> - UB, // &Upper
>> - ST // &Stride
>> + IL.getPointer(), // &isLastIter
>> + LB.getPointer(), // &Lower
>> + UB.getPointer(), // &Upper
>> + ST.getPointer() // &Stride
>> };
>> llvm::Value *Call =
>> CGF.EmitRuntimeCall(createDispatchNextFunction(IVSize, IVSigned), Args);
>> @@ -1921,10 +1965,9 @@ emitProxyTaskFunction(CodeGenModule &CGM
>> // TaskFunction(gtid, tt->task_data.part_id, &tt->privates, task_privates_map,
>> // tt->task_data.shareds);
>> auto *GtidParam = CGF.EmitLoadOfScalar(
>> - CGF.GetAddrOfLocalVar(&GtidArg), /*Volatile=*/false,
>> - C.getTypeAlignInChars(KmpInt32Ty).getQuantity(), KmpInt32Ty, Loc);
>> - auto *TaskTypeArgAddr = CGF.Builder.CreateAlignedLoad(
>> - CGF.GetAddrOfLocalVar(&TaskTypeArg), CGM.PointerAlignInBytes);
>> + CGF.GetAddrOfLocalVar(&GtidArg), /*Volatile=*/false, KmpInt32Ty, Loc);
>> + auto *TaskTypeArgAddr = CGF.Builder.CreateLoad(
>> + CGF.GetAddrOfLocalVar(&TaskTypeArg));
>> LValue TDBase =
>> CGF.MakeNaturalAlignAddrLValue(TaskTypeArgAddr, KmpTaskTWithPrivatesQTy);
>> auto *KmpTaskTWithPrivatesQTyRD =
>> @@ -1947,7 +1990,7 @@ emitProxyTaskFunction(CodeGenModule &CGM
>> if (PrivatesFI != KmpTaskTWithPrivatesQTyRD->field_end()) {
>> auto PrivatesLVal = CGF.EmitLValueForField(TDBase, *PrivatesFI);
>> PrivatesParam = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - PrivatesLVal.getAddress(), CGF.VoidPtrTy);
>> + PrivatesLVal.getPointer(), CGF.VoidPtrTy);
>> } else {
>> PrivatesParam = llvm::ConstantPointerNull::get(CGF.VoidPtrTy);
>> }
>> @@ -1957,7 +2000,7 @@ emitProxyTaskFunction(CodeGenModule &CGM
>> CGF.EmitCallOrInvoke(TaskFunction, CallArgs);
>> CGF.EmitStoreThroughLValue(
>> RValue::get(CGF.Builder.getInt32(/*C=*/0)),
>> - CGF.MakeNaturalAlignAddrLValue(CGF.ReturnValue, KmpInt32Ty));
>> + CGF.MakeAddrLValue(CGF.ReturnValue, KmpInt32Ty));
>> CGF.FinishFunction();
>> return TaskEntry;
>> }
>> @@ -1988,8 +2031,8 @@ static llvm::Value *emitDestructorsFunct
>> CGF.StartFunction(GlobalDecl(), KmpInt32Ty, DestructorFn, DestructorFnInfo,
>> Args);
>>
>> - auto *TaskTypeArgAddr = CGF.Builder.CreateAlignedLoad(
>> - CGF.GetAddrOfLocalVar(&TaskTypeArg), CGM.PointerAlignInBytes);
>> + auto *TaskTypeArgAddr = CGF.Builder.CreateLoad(
>> + CGF.GetAddrOfLocalVar(&TaskTypeArg));
>> LValue Base =
>> CGF.MakeNaturalAlignAddrLValue(TaskTypeArgAddr, KmpTaskTWithPrivatesQTy);
>> auto *KmpTaskTWithPrivatesQTyRD =
>> @@ -2069,8 +2112,8 @@ emitTaskPrivateMappingFunction(CodeGenMo
>> TaskPrivatesMapFnInfo, Args);
>>
>> // *privi = &.privates.privi;
>> - auto *TaskPrivatesArgAddr = CGF.Builder.CreateAlignedLoad(
>> - CGF.GetAddrOfLocalVar(&TaskPrivatesArg), CGM.PointerAlignInBytes);
>> + auto *TaskPrivatesArgAddr = CGF.Builder.CreateLoad(
>> + CGF.GetAddrOfLocalVar(&TaskPrivatesArg));
>> LValue Base =
>> CGF.MakeNaturalAlignAddrLValue(TaskPrivatesArgAddr, PrivatesQTy);
>> auto *PrivatesQTyRD = cast<RecordDecl>(PrivatesQTy->getAsTagDecl());
>> @@ -2078,11 +2121,10 @@ emitTaskPrivateMappingFunction(CodeGenMo
>> for (auto *Field : PrivatesQTyRD->fields()) {
>> auto FieldLVal = CGF.EmitLValueForField(Base, Field);
>> auto *VD = Args[PrivateVarsPos[Privates[Counter].second.Original]];
>> - auto RefLVal = CGF.MakeNaturalAlignAddrLValue(CGF.GetAddrOfLocalVar(VD),
>> - VD->getType());
>> + auto RefLVal = CGF.MakeAddrLValue(CGF.GetAddrOfLocalVar(VD), VD->getType());
>> auto RefLoadRVal = CGF.EmitLoadOfLValue(RefLVal, Loc);
>> CGF.EmitStoreOfScalar(
>> - FieldLVal.getAddress(),
>> + FieldLVal.getPointer(),
>> CGF.MakeNaturalAlignAddrLValue(RefLoadRVal.getScalarVal(),
>> RefLVal.getType()->getPointeeType()));
>> ++Counter;
>> @@ -2120,7 +2162,7 @@ static int array_pod_sort_comparator(con
>> void CGOpenMPRuntime::emitTaskCall(
>> CodeGenFunction &CGF, SourceLocation Loc, const OMPExecutableDirective &D,
>> bool Tied, llvm::PointerIntPair<llvm::Value *, 1, bool> Final,
>> - llvm::Value *TaskFunction, QualType SharedsTy, llvm::Value *Shareds,
>> + llvm::Value *TaskFunction, QualType SharedsTy, Address Shareds,
>> const Expr *IfCond, ArrayRef<const Expr *> PrivateVars,
>> ArrayRef<const Expr *> PrivateCopies,
>> ArrayRef<const Expr *> FirstprivateVars,
>> @@ -2227,12 +2269,12 @@ void CGOpenMPRuntime::emitTaskCall(
>> CGF.EmitLValueForField(Base, *KmpTaskTWithPrivatesQTyRD->field_begin());
>> // Fill the data in the resulting kmp_task_t record.
>> // Copy shareds if there are any.
>> - llvm::Value *KmpTaskSharedsPtr = nullptr;
>> + Address KmpTaskSharedsPtr = Address::invalid();
>> if (!SharedsTy->getAsStructureType()->getDecl()->field_empty()) {
>> - KmpTaskSharedsPtr = CGF.EmitLoadOfScalar(
>> + KmpTaskSharedsPtr = Address(CGF.EmitLoadOfScalar(
>> CGF.EmitLValueForField(
>> TDBase, *std::next(KmpTaskTQTyRD->field_begin(), KmpTaskTShareds)),
>> - Loc);
>> + Loc), CGF.getNaturalTypeAlignment(SharedsTy));
>> CGF.EmitAggregateCopy(KmpTaskSharedsPtr, Shareds, SharedsTy);
>> }
>> // Emit initial values for private copies (if any).
>> @@ -2243,7 +2285,7 @@ void CGOpenMPRuntime::emitTaskCall(
>> FI = cast<RecordDecl>(FI->getType()->getAsTagDecl())->field_begin();
>> LValue SharedsBase;
>> if (!FirstprivateVars.empty()) {
>> - SharedsBase = CGF.MakeNaturalAlignAddrLValue(
>> + SharedsBase = CGF.MakeAddrLValue(
>> CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> KmpTaskSharedsPtr, CGF.ConvertTypeForMem(SharedsPtrTy)),
>> SharedsTy);
>> @@ -2274,10 +2316,10 @@ void CGOpenMPRuntime::emitTaskCall(
>> CGF.EmitOMPAggregateAssign(
>> PrivateLValue.getAddress(), SharedRefLValue.getAddress(),
>> Type, [&CGF, Elem, Init, &CapturesInfo](
>> - llvm::Value *DestElement, llvm::Value *SrcElement) {
>> + Address DestElement, Address SrcElement) {
>> // Clean up any temporaries needed by the initialization.
>> CodeGenFunction::OMPPrivateScope InitScope(CGF);
>> - InitScope.addPrivate(Elem, [SrcElement]() -> llvm::Value *{
>> + InitScope.addPrivate(Elem, [SrcElement]() -> Address {
>> return SrcElement;
>> });
>> (void)InitScope.Privatize();
>> @@ -2291,7 +2333,7 @@ void CGOpenMPRuntime::emitTaskCall(
>> }
>> } else {
>> CodeGenFunction::OMPPrivateScope InitScope(CGF);
>> - InitScope.addPrivate(Elem, [SharedRefLValue]() -> llvm::Value *{
>> + InitScope.addPrivate(Elem, [SharedRefLValue]() -> Address {
>> return SharedRefLValue.getAddress();
>> });
>> (void)InitScope.Privatize();
>> @@ -2321,9 +2363,9 @@ void CGOpenMPRuntime::emitTaskCall(
>> Destructor);
>>
>> // Process list of dependences.
>> - llvm::Value *DependInfo = nullptr;
>> - unsigned DependencesNumber = Dependences.size();
>> - if (!Dependences.empty()) {
>> + Address DependenciesArray = Address::invalid();
>> + unsigned NumDependencies = Dependences.size();
>> + if (NumDependencies) {
>> // Dependence kind for RTL.
>> enum RTLDependenceKindTy { DepIn = 1, DepOut = 2, DepInOut = 3 };
>> enum RTLDependInfoFieldsTy { BaseAddr, Len, Flags };
>> @@ -2342,37 +2384,39 @@ void CGOpenMPRuntime::emitTaskCall(
>> } else {
>> KmpDependInfoRD = cast<RecordDecl>(KmpDependInfoTy->getAsTagDecl());
>> }
>> + CharUnits DependencySize = C.getTypeSizeInChars(KmpDependInfoTy);
>> // Define type kmp_depend_info[<Dependences.size()>];
>> QualType KmpDependInfoArrayTy = C.getConstantArrayType(
>> - KmpDependInfoTy, llvm::APInt(/*numBits=*/64, Dependences.size()),
>> + KmpDependInfoTy, llvm::APInt(/*numBits=*/64, NumDependencies),
>> ArrayType::Normal, /*IndexTypeQuals=*/0);
>> // kmp_depend_info[<Dependences.size()>] deps;
>> - DependInfo = CGF.CreateMemTemp(KmpDependInfoArrayTy);
>> - for (unsigned i = 0; i < DependencesNumber; ++i) {
>> - auto *E = Dependences[i].second;
>> - LValue Addr = CGF.EmitLValue(E);
>> + DependenciesArray = CGF.CreateMemTemp(KmpDependInfoArrayTy);
>> + for (unsigned i = 0; i < NumDependencies; ++i) {
>> + const Expr *E = Dependences[i].second;
>> + auto Addr = CGF.EmitLValue(E);
>> llvm::Value *Size;
>> QualType Ty = E->getType();
>> - auto *DestAddr = Addr.getAddress();
>> if (auto *ASE = dyn_cast<OMPArraySectionExpr>(E->IgnoreParenImpCasts())) {
>> LValue UpAddrLVal =
>> CGF.EmitOMPArraySectionExpr(ASE, /*LowerBound=*/false);
>> llvm::Value *UpAddr =
>> - CGF.Builder.CreateConstGEP1_32(UpAddrLVal.getAddress(), /*Idx0=*/1);
>> + CGF.Builder.CreateConstGEP1_32(UpAddrLVal.getPointer(), /*Idx0=*/1);
>> llvm::Value *LowIntPtr =
>> - CGF.Builder.CreatePtrToInt(DestAddr, CGM.SizeTy);
>> + CGF.Builder.CreatePtrToInt(Addr.getPointer(), CGM.SizeTy);
>> llvm::Value *UpIntPtr = CGF.Builder.CreatePtrToInt(UpAddr, CGM.SizeTy);
>> Size = CGF.Builder.CreateNUWSub(UpIntPtr, LowIntPtr);
>> - } else
>> + } else {
>> Size = getTypeSize(CGF, Ty);
>> - auto Base = CGF.MakeNaturalAlignAddrLValue(
>> - CGF.Builder.CreateStructGEP(/*Ty=*/nullptr, DependInfo, i),
>> + }
>> + auto Base = CGF.MakeAddrLValue(
>> + CGF.Builder.CreateConstArrayGEP(DependenciesArray, i, DependencySize),
>> KmpDependInfoTy);
>> // deps[i].base_addr = &<Dependences[i].second>;
>> auto BaseAddrLVal = CGF.EmitLValueForField(
>> Base, *std::next(KmpDependInfoRD->field_begin(), BaseAddr));
>> - CGF.EmitStoreOfScalar(CGF.Builder.CreatePtrToInt(DestAddr, CGF.IntPtrTy),
>> - BaseAddrLVal);
>> + CGF.EmitStoreOfScalar(
>> + CGF.Builder.CreatePtrToInt(Addr.getPointer(), CGF.IntPtrTy),
>> + BaseAddrLVal);
>> // deps[i].len = sizeof(<Dependences[i].second>);
>> auto LenLVal = CGF.EmitLValueForField(
>> Base, *std::next(KmpDependInfoRD->field_begin(), Len));
>> @@ -2397,8 +2441,8 @@ void CGOpenMPRuntime::emitTaskCall(
>> CGF.EmitStoreOfScalar(llvm::ConstantInt::get(LLVMFlagsTy, DepKind),
>> FlagsLVal);
>> }
>> - DependInfo = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateStructGEP(/*Ty=*/nullptr, DependInfo, 0),
>> + DependenciesArray = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> + CGF.Builder.CreateStructGEP(DependenciesArray, 0, CharUnits::Zero()),
>> CGF.VoidPtrTy);
>> }
>>
>> @@ -2412,40 +2456,48 @@ void CGOpenMPRuntime::emitTaskCall(
>> // list is not empty
>> auto *ThreadID = getThreadID(CGF, Loc);
>> auto *UpLoc = emitUpdateLocation(CGF, Loc);
>> - llvm::Value *TaskArgs[] = {UpLoc, ThreadID, NewTask};
>> - llvm::Value *DepTaskArgs[] = {
>> - UpLoc,
>> - ThreadID,
>> - NewTask,
>> - DependInfo ? CGF.Builder.getInt32(DependencesNumber) : nullptr,
>> - DependInfo,
>> - DependInfo ? CGF.Builder.getInt32(0) : nullptr,
>> - DependInfo ? llvm::ConstantPointerNull::get(CGF.VoidPtrTy) : nullptr};
>> - auto &&ThenCodeGen = [this, DependInfo, &TaskArgs,
>> - &DepTaskArgs](CodeGenFunction &CGF) {
>> - // TODO: add check for untied tasks.
>> - CGF.EmitRuntimeCall(
>> - createRuntimeFunction(DependInfo ? OMPRTL__kmpc_omp_task_with_deps
>> - : OMPRTL__kmpc_omp_task),
>> - DependInfo ? makeArrayRef(DepTaskArgs) : makeArrayRef(TaskArgs));
>> + llvm::Value *TaskArgs[] = { UpLoc, ThreadID, NewTask };
>> + llvm::Value *DepTaskArgs[7];
>> + if (NumDependencies) {
>> + DepTaskArgs[0] = UpLoc;
>> + DepTaskArgs[1] = ThreadID;
>> + DepTaskArgs[2] = NewTask;
>> + DepTaskArgs[3] = CGF.Builder.getInt32(NumDependencies);
>> + DepTaskArgs[4] = DependenciesArray.getPointer();
>> + DepTaskArgs[5] = CGF.Builder.getInt32(0);
>> + DepTaskArgs[6] = llvm::ConstantPointerNull::get(CGF.VoidPtrTy);
>> + }
>> + auto &&ThenCodeGen = [this, NumDependencies,
>> + &TaskArgs, &DepTaskArgs](CodeGenFunction &CGF) {
>> + // TODO: add check for untied tasks.
>> + if (NumDependencies) {
>> + CGF.EmitRuntimeCall(createRuntimeFunction(OMPRTL__kmpc_omp_task_with_deps),
>> + DepTaskArgs);
>> + } else {
>> + CGF.EmitRuntimeCall(createRuntimeFunction(OMPRTL__kmpc_omp_task),
>> + TaskArgs);
>> + }
>> };
>> typedef CallEndCleanup<std::extent<decltype(TaskArgs)>::value>
>> IfCallEndCleanup;
>> - llvm::Value *DepWaitTaskArgs[] = {
>> - UpLoc,
>> - ThreadID,
>> - DependInfo ? CGF.Builder.getInt32(DependencesNumber) : nullptr,
>> - DependInfo,
>> - DependInfo ? CGF.Builder.getInt32(0) : nullptr,
>> - DependInfo ? llvm::ConstantPointerNull::get(CGF.VoidPtrTy) : nullptr};
>> +
>> + llvm::Value *DepWaitTaskArgs[6];
>> + if (NumDependencies) {
>> + DepWaitTaskArgs[0] = UpLoc;
>> + DepWaitTaskArgs[1] = ThreadID;
>> + DepWaitTaskArgs[2] = CGF.Builder.getInt32(NumDependencies);
>> + DepWaitTaskArgs[3] = DependenciesArray.getPointer();
>> + DepWaitTaskArgs[4] = CGF.Builder.getInt32(0);
>> + DepWaitTaskArgs[5] = llvm::ConstantPointerNull::get(CGF.VoidPtrTy);
>> + }
>> auto &&ElseCodeGen = [this, &TaskArgs, ThreadID, NewTaskNewTaskTTy, TaskEntry,
>> - DependInfo, &DepWaitTaskArgs](CodeGenFunction &CGF) {
>> + NumDependencies, &DepWaitTaskArgs](CodeGenFunction &CGF) {
>> CodeGenFunction::RunCleanupsScope LocalScope(CGF);
>> // Build void __kmpc_omp_wait_deps(ident_t *, kmp_int32 gtid,
>> // kmp_int32 ndeps, kmp_depend_info_t *dep_list, kmp_int32
>> // ndeps_noalias, kmp_depend_info_t *noalias_dep_list); if dependence info
>> // is specified.
>> - if (DependInfo)
>> + if (NumDependencies)
>> CGF.EmitRuntimeCall(createRuntimeFunction(OMPRTL__kmpc_omp_wait_deps),
>> DepWaitTaskArgs);
>> // Build void __kmpc_omp_task_begin_if0(ident_t *, kmp_int32 gtid,
>> @@ -2463,6 +2515,7 @@ void CGOpenMPRuntime::emitTaskCall(
>> llvm::Value *OutlinedFnArgs[] = {ThreadID, NewTaskNewTaskTTy};
>> CGF.EmitCallOrInvoke(TaskEntry, OutlinedFnArgs);
>> };
>> +
>> if (IfCond) {
>> emitOMPIfClause(CGF, IfCond, ThenCodeGen, ElseCodeGen);
>> } else {
>> @@ -2498,38 +2551,26 @@ static llvm::Value *emitReductionFunctio
>>
>> // Dst = (void*[n])(LHSArg);
>> // Src = (void*[n])(RHSArg);
>> - auto *LHS = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(CGF.GetAddrOfLocalVar(&LHSArg),
>> - CGF.PointerAlignInBytes),
>> - ArgsType);
>> - auto *RHS = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(CGF.GetAddrOfLocalVar(&RHSArg),
>> - CGF.PointerAlignInBytes),
>> - ArgsType);
>> + Address LHS(CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> + CGF.Builder.CreateLoad(CGF.GetAddrOfLocalVar(&LHSArg)),
>> + ArgsType), CGF.getPointerAlign());
>> + Address RHS(CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> + CGF.Builder.CreateLoad(CGF.GetAddrOfLocalVar(&RHSArg)),
>> + ArgsType), CGF.getPointerAlign());
>>
>> // ...
>> // *(Type<i>*)lhs[i] = RedOp<i>(*(Type<i>*)lhs[i], *(Type<i>*)rhs[i]);
>> // ...
>> CodeGenFunction::OMPPrivateScope Scope(CGF);
>> for (unsigned I = 0, E = ReductionOps.size(); I < E; ++I) {
>> - Scope.addPrivate(
>> - cast<VarDecl>(cast<DeclRefExpr>(RHSExprs[I])->getDecl()),
>> - [&]() -> llvm::Value *{
>> - return CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(
>> - CGF.Builder.CreateStructGEP(/*Ty=*/nullptr, RHS, I),
>> - CGM.PointerAlignInBytes),
>> - CGF.ConvertTypeForMem(C.getPointerType(RHSExprs[I]->getType())));
>> - });
>> - Scope.addPrivate(
>> - cast<VarDecl>(cast<DeclRefExpr>(LHSExprs[I])->getDecl()),
>> - [&]() -> llvm::Value *{
>> - return CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.Builder.CreateAlignedLoad(
>> - CGF.Builder.CreateStructGEP(/*Ty=*/nullptr, LHS, I),
>> - CGM.PointerAlignInBytes),
>> - CGF.ConvertTypeForMem(C.getPointerType(LHSExprs[I]->getType())));
>> - });
>> + auto RHSVar = cast<VarDecl>(cast<DeclRefExpr>(RHSExprs[I])->getDecl());
>> + Scope.addPrivate(RHSVar, [&]() -> Address {
>> + return emitAddrOfVarFromArray(CGF, RHS, I, RHSVar);
>> + });
>> + auto LHSVar = cast<VarDecl>(cast<DeclRefExpr>(LHSExprs[I])->getDecl());
>> + Scope.addPrivate(LHSVar, [&]() -> Address {
>> + return emitAddrOfVarFromArray(CGF, LHS, I, LHSVar);
>> + });
>> }
>> Scope.Privatize();
>> for (auto *E : ReductionOps) {
>> @@ -2596,14 +2637,15 @@ void CGOpenMPRuntime::emitReduction(Code
>> QualType ReductionArrayTy =
>> C.getConstantArrayType(C.VoidPtrTy, ArraySize, ArrayType::Normal,
>> /*IndexTypeQuals=*/0);
>> - auto *ReductionList =
>> + Address ReductionList =
>> CGF.CreateMemTemp(ReductionArrayTy, ".omp.reduction.red_list");
>> for (unsigned I = 0, E = RHSExprs.size(); I < E; ++I) {
>> - auto *Elem = CGF.Builder.CreateStructGEP(/*Ty=*/nullptr, ReductionList, I);
>> - CGF.Builder.CreateAlignedStore(
>> + Address Elem =
>> + CGF.Builder.CreateConstArrayGEP(ReductionList, I, CGF.getPointerSize());
>> + CGF.Builder.CreateStore(
>> CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(
>> - CGF.EmitLValue(RHSExprs[I]).getAddress(), CGF.VoidPtrTy),
>> - Elem, CGM.PointerAlignInBytes);
>> + CGF.EmitLValue(RHSExprs[I]).getPointer(), CGF.VoidPtrTy),
>> + Elem);
>> }
>>
>> // 2. Emit reduce_func().
>> @@ -2622,8 +2664,9 @@ void CGOpenMPRuntime::emitReduction(Code
>> auto *ThreadId = getThreadID(CGF, Loc);
>> auto *ReductionArrayTySize = llvm::ConstantInt::get(
>> CGM.SizeTy, C.getTypeSizeInChars(ReductionArrayTy).getQuantity());
>> - auto *RL = CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(ReductionList,
>> - CGF.VoidPtrTy);
>> + auto *RL =
>> + CGF.Builder.CreatePointerBitCastOrAddrSpaceCast(ReductionList.getPointer(),
>> + CGF.VoidPtrTy);
>> llvm::Value *Args[] = {
>> IdentTLoc, // ident_t *<loc>
>> ThreadId, // i32 <gtid>
>> @@ -2736,11 +2779,11 @@ void CGOpenMPRuntime::emitReduction(Code
>> [&CGF, UpExpr, VD](RValue XRValue) {
>> CodeGenFunction::OMPPrivateScope PrivateScope(CGF);
>> PrivateScope.addPrivate(
>> - VD, [&CGF, VD, XRValue]() -> llvm::Value *{
>> - auto *LHSTemp = CGF.CreateMemTemp(VD->getType());
>> + VD, [&CGF, VD, XRValue]() -> Address {
>> + Address LHSTemp = CGF.CreateMemTemp(VD->getType());
>> CGF.EmitStoreThroughLValue(
>> XRValue,
>> - CGF.MakeNaturalAlignAddrLValue(LHSTemp, VD->getType()));
>> + CGF.MakeAddrLValue(LHSTemp, VD->getType()));
>> return LHSTemp;
>> });
>> (void)PrivateScope.Privatize();
>>
>> Modified: cfe/trunk/lib/CodeGen/CGOpenMPRuntime.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGOpenMPRuntime.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGOpenMPRuntime.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGOpenMPRuntime.h Tue Sep 8 03:05:57 2015
>> @@ -39,7 +39,7 @@ class OMPExecutableDirective;
>> class VarDecl;
>>
>> namespace CodeGen {
>> -
>> +class Address;
>> class CodeGenFunction;
>> class CodeGenModule;
>>
>> @@ -184,7 +184,9 @@ private:
>> /// \brief Map of flags and corresponding default locations.
>> typedef llvm::DenseMap<unsigned, llvm::Value *> OpenMPDefaultLocMapTy;
>> OpenMPDefaultLocMapTy OpenMPDefaultLocMap;
>> - llvm::Value *getOrCreateDefaultLocation(OpenMPLocationFlags Flags);
>> + Address getOrCreateDefaultLocation(OpenMPLocationFlags Flags);
>> +
>> +public:
>> /// \brief Describes ident structure that describes a source location.
>> /// All descriptions are taken from
>> /// http://llvm.org/svn/llvm-project/openmp/trunk/runtime/src/kmp.h
>> @@ -225,6 +227,7 @@ private:
>> /// and a pair of line numbers that delimit the construct.
>> IdentField_PSource
>> };
>> +private:
>> llvm::StructType *IdentTy;
>> /// \brief Map for SourceLocation and OpenMP runtime library debug locations.
>> typedef llvm::DenseMap<unsigned, llvm::Value *> OpenMPDebugLocMapTy;
>> @@ -321,8 +324,7 @@ private:
>>
>> /// \brief Emits address of the word in a memory where current thread id is
>> /// stored.
>> - virtual llvm::Value *emitThreadIDAddress(CodeGenFunction &CGF,
>> - SourceLocation Loc);
>> + virtual Address emitThreadIDAddress(CodeGenFunction &CGF, SourceLocation Loc);
>>
>> /// \brief Gets thread id value for the current thread.
>> ///
>> @@ -346,7 +348,7 @@ private:
>> /// \param CopyCtor Pointer to a global copy function for \a VD.
>> /// \param Dtor Pointer to a global destructor function for \a VD.
>> /// \param Loc Location of threadprivate declaration.
>> - void emitThreadPrivateVarInit(CodeGenFunction &CGF, llvm::Value *VDAddr,
>> + void emitThreadPrivateVarInit(CodeGenFunction &CGF, Address VDAddr,
>> llvm::Value *Ctor, llvm::Value *CopyCtor,
>> llvm::Value *Dtor, SourceLocation Loc);
>>
>> @@ -403,8 +405,7 @@ public:
>> ///
>> virtual void emitParallelCall(CodeGenFunction &CGF, SourceLocation Loc,
>> llvm::Value *OutlinedFn,
>> - llvm::Value *CapturedStruct,
>> - const Expr *IfCond);
>> + Address CapturedStruct, const Expr *IfCond);
>>
>> /// \brief Emits a critical region.
>> /// \param CriticalName Name of the critical region.
>> @@ -497,11 +498,17 @@ public:
>> /// \param Chunk Value of the chunk for the static_chunked scheduled loop.
>> /// For the default (nullptr) value, the chunk 1 will be used.
>> ///
>> - virtual void emitForInit(CodeGenFunction &CGF, SourceLocation Loc,
>> - OpenMPScheduleClauseKind SchedKind, unsigned IVSize,
>> - bool IVSigned, bool Ordered, llvm::Value *IL,
>> - llvm::Value *LB, llvm::Value *UB, llvm::Value *ST,
>> - llvm::Value *Chunk = nullptr);
>> + virtual void emitForDispatchInit(CodeGenFunction &CGF, SourceLocation Loc,
>> + OpenMPScheduleClauseKind SchedKind,
>> + unsigned IVSize, bool IVSigned,
>> + bool Ordered, llvm::Value *UB,
>> + llvm::Value *Chunk = nullptr);
>> + virtual void emitForStaticInit(CodeGenFunction &CGF, SourceLocation Loc,
>> + OpenMPScheduleClauseKind SchedKind,
>> + unsigned IVSize, bool IVSigned, bool Ordered,
>> + Address IL, Address LB,
>> + Address UB, Address ST,
>> + llvm::Value *Chunk = nullptr);
>>
>> /// \brief Call the appropriate runtime routine to notify that we finished
>> /// iteration of the ordered loop with the dynamic scheduling.
>> @@ -539,8 +546,8 @@ public:
>> /// returned.
>> virtual llvm::Value *emitForNext(CodeGenFunction &CGF, SourceLocation Loc,
>> unsigned IVSize, bool IVSigned,
>> - llvm::Value *IL, llvm::Value *LB,
>> - llvm::Value *UB, llvm::Value *ST);
>> + Address IL, Address LB,
>> + Address UB, Address ST);
>>
>> /// \brief Emits call to void __kmpc_push_num_threads(ident_t *loc, kmp_int32
>> /// global_tid, kmp_int32 num_threads) to generate code for 'num_threads'
>> @@ -562,10 +569,10 @@ public:
>> /// \param VDAddr Address of the global variable \a VD.
>> /// \param Loc Location of the reference to threadprivate var.
>> /// \return Address of the threadprivate variable for the current thread.
>> - virtual llvm::Value *getAddrOfThreadPrivate(CodeGenFunction &CGF,
>> - const VarDecl *VD,
>> - llvm::Value *VDAddr,
>> - SourceLocation Loc);
>> + virtual Address getAddrOfThreadPrivate(CodeGenFunction &CGF,
>> + const VarDecl *VD,
>> + Address VDAddr,
>> + SourceLocation Loc);
>>
>> /// \brief Emit a code for initialization of threadprivate variable. It emits
>> /// a call to runtime library which adds initial value to the newly created
>> @@ -576,7 +583,7 @@ public:
>> /// \param Loc Location of threadprivate declaration.
>> /// \param PerformInit true if initialization expression is not constant.
>> virtual llvm::Function *
>> - emitThreadPrivateVarDefinition(const VarDecl *VD, llvm::Value *VDAddr,
>> + emitThreadPrivateVarDefinition(const VarDecl *VD, Address VDAddr,
>> SourceLocation Loc, bool PerformInit,
>> CodeGenFunction *CGF = nullptr);
>>
>> @@ -632,7 +639,7 @@ public:
>> virtual void emitTaskCall(
>> CodeGenFunction &CGF, SourceLocation Loc, const OMPExecutableDirective &D,
>> bool Tied, llvm::PointerIntPair<llvm::Value *, 1, bool> Final,
>> - llvm::Value *TaskFunction, QualType SharedsTy, llvm::Value *Shareds,
>> + llvm::Value *TaskFunction, QualType SharedsTy, Address Shareds,
>> const Expr *IfCond, ArrayRef<const Expr *> PrivateVars,
>> ArrayRef<const Expr *> PrivateCopies,
>> ArrayRef<const Expr *> FirstprivateVars,
>>
>> Modified: cfe/trunk/lib/CodeGen/CGStmt.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGStmt.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGStmt.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGStmt.cpp Tue Sep 8 03:05:57 2015
>> @@ -275,8 +275,8 @@ bool CodeGenFunction::EmitSimpleStmt(con
>> /// EmitCompoundStmt - Emit a compound statement {..} node. If GetLast is true,
>> /// this captures the expression result of the last sub-statement and returns it
>> /// (for use by the statement expression extension).
>> -llvm::Value* CodeGenFunction::EmitCompoundStmt(const CompoundStmt &S, bool GetLast,
>> - AggValueSlot AggSlot) {
>> +Address CodeGenFunction::EmitCompoundStmt(const CompoundStmt &S, bool GetLast,
>> + AggValueSlot AggSlot) {
>> PrettyStackTraceLoc CrashInfo(getContext().getSourceManager(),S.getLBracLoc(),
>> "LLVM IR generation of compound statement ('{}')");
>>
>> @@ -286,7 +286,7 @@ llvm::Value* CodeGenFunction::EmitCompou
>> return EmitCompoundStmtWithoutScope(S, GetLast, AggSlot);
>> }
>>
>> -llvm::Value*
>> +Address
>> CodeGenFunction::EmitCompoundStmtWithoutScope(const CompoundStmt &S,
>> bool GetLast,
>> AggValueSlot AggSlot) {
>> @@ -295,7 +295,7 @@ CodeGenFunction::EmitCompoundStmtWithout
>> E = S.body_end()-GetLast; I != E; ++I)
>> EmitStmt(*I);
>>
>> - llvm::Value *RetAlloca = nullptr;
>> + Address RetAlloca = Address::invalid();
>> if (GetLast) {
>> // We have to special case labels here. They are statements, but when put
>> // at the end of a statement expression, they yield the value of their
>> @@ -909,10 +909,9 @@ void CodeGenFunction::EmitReturnOfRValue
>> if (RV.isScalar()) {
>> Builder.CreateStore(RV.getScalarVal(), ReturnValue);
>> } else if (RV.isAggregate()) {
>> - EmitAggregateCopy(ReturnValue, RV.getAggregateAddr(), Ty);
>> + EmitAggregateCopy(ReturnValue, RV.getAggregateAddress(), Ty);
>> } else {
>> - EmitStoreOfComplex(RV.getComplexVal(),
>> - MakeNaturalAlignAddrLValue(ReturnValue, Ty),
>> + EmitStoreOfComplex(RV.getComplexVal(), MakeAddrLValue(ReturnValue, Ty),
>> /*init*/ true);
>> }
>> EmitBranchThroughCleanup(ReturnBlock);
>> @@ -953,8 +952,8 @@ void CodeGenFunction::EmitReturnStmt(con
>> // If there is an NRVO flag for this variable, set it to 1 into indicate
>> // that the cleanup code should not destroy the variable.
>> if (llvm::Value *NRVOFlag = NRVOFlags[S.getNRVOCandidate()])
>> - Builder.CreateStore(Builder.getTrue(), NRVOFlag);
>> - } else if (!ReturnValue || (RV && RV->getType()->isVoidType())) {
>> + Builder.CreateFlagStore(Builder.getTrue(), NRVOFlag);
>> + } else if (!ReturnValue.isValid() || (RV && RV->getType()->isVoidType())) {
>> // Make sure not to return anything, but evaluate the expression
>> // for side effects.
>> if (RV)
>> @@ -972,20 +971,17 @@ void CodeGenFunction::EmitReturnStmt(con
>> Builder.CreateStore(EmitScalarExpr(RV), ReturnValue);
>> break;
>> case TEK_Complex:
>> - EmitComplexExprIntoLValue(RV,
>> - MakeNaturalAlignAddrLValue(ReturnValue, RV->getType()),
>> + EmitComplexExprIntoLValue(RV, MakeAddrLValue(ReturnValue, RV->getType()),
>> /*isInit*/ true);
>> break;
>> - case TEK_Aggregate: {
>> - CharUnits Alignment = getContext().getTypeAlignInChars(RV->getType());
>> - EmitAggExpr(RV, AggValueSlot::forAddr(ReturnValue, Alignment,
>> + case TEK_Aggregate:
>> + EmitAggExpr(RV, AggValueSlot::forAddr(ReturnValue,
>> Qualifiers(),
>> AggValueSlot::IsDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsNotAliased));
>> break;
>> }
>> - }
>> }
>>
>> ++NumReturnExprs;
>> @@ -1640,12 +1636,12 @@ CodeGenFunction::EmitAsmInputLValue(cons
>> Arg = Builder.CreateLoad(Builder.CreateBitCast(InputValue.getAddress(),
>> Ty));
>> } else {
>> - Arg = InputValue.getAddress();
>> + Arg = InputValue.getPointer();
>> ConstraintStr += '*';
>> }
>> }
>> } else {
>> - Arg = InputValue.getAddress();
>> + Arg = InputValue.getPointer();
>> ConstraintStr += '*';
>> }
>>
>> @@ -1816,8 +1812,8 @@ void CodeGenFunction::EmitAsmStmt(const
>> << OutExpr->getType() << OutputConstraint;
>> }
>> } else {
>> - ArgTypes.push_back(Dest.getAddress()->getType());
>> - Args.push_back(Dest.getAddress());
>> + ArgTypes.push_back(Dest.getAddress().getType());
>> + Args.push_back(Dest.getPointer());
>> Constraints += "=*";
>> Constraints += OutputConstraint;
>> ReadOnly = ReadNone = false;
>> @@ -2049,8 +2045,8 @@ LValue CodeGenFunction::InitCapturedStru
>> QualType RecordTy = getContext().getRecordType(RD);
>>
>> // Initialize the captured struct.
>> - LValue SlotLV = MakeNaturalAlignAddrLValue(
>> - CreateMemTemp(RecordTy, "agg.captured"), RecordTy);
>> + LValue SlotLV =
>> + MakeAddrLValue(CreateMemTemp(RecordTy, "agg.captured"), RecordTy);
>>
>> RecordDecl::field_iterator CurField = RD->field_begin();
>> for (CapturedStmt::const_capture_init_iterator I = S.capture_init_begin(),
>> @@ -2081,13 +2077,12 @@ CodeGenFunction::EmitCapturedStmt(const
>> delete CGF.CapturedStmtInfo;
>>
>> // Emit call to the helper function.
>> - EmitCallOrInvoke(F, CapStruct.getAddress());
>> + EmitCallOrInvoke(F, CapStruct.getPointer());
>>
>> return F;
>> }
>>
>> -llvm::Value *
>> -CodeGenFunction::GenerateCapturedStmtArgument(const CapturedStmt &S) {
>> +Address CodeGenFunction::GenerateCapturedStmtArgument(const CapturedStmt &S) {
>> LValue CapStruct = InitCapturedStruct(S);
>> return CapStruct.getAddress();
>> }
>> @@ -2126,8 +2121,7 @@ CodeGenFunction::GenerateCapturedStmtFun
>> CD->getLocation(),
>> CD->getBody()->getLocStart());
>> // Set the context parameter in CapturedStmtInfo.
>> - llvm::Value *DeclPtr = LocalDeclMap[CD->getContextParam()];
>> - assert(DeclPtr && "missing context parameter for CapturedStmt");
>> + Address DeclPtr = GetAddrOfLocalVar(CD->getContextParam());
>> CapturedStmtInfo->setContextValue(Builder.CreateLoad(DeclPtr));
>>
>> // Initialize variable-length arrays.
>>
>> Modified: cfe/trunk/lib/CodeGen/CGStmtOpenMP.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGStmtOpenMP.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGStmtOpenMP.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGStmtOpenMP.cpp Tue Sep 8 03:05:57 2015
>> @@ -24,17 +24,19 @@ using namespace CodeGen;
>> // OpenMP Directive Emission
>> //===----------------------------------------------------------------------===//
>> void CodeGenFunction::EmitOMPAggregateAssign(
>> - llvm::Value *DestAddr, llvm::Value *SrcAddr, QualType OriginalType,
>> - const llvm::function_ref<void(llvm::Value *, llvm::Value *)> &CopyGen) {
>> + Address DestAddr, Address SrcAddr, QualType OriginalType,
>> + const llvm::function_ref<void(Address, Address)> &CopyGen) {
>> // Perform element-by-element initialization.
>> QualType ElementTy;
>> - auto SrcBegin = SrcAddr;
>> - auto DestBegin = DestAddr;
>> +
>> + // Drill down to the base element type on both arrays.
>> auto ArrayTy = OriginalType->getAsArrayTypeUnsafe();
>> - auto NumElements = emitArrayLength(ArrayTy, ElementTy, DestBegin);
>> + auto NumElements = emitArrayLength(ArrayTy, ElementTy, DestAddr);
>> + SrcAddr = Builder.CreateElementBitCast(SrcAddr, DestAddr.getElementType());
>> +
>> + auto SrcBegin = SrcAddr.getPointer();
>> + auto DestBegin = DestAddr.getPointer();
>> // Cast from pointer to array type to pointer to single element.
>> - SrcBegin = Builder.CreatePointerBitCastOrAddrSpaceCast(SrcBegin,
>> - DestBegin->getType());
>> auto DestEnd = Builder.CreateGEP(DestBegin, NumElements);
>> // The basic structure here is a while-do loop.
>> auto BodyBB = createBasicBlock("omp.arraycpy.body");
>> @@ -46,69 +48,77 @@ void CodeGenFunction::EmitOMPAggregateAs
>> // Enter the loop body, making that address the current address.
>> auto EntryBB = Builder.GetInsertBlock();
>> EmitBlock(BodyBB);
>> - auto SrcElementCurrent =
>> - Builder.CreatePHI(SrcBegin->getType(), 2, "omp.arraycpy.srcElementPast");
>> - SrcElementCurrent->addIncoming(SrcBegin, EntryBB);
>> - auto DestElementCurrent = Builder.CreatePHI(DestBegin->getType(), 2,
>> - "omp.arraycpy.destElementPast");
>> - DestElementCurrent->addIncoming(DestBegin, EntryBB);
>> +
>> + CharUnits ElementSize = getContext().getTypeSizeInChars(ElementTy);
>> +
>> + llvm::PHINode *SrcElementPHI =
>> + Builder.CreatePHI(SrcBegin->getType(), 2, "omp.arraycpy.srcElementPast");
>> + SrcElementPHI->addIncoming(SrcBegin, EntryBB);
>> + Address SrcElementCurrent =
>> + Address(SrcElementPHI,
>> + SrcAddr.getAlignment().alignmentOfArrayElement(ElementSize));
>> +
>> + llvm::PHINode *DestElementPHI =
>> + Builder.CreatePHI(DestBegin->getType(), 2, "omp.arraycpy.destElementPast");
>> + DestElementPHI->addIncoming(DestBegin, EntryBB);
>> + Address DestElementCurrent =
>> + Address(DestElementPHI,
>> + DestAddr.getAlignment().alignmentOfArrayElement(ElementSize));
>>
>> // Emit copy.
>> CopyGen(DestElementCurrent, SrcElementCurrent);
>>
>> // Shift the address forward by one element.
>> auto DestElementNext = Builder.CreateConstGEP1_32(
>> - DestElementCurrent, /*Idx0=*/1, "omp.arraycpy.dest.element");
>> + DestElementPHI, /*Idx0=*/1, "omp.arraycpy.dest.element");
>> auto SrcElementNext = Builder.CreateConstGEP1_32(
>> - SrcElementCurrent, /*Idx0=*/1, "omp.arraycpy.src.element");
>> + SrcElementPHI, /*Idx0=*/1, "omp.arraycpy.src.element");
>> // Check whether we've reached the end.
>> auto Done =
>> Builder.CreateICmpEQ(DestElementNext, DestEnd, "omp.arraycpy.done");
>> Builder.CreateCondBr(Done, DoneBB, BodyBB);
>> - DestElementCurrent->addIncoming(DestElementNext, Builder.GetInsertBlock());
>> - SrcElementCurrent->addIncoming(SrcElementNext, Builder.GetInsertBlock());
>> + DestElementPHI->addIncoming(DestElementNext, Builder.GetInsertBlock());
>> + SrcElementPHI->addIncoming(SrcElementNext, Builder.GetInsertBlock());
>>
>> // Done.
>> EmitBlock(DoneBB, /*IsFinished=*/true);
>> }
>>
>> -void CodeGenFunction::EmitOMPCopy(CodeGenFunction &CGF,
>> - QualType OriginalType, llvm::Value *DestAddr,
>> - llvm::Value *SrcAddr, const VarDecl *DestVD,
>> +void CodeGenFunction::EmitOMPCopy(QualType OriginalType, Address DestAddr,
>> + Address SrcAddr, const VarDecl *DestVD,
>> const VarDecl *SrcVD, const Expr *Copy) {
>> if (OriginalType->isArrayType()) {
>> auto *BO = dyn_cast<BinaryOperator>(Copy);
>> if (BO && BO->getOpcode() == BO_Assign) {
>> // Perform simple memcpy for simple copying.
>> - CGF.EmitAggregateAssign(DestAddr, SrcAddr, OriginalType);
>> + EmitAggregateAssign(DestAddr, SrcAddr, OriginalType);
>> } else {
>> // For arrays with complex element types perform element by element
>> // copying.
>> - CGF.EmitOMPAggregateAssign(
>> + EmitOMPAggregateAssign(
>> DestAddr, SrcAddr, OriginalType,
>> - [&CGF, Copy, SrcVD, DestVD](llvm::Value *DestElement,
>> - llvm::Value *SrcElement) {
>> + [this, Copy, SrcVD, DestVD](Address DestElement, Address SrcElement) {
>> // Working with the single array element, so have to remap
>> // destination and source variables to corresponding array
>> // elements.
>> - CodeGenFunction::OMPPrivateScope Remap(CGF);
>> - Remap.addPrivate(DestVD, [DestElement]() -> llvm::Value *{
>> + CodeGenFunction::OMPPrivateScope Remap(*this);
>> + Remap.addPrivate(DestVD, [DestElement]() -> Address {
>> return DestElement;
>> });
>> Remap.addPrivate(
>> - SrcVD, [SrcElement]() -> llvm::Value *{ return SrcElement; });
>> + SrcVD, [SrcElement]() -> Address { return SrcElement; });
>> (void)Remap.Privatize();
>> - CGF.EmitIgnoredExpr(Copy);
>> + EmitIgnoredExpr(Copy);
>> });
>> }
>> } else {
>> // Remap pseudo source variable to private copy.
>> - CodeGenFunction::OMPPrivateScope Remap(CGF);
>> - Remap.addPrivate(SrcVD, [SrcAddr]() -> llvm::Value *{ return SrcAddr; });
>> - Remap.addPrivate(DestVD, [DestAddr]() -> llvm::Value *{ return DestAddr; });
>> + CodeGenFunction::OMPPrivateScope Remap(*this);
>> + Remap.addPrivate(SrcVD, [SrcAddr]() -> Address { return SrcAddr; });
>> + Remap.addPrivate(DestVD, [DestAddr]() -> Address { return DestAddr; });
>> (void)Remap.Privatize();
>> // Emit copying of the whole variable.
>> - CGF.EmitIgnoredExpr(Copy);
>> + EmitIgnoredExpr(Copy);
>> }
>> }
>>
>> @@ -130,13 +140,13 @@ bool CodeGenFunction::EmitOMPFirstprivat
>> /*RefersToEnclosingVariableOrCapture=*/CapturedStmtInfo->lookup(
>> OrigVD) != nullptr,
>> (*IRef)->getType(), VK_LValue, (*IRef)->getExprLoc());
>> - auto *OriginalAddr = EmitLValue(&DRE).getAddress();
>> + Address OriginalAddr = EmitLValue(&DRE).getAddress();
>> QualType Type = OrigVD->getType();
>> if (Type->isArrayType()) {
>> // Emit VarDecl with copy init for arrays.
>> // Get the address of the original variable captured in current
>> // captured region.
>> - IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> llvm::Value *{
>> + IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> Address {
>> auto Emission = EmitAutoVarAlloca(*VD);
>> auto *Init = VD->getInit();
>> if (!isa<CXXConstructExpr>(Init) || isTrivialInitializer(Init)) {
>> @@ -146,12 +156,12 @@ bool CodeGenFunction::EmitOMPFirstprivat
>> } else {
>> EmitOMPAggregateAssign(
>> Emission.getAllocatedAddress(), OriginalAddr, Type,
>> - [this, VDInit, Init](llvm::Value *DestElement,
>> - llvm::Value *SrcElement) {
>> + [this, VDInit, Init](Address DestElement,
>> + Address SrcElement) {
>> // Clean up any temporaries needed by the initialization.
>> RunCleanupsScope InitScope(*this);
>> // Emit initialization for single element.
>> - LocalDeclMap[VDInit] = SrcElement;
>> + setAddrOfLocalVar(VDInit, SrcElement);
>> EmitAnyExprToMem(Init, DestElement,
>> Init->getType().getQualifiers(),
>> /*IsInitializer*/ false);
>> @@ -162,12 +172,12 @@ bool CodeGenFunction::EmitOMPFirstprivat
>> return Emission.getAllocatedAddress();
>> });
>> } else {
>> - IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> llvm::Value *{
>> + IsRegistered = PrivateScope.addPrivate(OrigVD, [&]() -> Address {
>> // Emit private VarDecl with copy init.
>> // Remap temp VDInit variable to the address of the original
>> // variable
>> // (for proper handling of captured global variables).
>> - LocalDeclMap[VDInit] = OriginalAddr;
>> + setAddrOfLocalVar(VDInit, OriginalAddr);
>> EmitDecl(*VD);
>> LocalDeclMap.erase(VDInit);
>> return GetAddrOfLocalVar(VD);
>> @@ -195,7 +205,7 @@ void CodeGenFunction::EmitOMPPrivateClau
>> if (EmittedAsPrivate.insert(OrigVD->getCanonicalDecl()).second) {
>> auto VD = cast<VarDecl>(cast<DeclRefExpr>(IInit)->getDecl());
>> bool IsRegistered =
>> - PrivateScope.addPrivate(OrigVD, [&]() -> llvm::Value *{
>> + PrivateScope.addPrivate(OrigVD, [&]() -> Address {
>> // Emit private VarDecl with copy init.
>> EmitDecl(*VD);
>> return GetAddrOfLocalVar(VD);
>> @@ -228,7 +238,7 @@ bool CodeGenFunction::EmitOMPCopyinClaus
>> // Get the address of the master variable. If we are emitting code with
>> // TLS support, the address is passed from the master as field in the
>> // captured declaration.
>> - llvm::Value *MasterAddr;
>> + Address MasterAddr = Address::invalid();
>> if (getLangOpts().OpenMPUseTLS &&
>> getContext().getTargetInfo().isTLSSupported()) {
>> assert(CapturedStmtInfo->lookup(VD) &&
>> @@ -237,11 +247,13 @@ bool CodeGenFunction::EmitOMPCopyinClaus
>> VK_LValue, (*IRef)->getExprLoc());
>> MasterAddr = EmitLValue(&DRE).getAddress();
>> } else {
>> - MasterAddr = VD->isStaticLocal() ? CGM.getStaticLocalDeclAddress(VD)
>> - : CGM.GetAddrOfGlobal(VD);
>> + MasterAddr =
>> + Address(VD->isStaticLocal() ? CGM.getStaticLocalDeclAddress(VD)
>> + : CGM.GetAddrOfGlobal(VD),
>> + getContext().getDeclAlign(VD));
>> }
>> // Get the address of the threadprivate variable.
>> - auto *PrivateAddr = EmitLValue(*IRef).getAddress();
>> + Address PrivateAddr = EmitLValue(*IRef).getAddress();
>> if (CopiedVars.size() == 1) {
>> // At first check if current thread is a master thread. If it is, no
>> // need to copy data.
>> @@ -249,15 +261,14 @@ bool CodeGenFunction::EmitOMPCopyinClaus
>> CopyEnd = createBasicBlock("copyin.not.master.end");
>> Builder.CreateCondBr(
>> Builder.CreateICmpNE(
>> - Builder.CreatePtrToInt(MasterAddr, CGM.IntPtrTy),
>> - Builder.CreatePtrToInt(PrivateAddr, CGM.IntPtrTy)),
>> + Builder.CreatePtrToInt(MasterAddr.getPointer(), CGM.IntPtrTy),
>> + Builder.CreatePtrToInt(PrivateAddr.getPointer(), CGM.IntPtrTy)),
>> CopyBegin, CopyEnd);
>> EmitBlock(CopyBegin);
>> }
>> auto *SrcVD = cast<VarDecl>(cast<DeclRefExpr>(*ISrcRef)->getDecl());
>> auto *DestVD = cast<VarDecl>(cast<DeclRefExpr>(*IDestRef)->getDecl());
>> - EmitOMPCopy(*this, Type, PrivateAddr, MasterAddr, DestVD, SrcVD,
>> - AssignOp);
>> + EmitOMPCopy(Type, PrivateAddr, MasterAddr, DestVD, SrcVD, AssignOp);
>> }
>> ++IRef;
>> ++ISrcRef;
>> @@ -286,7 +297,7 @@ bool CodeGenFunction::EmitOMPLastprivate
>> auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>(*IRef)->getDecl());
>> if (AlreadyEmittedVars.insert(OrigVD->getCanonicalDecl()).second) {
>> auto *DestVD = cast<VarDecl>(cast<DeclRefExpr>(*IDestRef)->getDecl());
>> - PrivateScope.addPrivate(DestVD, [this, OrigVD, IRef]() -> llvm::Value *{
>> + PrivateScope.addPrivate(DestVD, [this, OrigVD, IRef]() -> Address {
>> DeclRefExpr DRE(
>> const_cast<VarDecl *>(OrigVD),
>> /*RefersToEnclosingVariableOrCapture=*/CapturedStmtInfo->lookup(
>> @@ -300,7 +311,7 @@ bool CodeGenFunction::EmitOMPLastprivate
>> if (IInit) {
>> auto *VD = cast<VarDecl>(cast<DeclRefExpr>(IInit)->getDecl());
>> bool IsRegistered =
>> - PrivateScope.addPrivate(OrigVD, [&]() -> llvm::Value *{
>> + PrivateScope.addPrivate(OrigVD, [&]() -> Address {
>> // Emit private VarDecl with copy init.
>> EmitDecl(*VD);
>> return GetAddrOfLocalVar(VD);
>> @@ -380,17 +391,14 @@ void CodeGenFunction::EmitOMPLastprivate
>> auto *SrcVD = cast<VarDecl>(cast<DeclRefExpr>(*ISrcRef)->getDecl());
>> auto *DestVD = cast<VarDecl>(cast<DeclRefExpr>(*IDestRef)->getDecl());
>> // Get the address of the original variable.
>> - auto *OriginalAddr = GetAddrOfLocalVar(DestVD);
>> + Address OriginalAddr = GetAddrOfLocalVar(DestVD);
>> // Get the address of the private variable.
>> - auto *PrivateAddr = GetAddrOfLocalVar(PrivateVD);
>> - if (PrivateVD->getType()->isReferenceType())
>> + Address PrivateAddr = GetAddrOfLocalVar(PrivateVD);
>> + if (auto RefTy = PrivateVD->getType()->getAs<ReferenceType>())
>> PrivateAddr =
>> - EmitLoadOfLValue(MakeNaturalAlignAddrLValue(
>> - PrivateAddr, PrivateVD->getType()),
>> - (*IRef)->getExprLoc())
>> - .getScalarVal();
>> - EmitOMPCopy(*this, Type, OriginalAddr, PrivateAddr, DestVD, SrcVD,
>> - AssignOp);
>> + Address(Builder.CreateLoad(PrivateAddr),
>> + getNaturalTypeAlignment(RefTy->getPointeeType()));
>> + EmitOMPCopy(Type, OriginalAddr, PrivateAddr, DestVD, SrcVD, AssignOp);
>> }
>> ++IRef;
>> ++ISrcRef;
>> @@ -415,7 +423,7 @@ void CodeGenFunction::EmitOMPReductionCl
>> auto *PrivateVD = cast<VarDecl>(cast<DeclRefExpr>(*IRHS)->getDecl());
>> // Store the address of the original variable associated with the LHS
>> // implicit variable.
>> - PrivateScope.addPrivate(LHSVD, [this, OrigVD, IRef]() -> llvm::Value *{
>> + PrivateScope.addPrivate(LHSVD, [this, OrigVD, IRef]() -> Address {
>> DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
>> CapturedStmtInfo->lookup(OrigVD) != nullptr,
>> IRef->getType(), VK_LValue, IRef->getExprLoc());
>> @@ -423,7 +431,7 @@ void CodeGenFunction::EmitOMPReductionCl
>> });
>> // Emit reduction copy.
>> bool IsRegistered =
>> - PrivateScope.addPrivate(OrigVD, [this, PrivateVD]() -> llvm::Value *{
>> + PrivateScope.addPrivate(OrigVD, [this, PrivateVD]() -> Address {
>> // Emit private VarDecl with reduction init.
>> EmitDecl(*PrivateVD);
>> return GetAddrOfLocalVar(PrivateVD);
>> @@ -607,8 +615,7 @@ void CodeGenFunction::EmitOMPLinearClaus
>> VD->getInit()->getExprLoc());
>> AutoVarEmission Emission = EmitAutoVarAlloca(*VD);
>> EmitExprAsInit(&DRE, VD,
>> - MakeAddrLValue(Emission.getAllocatedAddress(),
>> - VD->getType(), Emission.Alignment),
>> + MakeAddrLValue(Emission.getAllocatedAddress(), VD->getType()),
>> /*capturedByInit=*/false);
>> EmitAutoVarCleanups(Emission);
>> }
>> @@ -633,10 +640,10 @@ static void emitLinearClauseFinal(CodeGe
>> DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
>> CGF.CapturedStmtInfo->lookup(OrigVD) != nullptr,
>> (*IC)->getType(), VK_LValue, (*IC)->getExprLoc());
>> - auto *OrigAddr = CGF.EmitLValue(&DRE).getAddress();
>> + Address OrigAddr = CGF.EmitLValue(&DRE).getAddress();
>> CodeGenFunction::OMPPrivateScope VarScope(CGF);
>> VarScope.addPrivate(OrigVD,
>> - [OrigAddr]() -> llvm::Value *{ return OrigAddr; });
>> + [OrigAddr]() -> Address { return OrigAddr; });
>> (void)VarScope.Privatize();
>> CGF.EmitIgnoredExpr(F);
>> ++IC;
>> @@ -683,15 +690,15 @@ static void emitPrivateLoopCounters(Code
>> for (auto *E : Counters) {
>> auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
>> auto *PrivateVD = cast<VarDecl>(cast<DeclRefExpr>(*I)->getDecl());
>> - llvm::Value *Addr;
>> - (void)LoopScope.addPrivate(PrivateVD, [&]() -> llvm::Value * {
>> + Address Addr = Address::invalid();
>> + (void)LoopScope.addPrivate(PrivateVD, [&]() -> Address {
>> // Emit var without initialization.
>> auto VarEmission = CGF.EmitAutoVarAlloca(*PrivateVD);
>> CGF.EmitAutoVarCleanups(VarEmission);
>> Addr = VarEmission.getAllocatedAddress();
>> return Addr;
>> });
>> - (void)LoopScope.addPrivate(VD, [&]() -> llvm::Value * { return Addr; });
>> + (void)LoopScope.addPrivate(VD, [&]() -> Address { return Addr; });
>> ++I;
>> }
>> }
>> @@ -722,7 +729,7 @@ emitPrivateLinearVars(CodeGenFunction &C
>> auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
>> auto *PrivateVD =
>> cast<VarDecl>(cast<DeclRefExpr>(*CurPrivate)->getDecl());
>> - bool IsRegistered = PrivateScope.addPrivate(VD, [&]() -> llvm::Value * {
>> + bool IsRegistered = PrivateScope.addPrivate(VD, [&]() -> Address {
>> // Emit private VarDecl with copy init.
>> CGF.EmitVarDecl(*PrivateVD);
>> return CGF.GetAddrOfLocalVar(PrivateVD);
>> @@ -769,14 +776,14 @@ void CodeGenFunction::EmitOMPSimdFinal(c
>> auto IC = D.counters().begin();
>> for (auto F : D.finals()) {
>> auto *OrigVD = cast<VarDecl>(cast<DeclRefExpr>((*IC))->getDecl());
>> - if (LocalDeclMap.lookup(OrigVD) || CapturedStmtInfo->lookup(OrigVD)) {
>> + if (LocalDeclMap.count(OrigVD) || CapturedStmtInfo->lookup(OrigVD)) {
>> DeclRefExpr DRE(const_cast<VarDecl *>(OrigVD),
>> CapturedStmtInfo->lookup(OrigVD) != nullptr,
>> (*IC)->getType(), VK_LValue, (*IC)->getExprLoc());
>> - auto *OrigAddr = EmitLValue(&DRE).getAddress();
>> + Address OrigAddr = EmitLValue(&DRE).getAddress();
>> OMPPrivateScope VarScope(*this);
>> VarScope.addPrivate(OrigVD,
>> - [OrigAddr]() -> llvm::Value *{ return OrigAddr; });
>> + [OrigAddr]() -> Address { return OrigAddr; });
>> (void)VarScope.Privatize();
>> EmitIgnoredExpr(F);
>> }
>> @@ -865,9 +872,9 @@ void CodeGenFunction::EmitOMPSimdDirecti
>> void CodeGenFunction::EmitOMPForOuterLoop(OpenMPScheduleClauseKind ScheduleKind,
>> const OMPLoopDirective &S,
>> OMPPrivateScope &LoopScope,
>> - bool Ordered, llvm::Value *LB,
>> - llvm::Value *UB, llvm::Value *ST,
>> - llvm::Value *IL, llvm::Value *Chunk) {
>> + bool Ordered, Address LB,
>> + Address UB, Address ST,
>> + Address IL, llvm::Value *Chunk) {
>> auto &RT = CGM.getOpenMPRuntime();
>>
>> // Dynamic scheduling of the outer loop (dynamic, guided, auto, runtime).
>> @@ -931,11 +938,14 @@ void CodeGenFunction::EmitOMPForOuterLoo
>> const unsigned IVSize = getContext().getTypeSize(IVExpr->getType());
>> const bool IVSigned = IVExpr->getType()->hasSignedIntegerRepresentation();
>>
>> - RT.emitForInit(
>> - *this, S.getLocStart(), ScheduleKind, IVSize, IVSigned, Ordered, IL, LB,
>> - (DynamicOrOrdered ? EmitAnyExpr(S.getLastIteration()).getScalarVal()
>> - : UB),
>> - ST, Chunk);
>> + if (DynamicOrOrdered) {
>> + llvm::Value *UBVal = EmitScalarExpr(S.getLastIteration());
>> + RT.emitForDispatchInit(*this, S.getLocStart(), ScheduleKind,
>> + IVSize, IVSigned, Ordered, UBVal, Chunk);
>> + } else {
>> + RT.emitForStaticInit(*this, S.getLocStart(), ScheduleKind,
>> + IVSize, IVSigned, Ordered, IL, LB, UB, ST, Chunk);
>> + }
>>
>> auto LoopExit = getJumpDestInCurrentScope("omp.dispatch.end");
>>
>> @@ -1044,8 +1054,8 @@ emitScheduleClause(CodeGenFunction &CGF,
>> CGF.EmitVarDecl(*ImpVar);
>> CGF.EmitStoreThroughLValue(
>> CGF.EmitAnyExpr(Ch),
>> - CGF.MakeNaturalAlignAddrLValue(CGF.GetAddrOfLocalVar(ImpVar),
>> - ImpVar->getType()));
>> + CGF.MakeAddrLValue(CGF.GetAddrOfLocalVar(ImpVar),
>> + ImpVar->getType()));
>> } else {
>> Ch = ImpRef;
>> }
>> @@ -1148,9 +1158,10 @@ bool CodeGenFunction::EmitOMPWorksharing
>> // chunks that are approximately equal in size, and at most one chunk is
>> // distributed to each thread. Note that the size of the chunks is
>> // unspecified in this case.
>> - RT.emitForInit(*this, S.getLocStart(), ScheduleKind, IVSize, IVSigned,
>> - Ordered, IL.getAddress(), LB.getAddress(),
>> - UB.getAddress(), ST.getAddress());
>> + RT.emitForStaticInit(*this, S.getLocStart(), ScheduleKind,
>> + IVSize, IVSigned, Ordered,
>> + IL.getAddress(), LB.getAddress(),
>> + UB.getAddress(), ST.getAddress());
>> auto LoopExit = getJumpDestInCurrentScope(createBasicBlock("omp.loop.exit"));
>> // UB = min(UB, GlobalUB);
>> EmitIgnoredExpr(S.getEnsureUpperBound());
>> @@ -1223,7 +1234,7 @@ void CodeGenFunction::EmitOMPForSimdDire
>> static LValue createSectionLVal(CodeGenFunction &CGF, QualType Ty,
>> const Twine &Name,
>> llvm::Value *Init = nullptr) {
>> - auto LVal = CGF.MakeNaturalAlignAddrLValue(CGF.CreateMemTemp(Ty, Name), Ty);
>> + auto LVal = CGF.MakeAddrLValue(CGF.CreateMemTemp(Ty, Name), Ty);
>> if (Init)
>> CGF.EmitScalarInit(Init, LVal);
>> return LVal;
>> @@ -1302,7 +1313,7 @@ CodeGenFunction::EmitSections(const OMPE
>> (void)LoopScope.Privatize();
>>
>> // Emit static non-chunked loop.
>> - CGF.CGM.getOpenMPRuntime().emitForInit(
>> + CGF.CGM.getOpenMPRuntime().emitForStaticInit(
>> CGF, S.getLocStart(), OMPC_SCHEDULE_static, /*IVSize=*/32,
>> /*IVSigned=*/true, /*Ordered=*/false, IL.getAddress(),
>> LB.getAddress(), UB.getAddress(), ST.getAddress());
>> @@ -1556,35 +1567,33 @@ void CodeGenFunction::EmitOMPTaskDirecti
>> auto *CS = cast<CapturedStmt>(S.getAssociatedStmt());
>> OMPPrivateScope Scope(CGF);
>> if (!PrivateVars.empty() || !FirstprivateVars.empty()) {
>> - auto *CopyFn = CGF.Builder.CreateAlignedLoad(
>> - CGF.GetAddrOfLocalVar(CS->getCapturedDecl()->getParam(3)),
>> - CGF.PointerAlignInBytes);
>> - auto *PrivatesPtr = CGF.Builder.CreateAlignedLoad(
>> - CGF.GetAddrOfLocalVar(CS->getCapturedDecl()->getParam(2)),
>> - CGF.PointerAlignInBytes);
>> + auto *CopyFn = CGF.Builder.CreateLoad(
>> + CGF.GetAddrOfLocalVar(CS->getCapturedDecl()->getParam(3)));
>> + auto *PrivatesPtr = CGF.Builder.CreateLoad(
>> + CGF.GetAddrOfLocalVar(CS->getCapturedDecl()->getParam(2)));
>> // Map privates.
>> - llvm::SmallVector<std::pair<const VarDecl *, llvm::Value *>, 16>
>> + llvm::SmallVector<std::pair<const VarDecl *, Address>, 16>
>> PrivatePtrs;
>> llvm::SmallVector<llvm::Value *, 16> CallArgs;
>> CallArgs.push_back(PrivatesPtr);
>> for (auto *E : PrivateVars) {
>> auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
>> - auto *PrivatePtr =
>> + Address PrivatePtr =
>> CGF.CreateMemTemp(CGF.getContext().getPointerType(E->getType()));
>> PrivatePtrs.push_back(std::make_pair(VD, PrivatePtr));
>> - CallArgs.push_back(PrivatePtr);
>> + CallArgs.push_back(PrivatePtr.getPointer());
>> }
>> for (auto *E : FirstprivateVars) {
>> auto *VD = cast<VarDecl>(cast<DeclRefExpr>(E)->getDecl());
>> - auto *PrivatePtr =
>> + Address PrivatePtr =
>> CGF.CreateMemTemp(CGF.getContext().getPointerType(E->getType()));
>> PrivatePtrs.push_back(std::make_pair(VD, PrivatePtr));
>> - CallArgs.push_back(PrivatePtr);
>> + CallArgs.push_back(PrivatePtr.getPointer());
>> }
>> CGF.EmitRuntimeCall(CopyFn, CallArgs);
>> for (auto &&Pair : PrivatePtrs) {
>> - auto *Replacement =
>> - CGF.Builder.CreateAlignedLoad(Pair.second, CGF.PointerAlignInBytes);
>> + Address Replacement(CGF.Builder.CreateLoad(Pair.second),
>> + CGF.getContext().getDeclAlign(Pair.first));
>> Scope.addPrivate(Pair.first, [Replacement]() { return Replacement; });
>> }
>> }
>> @@ -1787,8 +1796,8 @@ static std::pair<bool, RValue> emitOMPAt
>> !Update.getScalarVal()->getType()->isIntegerTy() ||
>> !X.isSimple() || (!isa<llvm::ConstantInt>(Update.getScalarVal()) &&
>> (Update.getScalarVal()->getType() !=
>> - X.getAddress()->getType()->getPointerElementType())) ||
>> - !X.getAddress()->getType()->getPointerElementType()->isIntegerTy() ||
>> + X.getAddress().getElementType())) ||
>> + !X.getAddress().getElementType()->isIntegerTy() ||
>> !Context.getTargetInfo().hasBuiltinAtomic(
>> Context.getTypeSize(X.getType()), Context.toBits(X.getAlignment())))
>> return std::make_pair(false, RValue::get(nullptr));
>> @@ -1859,10 +1868,10 @@ static std::pair<bool, RValue> emitOMPAt
>> auto *UpdateVal = Update.getScalarVal();
>> if (auto *IC = dyn_cast<llvm::ConstantInt>(UpdateVal)) {
>> UpdateVal = CGF.Builder.CreateIntCast(
>> - IC, X.getAddress()->getType()->getPointerElementType(),
>> + IC, X.getAddress().getElementType(),
>> X.getType()->hasSignedIntegerRepresentation());
>> }
>> - auto *Res = CGF.Builder.CreateAtomicRMW(RMWOp, X.getAddress(), UpdateVal, AO);
>> + auto *Res = CGF.Builder.CreateAtomicRMW(RMWOp, X.getPointer(), UpdateVal, AO);
>> return std::make_pair(true, RValue::get(Res));
>> }
>>
>>
>> Modified: cfe/trunk/lib/CodeGen/CGVTables.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGVTables.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGVTables.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CGVTables.cpp Tue Sep 8 03:05:57 2015
>> @@ -102,8 +102,11 @@ static RValue PerformReturnAdjustment(Co
>> CGF.EmitBlock(AdjustNotNull);
>> }
>>
>> - ReturnValue = CGF.CGM.getCXXABI().performReturnAdjustment(CGF, ReturnValue,
>> - Thunk.Return);
>> + auto ClassDecl = ResultType->getPointeeType()->getAsCXXRecordDecl();
>> + auto ClassAlign = CGF.CGM.getClassPointerAlignment(ClassDecl);
>> + ReturnValue = CGF.CGM.getCXXABI().performReturnAdjustment(CGF,
>> + Address(ReturnValue, ClassAlign),
>> + Thunk.Return);
>>
>> if (NullCheckValue) {
>> CGF.Builder.CreateBr(AdjustEnd);
>> @@ -171,11 +174,11 @@ CodeGenFunction::GenerateVarArgsThunk(ll
>>
>> // Find the first store of "this", which will be to the alloca associated
>> // with "this".
>> - llvm::Value *ThisPtr = &*AI;
>> + Address ThisPtr(&*AI, CGM.getClassPointerAlignment(MD->getParent()));
>> llvm::BasicBlock *EntryBB = Fn->begin();
>> llvm::Instruction *ThisStore =
>> std::find_if(EntryBB->begin(), EntryBB->end(), [&](llvm::Instruction &I) {
>> - return isa<llvm::StoreInst>(I) && I.getOperand(0) == ThisPtr;
>> + return isa<llvm::StoreInst>(I) && I.getOperand(0) == ThisPtr.getPointer();
>> });
>> assert(ThisStore && "Store of this should be in entry block?");
>> // Adjust "this", if necessary.
>> @@ -235,6 +238,17 @@ void CodeGenFunction::StartThunk(llvm::F
>> // Since we didn't pass a GlobalDecl to StartFunction, do this ourselves.
>> CGM.getCXXABI().EmitInstanceFunctionProlog(*this);
>> CXXThisValue = CXXABIThisValue;
>> + CurCodeDecl = MD;
>> + CurFuncDecl = MD;
>> +}
>> +
>> +void CodeGenFunction::FinishThunk() {
>> + // Clear these to restore the invariants expected by
>> + // StartFunction/FinishFunction.
>> + CurCodeDecl = nullptr;
>> + CurFuncDecl = nullptr;
>> +
>> + FinishFunction();
>> }
>>
>> void CodeGenFunction::EmitCallAndReturnForThunk(llvm::Value *Callee,
>> @@ -244,9 +258,10 @@ void CodeGenFunction::EmitCallAndReturnF
>> const CXXMethodDecl *MD = cast<CXXMethodDecl>(CurGD.getDecl());
>>
>> // Adjust the 'this' pointer if necessary
>> - llvm::Value *AdjustedThisPtr = Thunk ? CGM.getCXXABI().performThisAdjustment(
>> - *this, LoadCXXThis(), Thunk->This)
>> - : LoadCXXThis();
>> + llvm::Value *AdjustedThisPtr =
>> + Thunk ? CGM.getCXXABI().performThisAdjustment(
>> + *this, LoadCXXThisAddress(), Thunk->This)
>> + : LoadCXXThis();
>>
>> if (CurFnInfo->usesInAlloca()) {
>> // We don't handle return adjusting thunks, because they require us to call
>> @@ -321,7 +336,7 @@ void CodeGenFunction::EmitCallAndReturnF
>> // Disable the final ARC autorelease.
>> AutoreleaseResult = false;
>>
>> - FinishFunction();
>> + FinishThunk();
>> }
>>
>> void CodeGenFunction::EmitMustTailThunk(const CXXMethodDecl *MD,
>> @@ -346,9 +361,8 @@ void CodeGenFunction::EmitMustTailThunk(
>> Args[ThisArgNo] = AdjustedThisPtr;
>> } else {
>> assert(ThisAI.isInAlloca() && "this is passed directly or inalloca");
>> - llvm::Value *ThisAddr = GetAddrOfLocalVar(CXXABIThisDecl);
>> - llvm::Type *ThisType =
>> - cast<llvm::PointerType>(ThisAddr->getType())->getElementType();
>> + Address ThisAddr = GetAddrOfLocalVar(CXXABIThisDecl);
>> + llvm::Type *ThisType = ThisAddr.getElementType();
>> if (ThisType != AdjustedThisPtr->getType())
>> AdjustedThisPtr = Builder.CreateBitCast(AdjustedThisPtr, ThisType);
>> Builder.CreateStore(AdjustedThisPtr, ThisAddr);
>>
>> Modified: cfe/trunk/lib/CodeGen/CGValue.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGValue.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CGValue.h (original)
>> +++ cfe/trunk/lib/CodeGen/CGValue.h Tue Sep 8 03:05:57 2015
>> @@ -16,10 +16,10 @@
>> #define LLVM_CLANG_LIB_CODEGEN_CGVALUE_H
>>
>> #include "clang/AST/ASTContext.h"
>> -#include "clang/AST/CharUnits.h"
>> #include "clang/AST/Type.h"
>> #include "llvm/IR/Value.h"
>> #include "llvm/IR/Type.h"
>> +#include "Address.h"
>>
>> namespace llvm {
>> class Constant;
>> @@ -38,6 +38,10 @@ namespace CodeGen {
>> class RValue {
>> enum Flavor { Scalar, Complex, Aggregate };
>>
>> + // The shift to make to an aggregate's alignment to make it look
>> + // like a pointer.
>> + enum { AggAlignShift = 4 };
>> +
>> // Stores first value and flavor.
>> llvm::PointerIntPair<llvm::Value *, 2, Flavor> V1;
>> // Stores second value and volatility.
>> @@ -63,11 +67,21 @@ public:
>> }
>>
>> /// getAggregateAddr() - Return the Value* of the address of the aggregate.
>> - llvm::Value *getAggregateAddr() const {
>> + Address getAggregateAddress() const {
>> + assert(isAggregate() && "Not an aggregate!");
>> + auto align = reinterpret_cast<uintptr_t>(V2.getPointer()) >> AggAlignShift;
>> + return Address(V1.getPointer(), CharUnits::fromQuantity(align));
>> + }
>> + llvm::Value *getAggregatePointer() const {
>> assert(isAggregate() && "Not an aggregate!");
>> return V1.getPointer();
>> }
>>
>> + static RValue getIgnored() {
>> + // FIXME: should we make this a more explicit state?
>> + return get(nullptr);
>> + }
>> +
>> static RValue get(llvm::Value *V) {
>> RValue ER;
>> ER.V1.setPointer(V);
>> @@ -89,11 +103,14 @@ public:
>> // FIXME: Aggregate rvalues need to retain information about whether they are
>> // volatile or not. Remove default to find all places that probably get this
>> // wrong.
>> - static RValue getAggregate(llvm::Value *V, bool Volatile = false) {
>> + static RValue getAggregate(Address addr, bool isVolatile = false) {
>> RValue ER;
>> - ER.V1.setPointer(V);
>> + ER.V1.setPointer(addr.getPointer());
>> ER.V1.setInt(Aggregate);
>> - ER.V2.setInt(Volatile);
>> +
>> + auto align = static_cast<uintptr_t>(addr.getAlignment().getQuantity());
>> + ER.V2.setPointer(reinterpret_cast<llvm::Value*>(align << AggAlignShift));
>> + ER.V2.setInt(isVolatile);
>> return ER;
>> }
>> };
>> @@ -103,6 +120,32 @@ enum ARCPreciseLifetime_t {
>> ARCImpreciseLifetime, ARCPreciseLifetime
>> };
>>
>> +/// The source of the alignment of an l-value; an expression of
>> +/// confidence in the alignment actually matching the estimate.
>> +enum class AlignmentSource {
>> + /// The l-value was an access to a declared entity or something
>> + /// equivalently strong, like the address of an array allocated by a
>> + /// language runtime.
>> + Decl,
>> +
>> + /// The l-value was considered opaque, so the alignment was
>> + /// determined from a type, but that type was an explicitly-aligned
>> + /// typedef.
>> + AttributedType,
>> +
>> + /// The l-value was considered opaque, so the alignment was
>> + /// determined from a type.
>> + Type
>> +};
>> +
>> +/// Given that the base address has the given alignment source, what's
>> +/// our confidence in the alignment of the field?
>> +static inline AlignmentSource getFieldAlignmentSource(AlignmentSource Source) {
>> + // For now, we don't distinguish fields of opaque pointers from
>> + // top-level declarations, but maybe we should.
>> + return AlignmentSource::Decl;
>> +}
>> +
>> /// LValue - This represents an lvalue references. Because C/C++ allow
>> /// bitfields, this is not a simple LLVM pointer, it may be a pointer plus a
>> /// bitrange.
>> @@ -157,6 +200,8 @@ class LValue {
>> // to make the default bitfield pattern all-zeroes.
>> bool ImpreciseLifetime : 1;
>>
>> + unsigned AlignSource : 2;
>> +
>> Expr *BaseIvarExp;
>>
>> /// Used by struct-path-aware TBAA.
>> @@ -169,13 +214,16 @@ class LValue {
>>
>> private:
>> void Initialize(QualType Type, Qualifiers Quals,
>> - CharUnits Alignment,
>> + CharUnits Alignment, AlignmentSource AlignSource,
>> llvm::MDNode *TBAAInfo = nullptr) {
>> + assert((!Alignment.isZero() || Type->isIncompleteType()) &&
>> + "initializing l-value with zero alignment!");
>> this->Type = Type;
>> this->Quals = Quals;
>> this->Alignment = Alignment.getQuantity();
>> assert(this->Alignment == Alignment.getQuantity() &&
>> "Alignment exceeds allowed max!");
>> + this->AlignSource = unsigned(AlignSource);
>>
>> // Initialize Objective-C flags.
>> this->Ivar = this->ObjIsArray = this->NonGC = this->GlobalObjCRef = false;
>> @@ -261,29 +309,50 @@ public:
>> CharUnits getAlignment() const { return CharUnits::fromQuantity(Alignment); }
>> void setAlignment(CharUnits A) { Alignment = A.getQuantity(); }
>>
>> + AlignmentSource getAlignmentSource() const {
>> + return AlignmentSource(AlignSource);
>> + }
>> + void setAlignmentSource(AlignmentSource Source) {
>> + AlignSource = unsigned(Source);
>> + }
>> +
>> // simple lvalue
>> - llvm::Value *getAddress() const { assert(isSimple()); return V; }
>> - void setAddress(llvm::Value *address) {
>> + llvm::Value *getPointer() const {
>> assert(isSimple());
>> - V = address;
>> + return V;
>> + }
>> + Address getAddress() const { return Address(getPointer(), getAlignment()); }
>> + void setAddress(Address address) {
>> + assert(isSimple());
>> + V = address.getPointer();
>> + Alignment = address.getAlignment().getQuantity();
>> }
>>
>> // vector elt lvalue
>> - llvm::Value *getVectorAddr() const { assert(isVectorElt()); return V; }
>> + Address getVectorAddress() const {
>> + return Address(getVectorPointer(), getAlignment());
>> + }
>> + llvm::Value *getVectorPointer() const { assert(isVectorElt()); return V; }
>> llvm::Value *getVectorIdx() const { assert(isVectorElt()); return VectorIdx; }
>>
>> // extended vector elements.
>> - llvm::Value *getExtVectorAddr() const { assert(isExtVectorElt()); return V; }
>> + Address getExtVectorAddress() const {
>> + return Address(getExtVectorPointer(), getAlignment());
>> + }
>> + llvm::Value *getExtVectorPointer() const {
>> + assert(isExtVectorElt());
>> + return V;
>> + }
>> llvm::Constant *getExtVectorElts() const {
>> assert(isExtVectorElt());
>> return VectorElts;
>> }
>>
>> // bitfield lvalue
>> - llvm::Value *getBitFieldAddr() const {
>> - assert(isBitField());
>> - return V;
>> + Address getBitFieldAddress() const {
>> + return Address(getBitFieldPointer(), getAlignment());
>> }
>> + llvm::Value *getBitFieldPointer() const { assert(isBitField()); return V; }
>> const CGBitFieldInfo &getBitFieldInfo() const {
>> assert(isBitField());
>> return *BitFieldInfo;
>> @@ -292,37 +361,40 @@ public:
>> // global register lvalue
>> llvm::Value *getGlobalReg() const { assert(isGlobalReg()); return V; }
>>
>> - static LValue MakeAddr(llvm::Value *address, QualType type,
>> - CharUnits alignment, ASTContext &Context,
>> + static LValue MakeAddr(Address address, QualType type,
>> + ASTContext &Context,
>> + AlignmentSource alignSource,
>> llvm::MDNode *TBAAInfo = nullptr) {
>> Qualifiers qs = type.getQualifiers();
>> qs.setObjCGCAttr(Context.getObjCGCAttrKind(type));
>>
>> LValue R;
>> R.LVType = Simple;
>> - assert(address->getType()->isPointerTy());
>> - R.V = address;
>> - R.Initialize(type, qs, alignment, TBAAInfo);
>> + assert(address.getPointer()->getType()->isPointerTy());
>> + R.V = address.getPointer();
>> + R.Initialize(type, qs, address.getAlignment(), alignSource, TBAAInfo);
>> return R;
>> }
>>
>> - static LValue MakeVectorElt(llvm::Value *Vec, llvm::Value *Idx,
>> - QualType type, CharUnits Alignment) {
>> + static LValue MakeVectorElt(Address vecAddress, llvm::Value *Idx,
>> + QualType type, AlignmentSource alignSource) {
>> LValue R;
>> R.LVType = VectorElt;
>> - R.V = Vec;
>> + R.V = vecAddress.getPointer();
>> R.VectorIdx = Idx;
>> - R.Initialize(type, type.getQualifiers(), Alignment);
>> + R.Initialize(type, type.getQualifiers(), vecAddress.getAlignment(),
>> + alignSource);
>> return R;
>> }
>>
>> - static LValue MakeExtVectorElt(llvm::Value *Vec, llvm::Constant *Elts,
>> - QualType type, CharUnits Alignment) {
>> + static LValue MakeExtVectorElt(Address vecAddress, llvm::Constant *Elts,
>> + QualType type, AlignmentSource alignSource) {
>> LValue R;
>> R.LVType = ExtVectorElt;
>> - R.V = Vec;
>> + R.V = vecAddress.getPointer();
>> R.VectorElts = Elts;
>> - R.Initialize(type, type.getQualifiers(), Alignment);
>> + R.Initialize(type, type.getQualifiers(), vecAddress.getAlignment(),
>> + alignSource);
>> return R;
>> }
>>
>> @@ -332,29 +404,28 @@ public:
>> /// bit-field refers to.
>> /// \param Info - The information describing how to perform the bit-field
>> /// access.
>> - static LValue MakeBitfield(llvm::Value *Addr,
>> + static LValue MakeBitfield(Address Addr,
>> const CGBitFieldInfo &Info,
>> - QualType type, CharUnits Alignment) {
>> + QualType type,
>> + AlignmentSource alignSource) {
>> LValue R;
>> R.LVType = BitField;
>> - R.V = Addr;
>> + R.V = Addr.getPointer();
>> R.BitFieldInfo = &Info;
>> - R.Initialize(type, type.getQualifiers(), Alignment);
>> + R.Initialize(type, type.getQualifiers(), Addr.getAlignment(), alignSource);
>> return R;
>> }
>>
>> - static LValue MakeGlobalReg(llvm::Value *Reg,
>> - QualType type,
>> - CharUnits Alignment) {
>> + static LValue MakeGlobalReg(Address Reg, QualType type) {
>> LValue R;
>> R.LVType = GlobalReg;
>> - R.V = Reg;
>> - R.Initialize(type, type.getQualifiers(), Alignment);
>> + R.V = Reg.getPointer();
>> + R.Initialize(type, type.getQualifiers(), Reg.getAlignment(),
>> + AlignmentSource::Decl);
>> return R;
>> }
>>
>> RValue asAggregateRValue() const {
>> - // FIMXE: Alignment
>> return RValue::getAggregate(getAddress(), isVolatileQualified());
>> }
>> };
>> @@ -407,7 +478,7 @@ public:
>> /// ignored - Returns an aggregate value slot indicating that the
>> /// aggregate value is being ignored.
>> static AggValueSlot ignored() {
>> - return forAddr(nullptr, CharUnits(), Qualifiers(), IsNotDestructed,
>> + return forAddr(Address::invalid(), Qualifiers(), IsNotDestructed,
>> DoesNotNeedGCBarriers, IsNotAliased);
>> }
>>
>> @@ -421,15 +492,20 @@ public:
>> /// for calling destructors on this object
>> /// \param needsGC - true if the slot is potentially located
>> /// somewhere that ObjC GC calls should be emitted for
>> - static AggValueSlot forAddr(llvm::Value *addr, CharUnits align,
>> + static AggValueSlot forAddr(Address addr,
>> Qualifiers quals,
>> IsDestructed_t isDestructed,
>> NeedsGCBarriers_t needsGC,
>> IsAliased_t isAliased,
>> IsZeroed_t isZeroed = IsNotZeroed) {
>> AggValueSlot AV;
>> - AV.Addr = addr;
>> - AV.Alignment = align.getQuantity();
>> + if (addr.isValid()) {
>> + AV.Addr = addr.getPointer();
>> + AV.Alignment = addr.getAlignment().getQuantity();
>> + } else {
>> + AV.Addr = nullptr;
>> + AV.Alignment = 0;
>> + }
>> AV.Quals = quals;
>> AV.DestructedFlag = isDestructed;
>> AV.ObjCGCFlag = needsGC;
>> @@ -443,7 +519,7 @@ public:
>> NeedsGCBarriers_t needsGC,
>> IsAliased_t isAliased,
>> IsZeroed_t isZeroed = IsNotZeroed) {
>> - return forAddr(LV.getAddress(), LV.getAlignment(),
>> + return forAddr(LV.getAddress(),
>> LV.getQuals(), isDestructed, needsGC, isAliased, isZeroed);
>> }
>>
>> @@ -471,11 +547,15 @@ public:
>> NeedsGCBarriers_t requiresGCollection() const {
>> return NeedsGCBarriers_t(ObjCGCFlag);
>> }
>> -
>> - llvm::Value *getAddr() const {
>> +
>> + llvm::Value *getPointer() const {
>> return Addr;
>> }
>>
>> + Address getAddress() const {
>> + return Address(Addr, getAlignment());
>> + }
>> +
>> bool isIgnored() const {
>> return Addr == nullptr;
>> }
>> @@ -488,9 +568,12 @@ public:
>> return IsAliased_t(AliasedFlag);
>> }
>>
>> - // FIXME: Alignment?
>> RValue asRValue() const {
>> - return RValue::getAggregate(getAddr(), isVolatile());
>> + if (isIgnored()) {
>> + return RValue::getIgnored();
>> + } else {
>> + return RValue::getAggregate(getAddress(), isVolatile());
>> + }
>> }
>>
>> void setZeroed(bool V = true) { ZeroedFlag = V; }
>>
>> Modified: cfe/trunk/lib/CodeGen/CodeGenFunction.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CodeGenFunction.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CodeGenFunction.cpp Tue Sep 8 03:05:57 2015
>> @@ -12,6 +12,7 @@
>> //===----------------------------------------------------------------------===//
>>
>> #include "CodeGenFunction.h"
>> +#include "CGBlocks.h"
>> #include "CGCleanup.h"
>> #include "CGCUDARuntime.h"
>> #include "CGCXXABI.h"
>> @@ -37,12 +38,14 @@ using namespace CodeGen;
>>
>> CodeGenFunction::CodeGenFunction(CodeGenModule &cgm, bool suppressNewContext)
>> : CodeGenTypeCache(cgm), CGM(cgm), Target(cgm.getTarget()),
>> - Builder(cgm.getModule().getContext(), llvm::ConstantFolder(),
>> + Builder(cgm, cgm.getModule().getContext(), llvm::ConstantFolder(),
>> CGBuilderInserterTy(this)),
>> - CurFn(nullptr), CapturedStmtInfo(nullptr),
>> + CurFn(nullptr), ReturnValue(Address::invalid()),
>> + CapturedStmtInfo(nullptr),
>> SanOpts(CGM.getLangOpts().Sanitize), IsSanitizerScope(false),
>> CurFuncIsThunk(false), AutoreleaseResult(false), SawAsmBlock(false),
>> - IsOutlinedSEHHelper(false), BlockInfo(nullptr), BlockPointer(nullptr),
>> + IsOutlinedSEHHelper(false),
>> + BlockInfo(nullptr), BlockPointer(nullptr),
>> LambdaThisCaptureField(nullptr), NormalCleanupDest(nullptr),
>> NextCleanupDestIndex(1), FirstBlockInfo(nullptr), EHResumeBlock(nullptr),
>> ExceptionSlot(nullptr), EHSelectorSlot(nullptr),
>> @@ -52,7 +55,7 @@ CodeGenFunction::CodeGenFunction(CodeGen
>> CaseRangeBlock(nullptr), UnreachableBlock(nullptr), NumReturnExprs(0),
>> NumSimpleReturnExprs(0), CXXABIThisDecl(nullptr),
>> CXXABIThisValue(nullptr), CXXThisValue(nullptr),
>> - CXXDefaultInitExprThis(nullptr), CXXStructorImplicitParamDecl(nullptr),
>> + CXXStructorImplicitParamDecl(nullptr),
>> CXXStructorImplicitParamValue(nullptr), OutermostConditional(nullptr),
>> CurLexicalScope(nullptr), TerminateLandingPad(nullptr),
>> TerminateHandler(nullptr), TrapBB(nullptr) {
>> @@ -92,18 +95,69 @@ CodeGenFunction::~CodeGenFunction() {
>> }
>> }
>>
>> -LValue CodeGenFunction::MakeNaturalAlignAddrLValue(llvm::Value *V, QualType T) {
>> +CharUnits CodeGenFunction::getNaturalPointeeTypeAlignment(QualType T,
>> + AlignmentSource *Source) {
>> + return getNaturalTypeAlignment(T->getPointeeType(), Source,
>> + /*forPointee*/ true);
>> +}
>> +
>> +CharUnits CodeGenFunction::getNaturalTypeAlignment(QualType T,
>> + AlignmentSource *Source,
>> + bool forPointeeType) {
>> + // Honor alignment typedef attributes even on incomplete types.
>> + // We also honor them straight for C++ class types, even as pointees;
>> + // there's an expressivity gap here.
>> + if (auto TT = T->getAs<TypedefType>()) {
>> + if (auto Align = TT->getDecl()->getMaxAlignment()) {
>> + if (Source) *Source = AlignmentSource::AttributedType;
>> + return getContext().toCharUnitsFromBits(Align);
>> + }
>> + }
>> +
>> + if (Source) *Source = AlignmentSource::Type;
>> +
>> CharUnits Alignment;
>> - if (CGM.getCXXABI().isTypeInfoCalculable(T)) {
>> - Alignment = getContext().getTypeAlignInChars(T);
>> - unsigned MaxAlign = getContext().getLangOpts().MaxTypeAlign;
>> - if (MaxAlign && Alignment.getQuantity() > MaxAlign &&
>> - !getContext().isAlignmentRequired(T))
>> - Alignment = CharUnits::fromQuantity(MaxAlign);
>> + if (!CGM.getCXXABI().isTypeInfoCalculable(T)) {
>> + Alignment = CharUnits::One(); // Shouldn't be used, but pessimistic is best.
>> + } else {
>> + // For C++ class pointees, we don't know whether we're pointing at a
>> + // base or a complete object, so we generally need to use the
>> + // non-virtual alignment.
>> + const CXXRecordDecl *RD;
>> + if (forPointeeType && (RD = T->getAsCXXRecordDecl())) {
>> + Alignment = CGM.getClassPointerAlignment(RD);
>> + } else {
>> + Alignment = getContext().getTypeAlignInChars(T);
>> + }
>> +
>> + // Cap to the global maximum type alignment unless the alignment
>> + // was somehow explicit on the type.
>> + if (unsigned MaxAlign = getLangOpts().MaxTypeAlign) {
>> + if (Alignment.getQuantity() > MaxAlign &&
>> + !getContext().isAlignmentRequired(T))
>> + Alignment = CharUnits::fromQuantity(MaxAlign);
>> + }
>> }
>> - return LValue::MakeAddr(V, T, Alignment, getContext(), CGM.getTBAAInfo(T));
>> + return Alignment;
>> }
>>
>> +LValue CodeGenFunction::MakeNaturalAlignAddrLValue(llvm::Value *V, QualType T) {
>> + AlignmentSource AlignSource;
>> + CharUnits Alignment = getNaturalTypeAlignment(T, &AlignSource);
>> + return LValue::MakeAddr(Address(V, Alignment), T, getContext(), AlignSource,
>> + CGM.getTBAAInfo(T));
>> +}
>> +
>> +/// Given a value of type T* that may not be to a complete object,
>> +/// construct an l-value with the natural pointee alignment of T.
>> +LValue
>> +CodeGenFunction::MakeNaturalAlignPointeeAddrLValue(llvm::Value *V, QualType T) {
>> + AlignmentSource AlignSource;
>> + CharUnits Align = getNaturalTypeAlignment(T, &AlignSource, /*pointee*/ true);
>> + return MakeAddrLValue(Address(V, Align), T, AlignSource);
>> +}
>> +
>> +
>> llvm::Type *CodeGenFunction::ConvertTypeForMem(QualType T) {
>> return CGM.getTypes().ConvertTypeForMem(T);
>> }
>> @@ -296,7 +350,7 @@ void CodeGenFunction::FinishFunction(Sou
>> EscapeArgs[Pair.second] = Pair.first;
>> llvm::Function *FrameEscapeFn = llvm::Intrinsic::getDeclaration(
>> &CGM.getModule(), llvm::Intrinsic::localescape);
>> - CGBuilderTy(AllocaInsertPt).CreateCall(FrameEscapeFn, EscapeArgs);
>> + CGBuilderTy(*this, AllocaInsertPt).CreateCall(FrameEscapeFn, EscapeArgs);
>> }
>>
>> // Remove the AllocaInsertPt instruction, which is just a convenience for us.
>> @@ -697,7 +751,7 @@ void CodeGenFunction::StartFunction(Glob
>>
>> if (RetTy->isVoidType()) {
>> // Void type; nothing to return.
>> - ReturnValue = nullptr;
>> + ReturnValue = Address::invalid();
>>
>> // Count the implicit return.
>> if (!endsWithReturn(D))
>> @@ -709,7 +763,7 @@ void CodeGenFunction::StartFunction(Glob
>> auto AI = CurFn->arg_begin();
>> if (CurFnInfo->getReturnInfo().isSRetAfterThis())
>> ++AI;
>> - ReturnValue = AI;
>> + ReturnValue = Address(AI, CurFnInfo->getReturnInfo().getIndirectAlign());
>> } else if (CurFnInfo->getReturnInfo().getKind() == ABIArgInfo::InAlloca &&
>> !hasScalarEvaluationKind(CurFnInfo->getReturnType())) {
>> // Load the sret pointer from the argument struct and return into that.
>> @@ -717,7 +771,8 @@ void CodeGenFunction::StartFunction(Glob
>> llvm::Function::arg_iterator EI = CurFn->arg_end();
>> --EI;
>> llvm::Value *Addr = Builder.CreateStructGEP(nullptr, EI, Idx);
>> - ReturnValue = Builder.CreateLoad(Addr, "agg.result");
>> + Addr = Builder.CreateAlignedLoad(Addr, getPointerAlign(), "agg.result");
>> + ReturnValue = Address(Addr, getNaturalTypeAlignment(RetTy));
>> } else {
>> ReturnValue = CreateIRTemp(RetTy, "retval");
>>
>> @@ -1249,20 +1304,18 @@ void CodeGenFunction::ErrorUnsupported(c
>> /// base element of the array
>> /// \param sizeInChars - the total size of the VLA, in chars
>> static void emitNonZeroVLAInit(CodeGenFunction &CGF, QualType baseType,
>> - llvm::Value *dest, llvm::Value *src,
>> + Address dest, Address src,
>> llvm::Value *sizeInChars) {
>> - std::pair<CharUnits,CharUnits> baseSizeAndAlign
>> - = CGF.getContext().getTypeInfoInChars(baseType);
>> -
>> CGBuilderTy &Builder = CGF.Builder;
>>
>> + CharUnits baseSize = CGF.getContext().getTypeSizeInChars(baseType);
>> llvm::Value *baseSizeInChars
>> - = llvm::ConstantInt::get(CGF.IntPtrTy, baseSizeAndAlign.first.getQuantity());
>> + = llvm::ConstantInt::get(CGF.IntPtrTy, baseSize.getQuantity());
>>
>> - llvm::Type *i8p = Builder.getInt8PtrTy();
>> -
>> - llvm::Value *begin = Builder.CreateBitCast(dest, i8p, "vla.begin");
>> - llvm::Value *end = Builder.CreateInBoundsGEP(dest, sizeInChars, "vla.end");
>> + Address begin =
>> + Builder.CreateElementBitCast(dest, CGF.Int8Ty, "vla.begin");
>> + llvm::Value *end =
>> + Builder.CreateInBoundsGEP(begin.getPointer(), sizeInChars, "vla.end");
>>
>> llvm::BasicBlock *originBB = CGF.Builder.GetInsertBlock();
>> llvm::BasicBlock *loopBB = CGF.createBasicBlock("vla-init.loop");
>> @@ -1272,17 +1325,19 @@ static void emitNonZeroVLAInit(CodeGenFu
>> // count must be nonzero.
>> CGF.EmitBlock(loopBB);
>>
>> - llvm::PHINode *cur = Builder.CreatePHI(i8p, 2, "vla.cur");
>> - cur->addIncoming(begin, originBB);
>> + llvm::PHINode *cur = Builder.CreatePHI(begin.getType(), 2, "vla.cur");
>> + cur->addIncoming(begin.getPointer(), originBB);
>> +
>> + CharUnits curAlign =
>> + dest.getAlignment().alignmentOfArrayElement(baseSize);
>>
>> // memcpy the individual element bit-pattern.
>> - Builder.CreateMemCpy(cur, src, baseSizeInChars,
>> - baseSizeAndAlign.second.getQuantity(),
>> + Builder.CreateMemCpy(Address(cur, curAlign), src, baseSizeInChars,
>> /*volatile*/ false);
>>
>> // Go to the next element.
>> - llvm::Value *next = Builder.CreateConstInBoundsGEP1_32(Builder.getInt8Ty(),
>> - cur, 1, "vla.next");
>> + llvm::Value *next =
>> + Builder.CreateInBoundsGEP(CGF.Int8Ty, cur, baseSizeInChars, "vla.next");
>>
>> // Leave if that's the end of the VLA.
>> llvm::Value *done = Builder.CreateICmpEQ(next, end, "vla-init.isdone");
>> @@ -1293,7 +1348,7 @@ static void emitNonZeroVLAInit(CodeGenFu
>> }
>>
>> void
>> -CodeGenFunction::EmitNullInitialization(llvm::Value *DestPtr, QualType Ty) {
>> +CodeGenFunction::EmitNullInitialization(Address DestPtr, QualType Ty) {
>> // Ignore empty classes in C++.
>> if (getLangOpts().CPlusPlus) {
>> if (const RecordType *RT = Ty->getAs<RecordType>()) {
>> @@ -1303,23 +1358,17 @@ CodeGenFunction::EmitNullInitialization(
>> }
>>
>> // Cast the dest ptr to the appropriate i8 pointer type.
>> - unsigned DestAS =
>> - cast<llvm::PointerType>(DestPtr->getType())->getAddressSpace();
>> - llvm::Type *BP = Builder.getInt8PtrTy(DestAS);
>> - if (DestPtr->getType() != BP)
>> - DestPtr = Builder.CreateBitCast(DestPtr, BP);
>> + if (DestPtr.getElementType() != Int8Ty)
>> + DestPtr = Builder.CreateElementBitCast(DestPtr, Int8Ty);
>>
>> // Get size and alignment info for this aggregate.
>> - std::pair<CharUnits, CharUnits> TypeInfo =
>> - getContext().getTypeInfoInChars(Ty);
>> - CharUnits Size = TypeInfo.first;
>> - CharUnits Align = TypeInfo.second;
>> + CharUnits size = getContext().getTypeSizeInChars(Ty);
>>
>> llvm::Value *SizeVal;
>> const VariableArrayType *vla;
>>
>> // Don't bother emitting a zero-byte memset.
>> - if (Size.isZero()) {
>> + if (size.isZero()) {
>> // But note that getTypeInfo returns 0 for a VLA.
>> if (const VariableArrayType *vlaType =
>> dyn_cast_or_null<VariableArrayType>(
>> @@ -1337,7 +1386,7 @@ CodeGenFunction::EmitNullInitialization(
>> return;
>> }
>> } else {
>> - SizeVal = CGM.getSize(Size);
>> + SizeVal = CGM.getSize(size);
>> vla = nullptr;
>> }
>>
>> @@ -1356,21 +1405,22 @@ CodeGenFunction::EmitNullInitialization(
>> /*isConstant=*/true,
>> llvm::GlobalVariable::PrivateLinkage,
>> NullConstant, Twine());
>> - llvm::Value *SrcPtr =
>> - Builder.CreateBitCast(NullVariable, Builder.getInt8PtrTy());
>> + CharUnits NullAlign = DestPtr.getAlignment();
>> + NullVariable->setAlignment(NullAlign.getQuantity());
>> + Address SrcPtr(Builder.CreateBitCast(NullVariable, Builder.getInt8PtrTy()),
>> + NullAlign);
>>
>> if (vla) return emitNonZeroVLAInit(*this, Ty, DestPtr, SrcPtr, SizeVal);
>>
>> // Get and call the appropriate llvm.memcpy overload.
>> - Builder.CreateMemCpy(DestPtr, SrcPtr, SizeVal, Align.getQuantity(), false);
>> + Builder.CreateMemCpy(DestPtr, SrcPtr, SizeVal, false);
>> return;
>> }
>>
>> // Otherwise, just memset the whole thing to zero. This is legal
>> // because in LLVM, all default initializers (other than the ones we just
>> // handled above) are guaranteed to have a bit pattern of all zeros.
>> - Builder.CreateMemSet(DestPtr, Builder.getInt8(0), SizeVal,
>> - Align.getQuantity(), false);
>> + Builder.CreateMemSet(DestPtr, Builder.getInt8(0), SizeVal, false);
>> }
>>
>> llvm::BlockAddress *CodeGenFunction::GetAddrOfLabel(const LabelDecl *L) {
>> @@ -1389,7 +1439,7 @@ llvm::BasicBlock *CodeGenFunction::GetIn
>> // If we already made the indirect branch for indirect goto, return its block.
>> if (IndirectBranch) return IndirectBranch->getParent();
>>
>> - CGBuilderTy TmpBuilder(createBasicBlock("indirectgoto"));
>> + CGBuilderTy TmpBuilder(*this, createBasicBlock("indirectgoto"));
>>
>> // Create the PHI node that indirect gotos will add entries to.
>> llvm::Value *DestVal = TmpBuilder.CreatePHI(Int8PtrTy, 0,
>> @@ -1404,7 +1454,7 @@ llvm::BasicBlock *CodeGenFunction::GetIn
>> /// element type and a properly-typed first element pointer.
>> llvm::Value *CodeGenFunction::emitArrayLength(const ArrayType *origArrayType,
>> QualType &baseType,
>> - llvm::Value *&addr) {
>> + Address &addr) {
>> const ArrayType *arrayType = origArrayType;
>>
>> // If it's a VLA, we have to load the stored size. Note that
>> @@ -1443,8 +1493,7 @@ llvm::Value *CodeGenFunction::emitArrayL
>> QualType eltType;
>>
>> llvm::ArrayType *llvmArrayType =
>> - dyn_cast<llvm::ArrayType>(
>> - cast<llvm::PointerType>(addr->getType())->getElementType());
>> + dyn_cast<llvm::ArrayType>(addr.getElementType());
>> while (llvmArrayType) {
>> assert(isa<ConstantArrayType>(arrayType));
>> assert(cast<ConstantArrayType>(arrayType)->getSize().getZExtValue()
>> @@ -1472,12 +1521,13 @@ llvm::Value *CodeGenFunction::emitArrayL
>> arrayType = getContext().getAsArrayType(eltType);
>> }
>>
>> - unsigned AddressSpace = addr->getType()->getPointerAddressSpace();
>> - llvm::Type *BaseType = ConvertType(eltType)->getPointerTo(AddressSpace);
>> - addr = Builder.CreateBitCast(addr, BaseType, "array.begin");
>> + llvm::Type *baseType = ConvertType(eltType);
>> + addr = Builder.CreateElementBitCast(addr, baseType, "array.begin");
>> } else {
>> // Create the actual GEP.
>> - addr = Builder.CreateInBoundsGEP(addr, gepIndices, "array.begin");
>> + addr = Address(Builder.CreateInBoundsGEP(addr.getPointer(),
>> + gepIndices, "array.begin"),
>> + addr.getAlignment());
>> }
>>
>> baseType = eltType;
>> @@ -1662,9 +1712,9 @@ void CodeGenFunction::EmitVariablyModifi
>> } while (type->isVariablyModifiedType());
>> }
>>
>> -llvm::Value* CodeGenFunction::EmitVAListRef(const Expr* E) {
>> +Address CodeGenFunction::EmitVAListRef(const Expr* E) {
>> if (getContext().getBuiltinVaListType()->isArrayType())
>> - return EmitScalarExpr(E);
>> + return EmitPointerWithAlignment(E);
>> return EmitLValue(E).getAddress();
>> }
>>
>> @@ -1726,9 +1776,10 @@ void CodeGenFunction::EmitVarAnnotations
>> I->getAnnotation(), D->getLocation());
>> }
>>
>> -llvm::Value *CodeGenFunction::EmitFieldAnnotations(const FieldDecl *D,
>> - llvm::Value *V) {
>> +Address CodeGenFunction::EmitFieldAnnotations(const FieldDecl *D,
>> + Address Addr) {
>> assert(D->hasAttr<AnnotateAttr>() && "no annotate attribute");
>> + llvm::Value *V = Addr.getPointer();
>> llvm::Type *VTy = V->getType();
>> llvm::Value *F = CGM.getIntrinsic(llvm::Intrinsic::ptr_annotation,
>> CGM.Int8PtrTy);
>> @@ -1743,7 +1794,7 @@ llvm::Value *CodeGenFunction::EmitFieldA
>> V = Builder.CreateBitCast(V, VTy);
>> }
>>
>> - return V;
>> + return Address(V, Addr.getAlignment());
>> }
>>
>> CodeGenFunction::CGCapturedStmtInfo::~CGCapturedStmtInfo() { }
>>
>> Modified: cfe/trunk/lib/CodeGen/CodeGenFunction.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenFunction.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CodeGenFunction.h (original)
>> +++ cfe/trunk/lib/CodeGen/CodeGenFunction.h Tue Sep 8 03:05:57 2015
>> @@ -81,6 +81,8 @@ class CGFunctionInfo;
>> class CGRecordLayout;
>> class CGBlockInfo;
>> class CGCXXABI;
>> +class BlockByrefHelpers;
>> +class BlockByrefInfo;
>> class BlockFlags;
>> class BlockFieldFlags;
>>
>> @@ -160,9 +162,9 @@ public:
>> /// ReturnBlock - Unified return block.
>> JumpDest ReturnBlock;
>>
>> - /// ReturnValue - The temporary alloca to hold the return value. This is null
>> - /// iff the function has no return value.
>> - llvm::Value *ReturnValue;
>> + /// ReturnValue - The temporary alloca to hold the return
>> + /// value. This is invalid iff the function has no return value.
>> + Address ReturnValue;
>>
>> /// AllocaInsertPoint - This is an instruction in the entry block before which
>> /// we prefer to insert allocas.
>> @@ -328,7 +330,7 @@ public:
>> /// A stack of exception code slots. Entering an __except block pushes a slot
>> /// on the stack and leaving pops one. The __exception_code() intrinsic loads
>> /// a value from the top of the stack.
>> - SmallVector<llvm::Value *, 1> SEHCodeSlotStack;
>> + SmallVector<Address, 1> SEHCodeSlotStack;
>>
>> /// Value returned by __exception_info intrinsic.
>> llvm::Value *SEHInfo = nullptr;
>> @@ -420,13 +422,12 @@ public:
>> /// complete-object destructor of an object of the given type at the
>> /// given address. Does nothing if T is not a C++ class type with a
>> /// non-trivial destructor.
>> - void PushDestructorCleanup(QualType T, llvm::Value *Addr);
>> + void PushDestructorCleanup(QualType T, Address Addr);
>>
>> /// PushDestructorCleanup - Push a cleanup to call the
>> /// complete-object variant of the given destructor on the object at
>> /// the given address.
>> - void PushDestructorCleanup(const CXXDestructorDecl *Dtor,
>> - llvm::Value *Addr);
>> + void PushDestructorCleanup(const CXXDestructorDecl *Dtor, Address Addr);
>>
>> /// PopCleanupBlock - Will pop the cleanup entry on the stack and
>> /// process all branch fixups.
>> @@ -556,13 +557,14 @@ public:
>> void rescopeLabels();
>> };
>>
>> + typedef llvm::DenseMap<const Decl *, Address> DeclMapTy;
>> +
>> /// \brief The scope used to remap some variables as private in the OpenMP
>> /// loop body (or other captured region emitted without outlining), and to
>> /// restore old vars back on exit.
>> class OMPPrivateScope : public RunCleanupsScope {
>> - typedef llvm::DenseMap<const VarDecl *, llvm::Value *> VarDeclMapTy;
>> - VarDeclMapTy SavedLocals;
>> - VarDeclMapTy SavedPrivates;
>> + DeclMapTy SavedLocals;
>> + DeclMapTy SavedPrivates;
>>
>> private:
>> OMPPrivateScope(const OMPPrivateScope &) = delete;
>> @@ -579,21 +581,30 @@ public:
>> /// been privatized already.
>> bool
>> addPrivate(const VarDecl *LocalVD,
>> - const std::function<llvm::Value *()> &PrivateGen) {
>> + llvm::function_ref<Address()> PrivateGen) {
>> assert(PerformCleanup && "adding private to dead scope");
>> - if (SavedLocals.count(LocalVD) > 0) return false;
>> - SavedLocals[LocalVD] = CGF.LocalDeclMap.lookup(LocalVD);
>> - CGF.LocalDeclMap.erase(LocalVD);
>> - auto *V = PrivateGen();
>> +
>> + // Only save it once.
>> + if (SavedLocals.count(LocalVD)) return false;
>> +
>> + // Copy the existing local entry to SavedLocals.
>> + auto it = CGF.LocalDeclMap.find(LocalVD);
>> + if (it != CGF.LocalDeclMap.end()) {
>> + SavedLocals.insert({LocalVD, it->second});
>> + } else {
>> + SavedLocals.insert({LocalVD, Address::invalid()});
>> + }
>> +
>> + // Generate the private entry.
>> + Address Addr = PrivateGen();
>> QualType VarTy = LocalVD->getType();
>> if (VarTy->isReferenceType()) {
>> - auto *TempAlloca = CGF.CreateMemTemp(VarTy);
>> - LValue RefLVal = CGF.MakeNaturalAlignAddrLValue(TempAlloca, VarTy);
>> - CGF.EmitStoreOfScalar(V, RefLVal);
>> - V = TempAlloca;
>> + Address Temp = CGF.CreateMemTemp(VarTy);
>> + CGF.Builder.CreateStore(Addr.getPointer(), Temp);
>> + Addr = Temp;
>> }
>> - SavedPrivates[LocalVD] = V;
>> - CGF.LocalDeclMap[LocalVD] = SavedLocals[LocalVD];
>> + SavedPrivates.insert({LocalVD, Addr});
>> +
>> return true;
>> }
>>
>> @@ -606,19 +617,14 @@ public:
>> /// private copies.
>> /// \return true if at least one variable was privatized, false otherwise.
>> bool Privatize() {
>> - for (auto VDPair : SavedPrivates) {
>> - CGF.LocalDeclMap[VDPair.first] = VDPair.second;
>> - }
>> + copyInto(SavedPrivates, CGF.LocalDeclMap);
>> SavedPrivates.clear();
>> return !SavedLocals.empty();
>> }
>>
>> void ForceCleanup() {
>> RunCleanupsScope::ForceCleanup();
>> - // Remap vars back to the original values.
>> - for (auto I : SavedLocals) {
>> - CGF.LocalDeclMap[I.first] = I.second;
>> - }
>> + copyInto(SavedLocals, CGF.LocalDeclMap);
>> SavedLocals.clear();
>> }
>>
>> @@ -627,6 +633,25 @@ public:
>> if (PerformCleanup)
>> ForceCleanup();
>> }
>> +
>> + private:
>> + /// Copy all the entries in the source map over the corresponding
>> + /// entries in the destination, which must exist.
>> + static void copyInto(const DeclMapTy &src, DeclMapTy &dest) {
>> + for (auto &pair : src) {
>> + if (!pair.second.isValid()) {
>> + dest.erase(pair.first);
>> + continue;
>> + }
>> +
>> + auto it = dest.find(pair.first);
>> + if (it != dest.end()) {
>> + it->second = pair.second;
>> + } else {
>> + dest.insert(pair);
>> + }
>> + }
>> + }
>> };
>>
>> /// \brief Takes the old cleanup stack size and emits the cleanup blocks
>> @@ -707,10 +732,11 @@ public:
>> /// one branch or the other of a conditional expression.
>> bool isInConditionalBranch() const { return OutermostConditional != nullptr; }
>>
>> - void setBeforeOutermostConditional(llvm::Value *value, llvm::Value *addr) {
>> + void setBeforeOutermostConditional(llvm::Value *value, Address addr) {
>> assert(isInConditionalBranch());
>> llvm::BasicBlock *block = OutermostConditional->getStartingBlock();
>> - new llvm::StoreInst(value, addr, &block->back());
>> + auto store = new llvm::StoreInst(value, addr.getPointer(), &block->back());
>> + store->setAlignment(addr.getAlignment().getQuantity());
>> }
>>
>> /// An RAII object to record that we're evaluating a statement
>> @@ -869,15 +895,6 @@ public:
>> }
>> };
>>
>> - /// getByrefValueFieldNumber - Given a declaration, returns the LLVM field
>> - /// number that holds the value.
>> - std::pair<llvm::Type *, unsigned>
>> - getByRefValueLLVMField(const ValueDecl *VD) const;
>> -
>> - /// BuildBlockByrefAddress - Computes address location of the
>> - /// variable which is declared as __block.
>> - llvm::Value *BuildBlockByrefAddress(llvm::Value *BaseAddr,
>> - const VarDecl *V);
>> private:
>> CGDebugInfo *DebugInfo;
>> bool DisableDebugInfo;
>> @@ -894,7 +911,6 @@ private:
>>
>> /// LocalDeclMap - This keeps track of the LLVM allocas or globals for local C
>> /// decls.
>> - typedef llvm::DenseMap<const Decl*, llvm::Value*> DeclMapTy;
>> DeclMapTy LocalDeclMap;
>>
>> /// Track escaped local variables with auto storage. Used during SEH
>> @@ -994,7 +1010,7 @@ public:
>> /// if we need to evaluate a CXXDefaultInitExpr within the evaluation.
>> class FieldConstructionScope {
>> public:
>> - FieldConstructionScope(CodeGenFunction &CGF, llvm::Value *This)
>> + FieldConstructionScope(CodeGenFunction &CGF, Address This)
>> : CGF(CGF), OldCXXDefaultInitExprThis(CGF.CXXDefaultInitExprThis) {
>> CGF.CXXDefaultInitExprThis = This;
>> }
>> @@ -1004,7 +1020,7 @@ public:
>>
>> private:
>> CodeGenFunction &CGF;
>> - llvm::Value *OldCXXDefaultInitExprThis;
>> + Address OldCXXDefaultInitExprThis;
>> };
>>
>> /// The scope of a CXXDefaultInitExpr. Within this scope, the value of 'this'
>> @@ -1012,16 +1028,20 @@ public:
>> class CXXDefaultInitExprScope {
>> public:
>> CXXDefaultInitExprScope(CodeGenFunction &CGF)
>> - : CGF(CGF), OldCXXThisValue(CGF.CXXThisValue) {
>> - CGF.CXXThisValue = CGF.CXXDefaultInitExprThis;
>> + : CGF(CGF), OldCXXThisValue(CGF.CXXThisValue),
>> + OldCXXThisAlignment(CGF.CXXThisAlignment) {
>> + CGF.CXXThisValue = CGF.CXXDefaultInitExprThis.getPointer();
>> + CGF.CXXThisAlignment = CGF.CXXDefaultInitExprThis.getAlignment();
>> }
>> ~CXXDefaultInitExprScope() {
>> CGF.CXXThisValue = OldCXXThisValue;
>> + CGF.CXXThisAlignment = OldCXXThisAlignment;
>> }
>>
>> public:
>> CodeGenFunction &CGF;
>> llvm::Value *OldCXXThisValue;
>> + CharUnits OldCXXThisAlignment;
>> };
>>
>> private:
>> @@ -1030,10 +1050,12 @@ private:
>> ImplicitParamDecl *CXXABIThisDecl;
>> llvm::Value *CXXABIThisValue;
>> llvm::Value *CXXThisValue;
>> + CharUnits CXXABIThisAlignment;
>> + CharUnits CXXThisAlignment;
>>
>> /// The value of 'this' to use when evaluating CXXDefaultInitExprs within
>> /// this expression.
>> - llvm::Value *CXXDefaultInitExprThis;
>> + Address CXXDefaultInitExprThis = Address::invalid();
>>
>> /// CXXStructorImplicitParamDecl - When generating code for a constructor or
>> /// destructor, this will hold the implicit argument (e.g. VTT).
>> @@ -1052,10 +1074,9 @@ private:
>> /// handling code.
>> SourceLocation CurEHLocation;
>>
>> - /// ByrefValueInfoMap - For each __block variable, contains a pair of the LLVM
>> - /// type as well as the field number that contains the actual data.
>> - llvm::DenseMap<const ValueDecl *, std::pair<llvm::Type *,
>> - unsigned> > ByRefValueInfo;
>> + /// BlockByrefInfos - For each __block variable, contains
>> + /// information about the layout of the variable.
>> + llvm::DenseMap<const ValueDecl *, BlockByrefInfo> BlockByrefInfos;
>>
>> llvm::BasicBlock *TerminateLandingPad;
>> llvm::BasicBlock *TerminateHandler;
>> @@ -1096,15 +1117,15 @@ public:
>>
>> /// Returns a pointer to the function's exception object and selector slot,
>> /// which is assigned in every landing pad.
>> - llvm::Value *getExceptionSlot();
>> - llvm::Value *getEHSelectorSlot();
>> + Address getExceptionSlot();
>> + Address getEHSelectorSlot();
>>
>> /// Returns the contents of the function's exception object and selector
>> /// slots.
>> llvm::Value *getExceptionFromSlot();
>> llvm::Value *getSelectorFromSlot();
>>
>> - llvm::Value *getNormalCleanupDestSlot();
>> + Address getNormalCleanupDestSlot();
>>
>> llvm::BasicBlock *getUnreachableBlock() {
>> if (!UnreachableBlock) {
>> @@ -1131,38 +1152,41 @@ public:
>> // Cleanups
>> //===--------------------------------------------------------------------===//
>>
>> - typedef void Destroyer(CodeGenFunction &CGF, llvm::Value *addr, QualType ty);
>> + typedef void Destroyer(CodeGenFunction &CGF, Address addr, QualType ty);
>>
>> void pushIrregularPartialArrayCleanup(llvm::Value *arrayBegin,
>> - llvm::Value *arrayEndPointer,
>> + Address arrayEndPointer,
>> QualType elementType,
>> + CharUnits elementAlignment,
>> Destroyer *destroyer);
>> void pushRegularPartialArrayCleanup(llvm::Value *arrayBegin,
>> llvm::Value *arrayEnd,
>> QualType elementType,
>> + CharUnits elementAlignment,
>> Destroyer *destroyer);
>>
>> void pushDestroy(QualType::DestructionKind dtorKind,
>> - llvm::Value *addr, QualType type);
>> + Address addr, QualType type);
>> void pushEHDestroy(QualType::DestructionKind dtorKind,
>> - llvm::Value *addr, QualType type);
>> - void pushDestroy(CleanupKind kind, llvm::Value *addr, QualType type,
>> + Address addr, QualType type);
>> + void pushDestroy(CleanupKind kind, Address addr, QualType type,
>> Destroyer *destroyer, bool useEHCleanupForArray);
>> - void pushLifetimeExtendedDestroy(CleanupKind kind, llvm::Value *addr,
>> + void pushLifetimeExtendedDestroy(CleanupKind kind, Address addr,
>> QualType type, Destroyer *destroyer,
>> bool useEHCleanupForArray);
>> void pushCallObjectDeleteCleanup(const FunctionDecl *OperatorDelete,
>> llvm::Value *CompletePtr,
>> QualType ElementType);
>> - void pushStackRestore(CleanupKind kind, llvm::Value *SPMem);
>> - void emitDestroy(llvm::Value *addr, QualType type, Destroyer *destroyer,
>> + void pushStackRestore(CleanupKind kind, Address SPMem);
>> + void emitDestroy(Address addr, QualType type, Destroyer *destroyer,
>> bool useEHCleanupForArray);
>> - llvm::Function *generateDestroyHelper(llvm::Constant *addr, QualType type,
>> + llvm::Function *generateDestroyHelper(Address addr, QualType type,
>> Destroyer *destroyer,
>> bool useEHCleanupForArray,
>> const VarDecl *VD);
>> void emitArrayDestroy(llvm::Value *begin, llvm::Value *end,
>> - QualType type, Destroyer *destroyer,
>> + QualType elementType, CharUnits elementAlign,
>> + Destroyer *destroyer,
>> bool checkZeroLength, bool useEHCleanup);
>>
>> Destroyer *getDestroyer(QualType::DestructionKind destructionKind);
>> @@ -1248,15 +1272,25 @@ public:
>> void emitByrefStructureInit(const AutoVarEmission &emission);
>> void enterByrefCleanup(const AutoVarEmission &emission);
>>
>> - llvm::Value *LoadBlockStruct() {
>> - assert(BlockPointer && "no block pointer set!");
>> - return BlockPointer;
>> - }
>> + void setBlockContextParameter(const ImplicitParamDecl *D, unsigned argNum,
>> + llvm::Value *ptr);
>> +
>> + Address LoadBlockStruct();
>>
>> void AllocateBlockCXXThisPointer(const CXXThisExpr *E);
>> void AllocateBlockDecl(const DeclRefExpr *E);
>> - llvm::Value *GetAddrOfBlockDecl(const VarDecl *var, bool ByRef);
>> - llvm::Type *BuildByRefType(const VarDecl *var);
>> + Address GetAddrOfBlockDecl(const VarDecl *var, bool ByRef);
>> +
>> + /// BuildBlockByrefAddress - Computes the location of the
>> + /// data in a variable which is declared as __block.
>> + Address emitBlockByrefAddress(Address baseAddr, const VarDecl *V,
>> + bool followForward = true);
>> + Address emitBlockByrefAddress(Address baseAddr,
>> + const BlockByrefInfo &info,
>> + bool followForward,
>> + const llvm::Twine &name);
>> +
>> + const BlockByrefInfo &getBlockByrefInfo(const VarDecl *var);
>>
>> void GenerateCode(GlobalDecl GD, llvm::Function *Fn,
>> const CGFunctionInfo &FnInfo);
>> @@ -1300,6 +1334,8 @@ public:
>>
>> void EmitCallAndReturnForThunk(llvm::Value *Callee, const ThunkInfo *Thunk);
>>
>> + void FinishThunk();
>> +
>> /// Emit a musttail call for a thunk with a potentially adjusted this pointer.
>> void EmitMustTailThunk(const CXXMethodDecl *MD, llvm::Value *AdjustedThisPtr,
>> llvm::Value *Callee);
>> @@ -1338,7 +1374,7 @@ public:
>>
>> /// GetVTablePtr - Return the Value of the vtable pointer member pointed
>> /// to by This.
>> - llvm::Value *GetVTablePtr(llvm::Value *This, llvm::Type *Ty);
>> + llvm::Value *GetVTablePtr(Address This, llvm::Type *Ty);
>>
>> enum CFITypeCheckKind {
>> CFITCK_VCall,
>> @@ -1505,39 +1541,79 @@ public:
>> // Helpers
>> //===--------------------------------------------------------------------===//
>>
>> - LValue MakeAddrLValue(llvm::Value *V, QualType T,
>> - CharUnits Alignment = CharUnits()) {
>> - return LValue::MakeAddr(V, T, Alignment, getContext(),
>> + LValue MakeAddrLValue(Address Addr, QualType T,
>> + AlignmentSource AlignSource = AlignmentSource::Type) {
>> + return LValue::MakeAddr(Addr, T, getContext(), AlignSource,
>> CGM.getTBAAInfo(T));
>> }
>>
>> + LValue MakeAddrLValue(llvm::Value *V, QualType T, CharUnits Alignment,
>> + AlignmentSource AlignSource = AlignmentSource::Type) {
>> + return LValue::MakeAddr(Address(V, Alignment), T, getContext(),
>> + AlignSource, CGM.getTBAAInfo(T));
>> + }
>> +
>> + LValue MakeNaturalAlignPointeeAddrLValue(llvm::Value *V, QualType T);
>> LValue MakeNaturalAlignAddrLValue(llvm::Value *V, QualType T);
>> + CharUnits getNaturalTypeAlignment(QualType T,
>> + AlignmentSource *Source = nullptr,
>> + bool forPointeeType = false);
>> + CharUnits getNaturalPointeeTypeAlignment(QualType T,
>> + AlignmentSource *Source = nullptr);
>> +
>> + Address EmitLoadOfReference(Address Ref, const ReferenceType *RefTy,
>> + AlignmentSource *Source = nullptr);
>> + LValue EmitLoadOfReferenceLValue(Address Ref, const ReferenceType *RefTy);
>>
>> /// CreateTempAlloca - This creates a alloca and inserts it into the entry
>> /// block. The caller is responsible for setting an appropriate alignment on
>> /// the alloca.
>> llvm::AllocaInst *CreateTempAlloca(llvm::Type *Ty,
>> const Twine &Name = "tmp");
>> + Address CreateTempAlloca(llvm::Type *Ty, CharUnits align,
>> + const Twine &Name = "tmp");
>>
>> - /// InitTempAlloca - Provide an initial value for the given alloca.
>> - void InitTempAlloca(llvm::AllocaInst *Alloca, llvm::Value *Value);
>> + /// CreateDefaultAlignedTempAlloca - This creates an alloca with the
>> + /// default ABI alignment of the given LLVM type.
>> + ///
>> + /// IMPORTANT NOTE: This is *not* generally the right alignment for
>> + /// any given AST type that happens to have been lowered to the
>> + /// given IR type. This should only ever be used for function-local,
>> + /// IR-driven manipulations like saving and restoring a value. Do
>> + /// not hand this address off to arbitrary IRGen routines, and especially
>> + /// do not pass it as an argument to a function that might expect a
>> + /// properly ABI-aligned value.
>> + Address CreateDefaultAlignTempAlloca(llvm::Type *Ty,
>> + const Twine &Name = "tmp");
>> +
>> + /// InitTempAlloca - Provide an initial value for the given alloca which
>> + /// will be observable at all locations in the function.
>> + ///
>> + /// The address should be something that was returned from one of
>> + /// the CreateTempAlloca or CreateMemTemp routines, and the
>> + /// initializer must be valid in the entry block (i.e. it must
>> + /// either be a constant or an argument value).
>> + void InitTempAlloca(Address Alloca, llvm::Value *Value);
>>
>> /// CreateIRTemp - Create a temporary IR object of the given type, with
>> /// appropriate alignment. This routine should only be used when an temporary
>> /// value needs to be stored into an alloca (for example, to avoid explicit
>> /// PHI construction), but the type is the IR type, not the type appropriate
>> /// for storing in memory.
>> - llvm::AllocaInst *CreateIRTemp(QualType T, const Twine &Name = "tmp");
>> + ///
>> + /// That is, this is exactly equivalent to CreateMemTemp, but calling
>> + /// ConvertType instead of ConvertTypeForMem.
>> + Address CreateIRTemp(QualType T, const Twine &Name = "tmp");
>>
>> /// CreateMemTemp - Create a temporary memory object of the given type, with
>> /// appropriate alignment.
>> - llvm::AllocaInst *CreateMemTemp(QualType T, const Twine &Name = "tmp");
>> + Address CreateMemTemp(QualType T, const Twine &Name = "tmp");
>> + Address CreateMemTemp(QualType T, CharUnits Align, const Twine &Name = "tmp");
>>
>> /// CreateAggTemp - Create a temporary memory object for the given
>> /// aggregate type.
>> AggValueSlot CreateAggTemp(QualType T, const Twine &Name = "tmp") {
>> - CharUnits Alignment = getContext().getTypeAlignInChars(T);
>> - return AggValueSlot::forAddr(CreateMemTemp(T, Name), Alignment,
>> + return AggValueSlot::forAddr(CreateMemTemp(T, Name),
>> T.getQualifiers(),
>> AggValueSlot::IsNotDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> @@ -1570,7 +1646,7 @@ public:
>>
>> // EmitVAListRef - Emit a "reference" to a va_list; this is either the address
>> // or the value of the expression, depending on how va_list is defined.
>> - llvm::Value *EmitVAListRef(const Expr *E);
>> + Address EmitVAListRef(const Expr *E);
>>
>> /// EmitAnyExprToTemp - Similary to EmitAnyExpr(), however, the result will
>> /// always be accessible even if no aggregate location is provided.
>> @@ -1578,10 +1654,10 @@ public:
>>
>> /// EmitAnyExprToMem - Emits the code necessary to evaluate an
>> /// arbitrary expression into the given memory location.
>> - void EmitAnyExprToMem(const Expr *E, llvm::Value *Location,
>> + void EmitAnyExprToMem(const Expr *E, Address Location,
>> Qualifiers Quals, bool IsInitializer);
>>
>> - void EmitAnyExprToExn(const Expr *E, llvm::Value *Addr);
>> + void EmitAnyExprToExn(const Expr *E, Address Addr);
>>
>> /// EmitExprAsInit - Emits the code necessary to initialize a
>> /// location in memory with the given initializer.
>> @@ -1601,19 +1677,15 @@ public:
>> ///
>> /// The difference to EmitAggregateCopy is that tail padding is not copied.
>> /// This is required for correctness when assigning non-POD structures in C++.
>> - void EmitAggregateAssign(llvm::Value *DestPtr, llvm::Value *SrcPtr,
>> + void EmitAggregateAssign(Address DestPtr, Address SrcPtr,
>> QualType EltTy) {
>> bool IsVolatile = hasVolatileMember(EltTy);
>> - EmitAggregateCopy(DestPtr, SrcPtr, EltTy, IsVolatile, CharUnits::Zero(),
>> - true);
>> + EmitAggregateCopy(DestPtr, SrcPtr, EltTy, IsVolatile, true);
>> }
>>
>> - void EmitAggregateCopyCtor(llvm::Value *DestPtr, llvm::Value *SrcPtr,
>> - QualType DestTy, QualType SrcTy) {
>> - CharUnits DestTypeAlign = getContext().getTypeAlignInChars(DestTy);
>> - CharUnits SrcTypeAlign = getContext().getTypeAlignInChars(SrcTy);
>> + void EmitAggregateCopyCtor(Address DestPtr, Address SrcPtr,
>> + QualType DestTy, QualType SrcTy) {
>> EmitAggregateCopy(DestPtr, SrcPtr, SrcTy, /*IsVolatile=*/false,
>> - std::min(DestTypeAlign, SrcTypeAlign),
>> /*IsAssignment=*/false);
>> }
>>
>> @@ -1623,9 +1695,8 @@ public:
>> /// volatile.
>> /// \param isAssignment - If false, allow padding to be copied. This often
>> /// yields more efficient.
>> - void EmitAggregateCopy(llvm::Value *DestPtr, llvm::Value *SrcPtr,
>> + void EmitAggregateCopy(Address DestPtr, Address SrcPtr,
>> QualType EltTy, bool isVolatile=false,
>> - CharUnits Alignment = CharUnits::Zero(),
>> bool isAssignment = false);
>>
>> /// StartBlock - Start new block named N. If insert block is a dummy block
>> @@ -1633,10 +1704,11 @@ public:
>> void StartBlock(const char *N);
>>
>> /// GetAddrOfLocalVar - Return the address of a local variable.
>> - llvm::Value *GetAddrOfLocalVar(const VarDecl *VD) {
>> - llvm::Value *Res = LocalDeclMap[VD];
>> - assert(Res && "Invalid argument to GetAddrOfLocalVar(), no decl!");
>> - return Res;
>> + Address GetAddrOfLocalVar(const VarDecl *VD) {
>> + auto it = LocalDeclMap.find(VD);
>> + assert(it != LocalDeclMap.end() &&
>> + "Invalid argument to GetAddrOfLocalVar(), no decl!");
>> + return it->second;
>> }
>>
>> /// getOpaqueLValueMapping - Given an opaque value expression (which
>> @@ -1671,19 +1743,19 @@ public:
>> /// EmitNullInitialization - Generate code to set a value of the given type to
>> /// null, If the type contains data member pointers, they will be initialized
>> /// to -1 in accordance with the Itanium C++ ABI.
>> - void EmitNullInitialization(llvm::Value *DestPtr, QualType Ty);
>> + void EmitNullInitialization(Address DestPtr, QualType Ty);
>>
>> // EmitVAArg - Generate code to get an argument from the passed in pointer
>> // and update it accordingly. The return value is a pointer to the argument.
>> // FIXME: We should be able to get rid of this method and use the va_arg
>> // instruction in LLVM instead once it works well enough.
>> - llvm::Value *EmitVAArg(llvm::Value *VAListAddr, QualType Ty);
>> + Address EmitVAArg(Address VAListAddr, QualType Ty);
>>
>> /// emitArrayLength - Compute the length of an array, even if it's a
>> /// VLA, and drill down to the base element type.
>> llvm::Value *emitArrayLength(const ArrayType *arrayType,
>> QualType &baseType,
>> - llvm::Value *&addr);
>> + Address &addr);
>>
>> /// EmitVLASize - Capture all the sizes for the VLA expressions in
>> /// the given variably-modified type and store them in the VLASizeMap.
>> @@ -1704,6 +1776,7 @@ public:
>> assert(CXXThisValue && "no 'this' value for this function");
>> return CXXThisValue;
>> }
>> + Address LoadCXXThisAddress();
>>
>> /// LoadCXXVTT - Load the VTT parameter to base constructors/destructors have
>> /// virtual bases.
>> @@ -1724,25 +1797,27 @@ public:
>>
>> /// GetAddressOfBaseOfCompleteClass - Convert the given pointer to a
>> /// complete class to the given direct base.
>> - llvm::Value *
>> - GetAddressOfDirectBaseInCompleteClass(llvm::Value *Value,
>> + Address
>> + GetAddressOfDirectBaseInCompleteClass(Address Value,
>> const CXXRecordDecl *Derived,
>> const CXXRecordDecl *Base,
>> bool BaseIsVirtual);
>>
>> + static bool ShouldNullCheckClassCastValue(const CastExpr *Cast);
>> +
>> /// GetAddressOfBaseClass - This function will add the necessary delta to the
>> /// load of 'this' and returns address of the base class.
>> - llvm::Value *GetAddressOfBaseClass(llvm::Value *Value,
>> - const CXXRecordDecl *Derived,
>> - CastExpr::path_const_iterator PathBegin,
>> - CastExpr::path_const_iterator PathEnd,
>> - bool NullCheckValue, SourceLocation Loc);
>> -
>> - llvm::Value *GetAddressOfDerivedClass(llvm::Value *Value,
>> - const CXXRecordDecl *Derived,
>> - CastExpr::path_const_iterator PathBegin,
>> - CastExpr::path_const_iterator PathEnd,
>> - bool NullCheckValue);
>> + Address GetAddressOfBaseClass(Address Value,
>> + const CXXRecordDecl *Derived,
>> + CastExpr::path_const_iterator PathBegin,
>> + CastExpr::path_const_iterator PathEnd,
>> + bool NullCheckValue, SourceLocation Loc);
>> +
>> + Address GetAddressOfDerivedClass(Address Value,
>> + const CXXRecordDecl *Derived,
>> + CastExpr::path_const_iterator PathBegin,
>> + CastExpr::path_const_iterator PathEnd,
>> + bool NullCheckValue);
>>
>> /// GetVTTParameter - Return the VTT parameter that should be passed to a
>> /// base constructor/destructor with virtual bases.
>> @@ -1763,21 +1838,21 @@ public:
>> const FunctionArgList &Args);
>> void EmitCXXConstructorCall(const CXXConstructorDecl *D, CXXCtorType Type,
>> bool ForVirtualBase, bool Delegating,
>> - llvm::Value *This, const CXXConstructExpr *E);
>> + Address This, const CXXConstructExpr *E);
>>
>> void EmitSynthesizedCXXCopyCtorCall(const CXXConstructorDecl *D,
>> - llvm::Value *This, llvm::Value *Src,
>> - const CXXConstructExpr *E);
>> + Address This, Address Src,
>> + const CXXConstructExpr *E);
>>
>> void EmitCXXAggrConstructorCall(const CXXConstructorDecl *D,
>> const ConstantArrayType *ArrayTy,
>> - llvm::Value *ArrayPtr,
>> + Address ArrayPtr,
>> const CXXConstructExpr *E,
>> bool ZeroInitialization = false);
>>
>> void EmitCXXAggrConstructorCall(const CXXConstructorDecl *D,
>> llvm::Value *NumElements,
>> - llvm::Value *ArrayPtr,
>> + Address ArrayPtr,
>> const CXXConstructExpr *E,
>> bool ZeroInitialization = false);
>>
>> @@ -1785,15 +1860,15 @@ public:
>>
>> void EmitCXXDestructorCall(const CXXDestructorDecl *D, CXXDtorType Type,
>> bool ForVirtualBase, bool Delegating,
>> - llvm::Value *This);
>> + Address This);
>>
>> void EmitNewArrayInitializer(const CXXNewExpr *E, QualType elementType,
>> - llvm::Type *ElementTy, llvm::Value *NewPtr,
>> + llvm::Type *ElementTy, Address NewPtr,
>> llvm::Value *NumElements,
>> llvm::Value *AllocSizeWithoutCookie);
>>
>> void EmitCXXTemporary(const CXXTemporary *Temporary, QualType TempType,
>> - llvm::Value *Ptr);
>> + Address Ptr);
>>
>> llvm::Value *EmitLifetimeStart(uint64_t Size, llvm::Value *Addr);
>> void EmitLifetimeEnd(llvm::Value *Size, llvm::Value *Addr);
>> @@ -1807,9 +1882,9 @@ public:
>> RValue EmitBuiltinNewDeleteCall(const FunctionProtoType *Type,
>> const Expr *Arg, bool IsDelete);
>>
>> - llvm::Value* EmitCXXTypeidExpr(const CXXTypeidExpr *E);
>> - llvm::Value *EmitDynamicCast(llvm::Value *V, const CXXDynamicCastExpr *DCE);
>> - llvm::Value* EmitCXXUuidofExpr(const CXXUuidofExpr *E);
>> + llvm::Value *EmitCXXTypeidExpr(const CXXTypeidExpr *E);
>> + llvm::Value *EmitDynamicCast(Address V, const CXXDynamicCastExpr *DCE);
>> + Address EmitCXXUuidofExpr(const CXXUuidofExpr *E);
>>
>> /// \brief Situations in which we might emit a check for the suitability of a
>> /// pointer or glvalue.
>> @@ -1906,12 +1981,9 @@ public:
>>
>> const VarDecl *Variable;
>>
>> - /// The alignment of the variable.
>> - CharUnits Alignment;
>> -
>> - /// The address of the alloca. Null if the variable was emitted
>> + /// The address of the alloca. Invalid if the variable was emitted
>> /// as a global constant.
>> - llvm::Value *Address;
>> + Address Addr;
>>
>> llvm::Value *NRVOFlag;
>>
>> @@ -1926,14 +1998,14 @@ public:
>> llvm::Value *SizeForLifetimeMarkers;
>>
>> struct Invalid {};
>> - AutoVarEmission(Invalid) : Variable(nullptr) {}
>> + AutoVarEmission(Invalid) : Variable(nullptr), Addr(Address::invalid()) {}
>>
>> AutoVarEmission(const VarDecl &variable)
>> - : Variable(&variable), Address(nullptr), NRVOFlag(nullptr),
>> + : Variable(&variable), Addr(Address::invalid()), NRVOFlag(nullptr),
>> IsByRef(false), IsConstantAggregate(false),
>> SizeForLifetimeMarkers(nullptr) {}
>>
>> - bool wasEmittedAsGlobal() const { return Address == nullptr; }
>> + bool wasEmittedAsGlobal() const { return !Addr.isValid(); }
>>
>> public:
>> static AutoVarEmission invalid() { return AutoVarEmission(Invalid()); }
>> @@ -1948,19 +2020,17 @@ public:
>>
>> /// Returns the raw, allocated address, which is not necessarily
>> /// the address of the object itself.
>> - llvm::Value *getAllocatedAddress() const {
>> - return Address;
>> + Address getAllocatedAddress() const {
>> + return Addr;
>> }
>>
>> /// Returns the address of the object within this declaration.
>> /// Note that this does not chase the forwarding pointer for
>> /// __block decls.
>> - llvm::Value *getObjectAddress(CodeGenFunction &CGF) const {
>> - if (!IsByRef) return Address;
>> + Address getObjectAddress(CodeGenFunction &CGF) const {
>> + if (!IsByRef) return Addr;
>>
>> - auto F = CGF.getByRefValueLLVMField(Variable);
>> - return CGF.Builder.CreateStructGEP(F.first, Address, F.second,
>> - Variable->getNameAsString());
>> + return CGF.emitBlockByrefAddress(Addr, Variable, /*forward*/ false);
>> }
>> };
>> AutoVarEmission EmitAutoVarAlloca(const VarDecl &var);
>> @@ -1972,9 +2042,35 @@ public:
>> void EmitStaticVarDecl(const VarDecl &D,
>> llvm::GlobalValue::LinkageTypes Linkage);
>>
>> + class ParamValue {
>> + llvm::Value *Value;
>> + unsigned Alignment;
>> + ParamValue(llvm::Value *V, unsigned A) : Value(V), Alignment(A) {}
>> + public:
>> + static ParamValue forDirect(llvm::Value *value) {
>> + return ParamValue(value, 0);
>> + }
>> + static ParamValue forIndirect(Address addr) {
>> + assert(!addr.getAlignment().isZero());
>> + return ParamValue(addr.getPointer(), addr.getAlignment().getQuantity());
>> + }
>> +
>> + bool isIndirect() const { return Alignment != 0; }
>> + llvm::Value *getAnyValue() const { return Value; }
>> +
>> + llvm::Value *getDirectValue() const {
>> + assert(!isIndirect());
>> + return Value;
>> + }
>> +
>> + Address getIndirectAddress() const {
>> + assert(isIndirect());
>> + return Address(Value, CharUnits::fromQuantity(Alignment));
>> + }
>> + };
>> +
>> /// EmitParmDecl - Emit a ParmVarDecl or an ImplicitParamDecl.
>> - void EmitParmDecl(const VarDecl &D, llvm::Value *Arg, bool ArgIsPointer,
>> - unsigned ArgNo);
>> + void EmitParmDecl(const VarDecl &D, ParamValue Arg, unsigned ArgNo);
>>
>> /// protectFromPeepholes - Protect a value that we're intending to
>> /// store to the side, but which will probably be used later, from
>> @@ -2011,11 +2107,11 @@ public:
>> /// \return True if the statement was handled.
>> bool EmitSimpleStmt(const Stmt *S);
>>
>> - llvm::Value *EmitCompoundStmt(const CompoundStmt &S, bool GetLast = false,
>> - AggValueSlot AVS = AggValueSlot::ignored());
>> - llvm::Value *EmitCompoundStmtWithoutScope(const CompoundStmt &S,
>> - bool GetLast = false,
>> - AggValueSlot AVS =
>> + Address EmitCompoundStmt(const CompoundStmt &S, bool GetLast = false,
>> + AggValueSlot AVS = AggValueSlot::ignored());
>> + Address EmitCompoundStmtWithoutScope(const CompoundStmt &S,
>> + bool GetLast = false,
>> + AggValueSlot AVS =
>> AggValueSlot::ignored());
>>
>> /// EmitLabel - Emit the block for the given label. It is legal to call this
>> @@ -2085,9 +2181,9 @@ public:
>> /// either be an alloca or a call to llvm.localrecover if there are nested
>> /// outlined functions. ParentFP is the frame pointer of the outermost parent
>> /// frame.
>> - llvm::Value *recoverAddrOfEscapedLocal(CodeGenFunction &ParentCGF,
>> - llvm::Value *ParentVar,
>> - llvm::Value *ParentFP);
>> + Address recoverAddrOfEscapedLocal(CodeGenFunction &ParentCGF,
>> + Address ParentVar,
>> + llvm::Value *ParentFP);
>>
>> void EmitCXXForRangeStmt(const CXXForRangeStmt &S,
>> ArrayRef<const Attr *> Attrs = None);
>> @@ -2097,7 +2193,7 @@ public:
>> void GenerateCapturedStmtFunctionProlog(const CapturedStmt &S);
>> llvm::Function *GenerateCapturedStmtFunctionEpilog(const CapturedStmt &S);
>> llvm::Function *GenerateCapturedStmtFunction(const CapturedStmt &S);
>> - llvm::Value *GenerateCapturedStmtArgument(const CapturedStmt &S);
>> + Address GenerateCapturedStmtArgument(const CapturedStmt &S);
>> /// \brief Perform element by element copying of arrays with type \a
>> /// OriginalType from \a SrcAddr to \a DestAddr using copying procedure
>> /// generated by \a CopyGen.
>> @@ -2108,8 +2204,8 @@ public:
>> /// \param CopyGen Copying procedure that copies value of single array element
>> /// to another single array element.
>> void EmitOMPAggregateAssign(
>> - llvm::Value *DestAddr, llvm::Value *SrcAddr, QualType OriginalType,
>> - const llvm::function_ref<void(llvm::Value *, llvm::Value *)> &CopyGen);
>> + Address DestAddr, Address SrcAddr, QualType OriginalType,
>> + const llvm::function_ref<void(Address, Address)> &CopyGen);
>> /// \brief Emit proper copying of data from one variable to another.
>> ///
>> /// \param OriginalType Original type of the copied variables.
>> @@ -2121,8 +2217,8 @@ public:
>> /// the base array element).
>> /// \param Copy Actual copygin expression for copying data from \a SrcVD to \a
>> /// DestVD.
>> - void EmitOMPCopy(CodeGenFunction &CGF, QualType OriginalType,
>> - llvm::Value *DestAddr, llvm::Value *SrcAddr,
>> + void EmitOMPCopy(QualType OriginalType,
>> + Address DestAddr, Address SrcAddr,
>> const VarDecl *DestVD, const VarDecl *SrcVD,
>> const Expr *Copy);
>> /// \brief Emit atomic update code for constructs: \a X = \a X \a BO \a E or
>> @@ -2258,8 +2354,8 @@ private:
>> void EmitOMPForOuterLoop(OpenMPScheduleClauseKind ScheduleKind,
>> const OMPLoopDirective &S,
>> OMPPrivateScope &LoopScope, bool Ordered,
>> - llvm::Value *LB, llvm::Value *UB, llvm::Value *ST,
>> - llvm::Value *IL, llvm::Value *Chunk);
>> + Address LB, Address UB, Address ST,
>> + Address IL, llvm::Value *Chunk);
>> /// \brief Emit code for sections directive.
>> OpenMPDirectiveKind EmitSections(const OMPExecutableDirective &S);
>>
>> @@ -2306,7 +2402,7 @@ public:
>> /// that the address will be used to access the object.
>> LValue EmitCheckedLValue(const Expr *E, TypeCheckKind TCK);
>>
>> - RValue convertTempToRValue(llvm::Value *addr, QualType type,
>> + RValue convertTempToRValue(Address addr, QualType type,
>> SourceLocation Loc);
>>
>> void EmitAtomicInit(Expr *E, LValue lvalue);
>> @@ -2347,9 +2443,10 @@ public:
>> /// EmitLoadOfScalar - Load a scalar value from an address, taking
>> /// care to appropriately convert from the memory representation to
>> /// the LLVM value representation.
>> - llvm::Value *EmitLoadOfScalar(llvm::Value *Addr, bool Volatile,
>> - unsigned Alignment, QualType Ty,
>> + llvm::Value *EmitLoadOfScalar(Address Addr, bool Volatile, QualType Ty,
>> SourceLocation Loc,
>> + AlignmentSource AlignSource =
>> + AlignmentSource::Type,
>> llvm::MDNode *TBAAInfo = nullptr,
>> QualType TBAABaseTy = QualType(),
>> uint64_t TBAAOffset = 0);
>> @@ -2363,8 +2460,9 @@ public:
>> /// EmitStoreOfScalar - Store a scalar value to an address, taking
>> /// care to appropriately convert from the memory representation to
>> /// the LLVM value representation.
>> - void EmitStoreOfScalar(llvm::Value *Value, llvm::Value *Addr,
>> - bool Volatile, unsigned Alignment, QualType Ty,
>> + void EmitStoreOfScalar(llvm::Value *Value, Address Addr,
>> + bool Volatile, QualType Ty,
>> + AlignmentSource AlignSource = AlignmentSource::Type,
>> llvm::MDNode *TBAAInfo = nullptr, bool isInit = false,
>> QualType TBAABaseTy = QualType(),
>> uint64_t TBAAOffset = 0);
>> @@ -2433,10 +2531,13 @@ public:
>> LValue EmitMaterializeTemporaryExpr(const MaterializeTemporaryExpr *E);
>> LValue EmitOpaqueValueLValue(const OpaqueValueExpr *e);
>>
>> - llvm::Value *EmitExtVectorElementLValue(LValue V);
>> + Address EmitExtVectorElementLValue(LValue V);
>>
>> RValue EmitRValueForField(LValue LV, const FieldDecl *FD, SourceLocation Loc);
>>
>> + Address EmitArrayToPointerDecay(const Expr *Array,
>> + AlignmentSource *AlignSource = nullptr);
>> +
>> class ConstantEmission {
>> llvm::PointerIntPair<llvm::Constant*, 1, bool> ValueAndIsReference;
>> ConstantEmission(llvm::Constant *C, bool isReference)
>> @@ -2576,6 +2677,10 @@ public:
>> NestedNameSpecifier *Qualifier,
>> bool IsArrow, const Expr *Base);
>> // Compute the object pointer.
>> + Address EmitCXXMemberDataPointerAddress(const Expr *E, Address base,
>> + llvm::Value *memberPtr,
>> + const MemberPointerType *memberPtrType,
>> + AlignmentSource *AlignSource = nullptr);
>> RValue EmitCXXMemberPointerCallExpr(const CXXMemberCallExpr *E,
>> ReturnValueSlot ReturnValue);
>>
>> @@ -2610,7 +2715,7 @@ public:
>> unsigned Modifier,
>> const CallExpr *E,
>> SmallVectorImpl<llvm::Value *> &Ops,
>> - llvm::Value *Align = nullptr);
>> + Address PtrOp0, Address PtrOp1);
>> llvm::Function *LookupNeonLLVMIntrinsic(unsigned IntrinsicID,
>> unsigned Modifier, llvm::Type *ArgTy,
>> const CallExpr *E);
>> @@ -2656,24 +2761,23 @@ public:
>> }
>>
>> // ARC primitives.
>> - void EmitARCInitWeak(llvm::Value *value, llvm::Value *addr);
>> - void EmitARCDestroyWeak(llvm::Value *addr);
>> - llvm::Value *EmitARCLoadWeak(llvm::Value *addr);
>> - llvm::Value *EmitARCLoadWeakRetained(llvm::Value *addr);
>> - llvm::Value *EmitARCStoreWeak(llvm::Value *value, llvm::Value *addr,
>> - bool ignored);
>> - void EmitARCCopyWeak(llvm::Value *dst, llvm::Value *src);
>> - void EmitARCMoveWeak(llvm::Value *dst, llvm::Value *src);
>> + void EmitARCInitWeak(Address addr, llvm::Value *value);
>> + void EmitARCDestroyWeak(Address addr);
>> + llvm::Value *EmitARCLoadWeak(Address addr);
>> + llvm::Value *EmitARCLoadWeakRetained(Address addr);
>> + llvm::Value *EmitARCStoreWeak(Address addr, llvm::Value *value, bool ignored);
>> + void EmitARCCopyWeak(Address dst, Address src);
>> + void EmitARCMoveWeak(Address dst, Address src);
>> llvm::Value *EmitARCRetainAutorelease(QualType type, llvm::Value *value);
>> llvm::Value *EmitARCRetainAutoreleaseNonBlock(llvm::Value *value);
>> llvm::Value *EmitARCStoreStrong(LValue lvalue, llvm::Value *value,
>> bool resultIgnored);
>> - llvm::Value *EmitARCStoreStrongCall(llvm::Value *addr, llvm::Value *value,
>> + llvm::Value *EmitARCStoreStrongCall(Address addr, llvm::Value *value,
>> bool resultIgnored);
>> llvm::Value *EmitARCRetain(QualType type, llvm::Value *value);
>> llvm::Value *EmitARCRetainNonBlock(llvm::Value *value);
>> llvm::Value *EmitARCRetainBlock(llvm::Value *value, bool mandatory);
>> - void EmitARCDestroyStrong(llvm::Value *addr, ARCPreciseLifetime_t precise);
>> + void EmitARCDestroyStrong(Address addr, ARCPreciseLifetime_t precise);
>> void EmitARCRelease(llvm::Value *value, ARCPreciseLifetime_t precise);
>> llvm::Value *EmitARCAutorelease(llvm::Value *value);
>> llvm::Value *EmitARCAutoreleaseReturnValue(llvm::Value *value);
>> @@ -2765,6 +2869,9 @@ public:
>> /// EmitLoadOfComplex - Load a complex number from the specified l-value.
>> ComplexPairTy EmitLoadOfComplex(LValue src, SourceLocation loc);
>>
>> + Address emitAddrOfRealComponent(Address complex, QualType complexType);
>> + Address emitAddrOfImagComponent(Address complex, QualType complexType);
>> +
>> /// AddInitializerToStaticVarDecl - Add the initializer for 'D' to the
>> /// global variable that has already been created for it. If the initializer
>> /// has a different type than GV does, this may free GV and return a different
>> @@ -2799,7 +2906,7 @@ public:
>> /// variables.
>> void GenerateCXXGlobalInitFunc(llvm::Function *Fn,
>> ArrayRef<llvm::Function *> CXXThreadLocals,
>> - llvm::GlobalVariable *Guard = nullptr);
>> + Address Guard = Address::invalid());
>>
>> /// GenerateCXXGlobalDtorsFunc - Generates code for destroying global
>> /// variables.
>> @@ -2814,8 +2921,7 @@ public:
>>
>> void EmitCXXConstructExpr(const CXXConstructExpr *E, AggValueSlot Dest);
>>
>> - void EmitSynthesizedCXXCopyCtor(llvm::Value *Dest, llvm::Value *Src,
>> - const Expr *Exp);
>> + void EmitSynthesizedCXXCopyCtor(Address Dest, Address Src, const Expr *Exp);
>>
>> void enterFullExpression(const ExprWithCleanups *E) {
>> if (E->getNumObjects() == 0) return;
>> @@ -2827,7 +2933,7 @@ public:
>>
>> void EmitLambdaExpr(const LambdaExpr *E, AggValueSlot Dest);
>>
>> - RValue EmitAtomicExpr(AtomicExpr *E, llvm::Value *Dest = nullptr);
>> + RValue EmitAtomicExpr(AtomicExpr *E, Address Dest = Address::invalid());
>>
>> //===--------------------------------------------------------------------===//
>> // Annotations Emission
>> @@ -2844,7 +2950,7 @@ public:
>>
>> /// Emit field annotations for the given field & value. Returns the
>> /// annotation result.
>> - llvm::Value *EmitFieldAnnotations(const FieldDecl *D, llvm::Value *V);
>> + Address EmitFieldAnnotations(const FieldDecl *D, Address V);
>>
>> //===--------------------------------------------------------------------===//
>> // Internal Helpers
>> @@ -2932,6 +3038,12 @@ private:
>> llvm::SmallVector<std::pair<llvm::Instruction *, llvm::Value *>, 4>
>> DeferredReplacements;
>>
>> + /// Set the address of a local variable.
>> + void setAddrOfLocalVar(const VarDecl *VD, Address Addr) {
>> + assert(!LocalDeclMap.count(VD) && "Decl already exists in LocalDeclMap!");
>> + LocalDeclMap.insert({VD, Addr});
>> + }
>> +
>> /// ExpandTypeFromArgs - Reconstruct a structure of type \arg Ty
>> /// from function arguments into \arg Dst. See ABIArgInfo::Expand.
>> ///
>> @@ -3028,6 +3140,30 @@ public:
>> const FunctionDecl *CalleeDecl = nullptr,
>> unsigned ParamsToSkip = 0);
>>
>> + /// EmitPointerWithAlignment - Given an expression with a pointer
>> + /// type, emit the value and compute our best estimate of the
>> + /// alignment of the pointee.
>> + ///
>> + /// Note that this function will conservatively fall back on the type
>> + /// when it doesn't
>> + ///
>> + /// \param Source - If non-null, this will be initialized with
>> + /// information about the source of the alignment. Note that this
>> + /// function will conservatively fall back on the type when it
>> + /// doesn't recognize the expression, which means that sometimes
>> + ///
>> + /// a worst-case One
>> + /// reasonable way to use this information is when there's a
>> + /// language guarantee that the pointer must be aligned to some
>> + /// stricter value, and we're simply trying to ensure that
>> + /// sufficiently obvious uses of under-aligned objects don't get
>> + /// miscompiled; for example, a placement new into the address of
>> + /// a local variable. In such a case, it's quite reasonable to
>> + /// just ignore the returned alignment when it isn't from an
>> + /// explicit source.
>> + Address EmitPointerWithAlignment(const Expr *Addr,
>> + AlignmentSource *Source = nullptr);
>> +
>> private:
>> QualType getVarArgType(const Expr *Arg);
>>
>> @@ -3037,16 +3173,11 @@ private:
>>
>> void EmitDeclMetadata();
>>
>> - CodeGenModule::ByrefHelpers *
>> - buildByrefHelpers(llvm::StructType &byrefType,
>> - const AutoVarEmission &emission);
>> + BlockByrefHelpers *buildByrefHelpers(llvm::StructType &byrefType,
>> + const AutoVarEmission &emission);
>>
>> void AddObjCARCExceptionMetadata(llvm::Instruction *Inst);
>>
>> - /// GetPointeeAlignment - Given an expression with a pointer type, emit the
>> - /// value and compute our best estimate of the alignment of the pointee.
>> - std::pair<llvm::Value*, unsigned> EmitPointerWithAlignment(const Expr *Addr);
>> -
>> llvm::Value *GetValueForARMHint(unsigned BuiltinID);
>> };
>>
>> @@ -3069,17 +3200,23 @@ struct DominatingLLVMValue {
>> static saved_type save(CodeGenFunction &CGF, llvm::Value *value) {
>> if (!needsSaving(value)) return saved_type(value, false);
>>
>> - // Otherwise we need an alloca.
>> - llvm::Value *alloca =
>> - CGF.CreateTempAlloca(value->getType(), "cond-cleanup.save");
>> + // Otherwise, we need an alloca.
>> + auto align = CharUnits::fromQuantity(
>> + CGF.CGM.getDataLayout().getPrefTypeAlignment(value->getType()));
>> + Address alloca =
>> + CGF.CreateTempAlloca(value->getType(), align, "cond-cleanup.save");
>> CGF.Builder.CreateStore(value, alloca);
>>
>> - return saved_type(alloca, true);
>> + return saved_type(alloca.getPointer(), true);
>> }
>>
>> static llvm::Value *restore(CodeGenFunction &CGF, saved_type value) {
>> + // If the value says it wasn't saved, trust that it's still dominating.
>> if (!value.getInt()) return value.getPointer();
>> - return CGF.Builder.CreateLoad(value.getPointer());
>> +
>> + // Otherwise, it should be an alloca instruction, as set up in save().
>> + auto alloca = cast<llvm::AllocaInst>(value.getPointer());
>> + return CGF.Builder.CreateAlignedLoad(alloca, alloca->getAlignment());
>> }
>> };
>>
>> @@ -3092,6 +3229,28 @@ template <class T> struct DominatingPoin
>> }
>> };
>>
>> +/// A specialization of DominatingValue for Address.
>> +template <> struct DominatingValue<Address> {
>> + typedef Address type;
>> +
>> + struct saved_type {
>> + DominatingLLVMValue::saved_type SavedValue;
>> + CharUnits Alignment;
>> + };
>> +
>> + static bool needsSaving(type value) {
>> + return DominatingLLVMValue::needsSaving(value.getPointer());
>> + }
>> + static saved_type save(CodeGenFunction &CGF, type value) {
>> + return { DominatingLLVMValue::save(CGF, value.getPointer()),
>> + value.getAlignment() };
>> + }
>> + static type restore(CodeGenFunction &CGF, saved_type value) {
>> + return Address(DominatingLLVMValue::restore(CGF, value.SavedValue),
>> + value.Alignment);
>> + }
>> +};
>> +
>> /// A specialization of DominatingValue for RValue.
>> template <> struct DominatingValue<RValue> {
>> typedef RValue type;
>> @@ -3100,15 +3259,17 @@ template <> struct DominatingValue<RValu
>> AggregateAddress, ComplexAddress };
>>
>> llvm::Value *Value;
>> - Kind K;
>> - saved_type(llvm::Value *v, Kind k) : Value(v), K(k) {}
>> + unsigned K : 3;
>> + unsigned Align : 29;
>> + saved_type(llvm::Value *v, Kind k, unsigned a = 0)
>> + : Value(v), K(k), Align(a) {}
>>
>> public:
>> static bool needsSaving(RValue value);
>> static saved_type save(CodeGenFunction &CGF, RValue value);
>> RValue restore(CodeGenFunction &CGF);
>>
>> - // implementations in CGExprCXX.cpp
>> + // implementations in CGCleanup.cpp
>> };
>>
>> static bool needsSaving(type value) {
>>
>> Modified: cfe/trunk/lib/CodeGen/CodeGenModule.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CodeGenModule.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/CodeGenModule.cpp Tue Sep 8 03:05:57 2015
>> @@ -12,6 +12,7 @@
>> //===----------------------------------------------------------------------===//
>>
>> #include "CodeGenModule.h"
>> +#include "CGBlocks.h"
>> #include "CGCUDARuntime.h"
>> #include "CGCXXABI.h"
>> #include "CGCall.h"
>> @@ -106,7 +107,9 @@ CodeGenModule::CodeGenModule(ASTContext
>> DoubleTy = llvm::Type::getDoubleTy(LLVMContext);
>> PointerWidthInBits = C.getTargetInfo().getPointerWidth(0);
>> PointerAlignInBytes =
>> - C.toCharUnitsFromBits(C.getTargetInfo().getPointerAlign(0)).getQuantity();
>> + C.toCharUnitsFromBits(C.getTargetInfo().getPointerAlign(0)).getQuantity();
>> + IntAlignInBytes =
>> + C.toCharUnitsFromBits(C.getTargetInfo().getIntAlign()).getQuantity();
>> IntTy = llvm::IntegerType::get(LLVMContext, C.getTargetInfo().getIntWidth());
>> IntPtrTy = llvm::IntegerType::get(LLVMContext, PointerWidthInBits);
>> Int8PtrTy = Int8Ty->getPointerTo(0);
>> @@ -1303,7 +1306,7 @@ bool CodeGenModule::MayBeEmittedEagerly(
>> return true;
>> }
>>
>> -llvm::Constant *CodeGenModule::GetAddrOfUuidDescriptor(
>> +ConstantAddress CodeGenModule::GetAddrOfUuidDescriptor(
>> const CXXUuidofExpr* E) {
>> // Sema has verified that IIDSource has a __declspec(uuid()), and that its
>> // well-formed.
>> @@ -1311,9 +1314,12 @@ llvm::Constant *CodeGenModule::GetAddrOf
>> std::string Name = "_GUID_" + Uuid.lower();
>> std::replace(Name.begin(), Name.end(), '-', '_');
>>
>> + // Contains a 32-bit field.
>> + CharUnits Alignment = CharUnits::fromQuantity(4);
>> +
>> // Look for an existing global.
>> if (llvm::GlobalVariable *GV = getModule().getNamedGlobal(Name))
>> - return GV;
>> + return ConstantAddress(GV, Alignment);
>>
>> llvm::Constant *Init = EmitUuidofInitializer(Uuid);
>> assert(Init && "failed to initialize as constant");
>> @@ -1323,20 +1329,22 @@ llvm::Constant *CodeGenModule::GetAddrOf
>> /*isConstant=*/true, llvm::GlobalValue::LinkOnceODRLinkage, Init, Name);
>> if (supportsCOMDAT())
>> GV->setComdat(TheModule.getOrInsertComdat(GV->getName()));
>> - return GV;
>> + return ConstantAddress(GV, Alignment);
>> }
>>
>> -llvm::Constant *CodeGenModule::GetWeakRefReference(const ValueDecl *VD) {
>> +ConstantAddress CodeGenModule::GetWeakRefReference(const ValueDecl *VD) {
>> const AliasAttr *AA = VD->getAttr<AliasAttr>();
>> assert(AA && "No alias?");
>>
>> + CharUnits Alignment = getContext().getDeclAlign(VD);
>> llvm::Type *DeclTy = getTypes().ConvertTypeForMem(VD->getType());
>>
>> // See if there is already something with the target's name in the module.
>> llvm::GlobalValue *Entry = GetGlobalValue(AA->getAliasee());
>> if (Entry) {
>> unsigned AS = getContext().getTargetAddressSpace(VD->getType());
>> - return llvm::ConstantExpr::getBitCast(Entry, DeclTy->getPointerTo(AS));
>> + auto Ptr = llvm::ConstantExpr::getBitCast(Entry, DeclTy->getPointerTo(AS));
>> + return ConstantAddress(Ptr, Alignment);
>> }
>>
>> llvm::Constant *Aliasee;
>> @@ -1353,7 +1361,7 @@ llvm::Constant *CodeGenModule::GetWeakRe
>> F->setLinkage(llvm::Function::ExternalWeakLinkage);
>> WeakRefReferences.insert(F);
>>
>> - return Aliasee;
>> + return ConstantAddress(Aliasee, Alignment);
>> }
>>
>> void CodeGenModule::EmitGlobal(GlobalDecl GD) {
>> @@ -2732,7 +2740,7 @@ GetConstantStringEntry(llvm::StringMap<l
>> return *Map.insert(std::make_pair(String, nullptr)).first;
>> }
>>
>> -llvm::Constant *
>> +ConstantAddress
>> CodeGenModule::GetAddrOfConstantCFString(const StringLiteral *Literal) {
>> unsigned StringLength = 0;
>> bool isUTF16 = false;
>> @@ -2742,7 +2750,7 @@ CodeGenModule::GetAddrOfConstantCFString
>> StringLength);
>>
>> if (auto *C = Entry.second)
>> - return C;
>> + return ConstantAddress(C, CharUnits::fromQuantity(C->getAlignment()));
>>
>> llvm::Constant *Zero = llvm::Constant::getNullValue(Int32Ty);
>> llvm::Constant *Zeros[] = { Zero, Zero };
>> @@ -2819,25 +2827,28 @@ CodeGenModule::GetAddrOfConstantCFString
>> Ty = getTypes().ConvertType(getContext().LongTy);
>> Fields[3] = llvm::ConstantInt::get(Ty, StringLength);
>>
>> + CharUnits Alignment = getPointerAlign();
>> +
>> // The struct.
>> C = llvm::ConstantStruct::get(STy, Fields);
>> GV = new llvm::GlobalVariable(getModule(), C->getType(), true,
>> llvm::GlobalVariable::PrivateLinkage, C,
>> "_unnamed_cfstring_");
>> GV->setSection("__DATA,__cfstring");
>> + GV->setAlignment(Alignment.getQuantity());
>> Entry.second = GV;
>>
>> - return GV;
>> + return ConstantAddress(GV, Alignment);
>> }
>>
>> -llvm::GlobalVariable *
>> +ConstantAddress
>> CodeGenModule::GetAddrOfConstantString(const StringLiteral *Literal) {
>> unsigned StringLength = 0;
>> llvm::StringMapEntry<llvm::GlobalVariable *> &Entry =
>> GetConstantStringEntry(CFConstantStringMap, Literal, StringLength);
>>
>> if (auto *C = Entry.second)
>> - return C;
>> + return ConstantAddress(C, CharUnits::fromQuantity(C->getAlignment()));
>>
>> llvm::Constant *Zero = llvm::Constant::getNullValue(Int32Ty);
>> llvm::Constant *Zeros[] = { Zero, Zero };
>> @@ -2930,10 +2941,12 @@ CodeGenModule::GetAddrOfConstantString(c
>> Fields[2] = llvm::ConstantInt::get(Ty, StringLength);
>>
>> // The struct.
>> + CharUnits Alignment = getPointerAlign();
>> C = llvm::ConstantStruct::get(NSConstantStringType, Fields);
>> GV = new llvm::GlobalVariable(getModule(), C->getType(), true,
>> llvm::GlobalVariable::PrivateLinkage, C,
>> "_unnamed_nsstring_");
>> + GV->setAlignment(Alignment.getQuantity());
>> const char *NSStringSection = "__OBJC,__cstring_object,regular,no_dead_strip";
>> const char *NSStringNonFragileABISection =
>> "__DATA,__objc_stringobj,regular,no_dead_strip";
>> @@ -2943,7 +2956,7 @@ CodeGenModule::GetAddrOfConstantString(c
>> : NSStringSection);
>> Entry.second = GV;
>>
>> - return GV;
>> + return ConstantAddress(GV, Alignment);
>> }
>>
>> QualType CodeGenModule::getObjCFastEnumerationStateType() {
>> @@ -3022,7 +3035,7 @@ CodeGenModule::GetConstantArrayFromStrin
>> static llvm::GlobalVariable *
>> GenerateStringLiteral(llvm::Constant *C, llvm::GlobalValue::LinkageTypes LT,
>> CodeGenModule &CGM, StringRef GlobalName,
>> - unsigned Alignment) {
>> + CharUnits Alignment) {
>> // OpenCL v1.2 s6.5.3: a string literal is in the constant address space.
>> unsigned AddrSpace = 0;
>> if (CGM.getLangOpts().OpenCL)
>> @@ -3033,7 +3046,7 @@ GenerateStringLiteral(llvm::Constant *C,
>> auto *GV = new llvm::GlobalVariable(
>> M, C->getType(), !CGM.getLangOpts().WritableStrings, LT, C, GlobalName,
>> nullptr, llvm::GlobalVariable::NotThreadLocal, AddrSpace);
>> - GV->setAlignment(Alignment);
>> + GV->setAlignment(Alignment.getQuantity());
>> GV->setUnnamedAddr(true);
>> if (GV->isWeakForLinker()) {
>> assert(CGM.supportsCOMDAT() && "Only COFF uses weak string literals");
>> @@ -3045,20 +3058,19 @@ GenerateStringLiteral(llvm::Constant *C,
>>
>> /// GetAddrOfConstantStringFromLiteral - Return a pointer to a
>> /// constant array for the given string literal.
>> -llvm::GlobalVariable *
>> +ConstantAddress
>> CodeGenModule::GetAddrOfConstantStringFromLiteral(const StringLiteral *S,
>> StringRef Name) {
>> - auto Alignment =
>> - getContext().getAlignOfGlobalVarInChars(S->getType()).getQuantity();
>> + CharUnits Alignment = getContext().getAlignOfGlobalVarInChars(S->getType());
>>
>> llvm::Constant *C = GetConstantArrayFromStringLiteral(S);
>> llvm::GlobalVariable **Entry = nullptr;
>> if (!LangOpts.WritableStrings) {
>> Entry = &ConstantStringMap[C];
>> if (auto GV = *Entry) {
>> - if (Alignment > GV->getAlignment())
>> - GV->setAlignment(Alignment);
>> - return GV;
>> + if (Alignment.getQuantity() > GV->getAlignment())
>> + GV->setAlignment(Alignment.getQuantity());
>> + return ConstantAddress(GV, Alignment);
>> }
>> }
>>
>> @@ -3088,12 +3100,12 @@ CodeGenModule::GetAddrOfConstantStringFr
>>
>> SanitizerMD->reportGlobalToASan(GV, S->getStrTokenLoc(0), "<string literal>",
>> QualType());
>> - return GV;
>> + return ConstantAddress(GV, Alignment);
>> }
>>
>> /// GetAddrOfConstantStringFromObjCEncode - Return a pointer to a constant
>> /// array for the given ObjCEncodeExpr node.
>> -llvm::GlobalVariable *
>> +ConstantAddress
>> CodeGenModule::GetAddrOfConstantStringFromObjCEncode(const ObjCEncodeExpr *E) {
>> std::string Str;
>> getContext().getObjCEncodingForType(E->getEncodedType(), Str);
>> @@ -3104,14 +3116,11 @@ CodeGenModule::GetAddrOfConstantStringFr
>> /// GetAddrOfConstantCString - Returns a pointer to a character array containing
>> /// the literal and a terminating '\0' character.
>> /// The result has pointer to array type.
>> -llvm::GlobalVariable *CodeGenModule::GetAddrOfConstantCString(
>> - const std::string &Str, const char *GlobalName, unsigned Alignment) {
>> +ConstantAddress CodeGenModule::GetAddrOfConstantCString(
>> + const std::string &Str, const char *GlobalName) {
>> StringRef StrWithNull(Str.c_str(), Str.size() + 1);
>> - if (Alignment == 0) {
>> - Alignment = getContext()
>> - .getAlignOfGlobalVarInChars(getContext().CharTy)
>> - .getQuantity();
>> - }
>> + CharUnits Alignment =
>> + getContext().getAlignOfGlobalVarInChars(getContext().CharTy);
>>
>> llvm::Constant *C =
>> llvm::ConstantDataArray::getString(getLLVMContext(), StrWithNull, false);
>> @@ -3121,9 +3130,9 @@ llvm::GlobalVariable *CodeGenModule::Get
>> if (!LangOpts.WritableStrings) {
>> Entry = &ConstantStringMap[C];
>> if (auto GV = *Entry) {
>> - if (Alignment > GV->getAlignment())
>> - GV->setAlignment(Alignment);
>> - return GV;
>> + if (Alignment.getQuantity() > GV->getAlignment())
>> + GV->setAlignment(Alignment.getQuantity());
>> + return ConstantAddress(GV, Alignment);
>> }
>> }
>>
>> @@ -3135,10 +3144,10 @@ llvm::GlobalVariable *CodeGenModule::Get
>> GlobalName, Alignment);
>> if (Entry)
>> *Entry = GV;
>> - return GV;
>> + return ConstantAddress(GV, Alignment);
>> }
>>
>> -llvm::Constant *CodeGenModule::GetAddrOfGlobalTemporary(
>> +ConstantAddress CodeGenModule::GetAddrOfGlobalTemporary(
>> const MaterializeTemporaryExpr *E, const Expr *Init) {
>> assert((E->getStorageDuration() == SD_Static ||
>> E->getStorageDuration() == SD_Thread) && "not a global temporary");
>> @@ -3150,8 +3159,10 @@ llvm::Constant *CodeGenModule::GetAddrOf
>> if (Init == E->GetTemporaryExpr())
>> MaterializedType = E->getType();
>>
>> + CharUnits Align = getContext().getTypeAlignInChars(MaterializedType);
>> +
>> if (llvm::Constant *Slot = MaterializedGlobalTemporaryMap[E])
>> - return Slot;
>> + return ConstantAddress(Slot, Align);
>>
>> // FIXME: If an externally-visible declaration extends multiple temporaries,
>> // we need to give each temporary the same name in every translation unit (and
>> @@ -3215,14 +3226,13 @@ llvm::Constant *CodeGenModule::GetAddrOf
>> /*InsertBefore=*/nullptr, llvm::GlobalVariable::NotThreadLocal,
>> AddrSpace);
>> setGlobalVisibility(GV, VD);
>> - GV->setAlignment(
>> - getContext().getTypeAlignInChars(MaterializedType).getQuantity());
>> + GV->setAlignment(Align.getQuantity());
>> if (supportsCOMDAT() && GV->isWeakForLinker())
>> GV->setComdat(TheModule.getOrInsertComdat(GV->getName()));
>> if (VD->getTLSKind())
>> setTLSMode(GV, *VD);
>> MaterializedGlobalTemporaryMap[E] = GV;
>> - return GV;
>> + return ConstantAddress(GV, Align);
>> }
>>
>> /// EmitObjCPropertyImplementations - Emit information for synthesized
>> @@ -3676,7 +3686,7 @@ void CodeGenFunction::EmitDeclMetadata()
>>
>> for (auto &I : LocalDeclMap) {
>> const Decl *D = I.first;
>> - llvm::Value *Addr = I.second;
>> + llvm::Value *Addr = I.second.getPointer();
>> if (auto *Alloca = dyn_cast<llvm::AllocaInst>(Addr)) {
>> llvm::Value *DAddr = GetPointerConstant(getLLVMContext(), D);
>> Alloca->setMetadata(
>> @@ -3785,8 +3795,10 @@ void CodeGenModule::EmitOMPThreadPrivate
>> VD->getAnyInitializer() &&
>> !VD->getAnyInitializer()->isConstantInitializer(getContext(),
>> /*ForRef=*/false);
>> +
>> + Address Addr(GetAddrOfGlobalVar(VD), getContext().getDeclAlign(VD));
>> if (auto InitFunction = getOpenMPRuntime().emitThreadPrivateVarDefinition(
>> - VD, GetAddrOfGlobalVar(VD), RefExpr->getLocStart(), PerformInit))
>> + VD, Addr, RefExpr->getLocStart(), PerformInit))
>> CXXGlobalInits.push_back(InitFunction);
>> }
>> }
>>
>> Modified: cfe/trunk/lib/CodeGen/CodeGenModule.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenModule.h?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CodeGenModule.h (original)
>> +++ cfe/trunk/lib/CodeGen/CodeGenModule.h Tue Sep 8 03:05:57 2015
>> @@ -15,6 +15,7 @@
>> #define LLVM_CLANG_LIB_CODEGEN_CODEGENMODULE_H
>>
>> #include "CGVTables.h"
>> +#include "CodeGenTypeCache.h"
>> #include "CodeGenTypes.h"
>> #include "SanitizerMetadata.h"
>> #include "clang/AST/Attr.h"
>> @@ -30,7 +31,6 @@
>> #include "llvm/ADT/SetVector.h"
>> #include "llvm/ADT/SmallPtrSet.h"
>> #include "llvm/ADT/StringMap.h"
>> -#include "llvm/IR/CallingConv.h"
>> #include "llvm/IR/Module.h"
>> #include "llvm/IR/ValueHandle.h"
>>
>> @@ -79,6 +79,7 @@ class CoverageSourceInfo;
>>
>> namespace CodeGen {
>>
>> +class BlockByrefHelpers;
>> class CallArgList;
>> class CodeGenFunction;
>> class CodeGenTBAA;
>> @@ -108,54 +109,6 @@ struct OrderGlobalInits {
>> }
>> };
>>
>> -struct CodeGenTypeCache {
>> - /// void
>> - llvm::Type *VoidTy;
>> -
>> - /// i8, i16, i32, and i64
>> - llvm::IntegerType *Int8Ty, *Int16Ty, *Int32Ty, *Int64Ty;
>> - /// float, double
>> - llvm::Type *FloatTy, *DoubleTy;
>> -
>> - /// int
>> - llvm::IntegerType *IntTy;
>> -
>> - /// intptr_t, size_t, and ptrdiff_t, which we assume are the same size.
>> - union {
>> - llvm::IntegerType *IntPtrTy;
>> - llvm::IntegerType *SizeTy;
>> - llvm::IntegerType *PtrDiffTy;
>> - };
>> -
>> - /// void* in address space 0
>> - union {
>> - llvm::PointerType *VoidPtrTy;
>> - llvm::PointerType *Int8PtrTy;
>> - };
>> -
>> - /// void** in address space 0
>> - union {
>> - llvm::PointerType *VoidPtrPtrTy;
>> - llvm::PointerType *Int8PtrPtrTy;
>> - };
>> -
>> - /// The width of a pointer into the generic address space.
>> - unsigned char PointerWidthInBits;
>> -
>> - /// The size and alignment of a pointer into the generic address
>> - /// space.
>> - union {
>> - unsigned char PointerAlignInBytes;
>> - unsigned char PointerSizeInBytes;
>> - unsigned char SizeSizeInBytes; // sizeof(size_t)
>> - };
>> -
>> - llvm::CallingConv::ID RuntimeCC;
>> - llvm::CallingConv::ID getRuntimeCC() const { return RuntimeCC; }
>> - llvm::CallingConv::ID BuiltinCC;
>> - llvm::CallingConv::ID getBuiltinCC() const { return BuiltinCC; }
>> -};
>> -
>> struct RREntrypoints {
>> RREntrypoints() { memset(this, 0, sizeof(*this)); }
>> /// void objc_autoreleasePoolPop(void*);
>> @@ -735,13 +688,28 @@ public:
>> QualType CatchHandlerType);
>>
>> /// Get the address of a uuid descriptor .
>> - llvm::Constant *GetAddrOfUuidDescriptor(const CXXUuidofExpr* E);
>> + ConstantAddress GetAddrOfUuidDescriptor(const CXXUuidofExpr* E);
>>
>> /// Get the address of the thunk for the given global decl.
>> llvm::Constant *GetAddrOfThunk(GlobalDecl GD, const ThunkInfo &Thunk);
>>
>> /// Get a reference to the target of VD.
>> - llvm::Constant *GetWeakRefReference(const ValueDecl *VD);
>> + ConstantAddress GetWeakRefReference(const ValueDecl *VD);
>> +
>> + /// Returns the assumed alignment of an opaque pointer to the given class.
>> + CharUnits getClassPointerAlignment(const CXXRecordDecl *CD);
>> +
>> + /// Returns the assumed alignment of a virtual base of a class.
>> + CharUnits getVBaseAlignment(CharUnits DerivedAlign,
>> + const CXXRecordDecl *Derived,
>> + const CXXRecordDecl *VBase);
>> +
>> + /// Given a class pointer with an actual known alignment, and the
>> + /// expected alignment of an object at a dynamic offset w.r.t that
>> + /// pointer, return the alignment to assume at the offset.
>> + CharUnits getDynamicOffsetAlignment(CharUnits ActualAlign,
>> + const CXXRecordDecl *Class,
>> + CharUnits ExpectedTargetAlign);
>>
>> CharUnits
>> computeNonVirtualBaseClassOffset(const CXXRecordDecl *DerivedClass,
>> @@ -755,36 +723,7 @@ public:
>> CastExpr::path_const_iterator PathBegin,
>> CastExpr::path_const_iterator PathEnd);
>>
>> - /// A pair of helper functions for a __block variable.
>> - class ByrefHelpers : public llvm::FoldingSetNode {
>> - public:
>> - llvm::Constant *CopyHelper;
>> - llvm::Constant *DisposeHelper;
>> -
>> - /// The alignment of the field. This is important because
>> - /// different offsets to the field within the byref struct need to
>> - /// have different helper functions.
>> - CharUnits Alignment;
>> -
>> - ByrefHelpers(CharUnits alignment) : Alignment(alignment) {}
>> - ByrefHelpers(const ByrefHelpers &) = default;
>> - virtual ~ByrefHelpers();
>> -
>> - void Profile(llvm::FoldingSetNodeID &id) const {
>> - id.AddInteger(Alignment.getQuantity());
>> - profileImpl(id);
>> - }
>> - virtual void profileImpl(llvm::FoldingSetNodeID &id) const = 0;
>> -
>> - virtual bool needsCopy() const { return true; }
>> - virtual void emitCopy(CodeGenFunction &CGF,
>> - llvm::Value *dest, llvm::Value *src) = 0;
>> -
>> - virtual bool needsDispose() const { return true; }
>> - virtual void emitDispose(CodeGenFunction &CGF, llvm::Value *field) = 0;
>> - };
>> -
>> - llvm::FoldingSet<ByrefHelpers> ByrefHelpersCache;
>> + llvm::FoldingSet<BlockByrefHelpers> ByrefHelpersCache;
>>
>> /// Fetches the global unique block count.
>> int getUniqueBlockCount() { return ++Block.GlobalUniqueCount; }
>> @@ -799,23 +738,23 @@ public:
>> llvm::Constant *GetAddrOfGlobalBlock(const BlockExpr *BE, const char *);
>>
>> /// Return a pointer to a constant CFString object for the given string.
>> - llvm::Constant *GetAddrOfConstantCFString(const StringLiteral *Literal);
>> + ConstantAddress GetAddrOfConstantCFString(const StringLiteral *Literal);
>>
>> /// Return a pointer to a constant NSString object for the given string. Or a
>> /// user defined String object as defined via
>> /// -fconstant-string-class=class_name option.
>> - llvm::GlobalVariable *GetAddrOfConstantString(const StringLiteral *Literal);
>> + ConstantAddress GetAddrOfConstantString(const StringLiteral *Literal);
>>
>> /// Return a constant array for the given string.
>> llvm::Constant *GetConstantArrayFromStringLiteral(const StringLiteral *E);
>>
>> /// Return a pointer to a constant array for the given string literal.
>> - llvm::GlobalVariable *
>> + ConstantAddress
>> GetAddrOfConstantStringFromLiteral(const StringLiteral *S,
>> StringRef Name = ".str");
>>
>> /// Return a pointer to a constant array for the given ObjCEncodeExpr node.
>> - llvm::GlobalVariable *
>> + ConstantAddress
>> GetAddrOfConstantStringFromObjCEncode(const ObjCEncodeExpr *);
>>
>> /// Returns a pointer to a character array containing the literal and a
>> @@ -823,18 +762,17 @@ public:
>> ///
>> /// \param GlobalName If provided, the name to use for the global (if one is
>> /// created).
>> - llvm::GlobalVariable *
>> + ConstantAddress
>> GetAddrOfConstantCString(const std::string &Str,
>> - const char *GlobalName = nullptr,
>> - unsigned Alignment = 0);
>> + const char *GlobalName = nullptr);
>>
>> /// Returns a pointer to a constant global variable for the given file-scope
>> /// compound literal expression.
>> - llvm::Constant *GetAddrOfConstantCompoundLiteral(const CompoundLiteralExpr*E);
>> + ConstantAddress GetAddrOfConstantCompoundLiteral(const CompoundLiteralExpr*E);
>>
>> /// \brief Returns a pointer to a global variable representing a temporary
>> /// with static or thread storage duration.
>> - llvm::Constant *GetAddrOfGlobalTemporary(const MaterializeTemporaryExpr *E,
>> + ConstantAddress GetAddrOfGlobalTemporary(const MaterializeTemporaryExpr *E,
>> const Expr *Inner);
>>
>> /// \brief Retrieve the record type that describes the state of an
>>
>> Added: cfe/trunk/lib/CodeGen/CodeGenTypeCache.h
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CodeGenTypeCache.h?rev=246985&view=auto
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/CodeGenTypeCache.h (added)
>> +++ cfe/trunk/lib/CodeGen/CodeGenTypeCache.h Tue Sep 8 03:05:57 2015
>> @@ -0,0 +1,108 @@
>> +//===--- CodeGenTypeCache.h - Commonly used LLVM types and info -*- C++ -*-===//
>> +//
>> +// The LLVM Compiler Infrastructure
>> +//
>> +// This file is distributed under the University of Illinois Open Source
>> +// License. See LICENSE.TXT for details.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +//
>> +// This structure provides a set of common types useful during IR emission.
>> +//
>> +//===----------------------------------------------------------------------===//
>> +
>> +#ifndef LLVM_CLANG_LIB_CODEGEN_CODEGENTYPECACHE_H
>> +#define LLVM_CLANG_LIB_CODEGEN_CODEGENTYPECACHE_H
>> +
>> +#include "clang/AST/CharUnits.h"
>> +#include "llvm/IR/CallingConv.h"
>> +
>> +namespace llvm {
>> + class Type;
>> + class IntegerType;
>> + class PointerType;
>> +}
>> +
>> +namespace clang {
>> +namespace CodeGen {
>> +
>> +/// This structure provides a set of types that are commonly used
>> +/// during IR emission. It's initialized once in CodeGenModule's
>> +/// constructor and then copied around into new CodeGenFunctions.
>> +struct CodeGenTypeCache {
>> + /// void
>> + llvm::Type *VoidTy;
>> +
>> + /// i8, i16, i32, and i64
>> + llvm::IntegerType *Int8Ty, *Int16Ty, *Int32Ty, *Int64Ty;
>> + /// float, double
>> + llvm::Type *FloatTy, *DoubleTy;
>> +
>> + /// int
>> + llvm::IntegerType *IntTy;
>> +
>> + /// intptr_t, size_t, and ptrdiff_t, which we assume are the same size.
>> + union {
>> + llvm::IntegerType *IntPtrTy;
>> + llvm::IntegerType *SizeTy;
>> + llvm::IntegerType *PtrDiffTy;
>> + };
>> +
>> + /// void* in address space 0
>> + union {
>> + llvm::PointerType *VoidPtrTy;
>> + llvm::PointerType *Int8PtrTy;
>> + };
>> +
>> + /// void** in address space 0
>> + union {
>> + llvm::PointerType *VoidPtrPtrTy;
>> + llvm::PointerType *Int8PtrPtrTy;
>> + };
>> +
>> + /// The size and alignment of the builtin C type 'int'. This comes
>> + /// up enough in various ABI lowering tasks to be worth pre-computing.
>> + union {
>> + unsigned char IntSizeInBytes;
>> + unsigned char IntAlignInBytes;
>> + };
>> + CharUnits getIntSize() const {
>> + return CharUnits::fromQuantity(IntSizeInBytes);
>> + }
>> + CharUnits getIntAlign() const {
>> + return CharUnits::fromQuantity(IntAlignInBytes);
>> + }
>> +
>> + /// The width of a pointer into the generic address space.
>> + unsigned char PointerWidthInBits;
>> +
>> + /// The size and alignment of a pointer into the generic address space.
>> + union {
>> + unsigned char PointerAlignInBytes;
>> + unsigned char PointerSizeInBytes;
>> + unsigned char SizeSizeInBytes; // sizeof(size_t)
>> + unsigned char SizeAlignInBytes;
>> + };
>> + CharUnits getSizeSize() const {
>> + return CharUnits::fromQuantity(SizeSizeInBytes);
>> + }
>> + CharUnits getSizeAlign() const {
>> + return CharUnits::fromQuantity(SizeAlignInBytes);
>> + }
>> + CharUnits getPointerSize() const {
>> + return CharUnits::fromQuantity(PointerSizeInBytes);
>> + }
>> + CharUnits getPointerAlign() const {
>> + return CharUnits::fromQuantity(PointerAlignInBytes);
>> + }
>> +
>> + llvm::CallingConv::ID RuntimeCC;
>> + llvm::CallingConv::ID getRuntimeCC() const { return RuntimeCC; }
>> + llvm::CallingConv::ID BuiltinCC;
>> + llvm::CallingConv::ID getBuiltinCC() const { return BuiltinCC; }
>> +};
>> +
>> +} // end namespace CodeGen
>> +} // end namespace clang
>> +
>> +#endif
>>
>> Modified: cfe/trunk/lib/CodeGen/ItaniumCXXABI.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/ItaniumCXXABI.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/ItaniumCXXABI.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/ItaniumCXXABI.cpp Tue Sep 8 03:05:57 2015
>> @@ -69,6 +69,45 @@ public:
>> return RAA_Default;
>> }
>>
>> + bool isThisCompleteObject(GlobalDecl GD) const override {
>> + // The Itanium ABI has separate complete-object vs. base-object
>> + // variants of both constructors and destructors.
>> + if (isa<CXXDestructorDecl>(GD.getDecl())) {
>> + switch (GD.getDtorType()) {
>> + case Dtor_Complete:
>> + case Dtor_Deleting:
>> + return true;
>> +
>> + case Dtor_Base:
>> + return false;
>> +
>> + case Dtor_Comdat:
>> + llvm_unreachable("emitting dtor comdat as function?");
>> + }
>> + llvm_unreachable("bad dtor kind");
>> + }
>> + if (isa<CXXConstructorDecl>(GD.getDecl())) {
>> + switch (GD.getCtorType()) {
>> + case Ctor_Complete:
>> + return true;
>> +
>> + case Ctor_Base:
>> + return false;
>> +
>> + case Ctor_CopyingClosure:
>> + case Ctor_DefaultClosure:
>> + llvm_unreachable("closure ctors in Itanium ABI?");
>> +
>> + case Ctor_Comdat:
>> + llvm_unreachable("emitting ctor comdat as function?");
>> + }
>> + llvm_unreachable("bad dtor kind");
>> + }
>> +
>> + // No other kinds.
>> + return false;
>> + }
>> +
>> bool isZeroInitializable(const MemberPointerType *MPT) override;
>>
>> llvm::Type *ConvertMemberPointerType(const MemberPointerType *MPT) override;
>> @@ -76,13 +115,14 @@ public:
>> llvm::Value *
>> EmitLoadOfMemberFunctionPointer(CodeGenFunction &CGF,
>> const Expr *E,
>> - llvm::Value *&This,
>> + Address This,
>> + llvm::Value *&ThisPtrForCall,
>> llvm::Value *MemFnPtr,
>> const MemberPointerType *MPT) override;
>>
>> llvm::Value *
>> EmitMemberDataPointerAddress(CodeGenFunction &CGF, const Expr *E,
>> - llvm::Value *Base,
>> + Address Base,
>> llvm::Value *MemPtr,
>> const MemberPointerType *MPT) override;
>>
>> @@ -111,9 +151,22 @@ public:
>> const MemberPointerType *MPT) override;
>>
>> void emitVirtualObjectDelete(CodeGenFunction &CGF, const CXXDeleteExpr *DE,
>> - llvm::Value *Ptr, QualType ElementType,
>> + Address Ptr, QualType ElementType,
>> const CXXDestructorDecl *Dtor) override;
>>
>> + /// Itanium says that an _Unwind_Exception has to be "double-word"
>> + /// aligned (and thus the end of it is also so-aligned), meaning 16
>> + /// bytes. Of course, that was written for the actual Itanium,
>> + /// which is a 64-bit platform. Classically, the ABI doesn't really
>> + /// specify the alignment on other platforms, but in practice
>> + /// libUnwind declares the struct with __attribute__((aligned)), so
>> + /// we assume that alignment here. (It's generally 16 bytes, but
>> + /// some targets overwrite it.)
>> + CharUnits getAlignmentOfExnObject() {
>> + auto align = CGM.getContext().getTargetDefaultAlignForAttributeAligned();
>> + return CGM.getContext().toCharUnitsFromBits(align);
>> + }
>> +
>> void emitRethrow(CodeGenFunction &CGF, bool isNoReturn) override;
>> void emitThrow(CodeGenFunction &CGF, const CXXThrowExpr *E) override;
>>
>> @@ -135,25 +188,25 @@ public:
>> bool shouldTypeidBeNullChecked(bool IsDeref, QualType SrcRecordTy) override;
>> void EmitBadTypeidCall(CodeGenFunction &CGF) override;
>> llvm::Value *EmitTypeid(CodeGenFunction &CGF, QualType SrcRecordTy,
>> - llvm::Value *ThisPtr,
>> + Address ThisPtr,
>> llvm::Type *StdTypeInfoPtrTy) override;
>>
>> bool shouldDynamicCastCallBeNullChecked(bool SrcIsPtr,
>> QualType SrcRecordTy) override;
>>
>> - llvm::Value *EmitDynamicCastCall(CodeGenFunction &CGF, llvm::Value *Value,
>> + llvm::Value *EmitDynamicCastCall(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy, QualType DestTy,
>> QualType DestRecordTy,
>> llvm::BasicBlock *CastEnd) override;
>>
>> - llvm::Value *EmitDynamicCastToVoid(CodeGenFunction &CGF, llvm::Value *Value,
>> + llvm::Value *EmitDynamicCastToVoid(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy,
>> QualType DestTy) override;
>>
>> bool EmitBadCastCall(CodeGenFunction &CGF) override;
>>
>> llvm::Value *
>> - GetVirtualBaseClassOffset(CodeGenFunction &CGF, llvm::Value *This,
>> + GetVirtualBaseClassOffset(CodeGenFunction &CGF, Address This,
>> const CXXRecordDecl *ClassDecl,
>> const CXXRecordDecl *BaseClassDecl) override;
>>
>> @@ -185,7 +238,7 @@ public:
>>
>> void EmitDestructorCall(CodeGenFunction &CGF, const CXXDestructorDecl *DD,
>> CXXDtorType Type, bool ForVirtualBase,
>> - bool Delegating, llvm::Value *This) override;
>> + bool Delegating, Address This) override;
>>
>> void emitVTableDefinitions(CodeGenVTables &CGVT,
>> const CXXRecordDecl *RD) override;
>> @@ -203,14 +256,13 @@ public:
>> CharUnits VPtrOffset) override;
>>
>> llvm::Value *getVirtualFunctionPointer(CodeGenFunction &CGF, GlobalDecl GD,
>> - llvm::Value *This,
>> - llvm::Type *Ty,
>> + Address This, llvm::Type *Ty,
>> SourceLocation Loc) override;
>>
>> llvm::Value *EmitVirtualDestructorCall(CodeGenFunction &CGF,
>> const CXXDestructorDecl *Dtor,
>> CXXDtorType DtorType,
>> - llvm::Value *This,
>> + Address This,
>> const CXXMemberCallExpr *CE) override;
>>
>> void emitVirtualInheritanceTables(const CXXRecordDecl *RD) override;
>> @@ -225,10 +277,10 @@ public:
>> Thunk->setLinkage(llvm::GlobalValue::AvailableExternallyLinkage);
>> }
>>
>> - llvm::Value *performThisAdjustment(CodeGenFunction &CGF, llvm::Value *This,
>> + llvm::Value *performThisAdjustment(CodeGenFunction &CGF, Address This,
>> const ThisAdjustment &TA) override;
>>
>> - llvm::Value *performReturnAdjustment(CodeGenFunction &CGF, llvm::Value *Ret,
>> + llvm::Value *performReturnAdjustment(CodeGenFunction &CGF, Address Ret,
>> const ReturnAdjustment &RA) override;
>>
>> size_t getSrcArgforCopyCtor(const CXXConstructorDecl *,
>> @@ -242,13 +294,13 @@ public:
>> { return "__cxa_deleted_virtual"; }
>>
>> CharUnits getArrayCookieSizeImpl(QualType elementType) override;
>> - llvm::Value *InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *NewPtr,
>> - llvm::Value *NumElements,
>> - const CXXNewExpr *expr,
>> - QualType ElementType) override;
>> + Address InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address NewPtr,
>> + llvm::Value *NumElements,
>> + const CXXNewExpr *expr,
>> + QualType ElementType) override;
>> llvm::Value *readArrayCookieImpl(CodeGenFunction &CGF,
>> - llvm::Value *allocPtr,
>> + Address allocPtr,
>> CharUnits cookieSize) override;
>>
>> void EmitGuardedInit(CodeGenFunction &CGF, const VarDecl &D,
>> @@ -338,12 +390,12 @@ public:
>> QualType ResTy) override;
>>
>> CharUnits getArrayCookieSizeImpl(QualType elementType) override;
>> - llvm::Value *InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *NewPtr,
>> - llvm::Value *NumElements,
>> - const CXXNewExpr *expr,
>> - QualType ElementType) override;
>> - llvm::Value *readArrayCookieImpl(CodeGenFunction &CGF, llvm::Value *allocPtr,
>> + Address InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address NewPtr,
>> + llvm::Value *NumElements,
>> + const CXXNewExpr *expr,
>> + QualType ElementType) override;
>> + llvm::Value *readArrayCookieImpl(CodeGenFunction &CGF, Address allocPtr,
>> CharUnits cookieSize) override;
>> };
>>
>> @@ -439,7 +491,8 @@ ItaniumCXXABI::ConvertMemberPointerType(
>> /// If the member is non-virtual, memptr.ptr is the address of
>> /// the function to call.
>> llvm::Value *ItaniumCXXABI::EmitLoadOfMemberFunctionPointer(
>> - CodeGenFunction &CGF, const Expr *E, llvm::Value *&This,
>> + CodeGenFunction &CGF, const Expr *E, Address ThisAddr,
>> + llvm::Value *&ThisPtrForCall,
>> llvm::Value *MemFnPtr, const MemberPointerType *MPT) {
>> CGBuilderTy &Builder = CGF.Builder;
>>
>> @@ -468,9 +521,11 @@ llvm::Value *ItaniumCXXABI::EmitLoadOfMe
>>
>> // Apply the adjustment and cast back to the original struct type
>> // for consistency.
>> + llvm::Value *This = ThisAddr.getPointer();
>> llvm::Value *Ptr = Builder.CreateBitCast(This, Builder.getInt8PtrTy());
>> Ptr = Builder.CreateInBoundsGEP(Ptr, Adj);
>> This = Builder.CreateBitCast(Ptr, This->getType(), "this.adjusted");
>> + ThisPtrForCall = This;
>>
>> // Load the function pointer.
>> llvm::Value *FnAsInt = Builder.CreateExtractValue(MemFnPtr, 0, "memptr.ptr");
>> @@ -492,7 +547,11 @@ llvm::Value *ItaniumCXXABI::EmitLoadOfMe
>>
>> // Cast the adjusted this to a pointer to vtable pointer and load.
>> llvm::Type *VTableTy = Builder.getInt8PtrTy();
>> - llvm::Value *VTable = CGF.GetVTablePtr(This, VTableTy);
>> + CharUnits VTablePtrAlign =
>> + CGF.CGM.getDynamicOffsetAlignment(ThisAddr.getAlignment(), RD,
>> + CGF.getPointerAlign());
>> + llvm::Value *VTable =
>> + CGF.GetVTablePtr(Address(This, VTablePtrAlign), VTableTy);
>>
>> // Apply the offset.
>> llvm::Value *VTableOffset = FnAsInt;
>> @@ -502,7 +561,9 @@ llvm::Value *ItaniumCXXABI::EmitLoadOfMe
>>
>> // Load the virtual function to call.
>> VTable = Builder.CreateBitCast(VTable, FTy->getPointerTo()->getPointerTo());
>> - llvm::Value *VirtualFn = Builder.CreateLoad(VTable, "memptr.virtualfn");
>> + llvm::Value *VirtualFn =
>> + Builder.CreateAlignedLoad(VTable, CGF.getPointerAlign(),
>> + "memptr.virtualfn");
>> CGF.EmitBranch(FnEnd);
>>
>> // In the non-virtual path, the function pointer is actually a
>> @@ -522,24 +583,23 @@ llvm::Value *ItaniumCXXABI::EmitLoadOfMe
>> /// Compute an l-value by applying the given pointer-to-member to a
>> /// base object.
>> llvm::Value *ItaniumCXXABI::EmitMemberDataPointerAddress(
>> - CodeGenFunction &CGF, const Expr *E, llvm::Value *Base, llvm::Value *MemPtr,
>> + CodeGenFunction &CGF, const Expr *E, Address Base, llvm::Value *MemPtr,
>> const MemberPointerType *MPT) {
>> assert(MemPtr->getType() == CGM.PtrDiffTy);
>>
>> CGBuilderTy &Builder = CGF.Builder;
>>
>> - unsigned AS = Base->getType()->getPointerAddressSpace();
>> -
>> // Cast to char*.
>> - Base = Builder.CreateBitCast(Base, Builder.getInt8Ty()->getPointerTo(AS));
>> + Base = Builder.CreateElementBitCast(Base, CGF.Int8Ty);
>>
>> // Apply the offset, which we assume is non-null.
>> - llvm::Value *Addr = Builder.CreateInBoundsGEP(Base, MemPtr, "memptr.offset");
>> + llvm::Value *Addr =
>> + Builder.CreateInBoundsGEP(Base.getPointer(), MemPtr, "memptr.offset");
>>
>> // Cast the address to the appropriate pointer type, adopting the
>> // address space of the base pointer.
>> - llvm::Type *PType
>> - = CGF.ConvertTypeForMem(MPT->getPointeeType())->getPointerTo(AS);
>> + llvm::Type *PType = CGF.ConvertTypeForMem(MPT->getPointeeType())
>> + ->getPointerTo(Base.getAddressSpace());
>> return Builder.CreateBitCast(Addr, PType);
>> }
>>
>> @@ -893,7 +953,8 @@ bool ItaniumCXXABI::classifyReturnType(C
>> // FIXME: Use canCopyArgument() when it is fixed to handle lazily declared
>> // special members.
>> if (RD->hasNonTrivialDestructor() || RD->hasNonTrivialCopyConstructor()) {
>> - FI.getReturnInfo() = ABIArgInfo::getIndirect(0, /*ByVal=*/false);
>> + auto Align = CGM.getContext().getTypeAlignInChars(FI.getReturnType());
>> + FI.getReturnInfo() = ABIArgInfo::getIndirect(Align, /*ByVal=*/false);
>> return true;
>> }
>> return false;
>> @@ -909,7 +970,7 @@ bool ItaniumCXXABI::isZeroInitializable(
>> /// at entry -2 in the vtable.
>> void ItaniumCXXABI::emitVirtualObjectDelete(CodeGenFunction &CGF,
>> const CXXDeleteExpr *DE,
>> - llvm::Value *Ptr,
>> + Address Ptr,
>> QualType ElementType,
>> const CXXDestructorDecl *Dtor) {
>> bool UseGlobalDelete = DE->isGlobalDelete();
>> @@ -923,11 +984,12 @@ void ItaniumCXXABI::emitVirtualObjectDel
>> // Track back to entry -2 and pull out the offset there.
>> llvm::Value *OffsetPtr = CGF.Builder.CreateConstInBoundsGEP1_64(
>> VTable, -2, "complete-offset.ptr");
>> - llvm::LoadInst *Offset = CGF.Builder.CreateLoad(OffsetPtr);
>> - Offset->setAlignment(CGF.PointerAlignInBytes);
>> + llvm::Value *Offset =
>> + CGF.Builder.CreateAlignedLoad(OffsetPtr, CGF.getPointerAlign());
>>
>> // Apply the offset.
>> - llvm::Value *CompletePtr = CGF.Builder.CreateBitCast(Ptr, CGF.Int8PtrTy);
>> + llvm::Value *CompletePtr =
>> + CGF.Builder.CreateBitCast(Ptr.getPointer(), CGF.Int8PtrTy);
>> CompletePtr = CGF.Builder.CreateInBoundsGEP(CompletePtr, Offset);
>>
>> // If we're supposed to call the global delete, make sure we do so
>> @@ -989,7 +1051,8 @@ void ItaniumCXXABI::emitThrow(CodeGenFun
>> llvm::CallInst *ExceptionPtr = CGF.EmitNounwindRuntimeCall(
>> AllocExceptionFn, llvm::ConstantInt::get(SizeTy, TypeSize), "exception");
>>
>> - CGF.EmitAnyExprToExn(E->getSubExpr(), ExceptionPtr);
>> + CharUnits ExnAlign = getAlignmentOfExnObject();
>> + CGF.EmitAnyExprToExn(E->getSubExpr(), Address(ExceptionPtr, ExnAlign));
>>
>> // Now throw the exception.
>> llvm::Constant *TypeInfo = CGM.GetAddrOfRTTIDescriptor(ThrowType,
>> @@ -1113,14 +1176,14 @@ void ItaniumCXXABI::EmitBadTypeidCall(Co
>>
>> llvm::Value *ItaniumCXXABI::EmitTypeid(CodeGenFunction &CGF,
>> QualType SrcRecordTy,
>> - llvm::Value *ThisPtr,
>> + Address ThisPtr,
>> llvm::Type *StdTypeInfoPtrTy) {
>> llvm::Value *Value =
>> CGF.GetVTablePtr(ThisPtr, StdTypeInfoPtrTy->getPointerTo());
>>
>> // Load the type info.
>> Value = CGF.Builder.CreateConstInBoundsGEP1_64(Value, -1ULL);
>> - return CGF.Builder.CreateLoad(Value);
>> + return CGF.Builder.CreateAlignedLoad(Value, CGF.getPointerAlign());
>> }
>>
>> bool ItaniumCXXABI::shouldDynamicCastCallBeNullChecked(bool SrcIsPtr,
>> @@ -1129,7 +1192,7 @@ bool ItaniumCXXABI::shouldDynamicCastCal
>> }
>>
>> llvm::Value *ItaniumCXXABI::EmitDynamicCastCall(
>> - CodeGenFunction &CGF, llvm::Value *Value, QualType SrcRecordTy,
>> + CodeGenFunction &CGF, Address ThisAddr, QualType SrcRecordTy,
>> QualType DestTy, QualType DestRecordTy, llvm::BasicBlock *CastEnd) {
>> llvm::Type *PtrDiffLTy =
>> CGF.ConvertType(CGF.getContext().getPointerDiffType());
>> @@ -1148,6 +1211,7 @@ llvm::Value *ItaniumCXXABI::EmitDynamicC
>> computeOffsetHint(CGF.getContext(), SrcDecl, DestDecl).getQuantity());
>>
>> // Emit the call to __dynamic_cast.
>> + llvm::Value *Value = ThisAddr.getPointer();
>> Value = CGF.EmitCastToVoidPtr(Value);
>>
>> llvm::Value *args[] = {Value, SrcRTTI, DestRTTI, OffsetHint};
>> @@ -1171,7 +1235,7 @@ llvm::Value *ItaniumCXXABI::EmitDynamicC
>> }
>>
>> llvm::Value *ItaniumCXXABI::EmitDynamicCastToVoid(CodeGenFunction &CGF,
>> - llvm::Value *Value,
>> + Address ThisAddr,
>> QualType SrcRecordTy,
>> QualType DestTy) {
>> llvm::Type *PtrDiffLTy =
>> @@ -1179,14 +1243,17 @@ llvm::Value *ItaniumCXXABI::EmitDynamicC
>> llvm::Type *DestLTy = CGF.ConvertType(DestTy);
>>
>> // Get the vtable pointer.
>> - llvm::Value *VTable = CGF.GetVTablePtr(Value, PtrDiffLTy->getPointerTo());
>> + llvm::Value *VTable = CGF.GetVTablePtr(ThisAddr, PtrDiffLTy->getPointerTo());
>>
>> // Get the offset-to-top from the vtable.
>> llvm::Value *OffsetToTop =
>> CGF.Builder.CreateConstInBoundsGEP1_64(VTable, -2ULL);
>> - OffsetToTop = CGF.Builder.CreateLoad(OffsetToTop, "offset.to.top");
>> + OffsetToTop =
>> + CGF.Builder.CreateAlignedLoad(OffsetToTop, CGF.getPointerAlign(),
>> + "offset.to.top");
>>
>> // Finally, add the offset to the pointer.
>> + llvm::Value *Value = ThisAddr.getPointer();
>> Value = CGF.EmitCastToVoidPtr(Value);
>> Value = CGF.Builder.CreateInBoundsGEP(Value, OffsetToTop);
>>
>> @@ -1202,7 +1269,7 @@ bool ItaniumCXXABI::EmitBadCastCall(Code
>>
>> llvm::Value *
>> ItaniumCXXABI::GetVirtualBaseClassOffset(CodeGenFunction &CGF,
>> - llvm::Value *This,
>> + Address This,
>> const CXXRecordDecl *ClassDecl,
>> const CXXRecordDecl *BaseClassDecl) {
>> llvm::Value *VTablePtr = CGF.GetVTablePtr(This, CGM.Int8PtrTy);
>> @@ -1217,7 +1284,8 @@ ItaniumCXXABI::GetVirtualBaseClassOffset
>> CGM.PtrDiffTy->getPointerTo());
>>
>> llvm::Value *VBaseOffset =
>> - CGF.Builder.CreateLoad(VBaseOffsetPtr, "vbase.offset");
>> + CGF.Builder.CreateAlignedLoad(VBaseOffsetPtr, CGF.getPointerAlign(),
>> + "vbase.offset");
>>
>> return VBaseOffset;
>> }
>> @@ -1328,7 +1396,7 @@ unsigned ItaniumCXXABI::addImplicitConst
>> void ItaniumCXXABI::EmitDestructorCall(CodeGenFunction &CGF,
>> const CXXDestructorDecl *DD,
>> CXXDtorType Type, bool ForVirtualBase,
>> - bool Delegating, llvm::Value *This) {
>> + bool Delegating, Address This) {
>> GlobalDecl GD(DD, Type);
>> llvm::Value *VTT = CGF.GetVTTParameter(GD, ForVirtualBase, Delegating);
>> QualType VTTTy = getContext().getPointerType(getContext().VoidPtrTy);
>> @@ -1340,8 +1408,8 @@ void ItaniumCXXABI::EmitDestructorCall(C
>> if (!Callee)
>> Callee = CGM.getAddrOfCXXStructor(DD, getFromDtorType(Type));
>>
>> - CGF.EmitCXXMemberOrOperatorCall(DD, Callee, ReturnValueSlot(), This, VTT,
>> - VTTTy, nullptr);
>> + CGF.EmitCXXMemberOrOperatorCall(DD, Callee, ReturnValueSlot(),
>> + This.getPointer(), VTT, VTTTy, nullptr);
>> }
>>
>> void ItaniumCXXABI::emitVTableDefinitions(CodeGenVTables &CGVT,
>> @@ -1409,7 +1477,7 @@ llvm::Value *ItaniumCXXABI::getVTableAdd
>> VTT = CGF.Builder.CreateConstInBoundsGEP1_64(VTT, VirtualPointerIndex);
>>
>> // And load the address point from the VTT.
>> - VTableAddressPoint = CGF.Builder.CreateLoad(VTT);
>> + VTableAddressPoint = CGF.Builder.CreateAlignedLoad(VTT, CGF.getPointerAlign());
>> } else {
>> llvm::Constant *VTable =
>> CGM.getCXXABI().getAddrOfVTable(VTableClass, CharUnits());
>> @@ -1473,7 +1541,7 @@ llvm::GlobalVariable *ItaniumCXXABI::get
>>
>> llvm::Value *ItaniumCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
>> GlobalDecl GD,
>> - llvm::Value *This,
>> + Address This,
>> llvm::Type *Ty,
>> SourceLocation Loc) {
>> GD = GD.getCanonicalDecl();
>> @@ -1487,12 +1555,12 @@ llvm::Value *ItaniumCXXABI::getVirtualFu
>> uint64_t VTableIndex = CGM.getItaniumVTableContext().getMethodVTableIndex(GD);
>> llvm::Value *VFuncPtr =
>> CGF.Builder.CreateConstInBoundsGEP1_64(VTable, VTableIndex, "vfn");
>> - return CGF.Builder.CreateLoad(VFuncPtr);
>> + return CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
>> }
>>
>> llvm::Value *ItaniumCXXABI::EmitVirtualDestructorCall(
>> CodeGenFunction &CGF, const CXXDestructorDecl *Dtor, CXXDtorType DtorType,
>> - llvm::Value *This, const CXXMemberCallExpr *CE) {
>> + Address This, const CXXMemberCallExpr *CE) {
>> assert(CE == nullptr || CE->arg_begin() == CE->arg_end());
>> assert(DtorType == Dtor_Deleting || DtorType == Dtor_Complete);
>>
>> @@ -1503,8 +1571,9 @@ llvm::Value *ItaniumCXXABI::EmitVirtualD
>> getVirtualFunctionPointer(CGF, GlobalDecl(Dtor, DtorType), This, Ty,
>> CE ? CE->getLocStart() : SourceLocation());
>>
>> - CGF.EmitCXXMemberOrOperatorCall(Dtor, Callee, ReturnValueSlot(), This,
>> - /*ImplicitParam=*/nullptr, QualType(), CE);
>> + CGF.EmitCXXMemberOrOperatorCall(Dtor, Callee, ReturnValueSlot(),
>> + This.getPointer(), /*ImplicitParam=*/nullptr,
>> + QualType(), CE);
>> return nullptr;
>> }
>>
>> @@ -1528,29 +1597,28 @@ bool ItaniumCXXABI::canEmitAvailableExte
>> return !hasAnyUsedVirtualInlineFunction(RD);
>> }
>> static llvm::Value *performTypeAdjustment(CodeGenFunction &CGF,
>> - llvm::Value *Ptr,
>> + Address InitialPtr,
>> int64_t NonVirtualAdjustment,
>> int64_t VirtualAdjustment,
>> bool IsReturnAdjustment) {
>> if (!NonVirtualAdjustment && !VirtualAdjustment)
>> - return Ptr;
>> + return InitialPtr.getPointer();
>>
>> - llvm::Type *Int8PtrTy = CGF.Int8PtrTy;
>> - llvm::Value *V = CGF.Builder.CreateBitCast(Ptr, Int8PtrTy);
>> + Address V = CGF.Builder.CreateElementBitCast(InitialPtr, CGF.Int8Ty);
>>
>> + // In a base-to-derived cast, the non-virtual adjustment is applied first.
>> if (NonVirtualAdjustment && !IsReturnAdjustment) {
>> - // Perform the non-virtual adjustment for a base-to-derived cast.
>> - V = CGF.Builder.CreateConstInBoundsGEP1_64(V, NonVirtualAdjustment);
>> + V = CGF.Builder.CreateConstInBoundsByteGEP(V,
>> + CharUnits::fromQuantity(NonVirtualAdjustment));
>> }
>>
>> + // Perform the virtual adjustment if we have one.
>> + llvm::Value *ResultPtr;
>> if (VirtualAdjustment) {
>> llvm::Type *PtrDiffTy =
>> CGF.ConvertType(CGF.getContext().getPointerDiffType());
>>
>> - // Perform the virtual adjustment.
>> - llvm::Value *VTablePtrPtr =
>> - CGF.Builder.CreateBitCast(V, Int8PtrTy->getPointerTo());
>> -
>> + Address VTablePtrPtr = CGF.Builder.CreateElementBitCast(V, CGF.Int8PtrTy);
>> llvm::Value *VTablePtr = CGF.Builder.CreateLoad(VTablePtrPtr);
>>
>> llvm::Value *OffsetPtr =
>> @@ -1559,23 +1627,28 @@ static llvm::Value *performTypeAdjustmen
>> OffsetPtr = CGF.Builder.CreateBitCast(OffsetPtr, PtrDiffTy->getPointerTo());
>>
>> // Load the adjustment offset from the vtable.
>> - llvm::Value *Offset = CGF.Builder.CreateLoad(OffsetPtr);
>> + llvm::Value *Offset =
>> + CGF.Builder.CreateAlignedLoad(OffsetPtr, CGF.getPointerAlign());
>>
>> // Adjust our pointer.
>> - V = CGF.Builder.CreateInBoundsGEP(V, Offset);
>> + ResultPtr = CGF.Builder.CreateInBoundsGEP(V.getPointer(), Offset);
>> + } else {
>> + ResultPtr = V.getPointer();
>> }
>>
>> + // In a derived-to-base conversion, the non-virtual adjustment is
>> + // applied second.
>> if (NonVirtualAdjustment && IsReturnAdjustment) {
>> - // Perform the non-virtual adjustment for a derived-to-base cast.
>> - V = CGF.Builder.CreateConstInBoundsGEP1_64(V, NonVirtualAdjustment);
>> + ResultPtr = CGF.Builder.CreateConstInBoundsGEP1_64(ResultPtr,
>> + NonVirtualAdjustment);
>> }
>>
>> // Cast back to the original type.
>> - return CGF.Builder.CreateBitCast(V, Ptr->getType());
>> + return CGF.Builder.CreateBitCast(ResultPtr, InitialPtr.getType());
>> }
>>
>> llvm::Value *ItaniumCXXABI::performThisAdjustment(CodeGenFunction &CGF,
>> - llvm::Value *This,
>> + Address This,
>> const ThisAdjustment &TA) {
>> return performTypeAdjustment(CGF, This, TA.NonVirtual,
>> TA.Virtual.Itanium.VCallOffsetOffset,
>> @@ -1583,7 +1656,7 @@ llvm::Value *ItaniumCXXABI::performThisA
>> }
>>
>> llvm::Value *
>> -ItaniumCXXABI::performReturnAdjustment(CodeGenFunction &CGF, llvm::Value *Ret,
>> +ItaniumCXXABI::performReturnAdjustment(CodeGenFunction &CGF, Address Ret,
>> const ReturnAdjustment &RA) {
>> return performTypeAdjustment(CGF, Ret, RA.NonVirtual,
>> RA.Virtual.Itanium.VBaseOffsetOffset,
>> @@ -1596,8 +1669,7 @@ void ARMCXXABI::EmitReturnFromThunk(Code
>> return ItaniumCXXABI::EmitReturnFromThunk(CGF, RV, ResultType);
>>
>> // Destructor thunks in the ARM ABI have indeterminate results.
>> - llvm::Type *T =
>> - cast<llvm::PointerType>(CGF.ReturnValue->getType())->getElementType();
>> + llvm::Type *T = CGF.ReturnValue.getElementType();
>> RValue Undef = RValue::get(llvm::UndefValue::get(T));
>> return ItaniumCXXABI::EmitReturnFromThunk(CGF, Undef, ResultType);
>> }
>> @@ -1611,18 +1683,17 @@ CharUnits ItaniumCXXABI::getArrayCookieS
>> CGM.getContext().getTypeAlignInChars(elementType));
>> }
>>
>> -llvm::Value *ItaniumCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *NewPtr,
>> - llvm::Value *NumElements,
>> - const CXXNewExpr *expr,
>> - QualType ElementType) {
>> +Address ItaniumCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address NewPtr,
>> + llvm::Value *NumElements,
>> + const CXXNewExpr *expr,
>> + QualType ElementType) {
>> assert(requiresArrayCookie(expr));
>>
>> - unsigned AS = NewPtr->getType()->getPointerAddressSpace();
>> + unsigned AS = NewPtr.getAddressSpace();
>>
>> ASTContext &Ctx = getContext();
>> - QualType SizeTy = Ctx.getSizeType();
>> - CharUnits SizeSize = Ctx.getTypeSizeInChars(SizeTy);
>> + CharUnits SizeSize = CGF.getSizeSize();
>>
>> // The size of the cookie.
>> CharUnits CookieSize =
>> @@ -1630,49 +1701,45 @@ llvm::Value *ItaniumCXXABI::InitializeAr
>> assert(CookieSize == getArrayCookieSizeImpl(ElementType));
>>
>> // Compute an offset to the cookie.
>> - llvm::Value *CookiePtr = NewPtr;
>> + Address CookiePtr = NewPtr;
>> CharUnits CookieOffset = CookieSize - SizeSize;
>> if (!CookieOffset.isZero())
>> - CookiePtr = CGF.Builder.CreateConstInBoundsGEP1_64(CookiePtr,
>> - CookieOffset.getQuantity());
>> + CookiePtr = CGF.Builder.CreateConstInBoundsByteGEP(CookiePtr, CookieOffset);
>>
>> // Write the number of elements into the appropriate slot.
>> - llvm::Type *NumElementsTy = CGF.ConvertType(SizeTy)->getPointerTo(AS);
>> - llvm::Value *NumElementsPtr =
>> - CGF.Builder.CreateBitCast(CookiePtr, NumElementsTy);
>> + Address NumElementsPtr =
>> + CGF.Builder.CreateElementBitCast(CookiePtr, CGF.SizeTy);
>> llvm::Instruction *SI = CGF.Builder.CreateStore(NumElements, NumElementsPtr);
>> +
>> + // Handle the array cookie specially in ASan.
>> if (CGM.getLangOpts().Sanitize.has(SanitizerKind::Address) && AS == 0 &&
>> expr->getOperatorNew()->isReplaceableGlobalAllocationFunction()) {
>> // The store to the CookiePtr does not need to be instrumented.
>> CGM.getSanitizerMetadata()->disableSanitizerForInstruction(SI);
>> llvm::FunctionType *FTy =
>> - llvm::FunctionType::get(CGM.VoidTy, NumElementsTy, false);
>> + llvm::FunctionType::get(CGM.VoidTy, NumElementsPtr.getType(), false);
>> llvm::Constant *F =
>> CGM.CreateRuntimeFunction(FTy, "__asan_poison_cxx_array_cookie");
>> - CGF.Builder.CreateCall(F, NumElementsPtr);
>> + CGF.Builder.CreateCall(F, NumElementsPtr.getPointer());
>> }
>>
>> // Finally, compute a pointer to the actual data buffer by skipping
>> // over the cookie completely.
>> - return CGF.Builder.CreateConstInBoundsGEP1_64(NewPtr,
>> - CookieSize.getQuantity());
>> + return CGF.Builder.CreateConstInBoundsByteGEP(NewPtr, CookieSize);
>> }
>>
>> llvm::Value *ItaniumCXXABI::readArrayCookieImpl(CodeGenFunction &CGF,
>> - llvm::Value *allocPtr,
>> + Address allocPtr,
>> CharUnits cookieSize) {
>> // The element size is right-justified in the cookie.
>> - llvm::Value *numElementsPtr = allocPtr;
>> - CharUnits numElementsOffset =
>> - cookieSize - CharUnits::fromQuantity(CGF.SizeSizeInBytes);
>> + Address numElementsPtr = allocPtr;
>> + CharUnits numElementsOffset = cookieSize - CGF.getSizeSize();
>> if (!numElementsOffset.isZero())
>> numElementsPtr =
>> - CGF.Builder.CreateConstInBoundsGEP1_64(numElementsPtr,
>> - numElementsOffset.getQuantity());
>> + CGF.Builder.CreateConstInBoundsByteGEP(numElementsPtr, numElementsOffset);
>>
>> - unsigned AS = allocPtr->getType()->getPointerAddressSpace();
>> - numElementsPtr =
>> - CGF.Builder.CreateBitCast(numElementsPtr, CGF.SizeTy->getPointerTo(AS));
>> + unsigned AS = allocPtr.getAddressSpace();
>> + numElementsPtr = CGF.Builder.CreateElementBitCast(numElementsPtr, CGF.SizeTy);
>> if (!CGM.getLangOpts().Sanitize.has(SanitizerKind::Address) || AS != 0)
>> return CGF.Builder.CreateLoad(numElementsPtr);
>> // In asan mode emit a function call instead of a regular load and let the
>> @@ -1684,7 +1751,7 @@ llvm::Value *ItaniumCXXABI::readArrayCoo
>> llvm::FunctionType::get(CGF.SizeTy, CGF.SizeTy->getPointerTo(0), false);
>> llvm::Constant *F =
>> CGM.CreateRuntimeFunction(FTy, "__asan_load_cxx_array_cookie");
>> - return CGF.Builder.CreateCall(F, numElementsPtr);
>> + return CGF.Builder.CreateCall(F, numElementsPtr.getPointer());
>> }
>>
>> CharUnits ARMCXXABI::getArrayCookieSizeImpl(QualType elementType) {
>> @@ -1700,47 +1767,41 @@ CharUnits ARMCXXABI::getArrayCookieSizeI
>> CGM.getContext().getTypeAlignInChars(elementType));
>> }
>>
>> -llvm::Value *ARMCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *newPtr,
>> - llvm::Value *numElements,
>> - const CXXNewExpr *expr,
>> - QualType elementType) {
>> +Address ARMCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address newPtr,
>> + llvm::Value *numElements,
>> + const CXXNewExpr *expr,
>> + QualType elementType) {
>> assert(requiresArrayCookie(expr));
>>
>> - // NewPtr is a char*, but we generalize to arbitrary addrspaces.
>> - unsigned AS = newPtr->getType()->getPointerAddressSpace();
>> -
>> // The cookie is always at the start of the buffer.
>> - llvm::Value *cookie = newPtr;
>> + Address cookie = newPtr;
>>
>> // The first element is the element size.
>> - cookie = CGF.Builder.CreateBitCast(cookie, CGF.SizeTy->getPointerTo(AS));
>> + cookie = CGF.Builder.CreateElementBitCast(cookie, CGF.SizeTy);
>> llvm::Value *elementSize = llvm::ConstantInt::get(CGF.SizeTy,
>> getContext().getTypeSizeInChars(elementType).getQuantity());
>> CGF.Builder.CreateStore(elementSize, cookie);
>>
>> // The second element is the element count.
>> - cookie = CGF.Builder.CreateConstInBoundsGEP1_32(CGF.SizeTy, cookie, 1);
>> + cookie = CGF.Builder.CreateConstInBoundsGEP(cookie, 1, CGF.getSizeSize());
>> CGF.Builder.CreateStore(numElements, cookie);
>>
>> // Finally, compute a pointer to the actual data buffer by skipping
>> // over the cookie completely.
>> CharUnits cookieSize = ARMCXXABI::getArrayCookieSizeImpl(elementType);
>> - return CGF.Builder.CreateConstInBoundsGEP1_64(newPtr,
>> - cookieSize.getQuantity());
>> + return CGF.Builder.CreateConstInBoundsByteGEP(newPtr, cookieSize);
>> }
>>
>> llvm::Value *ARMCXXABI::readArrayCookieImpl(CodeGenFunction &CGF,
>> - llvm::Value *allocPtr,
>> + Address allocPtr,
>> CharUnits cookieSize) {
>> // The number of elements is at offset sizeof(size_t) relative to
>> // the allocated pointer.
>> - llvm::Value *numElementsPtr
>> - = CGF.Builder.CreateConstInBoundsGEP1_64(allocPtr, CGF.SizeSizeInBytes);
>> + Address numElementsPtr
>> + = CGF.Builder.CreateConstInBoundsByteGEP(allocPtr, CGF.getSizeSize());
>>
>> - unsigned AS = allocPtr->getType()->getPointerAddressSpace();
>> - numElementsPtr =
>> - CGF.Builder.CreateBitCast(numElementsPtr, CGF.SizeTy->getPointerTo(AS));
>> + numElementsPtr = CGF.Builder.CreateElementBitCast(numElementsPtr, CGF.SizeTy);
>> return CGF.Builder.CreateLoad(numElementsPtr);
>> }
>>
>> @@ -1810,12 +1871,21 @@ void ItaniumCXXABI::EmitGuardedInit(Code
>> bool useInt8GuardVariable = !threadsafe && var->hasInternalLinkage();
>>
>> llvm::IntegerType *guardTy;
>> + CharUnits guardAlignment;
>> if (useInt8GuardVariable) {
>> guardTy = CGF.Int8Ty;
>> + guardAlignment = CharUnits::One();
>> } else {
>> // Guard variables are 64 bits in the generic ABI and size width on ARM
>> // (i.e. 32-bit on AArch32, 64-bit on AArch64).
>> - guardTy = (UseARMGuardVarABI ? CGF.SizeTy : CGF.Int64Ty);
>> + if (UseARMGuardVarABI) {
>> + guardTy = CGF.SizeTy;
>> + guardAlignment = CGF.getSizeAlign();
>> + } else {
>> + guardTy = CGF.Int64Ty;
>> + guardAlignment = CharUnits::fromQuantity(
>> + CGM.getDataLayout().getABITypeAlignment(guardTy));
>> + }
>> }
>> llvm::PointerType *guardPtrTy = guardTy->getPointerTo();
>>
>> @@ -1839,6 +1909,7 @@ void ItaniumCXXABI::EmitGuardedInit(Code
>> guard->setVisibility(var->getVisibility());
>> // If the variable is thread-local, so is its guard variable.
>> guard->setThreadLocalMode(var->getThreadLocalMode());
>> + guard->setAlignment(guardAlignment.getQuantity());
>>
>> // The ABI says: "It is suggested that it be emitted in the same COMDAT
>> // group as the associated data object." In practice, this doesn't work for
>> @@ -1855,6 +1926,8 @@ void ItaniumCXXABI::EmitGuardedInit(Code
>> CGM.setStaticLocalDeclGuardAddress(&D, guard);
>> }
>>
>> + Address guardAddr = Address(guard, guardAlignment);
>> +
>> // Test whether the variable has completed initialization.
>> //
>> // Itanium C++ ABI 3.3.2:
>> @@ -1874,8 +1947,7 @@ void ItaniumCXXABI::EmitGuardedInit(Code
>>
>> // Load the first byte of the guard variable.
>> llvm::LoadInst *LI =
>> - Builder.CreateLoad(Builder.CreateBitCast(guard, CGM.Int8PtrTy));
>> - LI->setAlignment(1);
>> + Builder.CreateLoad(Builder.CreateElementBitCast(guardAddr, CGM.Int8Ty));
>>
>> // Itanium ABI:
>> // An implementation supporting thread-safety on multiprocessor
>> @@ -1945,9 +2017,10 @@ void ItaniumCXXABI::EmitGuardedInit(Code
>> CGF.PopCleanupBlock();
>>
>> // Call __cxa_guard_release. This cannot throw.
>> - CGF.EmitNounwindRuntimeCall(getGuardReleaseFn(CGM, guardPtrTy), guard);
>> + CGF.EmitNounwindRuntimeCall(getGuardReleaseFn(CGM, guardPtrTy),
>> + guardAddr.getPointer());
>> } else {
>> - Builder.CreateStore(llvm::ConstantInt::get(guardTy, 1), guard);
>> + Builder.CreateStore(llvm::ConstantInt::get(guardTy, 1), guardAddr);
>> }
>>
>> CGF.EmitBlock(EndBlock);
>> @@ -2090,8 +2163,13 @@ void ItaniumCXXABI::EmitThreadLocalInitF
>> llvm::GlobalVariable::InternalLinkage,
>> llvm::ConstantInt::get(CGM.Int8Ty, 0), "__tls_guard");
>> Guard->setThreadLocal(true);
>> +
>> + CharUnits GuardAlign = CharUnits::One();
>> + Guard->setAlignment(GuardAlign.getQuantity());
>> +
>> CodeGenFunction(CGM)
>> - .GenerateCXXGlobalInitFunc(InitFunc, CXXThreadLocalInits, Guard);
>> + .GenerateCXXGlobalInitFunc(InitFunc, CXXThreadLocalInits,
>> + Address(Guard, GuardAlign));
>> }
>> for (auto &I : CXXThreadLocals) {
>> const VarDecl *VD = I.first;
>> @@ -2137,7 +2215,7 @@ void ItaniumCXXABI::EmitThreadLocalInitF
>> llvm::Function *Wrapper = getOrCreateThreadLocalWrapper(VD, Var);
>> llvm::LLVMContext &Context = CGM.getModule().getContext();
>> llvm::BasicBlock *Entry = llvm::BasicBlock::Create(Context, "", Wrapper);
>> - CGBuilderTy Builder(Entry);
>> + CGBuilderTy Builder(CGM, Entry);
>> if (InitIsInitFunc) {
>> if (Init)
>> Builder.CreateCall(Init);
>> @@ -2159,9 +2237,8 @@ void ItaniumCXXABI::EmitThreadLocalInitF
>> // the referenced object.
>> llvm::Value *Val = Var;
>> if (VD->getType()->isReferenceType()) {
>> - llvm::LoadInst *LI = Builder.CreateLoad(Val);
>> - LI->setAlignment(CGM.getContext().getDeclAlign(VD).getQuantity());
>> - Val = LI;
>> + CharUnits Align = CGM.getContext().getDeclAlign(VD);
>> + Val = Builder.CreateAlignedLoad(Val, Align);
>> }
>> if (Val->getType() != Wrapper->getReturnType())
>> Val = Builder.CreatePointerBitCastOrAddrSpaceCast(
>> @@ -3418,7 +3495,7 @@ static llvm::Value *CallBeginCatch(CodeG
>> /// parameter during catch initialization.
>> static void InitCatchParam(CodeGenFunction &CGF,
>> const VarDecl &CatchParam,
>> - llvm::Value *ParamAddr,
>> + Address ParamAddr,
>> SourceLocation Loc) {
>> // Load the exception from where the landing pad saved it.
>> llvm::Value *Exn = CGF.getExceptionFromSlot();
>> @@ -3472,12 +3549,13 @@ static void InitCatchParam(CodeGenFuncti
>> cast<llvm::PointerType>(LLVMCatchTy)->getElementType();
>>
>> // Create the temporary and write the adjusted pointer into it.
>> - llvm::Value *ExnPtrTmp = CGF.CreateTempAlloca(PtrTy, "exn.byref.tmp");
>> + Address ExnPtrTmp =
>> + CGF.CreateTempAlloca(PtrTy, CGF.getPointerAlign(), "exn.byref.tmp");
>> llvm::Value *Casted = CGF.Builder.CreateBitCast(AdjustedExn, PtrTy);
>> CGF.Builder.CreateStore(Casted, ExnPtrTmp);
>>
>> // Bind the reference to the temporary.
>> - AdjustedExn = ExnPtrTmp;
>> + AdjustedExn = ExnPtrTmp.getPointer();
>> }
>> }
>>
>> @@ -3522,8 +3600,7 @@ static void InitCatchParam(CodeGenFuncti
>> llvm::Value *Cast = CGF.Builder.CreateBitCast(AdjustedExn, PtrTy);
>>
>> LValue srcLV = CGF.MakeNaturalAlignAddrLValue(Cast, CatchType);
>> - LValue destLV = CGF.MakeAddrLValue(ParamAddr, CatchType,
>> - CGF.getContext().getDeclAlign(&CatchParam));
>> + LValue destLV = CGF.MakeAddrLValue(ParamAddr, CatchType);
>> switch (TEK) {
>> case TEK_Complex:
>> CGF.EmitStoreOfComplex(CGF.EmitLoadOfComplex(srcLV, Loc), destLV,
>> @@ -3541,6 +3618,8 @@ static void InitCatchParam(CodeGenFuncti
>> }
>>
>> assert(isa<RecordType>(CatchType) && "unexpected catch type!");
>> + auto catchRD = CatchType->getAsCXXRecordDecl();
>> + CharUnits caughtExnAlignment = CGF.CGM.getClassPointerAlignment(catchRD);
>>
>> llvm::Type *PtrTy = LLVMCatchTy->getPointerTo(0); // addrspace 0 ok
>>
>> @@ -3549,7 +3628,8 @@ static void InitCatchParam(CodeGenFuncti
>> const Expr *copyExpr = CatchParam.getInit();
>> if (!copyExpr) {
>> llvm::Value *rawAdjustedExn = CallBeginCatch(CGF, Exn, true);
>> - llvm::Value *adjustedExn = CGF.Builder.CreateBitCast(rawAdjustedExn, PtrTy);
>> + Address adjustedExn(CGF.Builder.CreateBitCast(rawAdjustedExn, PtrTy),
>> + caughtExnAlignment);
>> CGF.EmitAggregateCopy(ParamAddr, adjustedExn, CatchType);
>> return;
>> }
>> @@ -3560,7 +3640,8 @@ static void InitCatchParam(CodeGenFuncti
>> CGF.EmitNounwindRuntimeCall(getGetExceptionPtrFn(CGF.CGM), Exn);
>>
>> // Cast that to the appropriate type.
>> - llvm::Value *adjustedExn = CGF.Builder.CreateBitCast(rawAdjustedExn, PtrTy);
>> + Address adjustedExn(CGF.Builder.CreateBitCast(rawAdjustedExn, PtrTy),
>> + caughtExnAlignment);
>>
>> // The copy expression is defined in terms of an OpaqueValueExpr.
>> // Find it and map it to the adjusted expression.
>> @@ -3572,9 +3653,8 @@ static void InitCatchParam(CodeGenFuncti
>> CGF.EHStack.pushTerminate();
>>
>> // Perform the copy construction.
>> - CharUnits Alignment = CGF.getContext().getDeclAlign(&CatchParam);
>> CGF.EmitAggExpr(copyExpr,
>> - AggValueSlot::forAddr(ParamAddr, Alignment, Qualifiers(),
>> + AggValueSlot::forAddr(ParamAddr, Qualifiers(),
>> AggValueSlot::IsNotDestructed,
>> AggValueSlot::DoesNotNeedGCBarriers,
>> AggValueSlot::IsNotAliased));
>> @@ -3658,7 +3738,7 @@ static llvm::Constant *getClangCallTermi
>> // Set up the function.
>> llvm::BasicBlock *entry =
>> llvm::BasicBlock::Create(CGM.getLLVMContext(), "", fn);
>> - CGBuilderTy builder(entry);
>> + CGBuilderTy builder(CGM, entry);
>>
>> // Pull the exception pointer out of the parameter list.
>> llvm::Value *exn = &*fn->arg_begin();
>>
>> Modified: cfe/trunk/lib/CodeGen/MicrosoftCXXABI.cpp
>> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/MicrosoftCXXABI.cpp?rev=246985&r1=246984&r2=246985&view=diff
>> ==============================================================================
>> --- cfe/trunk/lib/CodeGen/MicrosoftCXXABI.cpp (original)
>> +++ cfe/trunk/lib/CodeGen/MicrosoftCXXABI.cpp Tue Sep 8 03:05:57 2015
>> @@ -56,6 +56,27 @@ public:
>>
>> bool isSRetParameterAfterThis() const override { return true; }
>>
>> + bool isThisCompleteObject(GlobalDecl GD) const override {
>> + // The Microsoft ABI doesn't use separate complete-object vs.
>> + // base-object variants of constructors, but it does of destructors.
>> + if (isa<CXXDestructorDecl>(GD.getDecl())) {
>> + switch (GD.getDtorType()) {
>> + case Dtor_Complete:
>> + case Dtor_Deleting:
>> + return true;
>> +
>> + case Dtor_Base:
>> + return false;
>> +
>> + case Dtor_Comdat: llvm_unreachable("emitting dtor comdat as function?");
>> + }
>> + llvm_unreachable("bad dtor kind");
>> + }
>> +
>> + // No other kinds.
>> + return false;
>> + }
>> +
>> size_t getSrcArgforCopyCtor(const CXXConstructorDecl *CD,
>> FunctionArgList &Args) const override {
>> assert(Args.size() >= 2 &&
>> @@ -72,7 +93,7 @@ public:
>> StringRef GetDeletedVirtualCallName() override { return "_purecall"; }
>>
>> void emitVirtualObjectDelete(CodeGenFunction &CGF, const CXXDeleteExpr *DE,
>> - llvm::Value *Ptr, QualType ElementType,
>> + Address Ptr, QualType ElementType,
>> const CXXDestructorDecl *Dtor) override;
>>
>> void emitRethrow(CodeGenFunction &CGF, bool isNoReturn) override;
>> @@ -90,18 +111,18 @@ public:
>> bool shouldTypeidBeNullChecked(bool IsDeref, QualType SrcRecordTy) override;
>> void EmitBadTypeidCall(CodeGenFunction &CGF) override;
>> llvm::Value *EmitTypeid(CodeGenFunction &CGF, QualType SrcRecordTy,
>> - llvm::Value *ThisPtr,
>> + Address ThisPtr,
>> llvm::Type *StdTypeInfoPtrTy) override;
>>
>> bool shouldDynamicCastCallBeNullChecked(bool SrcIsPtr,
>> QualType SrcRecordTy) override;
>>
>> - llvm::Value *EmitDynamicCastCall(CodeGenFunction &CGF, llvm::Value *Value,
>> + llvm::Value *EmitDynamicCastCall(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy, QualType DestTy,
>> QualType DestRecordTy,
>> llvm::BasicBlock *CastEnd) override;
>>
>> - llvm::Value *EmitDynamicCastToVoid(CodeGenFunction &CGF, llvm::Value *Value,
>> + llvm::Value *EmitDynamicCastToVoid(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy,
>> QualType DestTy) override;
>>
>> @@ -112,7 +133,7 @@ public:
>> }
>>
>> llvm::Value *
>> - GetVirtualBaseClassOffset(CodeGenFunction &CGF, llvm::Value *This,
>> + GetVirtualBaseClassOffset(CodeGenFunction &CGF, Address This,
>> const CXXRecordDecl *ClassDecl,
>> const CXXRecordDecl *BaseClassDecl) override;
>>
>> @@ -186,9 +207,9 @@ public:
>> return MD->getParent();
>> }
>>
>> - llvm::Value *
>> + Address
>> adjustThisArgumentForVirtualFunctionCall(CodeGenFunction &CGF, GlobalDecl GD,
>> - llvm::Value *This,
>> + Address This,
>> bool VirtualCall) override;
>>
>> void addImplicitStructorParams(CodeGenFunction &CGF, QualType &ResTy,
>> @@ -207,7 +228,7 @@ public:
>>
>> void EmitDestructorCall(CodeGenFunction &CGF, const CXXDestructorDecl *DD,
>> CXXDtorType Type, bool ForVirtualBase,
>> - bool Delegating, llvm::Value *This) override;
>> + bool Delegating, Address This) override;
>>
>> void emitVTableBitSetEntries(VPtrInfo *Info, const CXXRecordDecl *RD,
>> llvm::GlobalVariable *VTable);
>> @@ -228,13 +249,13 @@ public:
>> CharUnits VPtrOffset) override;
>>
>> llvm::Value *getVirtualFunctionPointer(CodeGenFunction &CGF, GlobalDecl GD,
>> - llvm::Value *This, llvm::Type *Ty,
>> + Address This, llvm::Type *Ty,
>> SourceLocation Loc) override;
>>
>> llvm::Value *EmitVirtualDestructorCall(CodeGenFunction &CGF,
>> const CXXDestructorDecl *Dtor,
>> CXXDtorType DtorType,
>> - llvm::Value *This,
>> + Address This,
>> const CXXMemberCallExpr *CE) override;
>>
>> void adjustCallArgsForDestructorThunk(CodeGenFunction &CGF, GlobalDecl GD,
>> @@ -313,10 +334,10 @@ public:
>> Thunk->setLinkage(llvm::GlobalValue::LinkOnceODRLinkage);
>> }
>>
>> - llvm::Value *performThisAdjustment(CodeGenFunction &CGF, llvm::Value *This,
>> + llvm::Value *performThisAdjustment(CodeGenFunction &CGF, Address This,
>> const ThisAdjustment &TA) override;
>>
>> - llvm::Value *performReturnAdjustment(CodeGenFunction &CGF, llvm::Value *Ret,
>> + llvm::Value *performReturnAdjustment(CodeGenFunction &CGF, Address Ret,
>> const ReturnAdjustment &RA) override;
>>
>> void EmitThreadLocalInitFuncs(
>> @@ -363,13 +384,13 @@ public:
>> QualType elementType) override;
>> bool requiresArrayCookie(const CXXNewExpr *expr) override;
>> CharUnits getArrayCookieSizeImpl(QualType type) override;
>> - llvm::Value *InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *NewPtr,
>> - llvm::Value *NumElements,
>> - const CXXNewExpr *expr,
>> - QualType ElementType) override;
>> + Address InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address NewPtr,
>> + llvm::Value *NumElements,
>> + const CXXNewExpr *expr,
>> + QualType ElementType) override;
>> llvm::Value *readArrayCookieImpl(CodeGenFunction &CGF,
>> - llvm::Value *allocPtr,
>> + Address allocPtr,
>> CharUnits cookieSize) override;
>>
>> friend struct MSRTTIBuilder;
>> @@ -514,13 +535,13 @@ private:
>> /// the vbptr to the virtual base. Optionally returns the address of the
>> /// vbptr itself.
>> llvm::Value *GetVBaseOffsetFromVBPtr(CodeGenFunction &CGF,
>> - llvm::Value *Base,
>> + Address Base,
>> llvm::Value *VBPtrOffset,
>> llvm::Value *VBTableOffset,
>> llvm::Value **VBPtr = nullptr);
>>
>> llvm::Value *GetVBaseOffsetFromVBPtr(CodeGenFunction &CGF,
>> - llvm::Value *Base,
>> + Address Base,
>> int32_t VBPtrOffset,
>> int32_t VBTableOffset,
>> llvm::Value **VBPtr = nullptr) {
>> @@ -530,14 +551,14 @@ private:
>> return GetVBaseOffsetFromVBPtr(CGF, Base, VBPOffset, VBTOffset, VBPtr);
>> }
>>
>> - std::pair<llvm::Value *, llvm::Value *>
>> - performBaseAdjustment(CodeGenFunction &CGF, llvm::Value *Value,
>> + std::pair<Address, llvm::Value *>
>> + performBaseAdjustment(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy);
>>
>> /// \brief Performs a full virtual base adjustment. Used to dereference
>> /// pointers to members of virtual bases.
>> llvm::Value *AdjustVirtualBase(CodeGenFunction &CGF, const Expr *E,
>> - const CXXRecordDecl *RD, llvm::Value *Base,
>> + const CXXRecordDecl *RD, Address Base,
>> llvm::Value *VirtualBaseAdjustmentOffset,
>> llvm::Value *VBPtrOffset /* optional */);
>>
>> @@ -603,7 +624,7 @@ public:
>>
>> llvm::Value *
>> EmitMemberDataPointerAddress(CodeGenFunction &CGF, const Expr *E,
>> - llvm::Value *Base, llvm::Value *MemPtr,
>> + Address Base, llvm::Value *MemPtr,
>> const MemberPointerType *MPT) override;
>>
>> llvm::Value *EmitNonNullMemberPointerConversion(
>> @@ -626,7 +647,8 @@ public:
>>
>> llvm::Value *
>> EmitLoadOfMemberFunctionPointer(CodeGenFunction &CGF, const Expr *E,
>> - llvm::Value *&This, llvm::Value *MemPtr,
>> + Address This, llvm::Value *&ThisPtrForCall,
>> + llvm::Value *MemPtr,
>> const MemberPointerType *MPT) override;
>>
>> void emitCXXStructor(const CXXMethodDecl *MD, StructorType Type) override;
>> @@ -826,7 +848,7 @@ MicrosoftCXXABI::getRecordArgABI(const C
>>
>> void MicrosoftCXXABI::emitVirtualObjectDelete(CodeGenFunction &CGF,
>> const CXXDeleteExpr *DE,
>> - llvm::Value *Ptr,
>> + Address Ptr,
>> QualType ElementType,
>> const CXXDestructorDecl *Dtor) {
>> // FIXME: Provide a source location here even though there's no
>> @@ -899,39 +921,52 @@ void MicrosoftCXXABI::emitBeginCatch(Cod
>>
>> CodeGenFunction::AutoVarEmission var = CGF.EmitAutoVarAlloca(*CatchParam);
>> if (!NewEH) {
>> - llvm::Value *ParamAddr =
>> - CGF.Builder.CreateBitCast(var.getObjectAddress(CGF), CGF.Int8PtrTy);
>> - llvm::Value *Args[2] = {Exn, ParamAddr};
>> + Address ParamAddr =
>> + CGF.Builder.CreateElementBitCast(var.getObjectAddress(CGF), CGF.Int8Ty);
>> + llvm::Value *Args[2] = {Exn, ParamAddr.getPointer()};
>> CGF.EmitNounwindRuntimeCall(BeginCatch, Args);
>> } else {
>> - CPI->setArgOperand(1, var.getObjectAddress(CGF));
>> + CPI->setArgOperand(1, var.getObjectAddress(CGF).getPointer());
>> }
>> CGF.EHStack.pushCleanup<CallEndCatchMSVC>(NormalCleanup, CPI);
>> CGF.EmitAutoVarCleanups(var);
>> }
>>
>> -std::pair<llvm::Value *, llvm::Value *>
>> -MicrosoftCXXABI::performBaseAdjustment(CodeGenFunction &CGF, llvm::Value *Value,
>> +/// We need to perform a generic polymorphic operation (like a typeid
>> +/// or a cast), which requires an object with a vfptr. Adjust the
>> +/// address to point to an object with a vfptr.
>> +std::pair<Address, llvm::Value *>
>> +MicrosoftCXXABI::performBaseAdjustment(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy) {
>> Value = CGF.Builder.CreateBitCast(Value, CGF.Int8PtrTy);
>> const CXXRecordDecl *SrcDecl = SrcRecordTy->getAsCXXRecordDecl();
>> const ASTContext &Context = getContext();
>>
>> + // If the class itself has a vfptr, great. This check implicitly
>> + // covers non-virtual base subobjects: a class with its own virtual
>> + // functions would be a candidate to be a primary base.
>> if (Context.getASTRecordLayout(SrcDecl).hasExtendableVFPtr())
>> return std::make_pair(Value, llvm::ConstantInt::get(CGF.Int32Ty, 0));
>>
>> - // Perform a base adjustment.
>> - const CXXBaseSpecifier *PolymorphicBase = std::find_if(
>> - SrcDecl->vbases_begin(), SrcDecl->vbases_end(),
>> - [&](const CXXBaseSpecifier &Base) {
>> - const CXXRecordDecl *BaseDecl = Base.getType()->getAsCXXRecordDecl();
>> - return Context.getASTRecordLayout(BaseDecl).hasExtendableVFPtr();
>> - });
>> - llvm::Value *Offset = GetVirtualBaseClassOffset(
>> - CGF, Value, SrcDecl, PolymorphicBase->getType()->getAsCXXRecordDecl());
>> - Value = CGF.Builder.CreateInBoundsGEP(Value, Offset);
>> + // Okay, one of the vbases must have a vfptr, or else this isn't
>> + // actually a polymorphic class.
>> + const CXXRecordDecl *PolymorphicBase = nullptr;
>> + for (auto &Base : SrcDecl->vbases()) {
>> + const CXXRecordDecl *BaseDecl = Base.getType()->getAsCXXRecordDecl();
>> + if (Context.getASTRecordLayout(BaseDecl).hasExtendableVFPtr()) {
>> + PolymorphicBase = BaseDecl;
>> + break;
>> + }
>> + }
>> + assert(PolymorphicBase && "polymorphic class has no apparent vfptr?");
>> +
>> + llvm::Value *Offset =
>> + GetVirtualBaseClassOffset(CGF, Value, SrcDecl, PolymorphicBase);
>> + llvm::Value *Ptr = CGF.Builder.CreateInBoundsGEP(Value.getPointer(), Offset);
>> Offset = CGF.Builder.CreateTrunc(Offset, CGF.Int32Ty);
>> - return std::make_pair(Value, Offset);
>> + CharUnits VBaseAlign =
>> + CGF.CGM.getVBaseAlignment(Value.getAlignment(), SrcDecl, PolymorphicBase);
>> + return std::make_pair(Address(Ptr, VBaseAlign), Offset);
>> }
>>
>> bool MicrosoftCXXABI::shouldTypeidBeNullChecked(bool IsDeref,
>> @@ -960,12 +995,12 @@ void MicrosoftCXXABI::EmitBadTypeidCall(
>>
>> llvm::Value *MicrosoftCXXABI::EmitTypeid(CodeGenFunction &CGF,
>> QualType SrcRecordTy,
>> - llvm::Value *ThisPtr,
>> + Address ThisPtr,
>> llvm::Type *StdTypeInfoPtrTy) {
>> llvm::Value *Offset;
>> std::tie(ThisPtr, Offset) = performBaseAdjustment(CGF, ThisPtr, SrcRecordTy);
>> - return CGF.Builder.CreateBitCast(
>> - emitRTtypeidCall(CGF, ThisPtr).getInstruction(), StdTypeInfoPtrTy);
>> + auto Typeid = emitRTtypeidCall(CGF, ThisPtr.getPointer()).getInstruction();
>> + return CGF.Builder.CreateBitCast(Typeid, StdTypeInfoPtrTy);
>> }
>>
>> bool MicrosoftCXXABI::shouldDynamicCastCallBeNullChecked(bool SrcIsPtr,
>> @@ -976,7 +1011,7 @@ bool MicrosoftCXXABI::shouldDynamicCastC
>> }
>>
>> llvm::Value *MicrosoftCXXABI::EmitDynamicCastCall(
>> - CodeGenFunction &CGF, llvm::Value *Value, QualType SrcRecordTy,
>> + CodeGenFunction &CGF, Address This, QualType SrcRecordTy,
>> QualType DestTy, QualType DestRecordTy, llvm::BasicBlock *CastEnd) {
>> llvm::Type *DestLTy = CGF.ConvertType(DestTy);
>>
>> @@ -986,7 +1021,8 @@ llvm::Value *MicrosoftCXXABI::EmitDynami
>> CGF.CGM.GetAddrOfRTTIDescriptor(DestRecordTy.getUnqualifiedType());
>>
>> llvm::Value *Offset;
>> - std::tie(Value, Offset) = performBaseAdjustment(CGF, Value, SrcRecordTy);
>> + std::tie(This, Offset) = performBaseAdjustment(CGF, This, SrcRecordTy);
>> + llvm::Value *ThisPtr = This.getPointer();
>>
>> // PVOID __RTDynamicCast(
>> // PVOID inptr,
>> @@ -1000,14 +1036,14 @@ llvm::Value *MicrosoftCXXABI::EmitDynami
>> llvm::FunctionType::get(CGF.Int8PtrTy, ArgTypes, false),
>> "__RTDynamicCast");
>> llvm::Value *Args[] = {
>> - Value, Offset, SrcRTTI, DestRTTI,
>> + ThisPtr, Offset, SrcRTTI, DestRTTI,
>> llvm::ConstantInt::get(CGF.Int32Ty, DestTy->isReferenceType())};
>> - Value = CGF.EmitRuntimeCallOrInvoke(Function, Args).getInstruction();
>> - return CGF.Builder.CreateBitCast(Value, DestLTy);
>> + ThisPtr = CGF.EmitRuntimeCallOrInvoke(Function, Args).getInstruction();
>> + return CGF.Builder.CreateBitCast(ThisPtr, DestLTy);
>> }
>>
>> llvm::Value *
>> -MicrosoftCXXABI::EmitDynamicCastToVoid(CodeGenFunction &CGF, llvm::Value *Value,
>> +MicrosoftCXXABI::EmitDynamicCastToVoid(CodeGenFunction &CGF, Address Value,
>> QualType SrcRecordTy,
>> QualType DestTy) {
>> llvm::Value *Offset;
>> @@ -1019,7 +1055,7 @@ MicrosoftCXXABI::EmitDynamicCastToVoid(C
>> llvm::Constant *Function = CGF.CGM.CreateRuntimeFunction(
>> llvm::FunctionType::get(CGF.Int8PtrTy, ArgTypes, false),
>> "__RTCastToVoid");
>> - llvm::Value *Args[] = {Value};
>> + llvm::Value *Args[] = {Value.getPointer()};
>> return CGF.EmitRuntimeCall(Function, Args);
>> }
>>
>> @@ -1028,7 +1064,7 @@ bool MicrosoftCXXABI::EmitBadCastCall(Co
>> }
>>
>> llvm::Value *MicrosoftCXXABI::GetVirtualBaseClassOffset(
>> - CodeGenFunction &CGF, llvm::Value *This, const CXXRecordDecl *ClassDecl,
>> + CodeGenFunction &CGF, Address This, const CXXRecordDecl *ClassDecl,
>> const CXXRecordDecl *BaseClassDecl) {
>> const ASTContext &Context = getContext();
>> int64_t VBPtrChars =
>> @@ -1066,15 +1102,16 @@ bool MicrosoftCXXABI::classifyReturnType
>> if (!RD)
>> return false;
>>
>> + CharUnits Align = CGM.getContext().getTypeAlignInChars(FI.getReturnType());
>> if (FI.isInstanceMethod()) {
>> // If it's an instance method, aggregates are always returned indirectly via
>> // the second parameter.
>> - FI.getReturnInfo() = ABIArgInfo::getIndirect(0, /*ByVal=*/false);
>> + FI.getReturnInfo() = ABIArgInfo::getIndirect(Align, /*ByVal=*/false);
>> FI.getReturnInfo().setSRetAfterThis(FI.isInstanceMethod());
>> return true;
>> } else if (!RD->isPOD()) {
>> // If it's a free function, non-POD types are returned indirectly.
>> - FI.getReturnInfo() = ABIArgInfo::getIndirect(0, /*ByVal=*/false);
>> + FI.getReturnInfo() = ABIArgInfo::getIndirect(Align, /*ByVal=*/false);
>> return true;
>> }
>>
>> @@ -1126,8 +1163,7 @@ void MicrosoftCXXABI::initializeHiddenVi
>> const VBOffsets &VBaseMap = Layout.getVBaseOffsetsMap();
>> CGBuilderTy &Builder = CGF.Builder;
>>
>> - unsigned AS =
>> - cast<llvm::PointerType>(getThisValue(CGF)->getType())->getAddressSpace();
>> + unsigned AS = getThisAddress(CGF).getAddressSpace();
>> llvm::Value *Int8This = nullptr; // Initialize lazily.
>>
>> for (VBOffsets::const_iterator I = VBaseMap.begin(), E = VBaseMap.end();
>> @@ -1136,7 +1172,7 @@ void MicrosoftCXXABI::initializeHiddenVi
>> continue;
>>
>> llvm::Value *VBaseOffset =
>> - GetVirtualBaseClassOffset(CGF, getThisValue(CGF), RD, I->first);
>> + GetVirtualBaseClassOffset(CGF, getThisAddress(CGF), RD, I->first);
>> // FIXME: it doesn't look right that we SExt in GetVirtualBaseClassOffset()
>> // just to Trunc back immediately.
>> VBaseOffset = Builder.CreateTruncOrBitCast(VBaseOffset, CGF.Int32Ty);
>> @@ -1157,7 +1193,8 @@ void MicrosoftCXXABI::initializeHiddenVi
>> VtorDispPtr = Builder.CreateBitCast(
>> VtorDispPtr, CGF.Int32Ty->getPointerTo(AS), "vtordisp.ptr");
>>
>> - Builder.CreateStore(VtorDispValue, VtorDispPtr);
>> + Builder.CreateAlignedStore(VtorDispValue, VtorDispPtr,
>> + CharUnits::fromQuantity(4));
>> }
>> }
>>
>> @@ -1188,8 +1225,8 @@ void MicrosoftCXXABI::EmitCXXConstructor
>>
>> void MicrosoftCXXABI::EmitVBPtrStores(CodeGenFunction &CGF,
>> const CXXRecordDecl *RD) {
>> - llvm::Value *ThisInt8Ptr =
>> - CGF.Builder.CreateBitCast(getThisValue(CGF), CGM.Int8PtrTy, "this.int8");
>> + Address This = getThisAddress(CGF);
>> + This = CGF.Builder.CreateElementBitCast(This, CGM.Int8Ty, "this.int8");
>> const ASTContext &Context = getContext();
>> const ASTRecordLayout &Layout = Context.getASTRecordLayout(RD);
>>
>> @@ -1203,11 +1240,10 @@ void MicrosoftCXXABI::EmitVBPtrStores(Co
>> Offs += SubobjectLayout.getVBPtrOffset();
>> if (VBT->getVBaseWithVPtr())
>> Offs += Layout.getVBaseClassOffset(VBT->getVBaseWithVPtr());
>> - llvm::Value *VBPtr =
>> - CGF.Builder.CreateConstInBoundsGEP1_64(ThisInt8Ptr, Offs.getQuantity());
>> + Address VBPtr = CGF.Builder.CreateConstInBoundsByteGEP(This, Offs);
>> llvm::Value *GVPtr =
>> CGF.Builder.CreateConstInBoundsGEP2_32(GV->getValueType(), GV, 0, 0);
>> - VBPtr = CGF.Builder.CreateBitCast(VBPtr, GVPtr->getType()->getPointerTo(0),
>> + VBPtr = CGF.Builder.CreateElementBitCast(VBPtr, GVPtr->getType(),
>> "vbptr." + VBT->ReusingBase->getName());
>> CGF.Builder.CreateStore(GVPtr, VBPtr);
>> }
>> @@ -1281,8 +1317,9 @@ MicrosoftCXXABI::getVirtualFunctionProlo
>> return Adjustment;
>> }
>>
>> -llvm::Value *MicrosoftCXXABI::adjustThisArgumentForVirtualFunctionCall(
>> - CodeGenFunction &CGF, GlobalDecl GD, llvm::Value *This, bool VirtualCall) {
>> +Address MicrosoftCXXABI::adjustThisArgumentForVirtualFunctionCall(
>> + CodeGenFunction &CGF, GlobalDecl GD, Address This,
>> + bool VirtualCall) {
>> if (!VirtualCall) {
>> // If the call of a virtual function is not virtual, we just have to
>> // compensate for the adjustment the virtual function does in its prologue.
>> @@ -1290,11 +1327,9 @@ llvm::Value *MicrosoftCXXABI::adjustThis
>> if (Adjustment.isZero())
>> return This;
>>
>> - unsigned AS = cast<llvm::PointerType>(This->getType())->getAddressSpace();
>> - llvm::Type *charPtrTy = CGF.Int8Ty->getPointerTo(AS);
>> - This = CGF.Builder.CreateBitCast(This, charPtrTy);
>> + This = CGF.Builder.CreateElementBitCast(This, CGF.Int8Ty);
>> assert(Adjustment.isPositive());
>> - return CGF.Builder.CreateConstGEP1_32(This, Adjustment.getQuantity());
>> + return CGF.Builder.CreateConstByteGEP(This, Adjustment);
>> }
>>
>> GD = GD.getCanonicalDecl();
>> @@ -1314,8 +1349,6 @@ llvm::Value *MicrosoftCXXABI::adjustThis
>> MicrosoftVTableContext::MethodVFTableLocation ML =
>> CGM.getMicrosoftVTableContext().getMethodVFTableLocation(LookupGD);
>>
>> - unsigned AS = cast<llvm::PointerType>(This->getType())->getAddressSpace();
>> - llvm::Type *charPtrTy = CGF.Int8Ty->getPointerTo(AS);
>> CharUnits StaticOffset = ML.VFPtrOffset;
>>
>> // Base destructors expect 'this' to point to the beginning of the base
>> @@ -1324,27 +1357,34 @@ llvm::Value *MicrosoftCXXABI::adjustThis
>> if (isa<CXXDestructorDecl>(MD) && GD.getDtorType() == Dtor_Base)
>> StaticOffset = CharUnits::Zero();
>>
>> + Address Result = This;
>> if (ML.VBase) {
>> - This = CGF.Builder.CreateBitCast(This, charPtrTy);
>> + Result = CGF.Builder.CreateElementBitCast(Result, CGF.Int8Ty);
>> +
>> + const CXXRecordDecl *Derived = MD->getParent();
>> + const CXXRecordDecl *VBase = ML.VBase;
>> llvm::Value *VBaseOffset =
>> - GetVirtualBaseClassOffset(CGF, This, MD->getParent(), ML.VBase);
>> - This = CGF.Builder.CreateInBoundsGEP(This, VBaseOffset);
>> + GetVirtualBaseClassOffset(CGF, Result, Derived, VBase);
>> + llvm::Value *VBasePtr =
>> + CGF.Builder.CreateInBoundsGEP(Result.getPointer(), VBaseOffset);
>> + CharUnits VBaseAlign =
>> + CGF.CGM.getVBaseAlignment(Result.getAlignment(), Derived, VBase);
>> + Result = Address(VBasePtr, VBaseAlign);
>> }
>> if (!StaticOffset.isZero()) {
>> assert(StaticOffset.isPositive());
>> - This = CGF.Builder.CreateBitCast(This, charPtrTy);
>> + Result = CGF.Builder.CreateElementBitCast(Result, CGF.Int8Ty);
>> if (ML.VBase) {
>> // Non-virtual adjustment might result in a pointer outside the allocated
>> // object, e.g. if the final overrider class is laid out after the virtual
>> // base that declares a method in the most derived class.
>> // FIXME: Update the code that emits this adjustment in thunks prologues.
>> - This = CGF.Builder.CreateConstGEP1_32(This, StaticOffset.getQuantity());
>> + Result = CGF.Builder.CreateConstByteGEP(Result, StaticOffset);
>> } else {
>> - This = CGF.Builder.CreateConstInBoundsGEP1_32(CGF.Int8Ty, This,
>> - StaticOffset.getQuantity());
>> + Result = CGF.Builder.CreateConstInBoundsByteGEP(Result, StaticOffset);
>> }
>> }
>> - return This;
>> + return Result;
>> }
>>
>> void MicrosoftCXXABI::addImplicitStructorParams(CodeGenFunction &CGF,
>> @@ -1465,7 +1505,7 @@ unsigned MicrosoftCXXABI::addImplicitCon
>> void MicrosoftCXXABI::EmitDestructorCall(CodeGenFunction &CGF,
>> const CXXDestructorDecl *DD,
>> CXXDtorType Type, bool ForVirtualBase,
>> - bool Delegating, llvm::Value *This) {
>> + bool Delegating, Address This) {
>> llvm::Value *Callee = CGM.getAddrOfCXXStructor(DD, getFromDtorType(Type));
>>
>> if (DD->isVirtual()) {
>> @@ -1475,7 +1515,7 @@ void MicrosoftCXXABI::EmitDestructorCall
>> This, false);
>> }
>>
>> - CGF.EmitCXXStructorCall(DD, Callee, ReturnValueSlot(), This,
>> + CGF.EmitCXXStructorCall(DD, Callee, ReturnValueSlot(), This.getPointer(),
>> /*ImplicitParam=*/nullptr,
>> /*ImplicitParamTy=*/QualType(), nullptr,
>> getFromDtorType(Type));
>> @@ -1772,14 +1812,14 @@ getClassAtVTableLocation(ASTContext &Ctx
>>
>> llvm::Value *MicrosoftCXXABI::getVirtualFunctionPointer(CodeGenFunction &CGF,
>> GlobalDecl GD,
>> - llvm::Value *This,
>> + Address This,
>> llvm::Type *Ty,
>> SourceLocation Loc) {
>> GD = GD.getCanonicalDecl();
>> CGBuilderTy &Builder = CGF.Builder;
>>
>> Ty = Ty->getPointerTo()->getPointerTo();
>> - llvm::Value *VPtr =
>> + Address VPtr =
>> adjustThisArgumentForVirtualFunctionCall(CGF, GD, This, true);
>> llvm::Value *VTable = CGF.GetVTablePtr(VPtr, Ty);
>>
>> @@ -1791,12 +1831,12 @@ llvm::Value *MicrosoftCXXABI::getVirtual
>>
>> llvm::Value *VFuncPtr =
>> Builder.CreateConstInBoundsGEP1_64(VTable, ML.Index, "vfn");
>> - return Builder.CreateLoad(VFuncPtr);
>> + return Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
>> }
>>
>> llvm::Value *MicrosoftCXXABI::EmitVirtualDestructorCall(
>> CodeGenFunction &CGF, const CXXDestructorDecl *Dtor, CXXDtorType DtorType,
>> - llvm::Value *This, const CXXMemberCallExpr *CE) {
>> + Address This, const CXXMemberCallExpr *CE) {
>> assert(CE == nullptr || CE->arg_begin() == CE->arg_end());
>> assert(DtorType == Dtor_Deleting || DtorType == Dtor_Complete);
>>
>> @@ -1815,7 +1855,8 @@ llvm::Value *MicrosoftCXXABI::EmitVirtua
>> DtorType == Dtor_Deleting);
>>
>> This = adjustThisArgumentForVirtualFunctionCall(CGF, GD, This, true);
>> - RValue RV = CGF.EmitCXXStructorCall(Dtor, Callee, ReturnValueSlot(), This,
>> + RValue RV = CGF.EmitCXXStructorCall(Dtor, Callee, ReturnValueSlot(),
>> + This.getPointer(),
>> ImplicitParam, Context.IntTy, CE,
>> StructorType::Deleting);
>> return RV.getScalarVal();
>> @@ -1907,10 +1948,11 @@ llvm::Function *MicrosoftCXXABI::EmitVir
>> // Load the vfptr and then callee from the vftable. The callee should have
>> // adjusted 'this' so that the vfptr is at offset zero.
>> llvm::Value *VTable = CGF.GetVTablePtr(
>> - getThisValue(CGF), ThunkTy->getPointerTo()->getPointerTo());
>> + getThisAddress(CGF), ThunkTy->getPointerTo()->getPointerTo());
>> llvm::Value *VFuncPtr =
>> CGF.Builder.CreateConstInBoundsGEP1_64(VTable, ML.Index, "vfn");
>> - llvm::Value *Callee = CGF.Builder.CreateLoad(VFuncPtr);
>> + llvm::Value *Callee =
>> + CGF.Builder.CreateAlignedLoad(VFuncPtr, CGF.getPointerAlign());
>>
>> CGF.EmitMustTailThunk(MD, getThisValue(CGF), Callee);
>>
>> @@ -2002,22 +2044,30 @@ void MicrosoftCXXABI::emitVBTableDefinit
>> }
>>
>> llvm::Value *MicrosoftCXXABI::performThisAdjustment(CodeGenFunction &CGF,
>> - llvm::Value *This,
>> + Address This,
>> const ThisAdjustment &TA) {
>> if (TA.isEmpty())
>> - return This;
>> + return This.getPointer();
>>
>> - llvm::Value *V = CGF.Builder.CreateBitCast(This, CGF.Int8PtrTy);
>> + This = CGF.Builder.CreateElementBitCast(This, CGF.Int8Ty);
>>
>> - if (!TA.Virtual.isEmpty()) {
>> + llvm::Value *V;
>> + if (TA.Virtual.isEmpty()) {
>> + V = This.getPointer();
>> + } else {
>> assert(TA.Virtual.Microsoft.VtordispOffset < 0);
>> // Adjust the this argument based on the vtordisp value.
>> - llvm::Value *VtorDispPtr =
>> - CGF.Builder.CreateConstGEP1_32(V, TA.Virtual.Microsoft.VtordispOffset);
>> - VtorDispPtr =
>> - CGF.Builder.CreateBitCast(VtorDispPtr, CGF.Int32Ty->getPointerTo());
>> + Address VtorDispPtr =
>> + CGF.Builder.CreateConstInBoundsByteGEP(This,
>> + CharUnits::fromQuantity(TA.Virtual.Microsoft.VtordispOffset));
>> + VtorDispPtr = CGF.Builder.CreateElementBitCast(VtorDispPtr, CGF.Int32Ty);
>> llvm::Value *VtorDisp = CGF.Builder.CreateLoad(VtorDispPtr, "vtordisp");
>> - V = CGF.Builder.CreateGEP(V, CGF.Builder.CreateNeg(VtorDisp));
>> + V = CGF.Builder.CreateGEP(This.getPointer(),
>> + CGF.Builder.CreateNeg(VtorDisp));
>> +
>> + // Unfortunately, having applied the vtordisp means that we no
>> + // longer really have a known alignment for the vbptr step.
>> + // We'll assume the vbptr is pointer-aligned.
>>
>> if (TA.Virtual.Microsoft.VBPtrOffset) {
>> // If the final overrider is defined in a virtual base other than the one
>> @@ -2027,7 +2077,8 @@ llvm::Value *MicrosoftCXXABI::performThi
>> assert(TA.Virtual.Microsoft.VBOffsetOffset >= 0);
>> llvm::Value *VBPtr;
>> llvm::Value *VBaseOffset =
>> - GetVBaseOffsetFromVBPtr(CGF, V, -TA.Virtual.Microsoft.VBPtrOffset,
>> + GetVBaseOffsetFromVBPtr(CGF, Address(V, CGF.getPointerAlign()),
>> + -TA.Virtual.Microsoft.VBPtrOffset,
>> TA.Virtual.Microsoft.VBOffsetOffset, &VBPtr);
>> V = CGF.Builder.CreateInBoundsGEP(VBPtr, VBaseOffset);
>> }
>> @@ -2045,20 +2096,21 @@ llvm::Value *MicrosoftCXXABI::performThi
>> }
>>
>> llvm::Value *
>> -MicrosoftCXXABI::performReturnAdjustment(CodeGenFunction &CGF, llvm::Value *Ret,
>> +MicrosoftCXXABI::performReturnAdjustment(CodeGenFunction &CGF, Address Ret,
>> const ReturnAdjustment &RA) {
>> if (RA.isEmpty())
>> - return Ret;
>> + return Ret.getPointer();
>>
>> - llvm::Value *V = CGF.Builder.CreateBitCast(Ret, CGF.Int8PtrTy);
>> + auto OrigTy = Ret.getType();
>> + Ret = CGF.Builder.CreateElementBitCast(Ret, CGF.Int8Ty);
>>
>> + llvm::Value *V = Ret.getPointer();
>> if (RA.Virtual.Microsoft.VBIndex) {
>> assert(RA.Virtual.Microsoft.VBIndex > 0);
>> - const ASTContext &Context = getContext();
>> - int32_t IntSize = Context.getTypeSizeInChars(Context.IntTy).getQuantity();
>> + int32_t IntSize = CGF.getIntSize().getQuantity();
>> llvm::Value *VBPtr;
>> llvm::Value *VBaseOffset =
>> - GetVBaseOffsetFromVBPtr(CGF, V, RA.Virtual.Microsoft.VBPtrOffset,
>> + GetVBaseOffsetFromVBPtr(CGF, Ret, RA.Virtual.Microsoft.VBPtrOffset,
>> IntSize * RA.Virtual.Microsoft.VBIndex, &VBPtr);
>> V = CGF.Builder.CreateInBoundsGEP(VBPtr, VBaseOffset);
>> }
>> @@ -2067,7 +2119,7 @@ MicrosoftCXXABI::performReturnAdjustment
>> V = CGF.Builder.CreateConstInBoundsGEP1_32(CGF.Int8Ty, V, RA.NonVirtual);
>>
>> // Cast back to the original type.
>> - return CGF.Builder.CreateBitCast(V, Ret->getType());
>> + return CGF.Builder.CreateBitCast(V, OrigTy);
>> }
>>
>> bool MicrosoftCXXABI::requiresArrayCookie(const CXXDeleteExpr *expr,
>> @@ -2092,37 +2144,34 @@ CharUnits MicrosoftCXXABI::getArrayCooki
>> }
>>
>> llvm::Value *MicrosoftCXXABI::readArrayCookieImpl(CodeGenFunction &CGF,
>> - llvm::Value *allocPtr,
>> + Address allocPtr,
>> CharUnits cookieSize) {
>> - unsigned AS = allocPtr->getType()->getPointerAddressSpace();
>> - llvm::Value *numElementsPtr =
>> - CGF.Builder.CreateBitCast(allocPtr, CGF.SizeTy->getPointerTo(AS));
>> + Address numElementsPtr =
>> + CGF.Builder.CreateElementBitCast(allocPtr, CGF.SizeTy);
>> return CGF.Builder.CreateLoad(numElementsPtr);
>> }
>>
>> -llvm::Value* MicrosoftCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> - llvm::Value *newPtr,
>> - llvm::Value *numElements,
>> - const CXXNewExpr *expr,
>> - QualType elementType) {
>> +Address MicrosoftCXXABI::InitializeArrayCookie(CodeGenFunction &CGF,
>> + Address newPtr,
>> + llvm::Value *numElements,
>> + const CXXNewExpr *expr,
>> + QualType elementType) {
>> assert(requiresArrayCookie(expr));
>>
>> // The size of the cookie.
>> CharUnits cookieSize = getArrayCookieSizeImpl(elementType);
>>
>> // Compute an offset to the cookie.
>> - llvm::Value *cookiePtr = newPtr;
>> + Address cookiePtr = newPtr;
>>
>> // Write the number of elements into the appropriate slot.
>> - unsigned AS = newPtr->getType()->getPointerAddressSpace();
>> - llvm::Value *numElementsPtr
>> - = CGF.Builder.CreateBitCast(cookiePtr, CGF.SizeTy->getPointerTo(AS));
>> + Address numElementsPtr
>> + = CGF.Builder.CreateElementBitCast(cookiePtr, CGF.SizeTy);
>> CGF.Builder.CreateStore(numElements, numElementsPtr);
>>
>> // Finally, compute a pointer to the actual data buffer by skipping
>> // over the cookie completely.
>> - return CGF.Builder.CreateConstInBoundsGEP1_64(newPtr,
>> - cookieSize.getQuantity());
>> + return CGF.Builder.CreateConstInBoundsByteGEP(newPtr, cookieSize);
>> }
>>
>> static void emitGlobalDtorWithTLRegDtor(CodeGenFunction &CGF, const VarDecl &VD,
>> @@ -2205,17 +2254,18 @@ LValue MicrosoftCXXABI::EmitThreadLocalV
>> return LValue();
>> }
>>
>> -static llvm::GlobalVariable *getInitThreadEpochPtr(CodeGenModule &CGM) {
>> +static ConstantAddress getInitThreadEpochPtr(CodeGenModule &CGM) {
>> StringRef VarName("_Init_thread_epoch");
>> + CharUnits Align = CGM.getIntAlign();
>> if (auto *GV = CGM.getModule().getNamedGlobal(VarName))
>> - return GV;
>> + return ConstantAddress(GV, Align);
>> auto *GV = new llvm::GlobalVariable(
>> CGM.getModule(), CGM.IntTy,
>> /*Constant=*/false, llvm::GlobalVariable::ExternalLinkage,
>> /*Initializer=*/nullptr, VarName,
>> /*InsertBefore=*/nullptr, llvm::GlobalVariable::GeneralDynamicTLSModel);
>> - GV->setAlignment(CGM.getTarget().getIntAlign() / 8);
>> - return GV;
>> + GV->setAlignment(Align.getQuantity());
>> + return ConstantAddress(GV, Align);
>> }
>>
>> static llvm::Constant *getInitThreadHeaderFn(CodeGenModule &CGM) {
>> @@ -2253,9 +2303,9 @@ static llvm::Constant *getInitThreadAbor
>>
>> namespace {
>> struct ResetGuardBit final : EHScopeStack::Cleanup {
>> - llvm::GlobalVariable *Guard;
>> + Address Guard;
>> unsigned GuardNum;
>> - ResetGuardBit(llvm::GlobalVariable *Guard, unsigned GuardNum)
>> + ResetGuardBit(Address Guard, unsigned GuardNum)
>> : Guard(Guard), GuardNum(GuardNum) {}
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> @@ -2270,8 +2320,8 @@ struct ResetGuardBit final : EHScopeStac
>> };
>>
>> struct CallInitThreadAbort final : EHScopeStack::Cleanup {
>> - llvm::GlobalVariable *Guard;
>> - CallInitThreadAbort(llvm::GlobalVariable *Guard) : Guard(Guard) {}
>> + llvm::Value *Guard;
>> + CallInitThreadAbort(Address Guard) : Guard(Guard.getPointer()) {}
>>
>> void Emit(CodeGenFunction &CGF, Flags flags) override {
>> // Calling _Init_thread_abort will reset the guard's state.
>> @@ -2304,6 +2354,7 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>> CGBuilderTy &Builder = CGF.Builder;
>> llvm::IntegerType *GuardTy = CGF.Int32Ty;
>> llvm::ConstantInt *Zero = llvm::ConstantInt::get(GuardTy, 0);
>> + CharUnits GuardAlign = CharUnits::fromQuantity(4);
>>
>> // Get the guard variable for this function if we have one already.
>> GuardInfo *GI = nullptr;
>> @@ -2353,6 +2404,7 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>> GV->getLinkage(), Zero, GuardName.str());
>> GuardVar->setVisibility(GV->getVisibility());
>> GuardVar->setDLLStorageClass(GV->getDLLStorageClass());
>> + GuardVar->setAlignment(GuardAlign.getQuantity());
>> if (GuardVar->isWeakForLinker())
>> GuardVar->setComdat(
>> CGM.getModule().getOrInsertComdat(GuardVar->getName()));
>> @@ -2362,6 +2414,8 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>> GI->Guard = GuardVar;
>> }
>>
>> + ConstantAddress GuardAddr(GuardVar, GuardAlign);
>> +
>> assert(GuardVar->getLinkage() == GV->getLinkage() &&
>> "static local from the same function had different linkage");
>>
>> @@ -2374,7 +2428,7 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>>
>> // Test our bit from the guard variable.
>> llvm::ConstantInt *Bit = llvm::ConstantInt::get(GuardTy, 1U << GuardNum);
>> - llvm::LoadInst *LI = Builder.CreateLoad(GuardVar);
>> + llvm::LoadInst *LI = Builder.CreateLoad(GuardAddr);
>> llvm::Value *IsInitialized =
>> Builder.CreateICmpNE(Builder.CreateAnd(LI, Bit), Zero);
>> llvm::BasicBlock *InitBlock = CGF.createBasicBlock("init");
>> @@ -2384,8 +2438,8 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>> // Set our bit in the guard variable and emit the initializer and add a global
>> // destructor if appropriate.
>> CGF.EmitBlock(InitBlock);
>> - Builder.CreateStore(Builder.CreateOr(LI, Bit), GuardVar);
>> - CGF.EHStack.pushCleanup<ResetGuardBit>(EHCleanup, GuardVar, GuardNum);
>> + Builder.CreateStore(Builder.CreateOr(LI, Bit), GuardAddr);
>> + CGF.EHStack.pushCleanup<ResetGuardBit>(EHCleanup, GuardAddr, GuardNum);
>> CGF.EmitCXXGlobalVarDeclInit(D, GV, PerformInit);
>> CGF.PopCleanupBlock();
>> Builder.CreateBr(EndBlock);
>> @@ -2405,11 +2459,8 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>> // The algorithm is almost identical to what can be found in the appendix
>> // found in N2325.
>>
>> - unsigned IntAlign = CGM.getTarget().getIntAlign() / 8;
>> -
>> // This BasicBLock determines whether or not we have any work to do.
>> - llvm::LoadInst *FirstGuardLoad =
>> - Builder.CreateAlignedLoad(GuardVar, IntAlign);
>> + llvm::LoadInst *FirstGuardLoad = Builder.CreateLoad(GuardAddr);
>> FirstGuardLoad->setOrdering(llvm::AtomicOrdering::Unordered);
>> llvm::LoadInst *InitThreadEpoch =
>> Builder.CreateLoad(getInitThreadEpochPtr(CGM));
>> @@ -2422,9 +2473,9 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>> // This BasicBlock attempts to determine whether or not this thread is
>> // responsible for doing the initialization.
>> CGF.EmitBlock(AttemptInitBlock);
>> - CGF.EmitNounwindRuntimeCall(getInitThreadHeaderFn(CGM), GuardVar);
>> - llvm::LoadInst *SecondGuardLoad =
>> - Builder.CreateAlignedLoad(GuardVar, IntAlign);
>> + CGF.EmitNounwindRuntimeCall(getInitThreadHeaderFn(CGM),
>> + GuardAddr.getPointer());
>> + llvm::LoadInst *SecondGuardLoad = Builder.CreateLoad(GuardAddr);
>> SecondGuardLoad->setOrdering(llvm::AtomicOrdering::Unordered);
>> llvm::Value *ShouldDoInit =
>> Builder.CreateICmpEQ(SecondGuardLoad, getAllOnesInt());
>> @@ -2433,10 +2484,11 @@ void MicrosoftCXXABI::EmitGuardedInit(Co
>>
>> // Ok, we ended up getting selected as the initializing thread.
>> CGF.EmitBlock(InitBlock);
>> - CGF.EHStack.pushCleanup<CallInitThreadAbort>(EHCleanup, GuardVar);
>> + CGF.EHStack.pushCleanup<CallInitThreadAbort>(EHCleanup, GuardAddr);
>> CGF.EmitCXXGlobalVarDeclInit(D, GV, PerformInit);
>> CGF.PopCleanupBlock();
>> - CGF.EmitNounwindRuntimeCall(getInitThreadFooterFn(CGM), GuardVar);
>> + CGF.EmitNounwindRuntimeCall(getInitThreadFooterFn(CGM),
>> + GuardAddr.getPointer());
>> Builder.CreateBr(EndBlock);
>>
>> CGF.EmitBlock(EndBlock);
>> @@ -2791,19 +2843,28 @@ bool MicrosoftCXXABI::MemberPointerConst
>>
>> llvm::Value *
>> MicrosoftCXXABI::GetVBaseOffsetFromVBPtr(CodeGenFunction &CGF,
>> - llvm::Value *This,
>> + Address This,
>> llvm::Value *VBPtrOffset,
>> llvm::Value *VBTableOffset,
>> llvm::Value **VBPtrOut) {
>> CGBuilderTy &Builder = CGF.Builder;
>> // Load the vbtable pointer from the vbptr in the instance.
>> - This = Builder.CreateBitCast(This, CGM.Int8PtrTy);
>> + This = Builder.CreateElementBitCast(This, CGM.Int8Ty);
>> llvm::Value *VBPtr =
>> - Builder.CreateInBoundsGEP(This, VBPtrOffset, "vbptr");
>> + Builder.CreateInBoundsGEP(This.getPointer(), VBPtrOffset, "vbptr");
>> if (VBPtrOut) *VBPtrOut = VBPtr;
>> VBPtr = Builder.CreateBitCast(VBPtr,
>> - CGM.Int32Ty->getPointerTo(0)->getPointerTo(0));
>> - llvm::Value *VBTable = Builder.CreateLoad(VBPtr, "vbtable");
>> + CGM.Int32Ty->getPointerTo(0)->getPointerTo(This.getAddressSpace()));
>> +
>> + CharUnits VBPtrAlign;
>> + if (auto CI = dyn_cast<llvm::ConstantInt>(VBPtrOffset)) {
>> + VBPtrAlign = This.getAlignment().alignmentAtOffset(
>> + CharUnits::fromQuantity(CI->getSExtValue()));
>> + } else {
>> + VBPtrAlign = CGF.getPointerAlign();
>> + }
>> +
>> + llvm::Value *VBTable = Builder.CreateAlignedLoad(VBPtr, VBPtrAlign, "vbtable");
>>
>> // Translate from byte offset to table index. It improves analyzability.
>> llvm::Value *VBTableIndex = Builder.CreateAShr(
>> @@ -2813,16 +2874,17 @@ MicrosoftCXXABI::GetVBaseOffsetFromVBPtr
>> // Load an i32 offset from the vb-table.
>> llvm::Value *VBaseOffs = Builder.CreateInBoundsGEP(VBTable, VBTableIndex);
>> VBaseOffs = Builder.CreateBitCast(VBaseOffs, CGM.Int32Ty->getPointerTo(0));
>> - return Builder.CreateLoad(VBaseOffs, "vbase_offs");
>> + return Builder.CreateAlignedLoad(VBaseOffs, CharUnits::fromQuantity(4),
>> + "vbase_offs");
>> }
>>
>> // Returns an adjusted base cast to i8*, since we do more address arithmetic on
>> // it.
>> llvm::Value *MicrosoftCXXABI::AdjustVirtualBase(
>> CodeGenFunction &CGF, const Expr *E, const CXXRecordDecl *RD,
>> - llvm::Value *Base, llvm::Value *VBTableOffset, llvm::Value *VBPtrOffset) {
>> + Address Base, llvm::Value *VBTableOffset, llvm::Value *VBPtrOffset) {
>> CGBuilderTy &Builder = CGF.Builder;
>> - Base = Builder.CreateBitCast(Base, CGM.Int8PtrTy);
>> + Base = Builder.CreateElementBitCast(Base, CGM.Int8Ty);
>> llvm::BasicBlock *OriginalBB = nullptr;
>> llvm::BasicBlock *SkipAdjustBB = nullptr;
>> llvm::BasicBlock *VBaseAdjustBB = nullptr;
>> @@ -2867,7 +2929,7 @@ llvm::Value *MicrosoftCXXABI::AdjustVirt
>> Builder.CreateBr(SkipAdjustBB);
>> CGF.EmitBlock(SkipAdjustBB);
>> llvm::PHINode *Phi = Builder.CreatePHI(CGM.Int8PtrTy, 2, "memptr.base");
>> - Phi->addIncoming(Base, OriginalBB);
>> + Phi->addIncoming(Base.getPointer(), OriginalBB);
>> Phi->addIncoming(AdjustedBase, VBaseAdjustBB);
>> return Phi;
>> }
>> @@ -2875,10 +2937,10 @@ llvm::Value *MicrosoftCXXABI::AdjustVirt
>> }
>>
>> llvm::Value *MicrosoftCXXABI::EmitMemberDataPointerAddress(
>> - CodeGenFunction &CGF, const Expr *E, llvm::Value *Base, llvm::Value *MemPtr,
>> + CodeGenFunction &CGF, const Expr *E, Address Base, llvm::Value *MemPtr,
>> const MemberPointerType *MPT) {
>> assert(MPT->isMemberDataPointer());
>> - unsigned AS = Base->getType()->getPointerAddressSpace();
>> + unsigned AS = Base.getAddressSpace();
>> llvm::Type *PType =
>> CGF.ConvertTypeForMem(MPT->getPointeeType())->getPointerTo(AS);
>> CGBuilderTy &Builder = CGF.Builder;
>> @@ -2900,17 +2962,19 @@ llvm::Value *MicrosoftCXXABI::EmitMember
>> VirtualBaseAdjustmentOffset = Builder.CreateExtractValue(MemPtr, I++);
>> }
>>
>> + llvm::Value *Addr;
>> if (VirtualBaseAdjustmentOffset) {
>> - Base = AdjustVirtualBase(CGF, E, RD, Base, VirtualBaseAdjustmentOffset,
>> + Addr = AdjustVirtualBase(CGF, E, RD, Base, VirtualBaseAdjustmentOffset,
>> VBPtrOffset);
>> + } else {
>> + Addr = Base.getPointer();
>> }
>>
>> // Cast to char*.
>> - Base = Builder.CreateBitCast(Base, Builder.getInt8Ty()->getPointerTo(AS));
>> + Addr = Builder.CreateBitCast(Addr, CGF.Int8Ty->getPointerTo(AS));
>>
>> // Apply the offset, which we assume is non-null.
>> - llvm::Value *Addr =
>> - Builder.CreateInBoundsGEP(Base, FieldOffset, "memptr.offset");
>> + Addr = Builder.CreateInBoundsGEP(Addr, FieldOffset, "memptr.offset");
>>
>> // Cast the address to the appropriate pointer type, adopting the address
>> // space of the base pointer.
>> @@ -3073,7 +3137,8 @@ llvm::Value *MicrosoftCXXABI::EmitNonNul
>> } else {
>> llvm::Value *Idxs[] = {getZeroInt(), VBIndex};
>> VirtualBaseAdjustmentOffset =
>> - Builder.CreateLoad(Builder.CreateInBoundsGEP(VDispMap, Idxs));
>> + Builder.CreateAlignedLoad(Builder.CreateInBoundsGEP(VDispMap, Idxs),
>> + CharUnits::fromQuantity(4));
>> }
>>
>> DstVBIndexEqZero =
>> @@ -3154,7 +3219,7 @@ llvm::Constant *MicrosoftCXXABI::EmitMem
>> if (CK == CK_ReinterpretMemberPointer)
>> return Src;
>>
>> - CGBuilderTy Builder(CGM.getLLVMContext());
>> + CGBuilderTy Builder(CGM, CGM.getLLVMContext());
>> auto *Dst = cast<llvm::Constant>(EmitNonNullMemberPointerConversion(
>> SrcTy, DstTy, CK, PathBegin, PathEnd, Src, Builder));
>>
>> @@ -3162,8 +3227,9 @@ llvm::Constant *MicrosoftCXXABI::EmitMem
>> }
>>
>> llvm::Value *MicrosoftCXXABI::EmitLoadOfMemberFunctionPointer(
>> - CodeGenFunction &CGF, const Expr *E, llvm::Value *&This,
>> - llvm::Value *MemPtr, const MemberPointerType *MPT) {
>> + CodeGenFunction &CGF, const Expr *E, Address This,
>> + llvm::Value *&ThisPtrForCall, llvm::Value *MemPtr,
>> + const MemberPointerType *MPT) {
>> assert(MPT->isMemberFunctionPointer());
>> const FunctionProtoType *FPT =
>> MPT->getPointeeType()->castAs<FunctionProtoType>();
>> @@ -3194,15 +3260,18 @@ llvm::Value *MicrosoftCXXABI::EmitLoadOf
>> }
>>
>> if (VirtualBaseAdjustmentOffset) {
>> - This = AdjustVirtualBase(CGF, E, RD, This, VirtualBaseAdjustmentOffset,
>> - VBPtrOffset);
>> + ThisPtrForCall = AdjustVirtualBase(CGF, E, RD, This,
>> + VirtualBaseAdjustmentOffset, VBPtrOffset);
>> + } else {
>> + ThisPtrForCall = This.getPointer();
>> }
>>
>> if (NonVirtualBaseAdjustment) {
>> // Apply the adjustment and cast back to the original struct type.
>> - llvm::Value *Ptr = Builder.CreateBitCast(This, Builder.getInt8PtrTy());
>> + llvm::Value *Ptr = Builder.CreateBitCast(ThisPtrForCall, CGF.Int8PtrTy);
>> Ptr = Builder.CreateInBoundsGEP(Ptr, NonVirtualBaseAdjustment);
>> - This = Builder.CreateBitCast(Ptr, This->getType(), "this.adjusted");
>> + ThisPtrForCall = Builder.CreateBitCast(Ptr, ThisPtrForCall->getType(),
>> + "this.adjusted");
>> }
>>
>> return Builder.CreateBitCast(FunctionPointer, FTy->getPointerTo());
>> @@ -4122,7 +4191,7 @@ void MicrosoftCXXABI::emitThrow(CodeGenF
>> QualType ThrowType = SubExpr->getType();
>> // The exception object lives on the stack and it's address is passed to the
>> // runtime function.
>> - llvm::AllocaInst *AI = CGF.CreateMemTemp(ThrowType);
>> + Address AI = CGF.CreateMemTemp(ThrowType);
>> CGF.EmitAnyExprToMem(SubExpr, AI, ThrowType.getQualifiers(),
>> /*IsInit=*/true);
>>
>> @@ -4131,6 +4200,9 @@ void MicrosoftCXXABI::emitThrow(CodeGenF
>> llvm::GlobalVariable *TI = getThrowInfo(ThrowType);
>>
>> // Call into the runtime to throw the exception.
>> - llvm::Value *Args[] = {CGF.Builder.CreateBitCast(AI, CGM.Int8PtrTy), TI};
>> + llvm::Value *Args[] = {
>> + CGF.Builder.CreateBitCast(AI.getPointer(), CGM.Int8PtrTy),
>> + TI
>> + };
>> CGF.EmitNoreturnRuntimeCallOrInvoke(getThrowFn(), Args);
>> }
>>
>>
>> _______________________________________________
>> cfe-commits mailing list
>> cfe-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>
More information about the cfe-commits
mailing list