r349442 - Automatic variable initialization

JF Bastien via cfe-commits cfe-commits at lists.llvm.org
Mon Dec 17 21:12:22 PST 2018


Author: jfb
Date: Mon Dec 17 21:12:21 2018
New Revision: 349442

URL: http://llvm.org/viewvc/llvm-project?rev=349442&view=rev
Log:
Automatic variable initialization

Summary:
Add an option to initialize automatic variables with either a pattern or with
zeroes. The default is still that automatic variables are uninitialized. Also
add attributes to request uninitialized on a per-variable basis, mainly to disable
initialization of large stack arrays when deemed too expensive.

This isn't meant to change the semantics of C and C++. Rather, it's meant to be
a last-resort when programmers inadvertently have some undefined behavior in
their code. This patch aims to make undefined behavior hurt less, which
security-minded people will be very happy about. Notably, this means that
there's no inadvertent information leak when:

  - The compiler re-uses stack slots, and a value is used uninitialized.
  - The compiler re-uses a register, and a value is used uninitialized.
  - Stack structs / arrays / unions with padding are copied.

This patch only addresses stack and register information leaks. There's many
more infoleaks that we could address, and much more undefined behavior that
could be tamed. Let's keep this patch focused, and I'm happy to address related
issues elsewhere.

To keep the patch simple, only some `undef` is removed for now, see
`replaceUndef`. The padding-related infoleaks are therefore not all gone yet.
This will be addressed in a follow-up, mainly because addressing padding-related
leaks should be a stand-alone option which is implied by variable
initialization.

There are three options when it comes to automatic variable initialization:

  0. Uninitialized

    This is C and C++'s default. It's not changing. Depending on code
    generation, a programmer who runs into undefined behavior by using an
    uninialized automatic variable may observe any previous value (including
    program secrets), or any value which the compiler saw fit to materialize on
    the stack or in a register (this could be to synthesize an immediate, to
    refer to code or data locations, to generate cookies, etc).

  1. Pattern initialization

    This is the recommended initialization approach. Pattern initialization's
    goal is to initialize automatic variables with values which will likely
    transform logic bugs into crashes down the line, are easily recognizable in
    a crash dump, without being values which programmers can rely on for useful
    program semantics. At the same time, pattern initialization tries to
    generate code which will optimize well. You'll find the following details in
    `patternFor`:

    - Integers are initialized with repeated 0xAA bytes (infinite scream).
    - Vectors of integers are also initialized with infinite scream.
    - Pointers are initialized with infinite scream on 64-bit platforms because
      it's an unmappable pointer value on architectures I'm aware of. Pointers
      are initialize to 0x000000AA (small scream) on 32-bit platforms because
      32-bit platforms don't consistently offer unmappable pages. When they do
      it's usually the zero page. As people try this out, I expect that we'll
      want to allow different platforms to customize this, let's do so later.
    - Vectors of pointers are initialized the same way pointers are.
    - Floating point values and vectors are initialized with a negative quiet
      NaN with repeated 0xFF payload (e.g. 0xffffffff and 0xffffffffffffffff).
      NaNs are nice (here, anways) because they propagate on arithmetic, making
      it more likely that entire computations become NaN when a single
      uninitialized value sneaks in.
    - Arrays are initialized to their homogeneous elements' initialization
      value, repeated. Stack-based Variable-Length Arrays (VLAs) are
      runtime-initialized to the allocated size (no effort is made for negative
      size, but zero-sized VLAs are untouched even if technically undefined).
    - Structs are initialized to their heterogeneous element's initialization
      values. Zero-size structs are initialized as 0xAA since they're allocated
      a single byte.
    - Unions are initialized using the initialization for the largest member of
      the union.

    Expect the values used for pattern initialization to change over time, as we
    refine heuristics (both for performance and security). The goal is truly to
    avoid injecting semantics into undefined behavior, and we should be
    comfortable changing these values when there's a worthwhile point in doing
    so.

    Why so much infinite scream? Repeated byte patterns tend to be easy to
    synthesize on most architectures, and otherwise memset is usually very
    efficient. For values which aren't entirely repeated byte patterns, LLVM
    will often generate code which does memset + a few stores.

  2. Zero initialization

    Zero initialize all values. This has the unfortunate side-effect of
    providing semantics to otherwise undefined behavior, programs therefore
    might start to rely on this behavior, and that's sad. However, some
    programmers believe that pattern initialization is too expensive for them,
    and data might show that they're right. The only way to make these
    programmers wrong is to offer zero-initialization as an option, figure out
    where they are right, and optimize the compiler into submission. Until the
    compiler provides acceptable performance for all security-minded code, zero
    initialization is a useful (if blunt) tool.

I've been asked for a fourth initialization option: user-provided byte value.
This might be useful, and can easily be added later.

Why is an out-of band initialization mecanism desired? We could instead use
-Wuninitialized! Indeed we could, but then we're forcing the programmer to
provide semantics for something which doesn't actually have any (it's
uninitialized!). It's then unclear whether `int derp = 0;` lends meaning to `0`,
or whether it's just there to shut that warning up. It's also way easier to use
a compiler flag than it is to manually and intelligently initialize all values
in a program.

Why not just rely on static analysis? Because it cannot reason about all dynamic
code paths effectively, and it has false positives. It's a great tool, could get
even better, but it's simply incapable of catching all uses of uninitialized
values.

Why not just rely on memory sanitizer? Because it's not universally available,
has a 3x performance cost, and shouldn't be deployed in production. Again, it's
a great tool, it'll find the dynamic uses of uninitialized variables that your
test coverage hits, but it won't find the ones that you encounter in production.

What's the performance like? Not too bad! Previous publications [0] have cited
2.7 to 4.5% averages. We've commmitted a few patches over the last few months to
address specific regressions, both in code size and performance. In all cases,
the optimizations are generally useful, but variable initialization benefits
from them a lot more than regular code does. We've got a handful of other
optimizations in mind, but the code is in good enough shape and has found enough
latent issues that it's a good time to get the change reviewed, checked in, and
have others kick the tires. We'll continue reducing overheads as we try this out
on diverse codebases.

Is it a good idea? Security-minded folks think so, and apparently so does the
Microsoft Visual Studio team [1] who say "Between 2017 and mid 2018, this
feature would have killed 49 MSRC cases that involved uninitialized struct data
leaking across a trust boundary. It would have also mitigated a number of bugs
involving uninitialized struct data being used directly.". They seem to use pure
zero initialization, and claim to have taken the overheads down to within noise.
Don't just trust Microsoft though, here's another relevant person asking for
this [2]. It's been proposed for GCC [3] and LLVM [4] before.

What are the caveats? A few!

  - Variables declared in unreachable code, and used later, aren't initialized.
    This goto, Duff's device, other objectionable uses of switch. This should
    instead be a hard-error in any serious codebase.
  - Volatile stack variables are still weird. That's pre-existing, it's really
    the language's fault and this patch keeps it weird. We should deprecate
    volatile [5].
  - As noted above, padding isn't fully handled yet.

I don't think these caveats make the patch untenable because they can be
addressed separately.

Should this be on by default? Maybe, in some circumstances. It's a conversation
we can have when we've tried it out sufficiently, and we're confident that we've
eliminated enough of the overheads that most codebases would want to opt-in.
Let's keep our precious undefined behavior until that point in time.

How do I use it:

  1. On the command-line:

    -ftrivial-auto-var-init=uninitialized (the default)
    -ftrivial-auto-var-init=pattern
    -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang

  2. Using an attribute:

    int dont_initialize_me __attribute((uninitialized));

  [0]: https://users.elis.ugent.be/~jsartor/researchDocs/OOPSLA2011Zero-submit.pdf
  [1]: https://twitter.com/JosephBialek/status/1062774315098112001
  [2]: https://outflux.net/slides/2018/lss/danger.pdf
  [3]: https://gcc.gnu.org/ml/gcc-patches/2014-06/msg00615.html
  [4]: https://github.com/AndroidHardeningArchive/platform_external_clang/commit/776a0955ef6686d23a82d2e6a3cbd4a6a882c31c
  [5]: http://wg21.link/p1152

I've also posted an RFC to cfe-dev: http://lists.llvm.org/pipermail/cfe-dev/2018-November/060172.html

<rdar://problem/39131435>

Reviewers: pcc, kcc, rsmith

Subscribers: JDevlieghere, jkorous, dexonsmith, cfe-commits

Differential Revision: https://reviews.llvm.org/D54604

Added:
    cfe/trunk/test/CodeGenCXX/trivial-auto-var-init-attribute.cpp
    cfe/trunk/test/CodeGenCXX/trivial-auto-var-init.cpp
    cfe/trunk/test/Sema/attr-uninitialized.c
Modified:
    cfe/trunk/include/clang/Basic/Attr.td
    cfe/trunk/include/clang/Basic/AttrDocs.td
    cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td
    cfe/trunk/include/clang/Basic/LangOptions.def
    cfe/trunk/include/clang/Basic/LangOptions.h
    cfe/trunk/include/clang/Driver/Options.td
    cfe/trunk/include/clang/Driver/ToolChain.h
    cfe/trunk/lib/CodeGen/CGDecl.cpp
    cfe/trunk/lib/Driver/ToolChains/Clang.cpp
    cfe/trunk/lib/Frontend/CompilerInvocation.cpp
    cfe/trunk/lib/Sema/SemaDeclAttr.cpp
    cfe/trunk/test/CodeGenCXX/auto-var-init.cpp
    cfe/trunk/test/Driver/clang_f_opts.c
    cfe/trunk/test/Sema/uninit-variables.c

Modified: cfe/trunk/include/clang/Basic/Attr.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/Attr.td?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/Attr.td (original)
+++ cfe/trunk/include/clang/Basic/Attr.td Mon Dec 17 21:12:21 2018
@@ -3131,3 +3131,9 @@ def SpeculativeLoadHardening : Inheritab
   let Subjects = SubjectList<[Function, ObjCMethod], ErrorDiag>;
   let Documentation = [SpeculativeLoadHardeningDocs];
 }
+
+def Uninitialized : InheritableAttr {
+  let Spellings = [Clang<"uninitialized", 0>];
+  let Subjects = SubjectList<[LocalVar]>;
+  let Documentation = [UninitializedDocs];
+}

Modified: cfe/trunk/include/clang/Basic/AttrDocs.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/AttrDocs.td?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/AttrDocs.td (original)
+++ cfe/trunk/include/clang/Basic/AttrDocs.td Mon Dec 17 21:12:21 2018
@@ -3704,6 +3704,18 @@ invoking clang with -fno-c++-static-dest
   }];
 }
 
+def UninitializedDocs : Documentation {
+  let Category = DocCatVariable;
+  let Content = [{
+The command-line parameter ``-ftrivial-auto-var-init=*`` can be used to
+initialize trivial automatic stack variables. By default, trivial automatic
+stack variables are uninitialized. This attribute is used to override the
+command-line parameter, forcing variables to remain uninitialized. It has no
+semantic meaning in that using uninitialized values is undefined behavior,
+it rather documents the programmer's intent.
+  }];
+}
+
 def GnuInlineDocs : Documentation {
   let Category = DocCatFunction;
   let Content = [{

Modified: cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td (original)
+++ cfe/trunk/include/clang/Basic/DiagnosticDriverKinds.td Mon Dec 17 21:12:21 2018
@@ -409,4 +409,9 @@ def warn_drv_moutline_unsupported_opt :
 def warn_drv_darwin_sdk_invalid_settings : Warning<
   "SDK settings were ignored as 'SDKSettings.json' could not be parsed">,
   InGroup<DiagGroup<"darwin-sdk-settings">>;
+
+def err_drv_trivial_auto_var_init_zero_disabled : Error<
+  "-ftrivial-auto-var-init=zero hasn't been enabled. Enable it at your own peril for benchmarking purpose only with "
+  "-enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang">;
+
 }

Modified: cfe/trunk/include/clang/Basic/LangOptions.def
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/LangOptions.def?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/LangOptions.def (original)
+++ cfe/trunk/include/clang/Basic/LangOptions.def Mon Dec 17 21:12:21 2018
@@ -263,6 +263,8 @@ ENUM_LANGOPT(TypeVisibilityMode, Visibil
              "type symbol visibility")
 ENUM_LANGOPT(StackProtector, StackProtectorMode, 2, SSPOff,
              "stack protector mode")
+ENUM_LANGOPT(TrivialAutoVarInit, TrivialAutoVarInitKind, 2, TrivialAutoVarInitKind::Uninitialized,
+             "trivial automatic variable initialization")
 ENUM_LANGOPT(SignedOverflowBehavior, SignedOverflowBehaviorTy, 2, SOB_Undefined,
              "signed integer overflow handling")
 

Modified: cfe/trunk/include/clang/Basic/LangOptions.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Basic/LangOptions.h?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/include/clang/Basic/LangOptions.h (original)
+++ cfe/trunk/include/clang/Basic/LangOptions.h Mon Dec 17 21:12:21 2018
@@ -54,6 +54,11 @@ public:
   enum GCMode { NonGC, GCOnly, HybridGC };
   enum StackProtectorMode { SSPOff, SSPOn, SSPStrong, SSPReq };
 
+  // Automatic variables live on the stack, and when trivial they're usually
+  // uninitialized because it's undefined behavior to use them without
+  // initializing them.
+  enum class TrivialAutoVarInitKind { Uninitialized, Zero, Pattern };
+
   enum SignedOverflowBehaviorTy {
     // Default C standard behavior.
     SOB_Undefined,

Modified: cfe/trunk/include/clang/Driver/Options.td
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/Options.td?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/Options.td (original)
+++ cfe/trunk/include/clang/Driver/Options.td Mon Dec 17 21:12:21 2018
@@ -1651,6 +1651,12 @@ def fstack_protector : Flag<["-"], "fsta
            "if they contain a char (or 8bit integer) array or constant sized calls to "
            "alloca, which are of greater size than ssp-buffer-size (default: 8 bytes). "
            "All variable sized calls to alloca are considered vulnerable">;
+def ftrivial_auto_var_init : Joined<["-"], "ftrivial-auto-var-init=">, Group<f_Group>,
+  Flags<[CC1Option]>, HelpText<"Initialize trivial automatic stack variables: uninitialized (default)"
+  " | pattern">, Values<"uninitialized,pattern">;
+def enable_trivial_var_init_zero : Joined<["-"], "enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang">,
+  Flags<[CC1Option]>,
+  HelpText<"Trivial automatic variable initialization to zero is only here for benchmarks, it'll eventually be removed, and I'm OK with that because I'm only using it to benchmark">;
 def fstandalone_debug : Flag<["-"], "fstandalone-debug">, Group<f_Group>, Flags<[CoreOption]>,
   HelpText<"Emit full debug info for all types used by the program">;
 def fno_standalone_debug : Flag<["-"], "fno-standalone-debug">, Group<f_Group>, Flags<[CoreOption]>,

Modified: cfe/trunk/include/clang/Driver/ToolChain.h
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/include/clang/Driver/ToolChain.h?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/include/clang/Driver/ToolChain.h (original)
+++ cfe/trunk/include/clang/Driver/ToolChain.h Mon Dec 17 21:12:21 2018
@@ -12,8 +12,8 @@
 
 #include "clang/Basic/DebugInfoOptions.h"
 #include "clang/Basic/LLVM.h"
+#include "clang/Basic/LangOptions.h"
 #include "clang/Basic/Sanitizers.h"
-#include "clang/Basic/DebugInfoOptions.h"
 #include "clang/Driver/Action.h"
 #include "clang/Driver/Multilib.h"
 #include "clang/Driver/Types.h"
@@ -350,6 +350,12 @@ public:
     return 0;
   }
 
+  /// Get the default trivial automatic variable initialization.
+  virtual LangOptions::TrivialAutoVarInitKind
+  GetDefaultTrivialAutoVarInit() const {
+    return LangOptions::TrivialAutoVarInitKind::Uninitialized;
+  }
+
   /// GetDefaultLinker - Get the default linker to use.
   virtual const char *getDefaultLinker() const { return "ld"; }
 

Modified: cfe/trunk/lib/CodeGen/CGDecl.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/CodeGen/CGDecl.cpp?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/lib/CodeGen/CGDecl.cpp (original)
+++ cfe/trunk/lib/CodeGen/CGDecl.cpp Mon Dec 17 21:12:21 2018
@@ -963,6 +963,85 @@ static llvm::Value *shouldUseMemSetToIni
   return llvm::isBytewiseValue(Init);
 }
 
+static llvm::Constant *patternFor(CodeGenModule &CGM, llvm::Type *Ty) {
+  // The following value is a guaranteed unmappable pointer value and has a
+  // repeated byte-pattern which makes it easier to synthesize. We use it for
+  // pointers as well as integers so that aggregates are likely to be
+  // initialized with this repeated value.
+  constexpr uint64_t LargeValue = 0xAAAAAAAAAAAAAAAAull;
+  // For 32-bit platforms it's a bit trickier because, across systems, only the
+  // zero page can reasonably be expected to be unmapped, and even then we need
+  // a very low address. We use a smaller value, and that value sadly doesn't
+  // have a repeated byte-pattern. We don't use it for integers.
+  constexpr uint32_t SmallValue = 0x000000AA;
+  // Floating-point values are initialized as NaNs because they propagate. Using
+  // a repeated byte pattern means that it will be easier to initialize
+  // all-floating-point aggregates and arrays with memset. Further, aggregates
+  // which mix integral and a few floats might also initialize with memset
+  // followed by a handful of stores for the floats. Using fairly unique NaNs
+  // also means they'll be easier to distinguish in a crash.
+  constexpr bool NegativeNaN = true;
+  constexpr uint64_t NaNPayload = 0xFFFFFFFFFFFFFFFFull;
+  if (Ty->isIntOrIntVectorTy()) {
+    unsigned BitWidth = cast<llvm::IntegerType>(
+                            Ty->isVectorTy() ? Ty->getVectorElementType() : Ty)
+                            ->getBitWidth();
+    if (BitWidth <= 64)
+      return llvm::ConstantInt::get(Ty, LargeValue);
+    return llvm::ConstantInt::get(
+        Ty, llvm::APInt::getSplat(BitWidth, llvm::APInt(64, LargeValue)));
+  }
+  if (Ty->isPtrOrPtrVectorTy()) {
+    auto *PtrTy = cast<llvm::PointerType>(
+        Ty->isVectorTy() ? Ty->getVectorElementType() : Ty);
+    unsigned PtrWidth = CGM.getContext().getTargetInfo().getPointerWidth(
+        PtrTy->getAddressSpace());
+    llvm::Type *IntTy = llvm::IntegerType::get(CGM.getLLVMContext(), PtrWidth);
+    uint64_t IntValue;
+    switch (PtrWidth) {
+    default:
+      llvm_unreachable("pattern initialization of unsupported pointer width");
+    case 64:
+      IntValue = LargeValue;
+      break;
+    case 32:
+      IntValue = SmallValue;
+      break;
+    }
+    auto *Int = llvm::ConstantInt::get(IntTy, IntValue);
+    return llvm::ConstantExpr::getIntToPtr(Int, PtrTy);
+  }
+  if (Ty->isFPOrFPVectorTy()) {
+    unsigned BitWidth = llvm::APFloat::semanticsSizeInBits(
+        (Ty->isVectorTy() ? Ty->getVectorElementType() : Ty)
+            ->getFltSemantics());
+    llvm::APInt Payload(64, NaNPayload);
+    if (BitWidth >= 64)
+      Payload = llvm::APInt::getSplat(BitWidth, Payload);
+    return llvm::ConstantFP::getQNaN(Ty, NegativeNaN, &Payload);
+  }
+  if (Ty->isArrayTy()) {
+    // Note: this doesn't touch tail padding (at the end of an object, before
+    // the next array object). It is instead handled by replaceUndef.
+    auto *ArrTy = cast<llvm::ArrayType>(Ty);
+    llvm::SmallVector<llvm::Constant *, 8> Element(
+        ArrTy->getNumElements(), patternFor(CGM, ArrTy->getElementType()));
+    return llvm::ConstantArray::get(ArrTy, Element);
+  }
+
+  // Note: this doesn't touch struct padding. It will initialize as much union
+  // padding as is required for the largest type in the union. Padding is
+  // instead handled by replaceUndef. Stores to structs with volatile members
+  // don't have a volatile qualifier when initialized according to C++. This is
+  // fine because stack-based volatiles don't really have volatile semantics
+  // anyways, and the initialization shouldn't be observable.
+  auto *StructTy = cast<llvm::StructType>(Ty);
+  llvm::SmallVector<llvm::Constant *, 8> Struct(StructTy->getNumElements());
+  for (unsigned El = 0; El != Struct.size(); ++El)
+    Struct[El] = patternFor(CGM, StructTy->getElementType(El));
+  return llvm::ConstantStruct::get(StructTy, Struct);
+}
+
 static Address createUnnamedGlobalFrom(CodeGenModule &CGM, const VarDecl &D,
                                        CGBuilderTy &Builder,
                                        llvm::Constant *Constant,
@@ -1010,13 +1089,20 @@ static void emitStoresForConstant(CodeGe
                                   Address Loc, bool isVolatile,
                                   CGBuilderTy &Builder,
                                   llvm::Constant *constant) {
+  auto *Ty = constant->getType();
+  bool isScalar = Ty->isIntOrIntVectorTy() || Ty->isPtrOrPtrVectorTy() ||
+                  Ty->isFPOrFPVectorTy();
+  if (isScalar) {
+    Builder.CreateStore(constant, Loc, isVolatile);
+    return;
+  }
+
   auto *Int8Ty = llvm::IntegerType::getInt8Ty(CGM.getLLVMContext());
   auto *IntPtrTy = CGM.getDataLayout().getIntPtrType(CGM.getLLVMContext());
 
   // If the initializer is all or mostly the same, codegen with bzero / memset
   // then do a few stores afterward.
-  uint64_t ConstantSize =
-      CGM.getDataLayout().getTypeAllocSize(constant->getType());
+  uint64_t ConstantSize = CGM.getDataLayout().getTypeAllocSize(Ty);
   auto *SizeVal = llvm::ConstantInt::get(IntPtrTy, ConstantSize);
   if (shouldUseBZeroPlusStoresToInitialize(constant, ConstantSize)) {
     Builder.CreateMemSet(Loc, llvm::ConstantInt::get(Int8Ty, 0), SizeVal,
@@ -1025,8 +1111,7 @@ static void emitStoresForConstant(CodeGe
     bool valueAlreadyCorrect =
         constant->isNullValue() || isa<llvm::UndefValue>(constant);
     if (!valueAlreadyCorrect) {
-      Loc = Builder.CreateBitCast(
-          Loc, constant->getType()->getPointerTo(Loc.getAddressSpace()));
+      Loc = Builder.CreateBitCast(Loc, Ty->getPointerTo(Loc.getAddressSpace()));
       emitStoresForInitAfterBZero(CGM, constant, Loc, isVolatile, Builder);
     }
     return;
@@ -1051,6 +1136,58 @@ static void emitStoresForConstant(CodeGe
       SizeVal, isVolatile);
 }
 
+static void emitStoresForZeroInit(CodeGenModule &CGM, const VarDecl &D,
+                                  Address Loc, bool isVolatile,
+                                  CGBuilderTy &Builder) {
+  llvm::Type *ElTy = Loc.getElementType();
+  llvm::Constant *constant = llvm::Constant::getNullValue(ElTy);
+  emitStoresForConstant(CGM, D, Loc, isVolatile, Builder, constant);
+}
+
+static void emitStoresForPatternInit(CodeGenModule &CGM, const VarDecl &D,
+                                     Address Loc, bool isVolatile,
+                                     CGBuilderTy &Builder) {
+  llvm::Type *ElTy = Loc.getElementType();
+  llvm::Constant *constant = patternFor(CGM, ElTy);
+  assert(!isa<llvm::UndefValue>(constant));
+  emitStoresForConstant(CGM, D, Loc, isVolatile, Builder, constant);
+}
+
+static bool containsUndef(llvm::Constant *constant) {
+  auto *Ty = constant->getType();
+  if (isa<llvm::UndefValue>(constant))
+    return true;
+  if (Ty->isStructTy() || Ty->isArrayTy() || Ty->isVectorTy())
+    for (llvm::Use &Op : constant->operands())
+      if (containsUndef(cast<llvm::Constant>(Op)))
+        return true;
+  return false;
+}
+
+static llvm::Constant *replaceUndef(llvm::Constant *constant) {
+  // FIXME: when doing pattern initialization, replace undef with 0xAA instead.
+  // FIXME: also replace padding between values by creating a new struct type
+  //        which has no padding.
+  auto *Ty = constant->getType();
+  if (isa<llvm::UndefValue>(constant))
+    return llvm::Constant::getNullValue(Ty);
+  if (!(Ty->isStructTy() || Ty->isArrayTy() || Ty->isVectorTy()))
+    return constant;
+  if (!containsUndef(constant))
+    return constant;
+  llvm::SmallVector<llvm::Constant *, 8> Values(constant->getNumOperands());
+  for (unsigned Op = 0, NumOp = constant->getNumOperands(); Op != NumOp; ++Op) {
+    auto *OpValue = cast<llvm::Constant>(constant->getOperand(Op));
+    Values[Op] = replaceUndef(OpValue);
+  }
+  if (Ty->isStructTy())
+    return llvm::ConstantStruct::get(cast<llvm::StructType>(Ty), Values);
+  if (Ty->isArrayTy())
+    return llvm::ConstantArray::get(cast<llvm::ArrayType>(Ty), Values);
+  assert(Ty->isVectorTy());
+  return llvm::ConstantVector::get(Values);
+}
+
 /// EmitAutoVarDecl - Emit code and set up an entry in LocalDeclMap for a
 /// variable declaration with auto, register, or no storage class specifier.
 /// These turn into simple stack objects, or GlobalValues depending on target.
@@ -1442,6 +1579,8 @@ void CodeGenFunction::EmitAutoVarInit(co
   auto DL = ApplyDebugLocation::CreateDefaultArtificial(*this, D.getLocation());
   QualType type = D.getType();
 
+  bool isVolatile = type.isVolatileQualified();
+
   // If this local has an initializer, emit it now.
   const Expr *Init = D.getInit();
 
@@ -1469,24 +1608,120 @@ void CodeGenFunction::EmitAutoVarInit(co
     return;
   }
 
-  if (isTrivialInitializer(Init))
-    return;
-
   // Check whether this is a byref variable that's potentially
   // captured and moved by its own initializer.  If so, we'll need to
   // emit the initializer first, then copy into the variable.
-  bool capturedByInit = emission.IsEscapingByRef && isCapturedBy(D, Init);
+  bool capturedByInit =
+      Init && emission.IsEscapingByRef && isCapturedBy(D, Init);
 
   Address Loc =
-    capturedByInit ? emission.Addr : emission.getObjectAddress(*this);
+      capturedByInit ? emission.Addr : emission.getObjectAddress(*this);
+
+  // Note: constexpr already initializes everything correctly.
+  LangOptions::TrivialAutoVarInitKind trivialAutoVarInit =
+      (D.isConstexpr()
+           ? LangOptions::TrivialAutoVarInitKind::Uninitialized
+           : (D.getAttr<UninitializedAttr>()
+                  ? LangOptions::TrivialAutoVarInitKind::Uninitialized
+                  : getContext().getLangOpts().getTrivialAutoVarInit()));
+
+  auto initializeWhatIsTechnicallyUninitialized = [&]() {
+    if (trivialAutoVarInit ==
+        LangOptions::TrivialAutoVarInitKind::Uninitialized)
+      return;
+
+    CharUnits Size = getContext().getTypeSizeInChars(type);
+    if (!Size.isZero()) {
+      switch (trivialAutoVarInit) {
+      case LangOptions::TrivialAutoVarInitKind::Uninitialized:
+        llvm_unreachable("Uninitialized handled above");
+      case LangOptions::TrivialAutoVarInitKind::Zero:
+        emitStoresForZeroInit(CGM, D, Loc, isVolatile, Builder);
+        break;
+      case LangOptions::TrivialAutoVarInitKind::Pattern:
+        emitStoresForPatternInit(CGM, D, Loc, isVolatile, Builder);
+        break;
+      }
+      return;
+    }
+
+    // VLAs look zero-sized to getTypeInfo. We can't emit constant stores to
+    // them, so emit a memcpy with the VLA size to initialize each element.
+    // Technically zero-sized or negative-sized VLAs are undefined, and UBSan
+    // will catch that code, but there exists code which generates zero-sized
+    // VLAs. Be nice and initialize whatever they requested.
+    const VariableArrayType *VlaType =
+        dyn_cast_or_null<VariableArrayType>(getContext().getAsArrayType(type));
+    if (!VlaType)
+      return;
+    auto VlaSize = getVLASize(VlaType);
+    auto SizeVal = VlaSize.NumElts;
+    CharUnits EltSize = getContext().getTypeSizeInChars(VlaSize.Type);
+    switch (trivialAutoVarInit) {
+    case LangOptions::TrivialAutoVarInitKind::Uninitialized:
+      llvm_unreachable("Uninitialized handled above");
+
+    case LangOptions::TrivialAutoVarInitKind::Zero:
+      if (!EltSize.isOne())
+        SizeVal = Builder.CreateNUWMul(SizeVal, CGM.getSize(EltSize));
+      Builder.CreateMemSet(Loc, llvm::ConstantInt::get(Int8Ty, 0), SizeVal,
+                           isVolatile);
+      break;
+
+    case LangOptions::TrivialAutoVarInitKind::Pattern: {
+      llvm::Type *ElTy = Loc.getElementType();
+      llvm::Constant *Constant = patternFor(CGM, ElTy);
+      CharUnits ConstantAlign = getContext().getTypeAlignInChars(VlaSize.Type);
+      llvm::BasicBlock *SetupBB = createBasicBlock("vla-setup.loop");
+      llvm::BasicBlock *LoopBB = createBasicBlock("vla-init.loop");
+      llvm::BasicBlock *ContBB = createBasicBlock("vla-init.cont");
+      llvm::Value *IsZeroSizedVLA = Builder.CreateICmpEQ(
+          SizeVal, llvm::ConstantInt::get(SizeVal->getType(), 0),
+          "vla.iszerosized");
+      Builder.CreateCondBr(IsZeroSizedVLA, ContBB, SetupBB);
+      EmitBlock(SetupBB);
+      if (!EltSize.isOne())
+        SizeVal = Builder.CreateNUWMul(SizeVal, CGM.getSize(EltSize));
+      llvm::Value *BaseSizeInChars =
+          llvm::ConstantInt::get(IntPtrTy, EltSize.getQuantity());
+      Address Begin = Builder.CreateElementBitCast(Loc, Int8Ty, "vla.begin");
+      llvm::Value *End =
+          Builder.CreateInBoundsGEP(Begin.getPointer(), SizeVal, "vla.end");
+      llvm::BasicBlock *OriginBB = Builder.GetInsertBlock();
+      EmitBlock(LoopBB);
+      llvm::PHINode *Cur = Builder.CreatePHI(Begin.getType(), 2, "vla.cur");
+      Cur->addIncoming(Begin.getPointer(), OriginBB);
+      CharUnits CurAlign = Loc.getAlignment().alignmentOfArrayElement(EltSize);
+      Builder.CreateMemCpy(
+          Address(Cur, CurAlign),
+          createUnnamedGlobalFrom(CGM, D, Builder, Constant, ConstantAlign),
+          BaseSizeInChars, isVolatile);
+      llvm::Value *Next =
+          Builder.CreateInBoundsGEP(Int8Ty, Cur, BaseSizeInChars, "vla.next");
+      llvm::Value *Done = Builder.CreateICmpEQ(Next, End, "vla-init.isdone");
+      Builder.CreateCondBr(Done, ContBB, LoopBB);
+      Cur->addIncoming(Next, LoopBB);
+      EmitBlock(ContBB);
+    } break;
+    }
+  };
+
+  if (isTrivialInitializer(Init)) {
+    initializeWhatIsTechnicallyUninitialized();
+    return;
+  }
 
   llvm::Constant *constant = nullptr;
   if (emission.IsConstantAggregate || D.isConstexpr()) {
     assert(!capturedByInit && "constant init contains a capturing block?");
     constant = ConstantEmitter(*this).tryEmitAbstractForInitializer(D);
+    if (constant && trivialAutoVarInit !=
+                        LangOptions::TrivialAutoVarInitKind::Uninitialized)
+      constant = replaceUndef(constant);
   }
 
   if (!constant) {
+    initializeWhatIsTechnicallyUninitialized();
     LValue lv = MakeAddrLValue(Loc, type);
     lv.setNonGC(true);
     return EmitExprAsInit(Init, &D, lv, capturedByInit);
@@ -1499,10 +1734,6 @@ void CodeGenFunction::EmitAutoVarInit(co
     return EmitStoreThroughLValue(RValue::get(constant), lv, true);
   }
 
-  // If this is a simple aggregate initialization, we can optimize it
-  // in various ways.
-  bool isVolatile = type.isVolatileQualified();
-
   llvm::Type *BP = CGM.Int8Ty->getPointerTo(Loc.getAddressSpace());
   if (Loc.getType() != BP)
     Loc = Builder.CreateBitCast(Loc, BP);

Modified: cfe/trunk/lib/Driver/ToolChains/Clang.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Driver/ToolChains/Clang.cpp?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/lib/Driver/ToolChains/Clang.cpp (original)
+++ cfe/trunk/lib/Driver/ToolChains/Clang.cpp Mon Dec 17 21:12:21 2018
@@ -2472,6 +2472,50 @@ static void RenderSSPOptions(const ToolC
   }
 }
 
+static void RenderTrivialAutoVarInitOptions(const Driver &D,
+                                            const ToolChain &TC,
+                                            const ArgList &Args,
+                                            ArgStringList &CmdArgs) {
+  auto DefaultTrivialAutoVarInit = TC.GetDefaultTrivialAutoVarInit();
+  StringRef TrivialAutoVarInit = "";
+
+  for (const Arg *A : Args) {
+    switch (A->getOption().getID()) {
+    default:
+      continue;
+    case options::OPT_ftrivial_auto_var_init: {
+      A->claim();
+      StringRef Val = A->getValue();
+      if (Val == "uninitialized" || Val == "zero" || Val == "pattern")
+        TrivialAutoVarInit = Val;
+      else
+        D.Diag(diag::err_drv_unsupported_option_argument)
+            << A->getOption().getName() << Val;
+      break;
+    }
+    }
+  }
+
+  if (TrivialAutoVarInit.empty())
+    switch (DefaultTrivialAutoVarInit) {
+    case LangOptions::TrivialAutoVarInitKind::Uninitialized:
+      break;
+    case LangOptions::TrivialAutoVarInitKind::Pattern:
+      TrivialAutoVarInit = "pattern";
+      break;
+    case LangOptions::TrivialAutoVarInitKind::Zero:
+      TrivialAutoVarInit = "zero";
+      break;
+    }
+
+  if (!TrivialAutoVarInit.empty()) {
+    if (TrivialAutoVarInit == "zero" && !Args.hasArg(options::OPT_enable_trivial_var_init_zero))
+      D.Diag(diag::err_drv_trivial_auto_var_init_zero_disabled);
+    CmdArgs.push_back(
+        Args.MakeArgString("-ftrivial-auto-var-init=" + TrivialAutoVarInit));
+  }
+}
+
 static void RenderOpenCLOptions(const ArgList &Args, ArgStringList &CmdArgs) {
   const unsigned ForwardedArguments[] = {
       options::OPT_cl_opt_disable,
@@ -4463,6 +4507,7 @@ void Clang::ConstructJob(Compilation &C,
     CmdArgs.push_back(Args.MakeArgString("-mspeculative-load-hardening"));
 
   RenderSSPOptions(TC, Args, CmdArgs, KernelOrKext);
+  RenderTrivialAutoVarInitOptions(D, TC, Args, CmdArgs);
 
   // Translate -mstackrealign
   if (Args.hasFlag(options::OPT_mstackrealign, options::OPT_mno_stackrealign,

Modified: cfe/trunk/lib/Frontend/CompilerInvocation.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Frontend/CompilerInvocation.cpp?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/lib/Frontend/CompilerInvocation.cpp (original)
+++ cfe/trunk/lib/Frontend/CompilerInvocation.cpp Mon Dec 17 21:12:21 2018
@@ -2938,6 +2938,19 @@ static void ParseLangArgs(LangOptions &O
   case 3: Opts.setStackProtector(LangOptions::SSPReq); break;
   }
 
+  if (Arg *A = Args.getLastArg(OPT_ftrivial_auto_var_init)) {
+    StringRef Val = A->getValue();
+    if (Val == "uninitialized")
+      Opts.setTrivialAutoVarInit(
+          LangOptions::TrivialAutoVarInitKind::Uninitialized);
+    else if (Val == "zero")
+      Opts.setTrivialAutoVarInit(LangOptions::TrivialAutoVarInitKind::Zero);
+    else if (Val == "pattern")
+      Opts.setTrivialAutoVarInit(LangOptions::TrivialAutoVarInitKind::Pattern);
+    else
+      Diags.Report(diag::err_drv_invalid_value) << A->getAsString(Args) << Val;
+  }
+
   // Parse -fsanitize= arguments.
   parseSanitizerKinds("-fsanitize=", Args.getAllArgValues(OPT_fsanitize_EQ),
                       Diags, Opts.Sanitize);

Modified: cfe/trunk/lib/Sema/SemaDeclAttr.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Sema/SemaDeclAttr.cpp?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/lib/Sema/SemaDeclAttr.cpp (original)
+++ cfe/trunk/lib/Sema/SemaDeclAttr.cpp Mon Dec 17 21:12:21 2018
@@ -6085,6 +6085,14 @@ static void handleDestroyAttr(Sema &S, D
     handleSimpleAttributeWithExclusions<NoDestroyAttr, AlwaysDestroyAttr>(S, D, A);
 }
 
+static void handleUninitializedAttr(Sema &S, Decl *D, const ParsedAttr &AL) {
+  assert(cast<VarDecl>(D)->getStorageDuration() == SD_Automatic &&
+         "uninitialized is only valid on automatic duration variables");
+  unsigned Index = AL.getAttributeSpellingListIndex();
+  D->addAttr(::new (S.Context)
+                 UninitializedAttr(AL.getLoc(), S.Context, Index));
+}
+
 //===----------------------------------------------------------------------===//
 // Top Level Sema Entry Points
 //===----------------------------------------------------------------------===//
@@ -6776,6 +6784,10 @@ static void ProcessDeclAttribute(Sema &S
   case ParsedAttr::AT_NoDestroy:
     handleDestroyAttr(S, D, AL);
     break;
+
+  case ParsedAttr::AT_Uninitialized:
+    handleUninitializedAttr(S, D, AL);
+    break;
   }
 }
 

Modified: cfe/trunk/test/CodeGenCXX/auto-var-init.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/auto-var-init.cpp?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/test/CodeGenCXX/auto-var-init.cpp (original)
+++ cfe/trunk/test/CodeGenCXX/auto-var-init.cpp Mon Dec 17 21:12:21 2018
@@ -1,4 +1,6 @@
 // RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks %s -emit-llvm -o - | FileCheck %s
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -emit-llvm -o - | FileCheck %s -check-prefix=PATTERN
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -emit-llvm -o - | FileCheck %s -check-prefix=ZERO
 
 template<typename T> void used(T &) noexcept;
 
@@ -24,28 +26,108 @@ template<typename T> void used(T &) noex
     used(custom);                               \
   }
 
+// None of the synthesized globals should contain `undef`.
+// PATTERN-NOT: undef
+// ZERO-NOT: undef
+
+// PATTERN: @__const.test_empty_uninit.uninit = private unnamed_addr constant %struct.empty { i8 -86 }, align 1
 struct empty {};
+// PATTERN: @__const.test_small_uninit.uninit = private unnamed_addr constant %struct.small { i8 -86 }, align 1
+// PATTERN: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
+// ZERO: @__const.test_small_custom.custom = private unnamed_addr constant %struct.small { i8 42 }, align 1
 struct small { char c; };
+// PATTERN: @__const.test_smallinit_uninit.uninit = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
+// PATTERN: @__const.test_smallinit_braces.braces = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
+// PATTERN: @__const.test_smallinit_custom.custom = private unnamed_addr constant %struct.smallinit { i8 -86 }, align 1
 struct smallinit { char c = 42; };
+// PATTERN: @__const.test_smallpartinit_uninit.uninit = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
+// PATTERN: @__const.test_smallpartinit_braces.braces = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
+// PATTERN: @__const.test_smallpartinit_custom.custom = private unnamed_addr constant %struct.smallpartinit { i8 -86, i8 -86 }, align 1
 struct smallpartinit { char c = 42, d; };
+// PATTERN: @__const.test_nullinit_uninit.uninit = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
+// PATTERN: @__const.test_nullinit_braces.braces = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
+// PATTERN: @__const.test_nullinit_custom.custom = private unnamed_addr constant %struct.nullinit { i8* inttoptr (i64 -6148914691236517206 to i8*) }, align 8
 struct nullinit { char* null = nullptr; };
+// PATTERN: @__const.test_padded_uninit.uninit = private unnamed_addr constant %struct.padded { i8 -86, i32 -1431655766 }, align 4
+// PATTERN: @__const.test_padded_custom.custom = private unnamed_addr constant %struct.padded { i8 42, i32 13371337 }, align 4
+// ZERO: @__const.test_padded_custom.custom = private unnamed_addr constant %struct.padded { i8 42, i32 13371337 }, align 4
 struct padded { char c; int i; };
+// PATTERN: @__const.test_paddednullinit_uninit.uninit = private unnamed_addr constant %struct.paddednullinit { i8 -86, i32 -1431655766 }, align 4
+// PATTERN: @__const.test_paddednullinit_braces.braces = private unnamed_addr constant %struct.paddednullinit { i8 -86, i32 -1431655766 }, align 4
+// PATTERN: @__const.test_paddednullinit_custom.custom = private unnamed_addr constant %struct.paddednullinit { i8 -86, i32 -1431655766 }, align 4
 struct paddednullinit { char c = 0; int i = 0; };
+// PATTERN: @__const.test_bitfield_uninit.uninit = private unnamed_addr constant %struct.bitfield { i8 -86, [3 x i8] c"\AA\AA\AA" }, align 4
+// PATTERN: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] zeroinitializer }, align 4
+// ZERO: @__const.test_bitfield_custom.custom = private unnamed_addr constant %struct.bitfield { i8 20, [3 x i8] zeroinitializer }, align 4
 struct bitfield { int i : 4; int j : 2; };
+// PATTERN: @__const.test_bitfieldaligned_uninit.uninit = private unnamed_addr constant %struct.bitfieldaligned { i8 -86, [3 x i8] c"\AA\AA\AA", i8 -86, [3 x i8] c"\AA\AA\AA" }, align 4
+// PATTERN: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] zeroinitializer, i8 1, [3 x i8] zeroinitializer }, align 4
+// ZERO: @__const.test_bitfieldaligned_custom.custom = private unnamed_addr constant %struct.bitfieldaligned { i8 4, [3 x i8] zeroinitializer, i8 1, [3 x i8] zeroinitializer }, align 4
 struct bitfieldaligned { int i : 4; int : 0; int j : 2; };
 struct big { unsigned a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z; };
+// PATTERN: @__const.test_arraytail_uninit.uninit = private unnamed_addr constant %struct.arraytail { i32 -1431655766, [0 x i32] zeroinitializer }, align 4
+// PATTERN: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
+// ZERO: @__const.test_arraytail_custom.custom = private unnamed_addr constant %struct.arraytail { i32 57005, [0 x i32] zeroinitializer }, align 4
 struct arraytail { int i; int arr[]; };
+// PATTERN: @__const.test_int1_uninit.uninit = private unnamed_addr constant [1 x i32] [i32 -1431655766], align 4
+// PATTERN: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
+// ZERO: @__const.test_int1_custom.custom = private unnamed_addr constant [1 x i32] [i32 858993459], align 4
+// PATTERN: @__const.test_bool4_uninit.uninit = private unnamed_addr constant [4 x i8] c"\AA\AA\AA\AA", align 1
+// PATTERN: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
+// ZERO: @__const.test_bool4_custom.custom = private unnamed_addr constant [4 x i8] c"\01\01\01\01", align 1
+// PATTERN: @__const.test_intptr4_uninit.uninit = private unnamed_addr constant [4 x i32*] [i32* inttoptr (i64 -6148914691236517206 to i32*), i32* inttoptr (i64 -6148914691236517206 to i32*), i32* inttoptr (i64 -6148914691236517206 to i32*), i32* inttoptr (i64 -6148914691236517206 to i32*)], align 16
+// PATTERN: @__const.test_intptr4_custom.custom = private unnamed_addr constant [4 x i32*] [i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*)], align 16
+// ZERO: @__const.test_intptr4_custom.custom = private unnamed_addr constant [4 x i32*] [i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*), i32* inttoptr (i64 572662306 to i32*)], align 16
+// PATTERN: @__const.test_tailpad4_uninit.uninit = private unnamed_addr constant [4 x %struct.tailpad] [%struct.tailpad { i16 -21846, i8 -86 }, %struct.tailpad { i16 -21846, i8 -86 }, %struct.tailpad { i16 -21846, i8 -86 }, %struct.tailpad { i16 -21846, i8 -86 }], align 16
+// PATTERN: @__const.test_tailpad4_custom.custom = private unnamed_addr constant [4 x %struct.tailpad] [%struct.tailpad { i16 257, i8 1 }, %struct.tailpad { i16 257, i8 1 }, %struct.tailpad { i16 257, i8 1 }, %struct.tailpad { i16 257, i8 1 }], align 16
+// ZERO: @__const.test_tailpad4_custom.custom = private unnamed_addr constant [4 x %struct.tailpad] [%struct.tailpad { i16 257, i8 1 }, %struct.tailpad { i16 257, i8 1 }, %struct.tailpad { i16 257, i8 1 }, %struct.tailpad { i16 257, i8 1 }], align 16
 struct tailpad { short s; char c; };
+// PATTERN: @__const.test_atomicnotlockfree_uninit.uninit = private unnamed_addr constant %struct.notlockfree { [4 x i64] [i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206] }, align 8
 struct notlockfree { long long a[4]; };
+// PATTERN: @__const.test_atomicpadded_uninit.uninit = private unnamed_addr constant %struct.padded { i8 -86, i32 -1431655766 }, align 8
+// PATTERN: @__const.test_atomictailpad_uninit.uninit = private unnamed_addr constant %struct.tailpad { i16 -21846, i8 -86 }, align 4
+// PATTERN: @__const.test_complexfloat_uninit.uninit = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN: @__const.test_complexfloat_braces.braces = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN: @__const.test_complexfloat_custom.custom = private unnamed_addr constant { float, float } { float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN: @__const.test_complexdouble_uninit.uninit = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN: @__const.test_complexdouble_braces.braces = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN: @__const.test_complexdouble_custom.custom = private unnamed_addr constant { double, double } { double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN: @__const.test_semivolatile_uninit.uninit = private unnamed_addr constant %struct.semivolatile { i32 -1431655766, i32 -1431655766 }, align 4
+// PATTERN: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
 struct semivolatile { int i; volatile int vi; };
+// PATTERN: @__const.test_semivolatileinit_uninit.uninit = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
+// PATTERN: @__const.test_semivolatileinit_braces.braces = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
+// PATTERN: @__const.test_semivolatileinit_custom.custom = private unnamed_addr constant %struct.semivolatileinit { i32 -1431655766, i32 -1431655766 }, align 4
+// ZERO: @__const.test_semivolatile_custom.custom = private unnamed_addr constant %struct.semivolatile { i32 1145324612, i32 1145324612 }, align 4
 struct semivolatileinit { int i = 0x11111111; volatile int vi = 0x11111111; };
+// PATTERN: @__const.test_base_uninit.uninit = private unnamed_addr constant %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, align 8
+// PATTERN: @__const.test_base_braces.braces = private unnamed_addr constant %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, align 8
 struct base { virtual ~base(); };
+// PATTERN: @__const.test_derived_uninit.uninit = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } }, align 8
+// PATTERN: @__const.test_derived_braces.braces = private unnamed_addr constant %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } }, align 8
 struct derived : public base {};
+// PATTERN: @__const.test_virtualderived_uninit.uninit = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } } }, align 8
+// PATTERN: @__const.test_virtualderived_braces.braces = private unnamed_addr constant %struct.virtualderived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) }, %struct.derived { %struct.base { i32 (...)** inttoptr (i64 -6148914691236517206 to i32 (...)**) } } }, align 8
 struct virtualderived : public virtual base, public virtual derived {};
+// PATTERN: @__const.test_matching_uninit.uninit = private unnamed_addr constant %union.matching { i32 -1431655766 }, align 4
+// PATTERN: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
 union matching { int i; float f; };
+// PATTERN: @__const.test_matchingreverse_uninit.uninit = private unnamed_addr constant %union.matchingreverse { float 0xFFFFFFFFE0000000 }, align 4
+// PATTERN: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
+// ZERO: @__const.test_matching_custom.custom = private unnamed_addr constant { float } { float 6.145500e+04 }, align 4
 union matchingreverse { float f; int i; };
+// PATTERN: @__const.test_unmatched_uninit.uninit = private unnamed_addr constant %union.unmatched { i32 -1431655766 }, align 4
+// PATTERN: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
+// ZERO: @__const.test_matchingreverse_custom.custom = private unnamed_addr constant { i32 } { i32 61455 }, align 4
 union unmatched { char c; int i; };
+// PATTERN: @__const.test_unmatchedreverse_uninit.uninit = private unnamed_addr constant %union.unmatchedreverse { i32 -1431655766 }, align 4
+// PATTERN: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] zeroinitializer }, align 4
+// ZERO: @__const.test_unmatched_custom.custom = private unnamed_addr constant %union.unmatched { i32 1001242351 }, align 4
 union unmatchedreverse { int i; char c; };
+// PATTERN: @__const.test_unmatchedfp_uninit.uninit = private unnamed_addr constant %union.unmatchedfp { double 0xFFFFFFFFFFFFFFFF }, align 8
+// PATTERN: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align 8
+// ZERO: @__const.test_unmatchedreverse_custom.custom = private unnamed_addr constant { i8, [3 x i8] } { i8 42, [3 x i8] zeroinitializer }, align 4
+// ZERO: @__const.test_unmatchedfp_custom.custom = private unnamed_addr constant %union.unmatchedfp { double 0x400921FB54442D18 }, align 8
 union unmatchedfp { float f; double d; };
 enum emptyenum {};
 enum smallenum { VALUE };
@@ -56,6 +138,10 @@ TEST_UNINIT(char, char);
 // CHECK-LABEL: @test_char_uninit()
 // CHECK:       %uninit = alloca i8, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_char_uninit()
+// PATTERN: store i8 -86, i8* %uninit, align 1
+// ZERO-LABEL: @test_char_uninit()
+// ZERO: store i8 0, i8* %uninit, align 1
 
 TEST_BRACES(char, char);
 // CHECK-LABEL: @test_char_braces()
@@ -67,6 +153,10 @@ TEST_UNINIT(uchar, unsigned char);
 // CHECK-LABEL: @test_uchar_uninit()
 // CHECK:       %uninit = alloca i8, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_uchar_uninit()
+// PATTERN: store i8 -86, i8* %uninit, align 1
+// ZERO-LABEL: @test_uchar_uninit()
+// ZERO: store i8 0, i8* %uninit, align 1
 
 TEST_BRACES(uchar, unsigned char);
 // CHECK-LABEL: @test_uchar_braces()
@@ -78,6 +168,10 @@ TEST_UNINIT(schar, signed char);
 // CHECK-LABEL: @test_schar_uninit()
 // CHECK:       %uninit = alloca i8, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_schar_uninit()
+// PATTERN: store i8 -86, i8* %uninit, align 1
+// ZERO-LABEL: @test_schar_uninit()
+// ZERO: store i8 0, i8* %uninit, align 1
 
 TEST_BRACES(schar, signed char);
 // CHECK-LABEL: @test_schar_braces()
@@ -89,6 +183,10 @@ TEST_UNINIT(wchar_t, wchar_t);
 // CHECK-LABEL: @test_wchar_t_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_wchar_t_uninit()
+// PATTERN: store i32 -1431655766, i32* %uninit, align
+// ZERO-LABEL: @test_wchar_t_uninit()
+// ZERO: store i32 0, i32* %uninit, align
 
 TEST_BRACES(wchar_t, wchar_t);
 // CHECK-LABEL: @test_wchar_t_braces()
@@ -100,6 +198,10 @@ TEST_UNINIT(short, short);
 // CHECK-LABEL: @test_short_uninit()
 // CHECK:       %uninit = alloca i16, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_short_uninit()
+// PATTERN: store i16 -21846, i16* %uninit, align
+// ZERO-LABEL: @test_short_uninit()
+// ZERO: store i16 0, i16* %uninit, align
 
 TEST_BRACES(short, short);
 // CHECK-LABEL: @test_short_braces()
@@ -111,6 +213,10 @@ TEST_UNINIT(ushort, unsigned short);
 // CHECK-LABEL: @test_ushort_uninit()
 // CHECK:       %uninit = alloca i16, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_ushort_uninit()
+// PATTERN: store i16 -21846, i16* %uninit, align
+// ZERO-LABEL: @test_ushort_uninit()
+// ZERO: store i16 0, i16* %uninit, align
 
 TEST_BRACES(ushort, unsigned short);
 // CHECK-LABEL: @test_ushort_braces()
@@ -122,6 +228,10 @@ TEST_UNINIT(int, int);
 // CHECK-LABEL: @test_int_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_int_uninit()
+// PATTERN: store i32 -1431655766, i32* %uninit, align
+// ZERO-LABEL: @test_int_uninit()
+// ZERO: store i32 0, i32* %uninit, align
 
 TEST_BRACES(int, int);
 // CHECK-LABEL: @test_int_braces()
@@ -133,6 +243,10 @@ TEST_UNINIT(unsigned, unsigned);
 // CHECK-LABEL: @test_unsigned_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_unsigned_uninit()
+// PATTERN: store i32 -1431655766, i32* %uninit, align
+// ZERO-LABEL: @test_unsigned_uninit()
+// ZERO: store i32 0, i32* %uninit, align
 
 TEST_BRACES(unsigned, unsigned);
 // CHECK-LABEL: @test_unsigned_braces()
@@ -144,6 +258,10 @@ TEST_UNINIT(long, long);
 // CHECK-LABEL: @test_long_uninit()
 // CHECK:       %uninit = alloca i64, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_long_uninit()
+// PATTERN: store i64 -6148914691236517206, i64* %uninit, align
+// ZERO-LABEL: @test_long_uninit()
+// ZERO: store i64 0, i64* %uninit, align
 
 TEST_BRACES(long, long);
 // CHECK-LABEL: @test_long_braces()
@@ -155,6 +273,10 @@ TEST_UNINIT(ulong, unsigned long);
 // CHECK-LABEL: @test_ulong_uninit()
 // CHECK:       %uninit = alloca i64, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_ulong_uninit()
+// PATTERN: store i64 -6148914691236517206, i64* %uninit, align
+// ZERO-LABEL: @test_ulong_uninit()
+// ZERO: store i64 0, i64* %uninit, align
 
 TEST_BRACES(ulong, unsigned long);
 // CHECK-LABEL: @test_ulong_braces()
@@ -166,6 +288,10 @@ TEST_UNINIT(longlong, long long);
 // CHECK-LABEL: @test_longlong_uninit()
 // CHECK:       %uninit = alloca i64, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_longlong_uninit()
+// PATTERN: store i64 -6148914691236517206, i64* %uninit, align
+// ZERO-LABEL: @test_longlong_uninit()
+// ZERO: store i64 0, i64* %uninit, align
 
 TEST_BRACES(longlong, long long);
 // CHECK-LABEL: @test_longlong_braces()
@@ -177,6 +303,10 @@ TEST_UNINIT(ulonglong, unsigned long lon
 // CHECK-LABEL: @test_ulonglong_uninit()
 // CHECK:       %uninit = alloca i64, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_ulonglong_uninit()
+// PATTERN: store i64 -6148914691236517206, i64* %uninit, align
+// ZERO-LABEL: @test_ulonglong_uninit()
+// ZERO: store i64 0, i64* %uninit, align
 
 TEST_BRACES(ulonglong, unsigned long long);
 // CHECK-LABEL: @test_ulonglong_braces()
@@ -188,6 +318,10 @@ TEST_UNINIT(int128, __int128);
 // CHECK-LABEL: @test_int128_uninit()
 // CHECK:       %uninit = alloca i128, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_int128_uninit()
+// PATTERN: store i128 -113427455640312821154458202477256070486, i128* %uninit, align
+// ZERO-LABEL: @test_int128_uninit()
+// ZERO: store i128 0, i128* %uninit, align
 
 TEST_BRACES(int128, __int128);
 // CHECK-LABEL: @test_int128_braces()
@@ -199,6 +333,10 @@ TEST_UNINIT(uint128, unsigned __int128);
 // CHECK-LABEL: @test_uint128_uninit()
 // CHECK:       %uninit = alloca i128, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_uint128_uninit()
+// PATTERN: store i128 -113427455640312821154458202477256070486, i128* %uninit, align
+// ZERO-LABEL: @test_uint128_uninit()
+// ZERO: store i128 0, i128* %uninit, align
 
 TEST_BRACES(uint128, unsigned __int128);
 // CHECK-LABEL: @test_uint128_braces()
@@ -211,6 +349,10 @@ TEST_UNINIT(fp16, __fp16);
 // CHECK-LABEL: @test_fp16_uninit()
 // CHECK:       %uninit = alloca half, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_fp16_uninit()
+// PATTERN: store half 0xHFFFF, half* %uninit, align
+// ZERO-LABEL: @test_fp16_uninit()
+// ZERO: store half 0xH0000, half* %uninit, align
 
 TEST_BRACES(fp16, __fp16);
 // CHECK-LABEL: @test_fp16_braces()
@@ -222,6 +364,10 @@ TEST_UNINIT(float, float);
 // CHECK-LABEL: @test_float_uninit()
 // CHECK:       %uninit = alloca float, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_float_uninit()
+// PATTERN: store float 0xFFFFFFFFE0000000, float* %uninit, align
+// ZERO-LABEL: @test_float_uninit()
+// ZERO: store float 0.000000e+00, float* %uninit, align
 
 TEST_BRACES(float, float);
 // CHECK-LABEL: @test_float_braces()
@@ -233,6 +379,10 @@ TEST_UNINIT(double, double);
 // CHECK-LABEL: @test_double_uninit()
 // CHECK:       %uninit = alloca double, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_double_uninit()
+// PATTERN: store double 0xFFFFFFFFFFFFFFFF, double* %uninit, align
+// ZERO-LABEL: @test_double_uninit()
+// ZERO: store double 0.000000e+00, double* %uninit, align
 
 TEST_BRACES(double, double);
 // CHECK-LABEL: @test_double_braces()
@@ -244,6 +394,10 @@ TEST_UNINIT(longdouble, long double);
 // CHECK-LABEL: @test_longdouble_uninit()
 // CHECK:       %uninit = alloca x86_fp80, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_longdouble_uninit()
+// PATTERN: store x86_fp80 0xKFFFFFFFFFFFFFFFFFFFF, x86_fp80* %uninit, align
+// ZERO-LABEL: @test_longdouble_uninit()
+// ZERO: store x86_fp80 0xK00000000000000000000, x86_fp80* %uninit, align
 
 TEST_BRACES(longdouble, long double);
 // CHECK-LABEL: @test_longdouble_braces()
@@ -256,6 +410,10 @@ TEST_UNINIT(intptr, int*);
 // CHECK-LABEL: @test_intptr_uninit()
 // CHECK:       %uninit = alloca i32*, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_intptr_uninit()
+// PATTERN: store i32* inttoptr (i64 -6148914691236517206 to i32*), i32** %uninit, align
+// ZERO-LABEL: @test_intptr_uninit()
+// ZERO: store i32* null, i32** %uninit, align
 
 TEST_BRACES(intptr, int*);
 // CHECK-LABEL: @test_intptr_braces()
@@ -267,6 +425,10 @@ TEST_UNINIT(intptrptr, int**);
 // CHECK-LABEL: @test_intptrptr_uninit()
 // CHECK:       %uninit = alloca i32**, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_intptrptr_uninit()
+// PATTERN: store i32** inttoptr (i64 -6148914691236517206 to i32**), i32*** %uninit, align
+// ZERO-LABEL: @test_intptrptr_uninit()
+// ZERO: store i32** null, i32*** %uninit, align
 
 TEST_BRACES(intptrptr, int**);
 // CHECK-LABEL: @test_intptrptr_braces()
@@ -278,6 +440,10 @@ TEST_UNINIT(function, void(*)());
 // CHECK-LABEL: @test_function_uninit()
 // CHECK:       %uninit = alloca void ()*, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_function_uninit()
+// PATTERN: store void ()* inttoptr (i64 -6148914691236517206 to void ()*), void ()** %uninit, align
+// ZERO-LABEL: @test_function_uninit()
+// ZERO: store void ()* null, void ()** %uninit, align
 
 TEST_BRACES(function, void(*)());
 // CHECK-LABEL: @test_function_braces()
@@ -289,6 +455,10 @@ TEST_UNINIT(bool, bool);
 // CHECK-LABEL: @test_bool_uninit()
 // CHECK:       %uninit = alloca i8, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_bool_uninit()
+// PATTERN: store i8 -86, i8* %uninit, align 1
+// ZERO-LABEL: @test_bool_uninit()
+// ZERO: store i8 0, i8* %uninit, align 1
 
 TEST_BRACES(bool, bool);
 // CHECK-LABEL: @test_bool_braces()
@@ -301,6 +471,10 @@ TEST_UNINIT(empty, empty);
 // CHECK-LABEL: @test_empty_uninit()
 // CHECK:       %uninit = alloca %struct.empty, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_empty_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_empty_uninit.uninit
+// ZERO-LABEL: @test_empty_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(empty, empty);
 // CHECK-LABEL: @test_empty_braces()
@@ -313,6 +487,10 @@ TEST_UNINIT(small, small);
 // CHECK-LABEL: @test_small_uninit()
 // CHECK:       %uninit = alloca %struct.small, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_small_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_small_uninit.uninit
+// ZERO-LABEL: @test_small_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(small, small);
 // CHECK-LABEL: @test_small_braces()
@@ -321,7 +499,7 @@ TEST_BRACES(small, small);
 // CHECK-NEXT:  call void @llvm.memset{{.*}}(i8* align [[ALIGN]] %{{.*}}, i8 0, i64 1, i1 false)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
-  TEST_CUSTOM(small, small, { 42 });
+TEST_CUSTOM(small, small, { 42 });
 // CHECK-LABEL: @test_small_custom()
 // CHECK:       %custom = alloca %struct.small, align
 // CHECK-NEXT:  bitcast
@@ -353,6 +531,10 @@ TEST_UNINIT(smallpartinit, smallpartinit
 // CHECK:       %uninit = alloca %struct.smallpartinit, align
 // CHECK-NEXT:  call void @{{.*}}smallpartinit{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_smallpartinit_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_smallpartinit_uninit.uninit
+// ZERO-LABEL: @test_smallpartinit_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(smallpartinit, smallpartinit);
 // CHECK-LABEL: @test_smallpartinit_braces()
@@ -396,6 +578,10 @@ TEST_UNINIT(padded, padded);
 // CHECK-LABEL: @test_padded_uninit()
 // CHECK:       %uninit = alloca %struct.padded, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_padded_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_padded_uninit.uninit
+// ZERO-LABEL: @test_padded_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(padded, padded);
 // CHECK-LABEL: @test_padded_braces()
@@ -416,6 +602,10 @@ TEST_UNINIT(paddednullinit, paddednullin
 // CHECK:       %uninit = alloca %struct.paddednullinit, align
 // CHECK-NEXT:  call void @{{.*}}paddednullinit{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_paddednullinit_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_paddednullinit_uninit.uninit
+// ZERO-LABEL: @test_paddednullinit_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(paddednullinit, paddednullinit);
 // CHECK-LABEL: @test_paddednullinit_braces()
@@ -439,6 +629,10 @@ TEST_UNINIT(bitfield, bitfield);
 // CHECK-LABEL: @test_bitfield_uninit()
 // CHECK:       %uninit = alloca %struct.bitfield, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_bitfield_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_bitfield_uninit.uninit
+// ZERO-LABEL: @test_bitfield_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(bitfield, bitfield);
 // CHECK-LABEL: @test_bitfield_braces()
@@ -458,6 +652,10 @@ TEST_UNINIT(bitfieldaligned, bitfieldali
 // CHECK-LABEL: @test_bitfieldaligned_uninit()
 // CHECK:       %uninit = alloca %struct.bitfieldaligned, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_bitfieldaligned_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_bitfieldaligned_uninit.uninit
+// ZERO-LABEL: @test_bitfieldaligned_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(bitfieldaligned, bitfieldaligned);
 // CHECK-LABEL: @test_bitfieldaligned_braces()
@@ -477,6 +675,10 @@ TEST_UNINIT(big, big);
 // CHECK-LABEL: @test_big_uninit()
 // CHECK:       %uninit = alloca %struct.big, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_big_uninit()
+// PATTERN: call void @llvm.memset{{.*}}, i8 -86,
+// ZERO-LABEL: @test_big_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(big, big);
 // CHECK-LABEL: @test_big_braces()
@@ -496,6 +698,10 @@ TEST_UNINIT(arraytail, arraytail);
 // CHECK-LABEL: @test_arraytail_uninit()
 // CHECK:       %uninit = alloca %struct.arraytail, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_arraytail_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_arraytail_uninit.uninit
+// ZERO-LABEL: @test_arraytail_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(arraytail, arraytail);
 // CHECK-LABEL: @test_arraytail_braces()
@@ -516,6 +722,12 @@ TEST_UNINIT(int0, int[0]);
 // CHECK-LABEL: @test_int0_uninit()
 // CHECK:       %uninit = alloca [0 x i32], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_int0_uninit()
+// PATTERN:       %uninit = alloca [0 x i32], align
+// PATTERN-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// ZERO-LABEL: @test_int0_uninit()
+// ZERO:       %uninit = alloca [0 x i32], align
+// ZERO-NEXT:  call void @{{.*}}used{{.*}}%uninit)
 
 TEST_BRACES(int0, int[0]);
 // CHECK-LABEL: @test_int0_braces()
@@ -528,6 +740,10 @@ TEST_UNINIT(int1, int[1]);
 // CHECK-LABEL: @test_int1_uninit()
 // CHECK:       %uninit = alloca [1 x i32], align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_int1_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_int1_uninit.uninit
+// ZERO-LABEL: @test_int1_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(int1, int[1]);
 // CHECK-LABEL: @test_int1_braces()
@@ -547,6 +763,10 @@ TEST_UNINIT(int64, int[64]);
 // CHECK-LABEL: @test_int64_uninit()
 // CHECK:       %uninit = alloca [64 x i32], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_int64_uninit()
+// PATTERN: call void @llvm.memset{{.*}}, i8 -86,
+// ZERO-LABEL: @test_int64_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(int64, int[64]);
 // CHECK-LABEL: @test_int64_braces()
@@ -566,6 +786,10 @@ TEST_UNINIT(bool4, bool[4]);
 // CHECK-LABEL: @test_bool4_uninit()
 // CHECK:       %uninit = alloca [4 x i8], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_bool4_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_bool4_uninit.uninit
+// ZERO-LABEL: @test_bool4_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(bool4, bool[4]);
 // CHECK-LABEL: @test_bool4_braces()
@@ -585,6 +809,10 @@ TEST_UNINIT(intptr4, int*[4]);
 // CHECK-LABEL: @test_intptr4_uninit()
 // CHECK:       %uninit = alloca [4 x i32*], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_intptr4_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_intptr4_uninit.uninit
+// ZERO-LABEL: @test_intptr4_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(intptr4, int*[4]);
 // CHECK-LABEL: @test_intptr4_braces()
@@ -604,6 +832,10 @@ TEST_UNINIT(tailpad4, tailpad[4]);
 // CHECK-LABEL: @test_tailpad4_uninit()
 // CHECK:       %uninit = alloca [4 x %struct.tailpad], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_tailpad4_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_tailpad4_uninit.uninit
+// ZERO-LABEL: @test_tailpad4_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(tailpad4, tailpad[4]);
 // CHECK-LABEL: @test_tailpad4_braces()
@@ -623,6 +855,10 @@ TEST_UNINIT(tailpad9, tailpad[9]);
 // CHECK-LABEL: @test_tailpad9_uninit()
 // CHECK:       %uninit = alloca [9 x %struct.tailpad], align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_tailpad9_uninit()
+// PATTERN: call void @llvm.memset{{.*}}, i8 -86,
+// ZERO-LABEL: @test_tailpad9_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(tailpad9, tailpad[9]);
 // CHECK-LABEL: @test_tailpad9_braces()
@@ -643,37 +879,65 @@ TEST_UNINIT(atomicbool, _Atomic(bool));
 // CHECK-LABEL: @test_atomicbool_uninit()
 // CHECK:       %uninit = alloca i8, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_atomicbool_uninit()
+// PATTERN: store i8 -86, i8* %uninit, align 1
+// ZERO-LABEL: @test_atomicbool_uninit()
+// ZERO: store i8 0, i8* %uninit, align 1
 
 TEST_UNINIT(atomicint, _Atomic(int));
 // CHECK-LABEL: @test_atomicint_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_atomicint_uninit()
+// PATTERN: store i32 -1431655766, i32* %uninit, align 4
+// ZERO-LABEL: @test_atomicint_uninit()
+// ZERO: store i32 0, i32* %uninit, align 4
 
 TEST_UNINIT(atomicdouble, _Atomic(double));
 // CHECK-LABEL: @test_atomicdouble_uninit()
 // CHECK:       %uninit = alloca double, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_atomicdouble_uninit()
+// PATTERN: store double 0xFFFFFFFFFFFFFFFF, double* %uninit, align 8
+// ZERO-LABEL: @test_atomicdouble_uninit()
+// ZERO: store double 0.000000e+00, double* %uninit, align 8
 
 TEST_UNINIT(atomicnotlockfree, _Atomic(notlockfree));
 // CHECK-LABEL: @test_atomicnotlockfree_uninit()
 // CHECK:       %uninit = alloca %struct.notlockfree, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_atomicnotlockfree_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_atomicnotlockfree_uninit.uninit
+// ZERO-LABEL: @test_atomicnotlockfree_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_UNINIT(atomicpadded, _Atomic(padded));
 // CHECK-LABEL: @test_atomicpadded_uninit()
 // CHECK:       %uninit = alloca %struct.padded, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_atomicpadded_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_atomicpadded_uninit.uninit
+// ZERO-LABEL: @test_atomicpadded_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_UNINIT(atomictailpad, _Atomic(tailpad));
 // CHECK-LABEL: @test_atomictailpad_uninit()
 // CHECK:       %uninit = alloca %struct.tailpad, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_atomictailpad_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_atomictailpad_uninit.uninit
+// ZERO-LABEL: @test_atomictailpad_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 
 TEST_UNINIT(complexfloat, _Complex float);
 // CHECK-LABEL: @test_complexfloat_uninit()
 // CHECK:       %uninit = alloca { float, float }, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_complexfloat_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_complexfloat_uninit.uninit
+// ZERO-LABEL: @test_complexfloat_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(complexfloat, _Complex float);
 // CHECK-LABEL: @test_complexfloat_braces()
@@ -697,6 +961,10 @@ TEST_UNINIT(complexdouble, _Complex doub
 // CHECK-LABEL: @test_complexdouble_uninit()
 // CHECK:       %uninit = alloca { double, double }, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_complexdouble_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_complexdouble_uninit.uninit
+// ZERO-LABEL: @test_complexdouble_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(complexdouble, _Complex double);
 // CHECK-LABEL: @test_complexdouble_braces()
@@ -721,6 +989,10 @@ TEST_UNINIT(volatileint, volatile int);
 // CHECK-LABEL: @test_volatileint_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_volatileint_uninit()
+// PATTERN: store volatile i32 -1431655766, i32* %uninit, align 4
+// ZERO-LABEL: @test_volatileint_uninit()
+// ZERO: store volatile i32 0, i32* %uninit, align 4
 
 TEST_BRACES(volatileint, volatile int);
 // CHECK-LABEL: @test_volatileint_braces()
@@ -732,6 +1004,10 @@ TEST_UNINIT(semivolatile, semivolatile);
 // CHECK-LABEL: @test_semivolatile_uninit()
 // CHECK:       %uninit = alloca %struct.semivolatile, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_semivolatile_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_semivolatile_uninit.uninit
+// ZERO-LABEL: @test_semivolatile_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(semivolatile, semivolatile);
 // CHECK-LABEL: @test_semivolatile_braces()
@@ -777,6 +1053,10 @@ TEST_UNINIT(base, base);
 // CHECK:       %uninit = alloca %struct.base, align
 // CHECK-NEXT:  call void @{{.*}}base{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_base_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_base_uninit.uninit
+// ZERO-LABEL: @test_base_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(base, base);
 // CHECK-LABEL: @test_base_braces()
@@ -791,6 +1071,10 @@ TEST_UNINIT(derived, derived);
 // CHECK:       %uninit = alloca %struct.derived, align
 // CHECK-NEXT:  call void @{{.*}}derived{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_derived_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_derived_uninit.uninit
+// ZERO-LABEL: @test_derived_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(derived, derived);
 // CHECK-LABEL: @test_derived_braces()
@@ -805,6 +1089,10 @@ TEST_UNINIT(virtualderived, virtualderiv
 // CHECK:       %uninit = alloca %struct.virtualderived, align
 // CHECK-NEXT:  call void @{{.*}}virtualderived{{.*}}%uninit)
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_virtualderived_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_virtualderived_uninit.uninit
+// ZERO-LABEL: @test_virtualderived_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(virtualderived, virtualderived);
 // CHECK-LABEL: @test_virtualderived_braces()
@@ -819,6 +1107,10 @@ TEST_UNINIT(matching, matching);
 // CHECK-LABEL: @test_matching_uninit()
 // CHECK:       %uninit = alloca %union.matching, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_matching_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_matching_uninit.uninit
+// ZERO-LABEL: @test_matching_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(matching, matching);
 // CHECK-LABEL: @test_matching_braces()
@@ -838,6 +1130,10 @@ TEST_UNINIT(matchingreverse, matchingrev
 // CHECK-LABEL: @test_matchingreverse_uninit()
 // CHECK:       %uninit = alloca %union.matchingreverse, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_matchingreverse_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_matchingreverse_uninit.uninit
+// ZERO-LABEL: @test_matchingreverse_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(matchingreverse, matchingreverse);
 // CHECK-LABEL: @test_matchingreverse_braces()
@@ -857,6 +1153,10 @@ TEST_UNINIT(unmatched, unmatched);
 // CHECK-LABEL: @test_unmatched_uninit()
 // CHECK:       %uninit = alloca %union.unmatched, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_unmatched_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_unmatched_uninit.uninit
+// ZERO-LABEL: @test_unmatched_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(unmatched, unmatched);
 // CHECK-LABEL: @test_unmatched_braces()
@@ -876,6 +1176,10 @@ TEST_UNINIT(unmatchedreverse, unmatchedr
 // CHECK-LABEL: @test_unmatchedreverse_uninit()
 // CHECK:       %uninit = alloca %union.unmatchedreverse, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_unmatchedreverse_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_unmatchedreverse_uninit.uninit
+// ZERO-LABEL: @test_unmatchedreverse_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(unmatchedreverse, unmatchedreverse);
 // CHECK-LABEL: @test_unmatchedreverse_braces()
@@ -895,6 +1199,10 @@ TEST_UNINIT(unmatchedfp, unmatchedfp);
 // CHECK-LABEL: @test_unmatchedfp_uninit()
 // CHECK:       %uninit = alloca %union.unmatchedfp, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_unmatchedfp_uninit()
+// PATTERN: call void @llvm.memcpy{{.*}} @__const.test_unmatchedfp_uninit.uninit
+// ZERO-LABEL: @test_unmatchedfp_uninit()
+// ZERO: call void @llvm.memset{{.*}}, i8 0,
 
 TEST_BRACES(unmatchedfp, unmatchedfp);
 // CHECK-LABEL: @test_unmatchedfp_braces()
@@ -915,6 +1223,10 @@ TEST_UNINIT(emptyenum, emptyenum);
 // CHECK-LABEL: @test_emptyenum_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_emptyenum_uninit()
+// PATTERN: store i32 -1431655766, i32* %braces, align 4
+// ZERO-LABEL: @test_emptyenum_uninit()
+// ZERO: store i32 0, i32* %braces, align 4
 
 TEST_BRACES(emptyenum, emptyenum);
 // CHECK-LABEL: @test_emptyenum_braces()
@@ -932,6 +1244,10 @@ TEST_UNINIT(smallenum, smallenum);
 // CHECK-LABEL: @test_smallenum_uninit()
 // CHECK:       %uninit = alloca i32, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_smallenum_uninit()
+// PATTERN: store i32 -1431655766, i32* %braces, align 4
+// ZERO-LABEL: @test_smallenum_uninit()
+// ZERO: store i32 0, i32* %braces, align 4
 
 TEST_BRACES(smallenum, smallenum);
 // CHECK-LABEL: @test_smallenum_braces()
@@ -950,6 +1266,10 @@ TEST_UNINIT(intvec16, int  __attribute__
 // CHECK-LABEL: @test_intvec16_uninit()
 // CHECK:       %uninit = alloca <4 x i32>, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_intvec16_uninit()
+// PATTERN: store <4 x i32> <i32 -1431655766, i32 -1431655766, i32 -1431655766, i32 -1431655766>, <4 x i32>* %uninit, align 16
+// ZERO-LABEL: @test_intvec16_uninit()
+// ZERO: store <4 x i32> zeroinitializer, <4 x i32>* %uninit, align 16
 
 TEST_BRACES(intvec16, int  __attribute__((vector_size(16))));
 // CHECK-LABEL: @test_intvec16_braces()
@@ -957,7 +1277,7 @@ TEST_BRACES(intvec16, int  __attribute__
 // CHECK-NEXT:  store <4 x i32> zeroinitializer, <4 x i32>* %braces, align [[ALIGN]]
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%braces)
 
-  TEST_CUSTOM(intvec16, int  __attribute__((vector_size(16))), { 0x44444444, 0x44444444, 0x44444444, 0x44444444 });
+TEST_CUSTOM(intvec16, int  __attribute__((vector_size(16))), { 0x44444444, 0x44444444, 0x44444444, 0x44444444 });
 // CHECK-LABEL: @test_intvec16_custom()
 // CHECK:       %custom = alloca <4 x i32>, align [[ALIGN:[0-9]*]]
 // CHECK-NEXT:  store <4 x i32> <i32 1145324612, i32 1145324612, i32 1145324612, i32 1145324612>, <4 x i32>* %custom, align [[ALIGN]]
@@ -967,6 +1287,10 @@ TEST_UNINIT(longlongvec32, long long  __
 // CHECK-LABEL: @test_longlongvec32_uninit()
 // CHECK:       %uninit = alloca <4 x i64>, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_longlongvec32_uninit()
+// PATTERN: store <4 x i64> <i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206, i64 -6148914691236517206>, <4 x i64>* %uninit, align 32
+// ZERO-LABEL: @test_longlongvec32_uninit()
+// ZERO: store <4 x i64> zeroinitializer, <4 x i64>* %uninit, align 32
 
 TEST_BRACES(longlongvec32, long long  __attribute__((vector_size(32))));
 // CHECK-LABEL: @test_longlongvec32_braces()
@@ -984,6 +1308,10 @@ TEST_UNINIT(floatvec16, float  __attribu
 // CHECK-LABEL: @test_floatvec16_uninit()
 // CHECK:       %uninit = alloca <4 x float>, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_floatvec16_uninit()
+// PATTERN: store <4 x float> <float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000, float 0xFFFFFFFFE0000000>, <4 x float>* %uninit, align 16
+// ZERO-LABEL: @test_floatvec16_uninit()
+// ZERO: store <4 x float> zeroinitializer, <4 x float>* %uninit, align 16
 
 TEST_BRACES(floatvec16, float  __attribute__((vector_size(16))));
 // CHECK-LABEL: @test_floatvec16_braces()
@@ -1001,6 +1329,10 @@ TEST_UNINIT(doublevec32, double  __attri
 // CHECK-LABEL: @test_doublevec32_uninit()
 // CHECK:       %uninit = alloca <4 x double>, align
 // CHECK-NEXT:  call void @{{.*}}used{{.*}}%uninit)
+// PATTERN-LABEL: @test_doublevec32_uninit()
+// PATTERN: store <4 x double> <double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF, double 0xFFFFFFFFFFFFFFFF>, <4 x double>* %uninit, align 32
+// ZERO-LABEL: @test_doublevec32_uninit()
+// ZERO: store <4 x double> zeroinitializer, <4 x double>* %uninit, align 32
 
 TEST_BRACES(doublevec32, double  __attribute__((vector_size(32))));
 // CHECK-LABEL: @test_doublevec32_braces()

Added: cfe/trunk/test/CodeGenCXX/trivial-auto-var-init-attribute.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/trivial-auto-var-init-attribute.cpp?rev=349442&view=auto
==============================================================================
--- cfe/trunk/test/CodeGenCXX/trivial-auto-var-init-attribute.cpp (added)
+++ cfe/trunk/test/CodeGenCXX/trivial-auto-var-init-attribute.cpp Mon Dec 17 21:12:21 2018
@@ -0,0 +1,23 @@
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks %s -emit-llvm -o - | FileCheck %s -check-prefix=UNINIT
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -emit-llvm -o - | FileCheck %s -check-prefix=PATTERN
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -emit-llvm -o - | FileCheck %s -check-prefix=ZERO
+
+template<typename T> void used(T &) noexcept;
+
+extern "C" {
+
+// UNINIT-LABEL:  test_attribute_uninitialized(
+// UNINIT:      alloca
+// UNINIT-NEXT: call void
+// ZERO-LABEL:    test_attribute_uninitialized(
+// ZERO:      alloca
+// ZERO-NEXT: call void
+// PATTERN-LABEL: test_attribute_uninitialized(
+// PATTERN:      alloca
+// PATTERN-NEXT: call void
+void test_attribute_uninitialized() {
+  [[clang::uninitialized]] int i;
+  used(i);
+}
+
+} // extern "C"

Added: cfe/trunk/test/CodeGenCXX/trivial-auto-var-init.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/CodeGenCXX/trivial-auto-var-init.cpp?rev=349442&view=auto
==============================================================================
--- cfe/trunk/test/CodeGenCXX/trivial-auto-var-init.cpp (added)
+++ cfe/trunk/test/CodeGenCXX/trivial-auto-var-init.cpp Mon Dec 17 21:12:21 2018
@@ -0,0 +1,216 @@
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks %s -emit-llvm -o - | FileCheck %s -check-prefix=UNINIT
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=pattern %s -emit-llvm -o - | FileCheck %s -check-prefix=PATTERN
+// RUN: %clang_cc1 -triple x86_64-unknown-unknown -fblocks -ftrivial-auto-var-init=zero %s -emit-llvm -o - | FileCheck %s -check-prefix=ZERO
+
+// None of the synthesized globals should contain `undef`.
+// PATTERN-NOT: undef
+// ZERO-NOT: undef
+
+template<typename T> void used(T &) noexcept;
+
+extern "C" {
+
+// UNINIT-LABEL:  test_selfinit(
+// ZERO-LABEL:    test_selfinit(
+// ZERO: store i32 0, i32* %self, align 4
+// PATTERN-LABEL: test_selfinit(
+// PATTERN: store i32 -1431655766, i32* %self, align 4
+void test_selfinit() {
+  int self = self + 1;
+  used(self);
+}
+
+// UNINIT-LABEL:  test_block(
+// ZERO-LABEL:    test_block(
+// ZERO: store i32 0, i32* %block, align 4
+// PATTERN-LABEL: test_block(
+// PATTERN: store i32 -1431655766, i32* %block, align 4
+void test_block() {
+  __block int block;
+  used(block);
+}
+
+// This type of code is currently not handled by zero / pattern initialization.
+// The test will break when that is fixed.
+// UNINIT-LABEL:  test_goto_unreachable_value(
+// ZERO-LABEL:    test_goto_unreachable_value(
+// ZERO-NOT: store {{.*}}%oops
+// PATTERN-LABEL: test_goto_unreachable_value(
+// PATTERN-NOT: store {{.*}}%oops
+void test_goto_unreachable_value() {
+  goto jump;
+  int oops;
+ jump:
+  used(oops);
+}
+
+// This type of code is currently not handled by zero / pattern initialization.
+// The test will break when that is fixed.
+// UNINIT-LABEL:  test_goto(
+// ZERO-LABEL:    test_goto(
+// ZERO: if.then:
+// ZERO: br label %jump
+// ZERO: store i32 0, i32* %oops, align 4
+// ZERO: br label %jump
+// ZERO: jump:
+// PATTERN-LABEL: test_goto(
+// PATTERN: if.then:
+// PATTERN: br label %jump
+// PATTERN: store i32 -1431655766, i32* %oops, align 4
+// PATTERN: br label %jump
+// PATTERN: jump:
+void test_goto(int i) {
+  if (i)
+    goto jump;
+  int oops;
+ jump:
+  used(oops);
+}
+
+// This type of code is currently not handled by zero / pattern initialization.
+// The test will break when that is fixed.
+// UNINIT-LABEL:  test_switch(
+// ZERO-LABEL:    test_switch(
+// ZERO:      sw.bb:
+// ZERO-NEXT: store i32 0, i32* %oops, align 4
+// ZERO:      sw.bb1:
+// ZERO-NEXT: call void @{{.*}}used
+// PATTERN-LABEL: test_switch(
+// PATTERN:      sw.bb:
+// PATTERN-NEXT: store i32 -1431655766, i32* %oops, align 4
+// PATTERN:      sw.bb1:
+// PATTERN-NEXT: call void @{{.*}}used
+void test_switch(int i) {
+  switch (i) {
+  case 0:
+    int oops;
+    break;
+  case 1:
+    used(oops);
+  }
+}
+
+// UNINIT-LABEL:  test_vla(
+// ZERO-LABEL:    test_vla(
+// ZERO:  %[[SIZE:[0-9]+]] = mul nuw i64 %{{.*}}, 4
+// ZERO:  call void @llvm.memset{{.*}}(i8* align 16 %{{.*}}, i8 0, i64 %[[SIZE]], i1 false)
+// PATTERN-LABEL: test_vla(
+// PATTERN:  %vla.iszerosized = icmp eq i64 %{{.*}}, 0
+// PATTERN:  br i1 %vla.iszerosized, label %vla-init.cont, label %vla-setup.loop
+// PATTERN: vla-setup.loop:
+// PATTERN:  %[[SIZE:[0-9]+]] = mul nuw i64 %{{.*}}, 4
+// PATTERN:  %vla.begin = bitcast i32* %vla to i8*
+// PATTERN:  %vla.end = getelementptr inbounds i8, i8* %vla.begin, i64 %[[SIZE]]
+// PATTERN:  br label %vla-init.loop
+// PATTERN: vla-init.loop:
+// PATTERN:  %vla.cur = phi i8* [ %vla.begin, %vla-setup.loop ], [ %vla.next, %vla-init.loop ]
+// PATTERN:  call void @llvm.memcpy{{.*}} %vla.cur, {{.*}}@__const.test_vla.vla
+// PATTERN:  %vla.next = getelementptr inbounds i8, i8* %vla.cur, i64 4
+// PATTERN:  %vla-init.isdone = icmp eq i8* %vla.next, %vla.end
+// PATTERN:  br i1 %vla-init.isdone, label %vla-init.cont, label %vla-init.loop
+// PATTERN: vla-init.cont:
+// PATTERN:  call void @{{.*}}used
+void test_vla(int size) {
+  // Variable-length arrays can't have a zero size according to C11 6.7.6.2/5.
+  // Neither can they be negative-sized.
+  //
+  // We don't use the former fact because some code creates zero-sized VLAs and
+  // doesn't use them. clang makes these share locations with other stack
+  // values, which leads to initialization of the wrong values.
+  //
+  // We rely on the later fact because it generates better code.
+  //
+  // Both cases are caught by UBSan.
+  int vla[size];
+  int *ptr = vla;
+  used(ptr);
+}
+
+// UNINIT-LABEL:  test_struct_vla(
+// ZERO-LABEL:    test_struct_vla(
+// ZERO:  %[[SIZE:[0-9]+]] = mul nuw i64 %{{.*}}, 16
+// ZERO:  call void @llvm.memset{{.*}}(i8* align 16 %{{.*}}, i8 0, i64 %[[SIZE]], i1 false)
+// PATTERN-LABEL: test_struct_vla(
+// PATTERN:  %vla.iszerosized = icmp eq i64 %{{.*}}, 0
+// PATTERN:  br i1 %vla.iszerosized, label %vla-init.cont, label %vla-setup.loop
+// PATTERN: vla-setup.loop:
+// PATTERN:  %[[SIZE:[0-9]+]] = mul nuw i64 %{{.*}}, 16
+// PATTERN:  %vla.begin = bitcast %struct.anon* %vla to i8*
+// PATTERN:  %vla.end = getelementptr inbounds i8, i8* %vla.begin, i64 %[[SIZE]]
+// PATTERN:  br label %vla-init.loop
+// PATTERN: vla-init.loop:
+// PATTERN:  %vla.cur = phi i8* [ %vla.begin, %vla-setup.loop ], [ %vla.next, %vla-init.loop ]
+// PATTERN:  call void @llvm.memcpy{{.*}} %vla.cur, {{.*}}@__const.test_struct_vla.vla
+// PATTERN:  %vla.next = getelementptr inbounds i8, i8* %vla.cur, i64 16
+// PATTERN:  %vla-init.isdone = icmp eq i8* %vla.next, %vla.end
+// PATTERN:  br i1 %vla-init.isdone, label %vla-init.cont, label %vla-init.loop
+// PATTERN: vla-init.cont:
+// PATTERN:  call void @{{.*}}used
+void test_struct_vla(int size) {
+  // Same as above, but with a struct that doesn't just memcpy.
+  struct {
+    float f;
+    char c;
+    void *ptr;
+  } vla[size];
+  void *ptr = static_cast<void*>(vla);
+  used(ptr);
+}
+
+// UNINIT-LABEL:  test_zsa(
+// ZERO-LABEL:    test_zsa(
+// ZERO: %zsa = alloca [0 x i32], align 4
+// ZERO-NOT: %zsa
+// ZERO:  call void @{{.*}}used
+// PATTERN-LABEL: test_zsa(
+// PATTERN: %zsa = alloca [0 x i32], align 4
+// PATTERN-NOT: %zsa
+// PATTERN:  call void @{{.*}}used
+void test_zsa(int size) {
+  // Technically not valid, but as long as clang accepts them we should do
+  // something sensible (i.e. not store to the zero-size array).
+  int zsa[0];
+  used(zsa);
+}
+  
+// UNINIT-LABEL:  test_huge_uninit(
+// ZERO-LABEL:    test_huge_uninit(
+// ZERO: call void @llvm.memset{{.*}}, i8 0, i64 65536,
+// PATTERN-LABEL: test_huge_uninit(
+// PATTERN: call void @llvm.memset{{.*}}, i8 -86, i64 65536,
+void test_huge_uninit() {
+  // We can't emit this as an inline constant to a store instruction because
+  // SDNode hits an internal size limit.
+  char big[65536];
+  used(big);
+}
+
+// UNINIT-LABEL:  test_huge_small_init(
+// ZERO-LABEL:    test_huge_small_init(
+// ZERO: call void @llvm.memset{{.*}}, i8 0, i64 65536,
+// ZERO: store i8 97,
+// ZERO: store i8 98,
+// ZERO: store i8 99,
+// ZERO: store i8 100,
+// PATTERN-LABEL: test_huge_small_init(
+// PATTERN: call void @llvm.memset{{.*}}, i8 0, i64 65536,
+// PATTERN: store i8 97,
+// PATTERN: store i8 98,
+// PATTERN: store i8 99,
+// PATTERN: store i8 100,
+void test_huge_small_init() {
+  char big[65536] = { 'a', 'b', 'c', 'd' };
+  used(big);
+}
+
+// UNINIT-LABEL:  test_huge_larger_init(
+// ZERO-LABEL:    test_huge_larger_init(
+// ZERO:  call void @llvm.memcpy{{.*}} @__const.test_huge_larger_init.big, {{.*}}, i64 65536,
+// PATTERN-LABEL: test_huge_larger_init(
+// PATTERN:  call void @llvm.memcpy{{.*}} @__const.test_huge_larger_init.big, {{.*}}, i64 65536,
+void test_huge_larger_init() {
+  char big[65536] = { 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' };
+  used(big);
+}
+
+} // extern "C"

Modified: cfe/trunk/test/Driver/clang_f_opts.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Driver/clang_f_opts.c?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/test/Driver/clang_f_opts.c (original)
+++ cfe/trunk/test/Driver/clang_f_opts.c Mon Dec 17 21:12:21 2018
@@ -557,3 +557,12 @@
 // CHECK-RECORD-GCC-SWITCHES: "-record-command-line"
 // CHECK-NO-RECORD-GCC-SWITCHES-NOT: "-record-command-line"
 // CHECK-RECORD-GCC-SWITCHES-ERROR: error: unsupported option '-frecord-command-line' for target
+
+// RUN: %clang -### -S -ftrivial-auto-var-init=uninitialized %s 2>&1 | FileCheck -check-prefix=CHECK-TRIVIAL-UNINIT %s
+// RUN: %clang -### -S -ftrivial-auto-var-init=pattern %s 2>&1 | FileCheck -check-prefix=CHECK-TRIVIAL-PATTERN %s
+// RUN: %clang -### -S -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang %s 2>&1 | FileCheck -check-prefix=CHECK-TRIVIAL-ZERO-GOOD %s
+// RUN: %clang -### -S -ftrivial-auto-var-init=zero %s 2>&1 | FileCheck -check-prefix=CHECK-TRIVIAL-ZERO-BAD %s
+// CHECK-TRIVIAL-UNINIT-NOT: hasn't been enabled
+// CHECK-TRIVIAL-PATTERN-NOT: hasn't been enabled
+// CHECK-TRIVIAL-ZERO-GOOD-NOT: hasn't been enabled
+// CHECK-TRIVIAL-ZERO-BAD: hasn't been enabled

Added: cfe/trunk/test/Sema/attr-uninitialized.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Sema/attr-uninitialized.c?rev=349442&view=auto
==============================================================================
--- cfe/trunk/test/Sema/attr-uninitialized.c (added)
+++ cfe/trunk/test/Sema/attr-uninitialized.c Mon Dec 17 21:12:21 2018
@@ -0,0 +1,21 @@
+// RUN: %clang_cc1 %s -verify -fsyntax-only
+
+void good() {
+  int dont_initialize_me __attribute((uninitialized));
+}
+
+void bad() {
+  int im_bad __attribute((uninitialized("zero")));  // expected-error {{'uninitialized' attribute takes no arguments}}
+  static int im_baaad __attribute((uninitialized)); // expected-warning {{'uninitialized' attribute only applies to local variables}}
+}
+
+extern int come_on __attribute((uninitialized));                    // expected-warning {{'uninitialized' attribute only applies to local variables}}
+int you_know __attribute((uninitialized));                          // expected-warning {{'uninitialized' attribute only applies to local variables}}
+static int and_the_whole_world_has_to __attribute((uninitialized)); // expected-warning {{'uninitialized' attribute only applies to local variables}}
+
+void answer_right_now() __attribute((uninitialized)) {}                        // expected-warning {{'uninitialized' attribute only applies to local variables}}
+void just_to_tell_you_once_again(__attribute((uninitialized)) int whos_bad) {} // expected-warning {{'uninitialized' attribute only applies to local variables}}
+
+struct TheWordIsOut {
+  __attribute((uninitialized)) int youre_doin_wrong; // expected-warning {{'uninitialized' attribute only applies to local variables}}
+} __attribute((uninitialized));                      // expected-warning {{'uninitialized' attribute only applies to local variables}}

Modified: cfe/trunk/test/Sema/uninit-variables.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Sema/uninit-variables.c?rev=349442&r1=349441&r2=349442&view=diff
==============================================================================
--- cfe/trunk/test/Sema/uninit-variables.c (original)
+++ cfe/trunk/test/Sema/uninit-variables.c Mon Dec 17 21:12:21 2018
@@ -1,4 +1,5 @@
 // RUN: %clang_cc1 -fsyntax-only -Wuninitialized -Wconditional-uninitialized -fsyntax-only -fblocks %s -verify
+// RUN: %clang_cc1 -fsyntax-only -Wuninitialized -Wconditional-uninitialized -ftrivial-auto-var-init=pattern -fsyntax-only -fblocks %s -verify
 
 typedef __typeof(sizeof(int)) size_t;
 void *malloc(size_t);




More information about the cfe-commits mailing list