<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">We currently do not document the automatic initialization flags because we haven’t had customers ask for it through our official channels.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">You are correct though, if we shipped this to customers they could do whatever they wanted with the flags. I’m simply stating how we are using the automatic initialization in our build environment.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">From my perspective, all compilers already handle undefined behavior differently. i.e. in MSVC we don’t optimize away signed overflows in many cases due to security consequences of doing so. If people want to take a dependency on undefined
behavior in the compiler they already can (and this makes this code less portable). We can give them the tools to avoid it though. In this case, if someone wants to make their code completely unportable by depending on zero init they can. It is their code
though so I don’t really feel too bad about that. The recommendation should be to not take this dependency though, and to use different pattern settings for debug/retail builds to ensure you aren’t taking a dependency.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Clang could also ban zero-init for debug builds and only allow it for retail. This does potentially make it harder to debug actual issues though.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Joe<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> David Blaikie <dblaikie@gmail.com> <br>
<b>Sent:</b> Wednesday, April 22, 2020 9:56 AM<br>
<b>To:</b> Joe Bialek <jobialek@microsoft.com><br>
<b>Cc:</b> Kees Cook <keescook@chromium.org>; Hubert Tong <hubert.reinterpretcast@gmail.com>; Clang Dev <cfe-dev@lists.llvm.org><br>
<b>Subject:</b> Re: [cfe-dev] [EXTERNAL] Re: making -ftrivial-auto-var-init=zero a first-class option<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Tue, Apr 21, 2020 at 9:18 PM Joe Bialek via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org">cfe-dev@lists.llvm.org</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal">To add in, we (Microsoft) currently use zero initialization technology in Visual Studio in a large amount of production code we ship to customers (all kernel components, a number of user-mode components). This code is both C and C++.<br>
<br>
We already have had multiple vulnerabilities killed because we shipped this technology in production. We received bug reports with repros that worked on older versions of Windows without the mitigation and new versions of Windows that do have it. The new versions
don't repro, the old ones do.<br>
<br>
Using this sort of technology only in development (and not in production) is not sufficient. Some of these bugs will never produce crashes, the uninitialized data is copied across a trust boundary (i.e. from the kernel in to a untrusted user-mode process).
This will never result in a crash but does result in a security issue. This is why shipping in production is a requirement even if you had perfect test coverage that exercises all code paths (which nobody has).<br>
<br>
We do enable pattern initialization for debug builds and internal retail builds (using a developer mode in the build environment). We do this to help prevent "forking of the language" and also to force non-determinism. If your code relies on the zero-init,
it won't work when we do pattern init. If your code only works with a non-zero value but doesn't care what that value is (Booleans, certain bit tests, etc.), it won't work with zero-init. Developers cannot depend on the automatic initialization for program
correctness.<o:p></o:p></p>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">How is this, as a compiler feature, shipped to users? Users don't have a direct toggle available, but they say "I want memory init" with some compiler flag, then if they're building in debug (any unoptimized build, or only those with debug
information emitted?) that translates to pattern, and in a build with any optimizations enabled (even the lowest level) they get zero init?<br>
<br>
While in a constrained environment within a company I can see how you could hold that bar - once you ship that to end users I'd be afraid that some of them would figure out the least-optimizing (most debuggable) way to get zero init and avoid the pattern init
behavior & write code that depended on that zero init - thus forking the language.<br>
<br>
- Dave<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal"><br>
Joe<br>
<br>
-----Original Message-----<br>
From: Kees Cook <<a href="mailto:keescook@chromium.org" target="_blank">keescook@chromium.org</a>>
<br>
Sent: Tuesday, April 21, 2020 4:20 PM<br>
To: Hubert Tong <<a href="mailto:hubert.reinterpretcast@gmail.com" target="_blank">hubert.reinterpretcast@gmail.com</a>><br>
Cc: Clang Dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>>; Joe Bialek <<a href="mailto:jobialek@microsoft.com" target="_blank">jobialek@microsoft.com</a>><br>
Subject: [EXTERNAL] Re: [cfe-dev] making -ftrivial-auto-var-init=zero a first-class option<br>
<br>
On Tue, Apr 21, 2020 at 06:29:07PM -0400, Hubert Tong wrote:<br>
> On Tue, Apr 21, 2020 at 5:20 PM Kees Cook via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br>
> > <a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcl" target="_blank">
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcl</a><br>
> > <a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fangbuiltlinux.github.io%2F&data=02%7C01%7Cjobialek%40microsoft.com%7C781cef573d6548f6066a08d7e6de16ab%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637231714923718204&sdata=J5a%2B038vvTIxuhVZCdQo6DHP4K1N0tknWEqmuM2ZGX0%3D&reserved=0" target="_blank">
angbuiltlinux.github.io</a>%2FCBL-meetup-2020-slides%2Fglider%2FFighting<br>
> > _uninitialized_memory_%2540_CBL_Meetup_2020.pdf&data=02%7C01%7Cj<br>
> > obialek%<a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40microsoft.com%2F&data=02%7C01%7Cjobialek%40microsoft.com%7C781cef573d6548f6066a08d7e6de16ab%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637231714923728200&sdata=9kjsGbW2k9IvCpwxKa7p64nwnltKEr8Oelr1Zu8t7Mg%3D&reserved=0" target="_blank">40microsoft.com</a>%7C4bce6c76554b4dcf4c2b08d7e64a7848%7C72f988b<br>
> > f86f141af91ab2d7cd011db47%7C1%7C0%7C637231080235713420&sdata=uMZ<br>
> > PAiQvnjfLxORQNOAdUHGLY8czk8Mlsxc8dXpLSYg%3D&reserved=0<br>
> > tl;dr: zero-init tended to be half the cost of pattern-init, though <br>
> > it varied based on workload, and binary size impact fell over 95% <br>
> > going from pattern-init to zero-init.<br>
> ><br>
> This does not seem to indicate why zero-init is preferred over a <br>
> default of using no explicit policy in production.<br>
<br>
Do you mean "leaving things uninitialized" when you say "no explicit policy"? Maybe I've misunderstood. Google's goal of using auto-init is to eliminate uninitialized variables in production as a security defense. When examining zero-init vs pattern-init, there
is a clear advantage on performance and size for zero-init.<br>
<br>
> > as good. There are certainly exceptions here, but the bulk of the <br>
> > historical record on how "uninitialized" variables have been used in<br>
> ><br>
> Maybe an explanation of the scare quotes around "uninitialized" would <br>
> help clarify your position.<br>
<br>
Ah, sorry, I always use quotes (they are not intended to scare but to<br>
clarify) when discussing uninitialized variables in real-world contexts, because they are, of course, not uninitialized in the sense of them not having a value. The RAM contents have a value. Many people without compiler backgrounds think of such variables
as being uncontrollable or meaningless, when in fact they are usually highly controllable by an attacker, etc.<br>
<br>
> > Google (Android, Chrome OS)<br>
> ><br>
> > Both Android and Chrome OS initially started using pattern-init, but <br>
> > due to each of: the performance characteristics, the binary size <br>
> > changes, and the less robust security stance, both projects have <br>
> > recently committed to switching to zero-init.<br>
> ><br>
> I'm not sure that this is clear in terms of whether the statements <br>
> apply to debug/development or production. I don't think pattern-init <br>
> is meant to be a tool for production builds, which leads me to think <br>
> that the above statement is about debug builds, at which point I'm <br>
> thinking that using zero-init only serves to hide problems.<br>
<br>
The context for Google's use of zero-init was meant here to be about production builds.<br>
<br>
> > Upstream Linux kernel<br>
> ><br>
> > Linus Torvalds has directly stated that he wants zero-init:<br>
> > "So I'd like the zeroing of local variables to be a native compiler <br>
> > option..."<br>
> > "This, btw, is why I also think that the "initialize with poison" is <br>
> > pointless and wrong."<br>
> ><br>
> > <a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flo" target="_blank">
https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flo</a><br>
> > <a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fre.kernel.org%2F&data=02%7C01%7Cjobialek%40microsoft.com%7C781cef573d6548f6066a08d7e6de16ab%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637231714923728200&sdata=DeChhM5lA7Fe%2F9FEtvdlQoC%2Ft7DWyyTLPLjXPwkTPo8%3D&reserved=0" target="_blank">
re.kernel.org</a>%2Flkml%2FCAHk-%3DwgTM%2BcN7zyUZacGQDv3DuuoA4LORNPWgb1Y<br>
> > _Z1p4iedNQ%<a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2F40mail.gmail.com%2F&data=02%7C01%7Cjobialek%40microsoft.com%7C781cef573d6548f6066a08d7e6de16ab%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637231714923738196&sdata=mhHTUphR%2BGDt1yZYKPgdxE9SY8dvJFaZ3CVN2xW11LM%3D&reserved=0" target="_blank">40mail.gmail.com</a>%2F&data=02%7C01%7Cjobialek%40microso<br>
> > <a href="https://nam06.safelinks.protection.outlook.com/?url=http%3A%2F%2Fft.com%2F&data=02%7C01%7Cjobialek%40microsoft.com%7C781cef573d6548f6066a08d7e6de16ab%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637231714923748191&sdata=QEjJwPKL1Tni68%2FtdomtwPclj%2FmlI%2BfTrzOpB6O0jdA%3D&reserved=0" target="_blank">
ft.com</a>%7C4bce6c76554b4dcf4c2b08d7e64a7848%7C72f988bf86f141af91ab2d7c<br>
> > d011db47%7C1%7C0%7C637231080235723408&sdata=256CyAUusLIf8IetQfyd<br>
> > 3KNAlIqVwV8uvjuPc6daP14%3D&reserved=0<br>
> > Unsurprisingly, I strongly agree. ;)<br>
> ><br>
> I don't see why claiming that pattern-init is bad helps make the case <br>
> for zero-init.<br>
<br>
Perhaps I did not express it well enough, but both have meaningful and important uses. My goal here is to illustrate how zero-init is being used (or preferred) in many real situations, as an argument for why it should not be hidden behind what some might see
as a scary enable flag.<br>
<br>
> > Apple<br>
> ><br>
> > I can't speak meaningfully here, but I've heard rumors that they are <br>
> > depending on zero-init as well. Perhaps someone there can clarify <br>
> > how they are using these features?<br>
> ><br>
> There's a difference between "depending on zero-init" (as in, the <br>
> group in question is okay with relying on implicit zeroing on code <br>
> reviews, etc.) and the use of zero-init as some sort of <br>
> defence-in-depth approach. Are these rumours clear as to which?<br>
<br>
My understanding was the latter, but I hope to find out for real via this thread! :) It's not clear to me either.<br>
<br>
> > So, while I understand the earlier objections to zero-init from a <br>
> > "language fork" concern, I think this isn't a position that can <br>
> > really stand up to the reality of how many projects are using the <br>
> > feature (even via non-Clang compilers). Given that so much code is <br>
> > going to be built using zero-init, what's the best way for Clang to adapt here?<br>
> <br>
> It happens that there is zero-init and it's at least close enough to <br>
> what these projects want, but it is actually what they need?<br>
<br>
Yes, it's expressly what is desired from a security perspective. (And quite to the relief of that same community, comes with the least performance impact, which is an unfortunately uncommon scenario in security flaw mitigations.) I tried to detail that earlier
in my email where it's directly what is indicated as a meaningful defense against the long history of real-world "uninitialized" variable attacks: setting everything to zero is the best defense for the entire class of flaws.<br>
<br>
--<br>
Kees Cook<br>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.llvm.org%2Fcgi-bin%2Fmailman%2Flistinfo%2Fcfe-dev&data=02%7C01%7Cjobialek%40microsoft.com%7C781cef573d6548f6066a08d7e6de16ab%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637231714923748191&sdata=mfl5z9S2SsPiAUqxbbmwC40VdQWpoq%2BST5hvhTbKm28%3D&reserved=0" target="_blank">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</body>
</html>