
<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class=""><br class=""><div><br class=""><blockquote type="cite" class=""><div class="">On Apr 21, 2020, at 3:29 PM, Hubert Tong via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" class="">cfe-dev@lists.llvm.org</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, Apr 21, 2020 at 5:20 PM Kees Cook via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" class="">cfe-dev@lists.llvm.org</a>> wrote:<br class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">Hi,<br class=""><br class="">tl;dr: I'd like to revisit making -ftrivial-auto-var-init=zero an expressly<br class="">supported option. To do this, I think we need to either entirely remove<br class="">"-enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang"<br class="">or rename it to something more directly reflecting the issue, like<br class="">"-enable-trivial-auto-var-init-zero-knowing-it-forks-the-language".<br class=""><br class="">This is currently open as<span class="Apple-converted-space"> </span><a href="https://bugs.llvm.org/show_bug.cgi?id=45497" rel="noreferrer" target="_blank" class="">https://bugs.llvm.org/show_bug.cgi?id=45497</a><br class=""><br class="">Here is the situation: -ftrivial-auto-var-init=pattern is great for<br class="">debugging, but -ftrivial-auto-var-init=zero is needed for production<br class="">systems for mainly two reasons, each of which I will try to express context<br class="">for:<br class=""><br class="">1) performance and size<br class=""><br class="">As measured by various Google folks across a few projects and in<br class="">various places, there's a fairly significant performance impact of<br class="">using pattern-init over zero-init. I can let other folks chime in<br class="">with their exact numbers, but I can at least share some measurements<br class="">Alexander Potapenko made with the Linux kernel (see "Performance costs"):<br class=""><a href="https://clangbuiltlinux.github.io/CBL-meetup-2020-slides/glider/Fighting_uninitialized_memory_%40_CBL_Meetup_2020.pdf" rel="noreferrer" target="_blank" class="">https://clangbuiltlinux.github.io/CBL-meetup-2020-slides/glider/Fighting_uninitialized_memory_%40_CBL_Meetup_2020.pdf</a><br class="">tl;dr: zero-init tended to be half the cost of pattern-init, though it<br class="">varied based on workload, and binary size impact fell over 95% going<br class="">from pattern-init to zero-init.<br class=""></blockquote><div class="">This does not seem to indicate why zero-init is preferred over a default of using no explicit policy in production.<br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class="">2) security<br class=""><br class="">Another driving factor (see below from various vendors/projects), is the<br class="">security stance. Putting non-zero values into most variables types ends<br class="">up making them arguably more dangerous than if they were zero-filled.<br class="">Most notably, sizes and indexes and less likely to be used out of bounds<br class="">if they are zero-initialized. The same holds for bool values that tend<br class="">to indicate success instead of failing safe with a false value. While<br class="">pointers in the non-canonical range are nice, zero tends to be just<br class="">as good. There are certainly exceptions here, but the bulk of the<br class="">historical record on how "uninitialized" variables have been used in<br class=""></blockquote><div class="">Maybe an explanation of the scare quotes around "uninitialized" would help clarify your position.<br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">real world exploitation involve their being non-zero, and analysis of<br class="">those bugs support that conclusion.<br class=""></blockquote><div class=""></div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class=""><br class="">Various positions from vendors and projects:<br class=""><br class="">Google (Android, Chrome OS)<br class=""><br class="">Both Android and Chrome OS initially started using pattern-init, but due<br class="">to each of: the performance characteristics, the binary size changes, and<br class="">the less robust security stance, both projects have recently committed<br class="">to switching to zero-init.<br class=""></blockquote><div class="">I'm not sure that this is clear in terms of whether the statements apply to debug/development or production. I don't think pattern-init is meant to be a tool for production builds, which leads me to think that the above statement is about debug builds, at which point I'm thinking that using zero-init only serves to hide problems.</div></div></div></div></blockquote><div><br class=""></div><div>The entire feature (including pattern init) is exactly designed to be a tool for production builds. It is in production at Apple as well as Google (not just their servers, but their devices as well), and I’ve heard the same (privately) from many others, bit and small. I covered why in my LLVM dev meeting talk: <a href="https://www.youtube.com/watch?v=I-XUHPimq3o" class="">https://www.youtube.com/watch?v=I-XUHPimq3o</a></div><div><br class=""></div><div>I’ve also written this out in details in prior discussions. I can repeat here if needed, but I’m not sure more text is helpful at the moment :-)</div><div><br class=""></div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><div class="gmail_quote"><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">Microsoft (Windows)<br class=""><br class="">I'm repeating what Joe Bialek has told me, so he can clarify if I'm not<br class="">representing this correctly... While not using Clang/LLVM, Microsoft is<br class="">part of the larger C/C++ ecosystem and has implemented both zero-init<br class="">(for production builds) and pattern-init (for debug builds) in their<br class="">compiler too. They also chose zero-init for production expressly due<br class="">to the security benefits.<br class=""><br class="">Some details of their work:<br class=""><a href="https://github.com/microsoft/MSRC-Security-Research/blob/master/presentations/2019_09_CppCon/CppCon2019%20-%20Killing%20Uninitialized%20Memory.pdf" rel="noreferrer" target="_blank" class="">https://github.com/microsoft/MSRC-Security-Research/blob/master/presentations/2019_09_CppCon/CppCon2019%20-%20Killing%20Uninitialized%20Memory.pdf</a><br class=""><br class=""><br class="">Upstream Linux kernel<br class=""><br class="">Linus Torvalds has directly stated that he wants zero-init:<br class="">"So I'd like the zeroing of local variables to be a native compiler<br class="">option..."<br class="">"This, btw, is why I also think that the "initialize with poison" is<br class="">pointless and wrong."<br class=""><a href="https://lore.kernel.org/lkml/CAHk-=wgTM+cN7zyUZacGQDv3DuuoA4LORNPWgb1Y_Z1p4iedNQ@mail.gmail.com/" rel="noreferrer" target="_blank" class="">https://lore.kernel.org/lkml/CAHk-=wgTM+cN7zyUZacGQDv3DuuoA4LORNPWgb1Y_Z1p4iedNQ@mail.gmail.com/</a><br class="">Unsurprisingly, I strongly agree. ;)<br class=""></blockquote><div class="">I don't see why claiming that pattern-init is bad helps make the case for zero-init.<br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class=""><br class="">GrapheneOS is using zero-init (rather than patching Clang as it used to, to get<br class="">the same result):<br class=""><a href="https://twitter.com/DanielMicay/status/1248384468181643272" rel="noreferrer" target="_blank" class="">https://twitter.com/DanielMicay/status/1248384468181643272</a><br class=""><br class=""><br class="">GCC<br class="">There's been mostly silence on the entire topic of automatic variable<br class="">initialization, though there have been patches proposed in the past for<br class="">zero-init:<br class=""><a href="https://gcc.gnu.org/legacy-ml/gcc-patches/2014-06/msg00615.html" rel="noreferrer" target="_blank" class="">https://gcc.gnu.org/legacy-ml/gcc-patches/2014-06/msg00615.html</a><br class=""><br class=""><br class="">Apple<br class=""><br class="">I can't speak meaningfully here, but I've heard rumors that they are<br class="">depending on zero-init as well. Perhaps someone there can clarify how<br class="">they are using these features?<br class=""></blockquote><div class="">There's a difference between "depending on zero-init" (as in, the group in question is okay with relying on implicit zeroing on code reviews, etc.) and the use of zero-init as some sort of defence-in-depth approach. Are these rumours clear as to which?<br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;"><br class=""><br class=""><br class="">So, while I understand the earlier objections to zero-init from a<br class="">"language fork" concern, I think this isn't a position that can really<br class="">stand up to the reality of how many projects are using the feature (even<br class="">via non-Clang compilers). Given that so much code is going to be built<br class="">using zero-init, what's the best way for Clang to adapt here?</blockquote><div class="">It happens that there is zero-init and it's at least close enough to what these projects want, but it is actually what they need?<br class=""></div><div class=""> </div><blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-style: solid; border-left-color: rgb(204, 204, 204); padding-left: 1ex;">I would<br class="">prefer to just drop the -enable... option entirely, but I think just<br class="">renaming it would be fine too.<br class=""><br class="">Thoughts/flames? ;)<br class=""><br class="">--<span class="Apple-converted-space"> </span><br class="">Kees Cook<br class="">_______________________________________________<br class="">cfe-dev mailing list<br class=""><a href="mailto:cfe-dev@lists.llvm.org" target="_blank" class="">cfe-dev@lists.llvm.org</a><br class=""><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br class=""></blockquote></div></div><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">_______________________________________________</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><span style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none; float: none; display: inline !important;" class="">cfe-dev mailing list</span><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="mailto:cfe-dev@lists.llvm.org" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">cfe-dev@lists.llvm.org</a><br style="caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class=""><a href="https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-size-adjust: auto; -webkit-text-stroke-width: 0px;" class="">https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a></div></blockquote></div><br class=""></body></html>