<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Mar 26, 2015, at 6:54 PM, Chandler Carruth <<a href="mailto:chandlerc@google.com" class="">chandlerc@google.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class="">FYI, as of r233348, ever source of non-deterministic output I can find by inspection and some targeted stress testing on the C++ side is fixed. I suspect we're pretty clase.</div></div></blockquote><div><br class=""></div>Woo!</div><div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">Ben, if you want to add test cases for objc stuff, that'd be awesome. I just fixed it all by inspection.</div></div></div></blockquote><div><br class=""></div><div>Okay, it will probably be a few days before I have time for this. However, as a quick check I turned off signatures and emitting the mtime for implicit modueles and built our Cocoa module and its dependencies a few times. It looks like 22/27 of the modules built identically each time, and the other 5 were reliably different so it should be easy to figure out what’s still having trouble. If you’re interested, a quick sample showed the differences came from the order of DECL_OBJC_METHOD and DECL_OBJC_PROPERTY records. I'll look into that when I start adding tests.</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">I think that what we should do is add a (potentially optional and only produced by an asserts build of Clang) record to the serialized AST which is a digest (MD5 or some such) of the entire AST up to that record. We could add this to the low layer of the AST writing so it would be easy to plumb through. This would make an actually *good* signature record and we could check it to ensure we don't waste time trying to debug failures if non-deterministic behavior creeps in again. Ben, could you look at making that change? I think that would be a *much* better design for the signature than a random number, and would serve the exact same purpose.</div></div></div></blockquote><div><br class=""></div><div>Richard and I previously discussed having the signature be a hash of:</div><div>a) the input files to the current module (path, size, modtime)</div><div>b) the signature of all the imports</div><div><br class=""></div><div>which should be cheap enough to build all the time. Having a separate more expensive hash of the whole AST in asserts builds would be good too.</div><div><br class=""></div><div>Ben</div><div><br class=""></div><blockquote type="cite" class=""><div class=""><div class="gmail_extra"><br class=""><div class="gmail_quote">On Tue, Mar 24, 2015 at 2:21 PM, Chandler Carruth <span dir="ltr" class=""><<a href="mailto:chandlerc@google.com" target="_blank" class="">chandlerc@google.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class=""><div class="gmail_extra"><span class=""><br class=""><div class="gmail_quote">On Mon, Mar 23, 2015 at 10:37 PM, Ben Langmuir <span dir="ltr" class=""><<a href="mailto:blangmuir@apple.com" target="_blank" class="">blangmuir@apple.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=""><blockquote type="cite" class=""><div class="">On Mar 23, 2015, at 10:21 PM, Chandler Carruth <<a href="mailto:chandlerc@google.com" target="_blank" class="">chandlerc@google.com</a>> wrote:</div><br class=""><div class=""><br class=""><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">On Mon, Mar 23, 2015 at 9:54 PM, Ben Langmuir<span class=""> </span></span><span dir="ltr" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><<a href="mailto:blangmuir@apple.com" target="_blank" class="">blangmuir@apple.com</a>></span><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class=""> </span><span style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;float:none;display:inline!important" class="">wrote:</span><br style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""><blockquote class="gmail_quote" style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px;margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><span class=""><blockquote type="cite" class=""><div class="">On Mar 23, 2015, at 5:38 PM, Chandler Carruth <<a href="mailto:chandlerc@google.com" target="_blank" class="">chandlerc@google.com</a>> wrote:</div><br class=""><div class=""><div dir="ltr" class="">Digging this thread back up, I'm starting to work on <span class="">deterministic</span> output of <span class="">modules</span>. This is *really* important. Unfortunately, it's almost impossible to make progress as long as this code is actually firing because it kind of makes things definitively non-<span class="">deterministic</span>. ;]</div></div></blockquote><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class=""><br class=""></div><div class="">Ben, do you have any concerns about only emitting the signature record for implicit <span class="">modules</span>? This would omit it for explicit <span class="">modules</span> (where I'm working), PCH files, etc. Based on your commit message and explanation, it doesn't really make sense outside of implicit <span class="">modules</span> AFAICT, and we're hoping it will go away completely.</div></div></div></blockquote><div class=""><br class=""></div></span><div class="">SGTM although I would prefer we continue to check the /expected/ signature when an AST file imports an implicit <span class="">module</span> (and in particular when a PCH imports an implicit <span class="">module</span>). Thanks for working on this!</div></blockquote><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class=""> </div><div style="font-family:Helvetica;font-size:12px;font-style:normal;font-variant:normal;font-weight:normal;letter-spacing:normal;line-height:normal;text-align:start;text-indent:0px;text-transform:none;white-space:normal;word-spacing:0px" class="">I'm not planning to change the checking code path at all. All the tests pass if I just restrict the emission to be when generating implicit <span class="">modules</span></div></div></blockquote><div class=""><br class=""></div></span><div class="">Perfect, that’s what I was hoping for. Sorry I didn’t phrase it clearly.</div></blockquote></div><br class=""></span>This is in as part of r233115 with the minimal test case for deterministic output. More to come.</div></div>
</blockquote></div><br class=""></div>
</div></blockquote></div><br class=""></body></html>