<div dir="ltr"><br><div class="gmail_extra"><br><br><div class="gmail_quote">On Fri, Nov 8, 2013 at 1:09 PM, Alp Toker <span dir="ltr"><<a href="mailto:alp@nuanti.com" target="_blank" class="cremed">alp@nuanti.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="im"><br>
On 08/11/2013 20:34, Daniel Jasper wrote:<br>
> The patch itself looks good (I don't know why we didn't do that in the<br>
> first place).<br>
><br>
><br>
> On Fri, Nov 8, 2013 at 12:07 AM, Alp Toker <<a href="mailto:alp@nuanti.com" class="cremed">alp@nuanti.com</a><br>
</div><div class="im">> <mailto:<a href="mailto:alp@nuanti.com" class="cremed">alp@nuanti.com</a>>> wrote:<br>
><br>
> Author: alp<br>
> Date: Fri Nov 8 02:07:19 2013<br>
> New Revision: 194250<br>
><br>
> URL: <a href="http://llvm.org/viewvc/llvm-project?rev=194250&view=rev" target="_blank" class="cremed">http://llvm.org/viewvc/llvm-project?rev=194250&view=rev</a><br>
> Log:<br>
> clang-format: Write files atomically<br>
><br>
> Switch clang-format over to Rewriter::overwriteChangedFiles().<br>
><br>
> The previous implementation was attempting to stream back directly<br>
> to the<br>
> original file and failing if it was already memory mapped by<br>
> MemoryBuffer,<br>
> an operation unsupported by Windows.<br>
><br>
> MemoryBuffer generally mmaps files larger than the physical page<br>
> size so<br>
> this will have been difficult to reproduce consistently.<br>
><br>
> This change also reduces flicker in code editors and IDEs on all<br>
> platforms<br>
> when reformatting in-place.<br>
><br>
><br>
> I think it is generally a bad idea for IDEs to use in-place<br>
> reformatting, but ok ..<br>
<br>
</div>This was referring to running clang-format on the commandline while you<br>
have files open in XCode. From time to time it went wrong, especially<br>
with unsaved changes.<br>
<div class="im"><br>
><br>
> Note that other incorrect uses of MemoryBuffer exist in LLVM/clang and<br>
> will need a similar fix.<br>
><br>
> A test should be added for Windows when libFormat performance<br>
> issues are<br>
> fixed (it takes longer than a day to format a 1MB file at present!)<br>
><br>
><br>
> Is it the write that takes so long (i.e. fixed with this patch)?<br>
><br>
> If not, can the file be shared (or changed enough so that it can be<br>
> shared)?<br>
<br>
</div>This happened with various internal sources pasted together, but I think<br>
it can also be reproduced with sqlite.c:<br>
<br>
<a href="http://www.sqlite.org/2013/sqlite-amalgamation-3080100.zip" target="_blank" class="cremed">http://www.sqlite.org/2013/sqlite-amalgamation-3080100.zip</a><br>
<br>
(My RewriteBuffer patch doesn't help with this one.)<br></blockquote><div><br></div><div>Thanks for the test case. The problem here is the (relatively) new logic we have to format complex #if/#else branches correctly even if they cut through statements. The file is not only large, but also has deeply nested #if/#else combinations (6+ levels). With this feature turned off, the file formats in ~10sec, which IMO is reasonable for a file of that size.</div>
<div><br></div><div>There are two routes forward:<br>a) There are quite a few low-hanging fruits in optimizing our current approach. Quite probably those will be sufficient.</div><div>b) Turn off this formatting feature for very large files as it only improves corner cases (possibly not even happing in this file).</div>
<div><br></div><div>We'll look into it (not top priority at the moment, though).</div><div><br></div><div>Cheers,</div><div>Daniel</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
For now I'm continuing to track down the MemoryBuffer issues in other<br>
parts of LLVM making the same mistake as clang-format.cpp. So won't be<br>
looking at this performance issue for a while -- feel free to pick it up..<br>
<br>
Alp.<br>
<div><div class="h5"><br>
><br>
> Modified:<br>
> cfe/trunk/tools/clang-format/ClangFormat.cpp<br>
><br>
> Modified: cfe/trunk/tools/clang-format/ClangFormat.cpp<br>
> URL:<br>
> <a href="http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-format/ClangFormat.cpp?rev=194250&r1=194249&r2=194250&view=diff" target="_blank" class="cremed">http://llvm.org/viewvc/llvm-project/cfe/trunk/tools/clang-format/ClangFormat.cpp?rev=194250&r1=194249&r2=194250&view=diff</a><br>
> ==============================================================================<br>
> --- cfe/trunk/tools/clang-format/ClangFormat.cpp (original)<br>
> +++ cfe/trunk/tools/clang-format/ClangFormat.cpp Fri Nov 8<br>
> 02:07:19 2013<br>
> @@ -213,18 +213,8 @@ static bool format(std::string FileName)<br>
> Rewriter Rewrite(Sources, LangOptions());<br>
> tooling::applyAllReplacements(Replaces, Rewrite);<br>
> if (Inplace) {<br>
> - if (Replaces.size() == 0)<br>
> - return false; // Nothing changed, don't touch the file.<br>
> -<br>
> - std::string ErrorInfo;<br>
> - llvm::raw_fd_ostream FileStream(FileName.c_str(), ErrorInfo,<br>
> - llvm::sys::fs::F_Binary);<br>
> - if (!ErrorInfo.empty()) {<br>
> - llvm::errs() << "Error while writing file: " << ErrorInfo<br>
> << "\n";<br>
> + if (Rewrite.overwriteChangedFiles())<br>
> return true;<br>
> - }<br>
> - Rewrite.getEditBuffer(ID).write(FileStream);<br>
> - FileStream.flush();<br>
> } else {<br>
> if (Cursor.getNumOccurrences() != 0)<br>
> outs() << "{ \"Cursor\": " << tooling::shiftedCodePosition(<br>
><br>
><br>
> _______________________________________________<br>
> cfe-commits mailing list<br>
</div></div>> <a href="mailto:cfe-commits@cs.uiuc.edu" class="cremed">cfe-commits@cs.uiuc.edu</a> <mailto:<a href="mailto:cfe-commits@cs.uiuc.edu" class="cremed">cfe-commits@cs.uiuc.edu</a>><br>
> <a href="http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits" target="_blank" class="cremed">http://lists.cs.uiuc.edu/mailman/listinfo/cfe-commits</a><br>
><br>
><br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
<a href="http://www.nuanti.com" target="_blank" class="cremed">http://www.nuanti.com</a><br>
the browser experts<br>
<br>
</font></span></blockquote></div><br></div></div>