[PATCH] D37491: [Preamble] Fixed preamble breaking with BOM presence (and particularly, fluctuating BOM presence)

[Cameron via Phabricator via cfe-commits]

cameron314 created this revision.

This patch fixes preamble skipping when the preamble region includes a byte order mark (BOM). Previously, parsing would fail if preamble PCH generation was enabled and a BOM was present.

This also fixes the preamble incorrectly being invalidated when a BOM appeared or disappeared, even if the contents after the BOM had not changed. This may seem to be an obscure edge case, but it is important for IDEs that pass buffer overrides that never (or always) have a BOM, yet the underlying file from the initial parse that generated a PCH might (or might not) have a BOM.

Test cases for these scenarios are included.

Note: This depends on the test infrastructure introduced in https://reviews.llvm.org/D37474.


https://reviews.llvm.org/D37491

Files:
  include/clang/Frontend/PrecompiledPreamble.h
  include/clang/Lex/Lexer.h
  include/clang/Lex/PreprocessorOptions.h
  lib/Frontend/ASTUnit.cpp
  lib/Frontend/FrontendActions.cpp
  lib/Frontend/PrecompiledPreamble.cpp
  lib/Lex/Lexer.cpp
  lib/Lex/Preprocessor.cpp
  unittests/Frontend/PchPreambleTest.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D37491.113898.patch
Type: text/x-patch
Size: 17349 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170905/b4b76138/attachment-0001.bin>