r311683 - [Preprocessor] Correct internal token parsing of newline characters in CRLF

[Stephen Hines via cfe-commits]

Ah, sorry I had to run off for a meeting. Yes, I was able to finally fix
this. For anyone else who hits it, you can "git reset --hard" back to the
CL before the initial change. Then you can "git pull" to move past the
issue. Thanks for fixing this.

Steve

On Thu, Aug 24, 2017 at 4:35 PM, Keane, Erich <erich.keane at intel.com> wrote:

> Alright, final update.  Thanks to some fantastic help on #llvm, I believe
> this is fixed.
>
>
>
> Stephen: You may have to do some shenanigans to fix your local stuff, but
> the monorepo and another repo both seem to work.  Sorry for this everyone :/
>
>
>
> *From:* Stephen Hines [mailto:srhines at google.com]
> *Sent:* Thursday, August 24, 2017 3:18 PM
> *To:* Keane, Erich <erich.keane at intel.com>
> *Cc:* cfe-commits <cfe-commits at lists.llvm.org>
> *Subject:* Re: r311683 - [Preprocessor] Correct internal token parsing of
> newline characters in CRLF
>
>
>
> This change seems to have broken the git mirrors, as I can no longer check
> out clang without having a merge conflict as git alters the line endings.
> Setting core.autocrlf to false doesn't help either. Does anyone have any
> idea how to fix this svn <-> git issue?
>
>
>
> Thanks,
>
> Steve
>
>
>
> On Thu, Aug 24, 2017 at 11:36 AM, Erich Keane via cfe-commits <
> cfe-commits at lists.llvm.org> wrote:
>
> Author: erichkeane
> Date: Thu Aug 24 11:36:07 2017
> New Revision: 311683
>
> URL: http://llvm.org/viewvc/llvm-project?rev=311683&view=rev
> Log:
> [Preprocessor] Correct internal token parsing of newline characters in CRLF
>
> Discovered due to a goofy git setup, the test system-headerline-directive.c
> (and a few others) failed because the token-consumption will consume only
> the
> '\r' in CRLF, making the preprocessor's printed value give the wrong line
> number
> when returning from an include. For example:
>
> (line 1):#include <noline.h>\r\n
>
> The "file exit" code causes the printer to try to print the 'returned to
> the
> main file' line. It looks up what the current line number is. However,
> since the
> current 'token' is the '\n' (since only the \r was consumed), it will give
> the
> line number as '1", not '2'. This results in a few failed tests, but more
> importantly, results in error messages being incorrect when compiling a
> previously preprocessed file.
>
> Differential Revision: https://reviews.llvm.org/D37079
>
> Added:
>     cfe/trunk/test/Frontend/.gitattributes
>     cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c
>  (with props)
> Modified:
>     cfe/trunk/lib/Lex/Lexer.cpp
>
> Modified: cfe/trunk/lib/Lex/Lexer.cpp
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/
> Lexer.cpp?rev=311683&r1=311682&r2=311683&view=diff
> ============================================================
> ==================
> --- cfe/trunk/lib/Lex/Lexer.cpp (original)
> +++ cfe/trunk/lib/Lex/Lexer.cpp Thu Aug 24 11:36:07 2017
> @@ -3073,6 +3073,8 @@ LexNextToken:
>
>    case '\n':
>    case '\r':
> +    if (CurPtr[0] != Char && (CurPtr[0] == '\n' || CurPtr[0] == '\r'))
> +      Char = getAndAdvanceChar(CurPtr, Result);
>      // If we are inside a preprocessor directive and we see the end of
> line,
>      // we know we are done with the directive, so return an EOD token.
>      if (ParsingPreprocessorDirective) {
>
> Added: cfe/trunk/test/Frontend/.gitattributes
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/
> Frontend/.gitattributes?rev=311683&view=auto
> ============================================================
> ==================
> --- cfe/trunk/test/Frontend/.gitattributes (added)
> +++ cfe/trunk/test/Frontend/.gitattributes Thu Aug 24 11:36:07 2017
> @@ -0,0 +1,2 @@
> +# Below test validates crlf line endings, so it should stay crlf.
> +system-header-line-directive-ms-lineendings.c text eol=crlf
>
> Added: cfe/trunk/test/Frontend/system-header-line-directive-
> ms-lineendings.c
> URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/
> Frontend/system-header-line-directive-ms-lineendings.c?
> rev=311683&view=auto
> ============================================================
> ==================
> --- cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c
> (added)
> +++ cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c
> Thu Aug 24 11:36:07 2017
> @@ -0,0 +1,21 @@
> +// RUN: %clang_cc1 %s -E -o - -I %S/Inputs -isystem
> %S/Inputs/SystemHeaderPrefix | FileCheck %s
> +#include <noline.h>
> +#include <line-directive-in-system.h>
> +
> +#include "line-directive.h"
> +
> +// This tests that the line numbers for the current file are correctly
> outputted
> +// for the include-file-completed test case.
> +
> +// CHECK: # 1 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
> +// CHECK: # 1 "{{.*}}noline.h" 1 3
> +// CHECK: foo();
> +// CHECK: # 3 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
> +// CHECK: # 1 "{{.*}}line-directive-in-system.h" 1 3
> +//      The "3" below indicates that "foo.h" is considered a system
> header.
> +// CHECK: # 1 "foo.h" 3
> +// CHECK: foo();
> +// CHECK: # 4 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
> +// CHECK: # 1 "{{.*}}line-directive.h" 1
> +// CHECK: # 10 "foo.h"{{$}}
> +// CHECK: # 6 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
>
> Propchange: cfe/trunk/test/Frontend/system-header-line-directive-
> ms-lineendings.c
> ------------------------------------------------------------
> ------------------
>     svn:eol-style = CRLF
>
>
> _______________________________________________
> cfe-commits mailing list
> cfe-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170824/506b24e7/attachment-0001.html>