r311683 - [Preprocessor] Correct internal token parsing of newline characters in CRLF

[Keane, Erich via cfe-commits]

Can you better clarify what went wrong?  I included a ‘.gitattriutes’ that matches only the new test that is supposed to keep it as CLRF


From: Stephen Hines [mailto:srhines at google.com]
Sent: Thursday, August 24, 2017 3:18 PM
To: Keane, Erich <erich.keane at intel.com>
Cc: cfe-commits <cfe-commits at lists.llvm.org>
Subject: Re: r311683 - [Preprocessor] Correct internal token parsing of newline characters in CRLF

This change seems to have broken the git mirrors, as I can no longer check out clang without having a merge conflict as git alters the line endings. Setting core.autocrlf to false doesn't help either. Does anyone have any idea how to fix this svn <-> git issue?

Thanks,
Steve

On Thu, Aug 24, 2017 at 11:36 AM, Erich Keane via cfe-commits <cfe-commits at lists.llvm.org<mailto:cfe-commits at lists.llvm.org>> wrote:
Author: erichkeane
Date: Thu Aug 24 11:36:07 2017
New Revision: 311683

URL: http://llvm.org/viewvc/llvm-project?rev=311683&view=rev
Log:
[Preprocessor] Correct internal token parsing of newline characters in CRLF

Discovered due to a goofy git setup, the test system-headerline-directive.c
(and a few others) failed because the token-consumption will consume only the
'\r' in CRLF, making the preprocessor's printed value give the wrong line number
when returning from an include. For example:

(line 1):#include <noline.h>\r\n

The "file exit" code causes the printer to try to print the 'returned to the
main file' line. It looks up what the current line number is. However, since the
current 'token' is the '\n' (since only the \r was consumed), it will give the
line number as '1", not '2'. This results in a few failed tests, but more
importantly, results in error messages being incorrect when compiling a
previously preprocessed file.

Differential Revision: https://reviews.llvm.org/D37079

Added:
    cfe/trunk/test/Frontend/.gitattributes
    cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c   (with props)
Modified:
    cfe/trunk/lib/Lex/Lexer.cpp

Modified: cfe/trunk/lib/Lex/Lexer.cpp
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/lib/Lex/Lexer.cpp?rev=311683&r1=311682&r2=311683&view=diff
==============================================================================
--- cfe/trunk/lib/Lex/Lexer.cpp (original)
+++ cfe/trunk/lib/Lex/Lexer.cpp Thu Aug 24 11:36:07 2017
@@ -3073,6 +3073,8 @@ LexNextToken:

   case '\n':
   case '\r':
+    if (CurPtr[0] != Char && (CurPtr[0] == '\n' || CurPtr[0] == '\r'))
+      Char = getAndAdvanceChar(CurPtr, Result);
     // If we are inside a preprocessor directive and we see the end of line,
     // we know we are done with the directive, so return an EOD token.
     if (ParsingPreprocessorDirective) {

Added: cfe/trunk/test/Frontend/.gitattributes
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Frontend/.gitattributes?rev=311683&view=auto
==============================================================================
--- cfe/trunk/test/Frontend/.gitattributes (added)
+++ cfe/trunk/test/Frontend/.gitattributes Thu Aug 24 11:36:07 2017
@@ -0,0 +1,2 @@
+# Below test validates crlf line endings, so it should stay crlf.
+system-header-line-directive-ms-lineendings.c text eol=crlf

Added: cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c
URL: http://llvm.org/viewvc/llvm-project/cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c?rev=311683&view=auto
==============================================================================
--- cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c (added)
+++ cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c Thu Aug 24 11:36:07 2017
@@ -0,0 +1,21 @@
+// RUN: %clang_cc1 %s -E -o - -I %S/Inputs -isystem %S/Inputs/SystemHeaderPrefix | FileCheck %s
+#include <noline.h>
+#include <line-directive-in-system.h>
+
+#include "line-directive.h"
+
+// This tests that the line numbers for the current file are correctly outputted
+// for the include-file-completed test case.
+
+// CHECK: # 1 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}noline.h" 1 3
+// CHECK: foo();
+// CHECK: # 3 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}line-directive-in-system.h" 1 3
+//      The "3" below indicates that "foo.h" is considered a system header.
+// CHECK: # 1 "foo.h" 3
+// CHECK: foo();
+// CHECK: # 4 "{{.*}}system-header-line-directive-ms-lineendings.c" 2
+// CHECK: # 1 "{{.*}}line-directive.h" 1
+// CHECK: # 10 "foo.h"{{$}}
+// CHECK: # 6 "{{.*}}system-header-line-directive-ms-lineendings.c" 2

Propchange: cfe/trunk/test/Frontend/system-header-line-directive-ms-lineendings.c
------------------------------------------------------------------------------
    svn:eol-style = CRLF


_______________________________________________
cfe-commits mailing list
cfe-commits at lists.llvm.org<mailto:cfe-commits at lists.llvm.org>
http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20170824/303690ad/attachment.html>