[llvm] r344095 - [git-llvm] Fix some issues surrouding EOL conversion on Windows.

[Zachary Turner via llvm-commits]

Author: zturner
Date: Tue Oct  9 16:42:28 2018
New Revision: 344095

URL: http://llvm.org/viewvc/llvm-project?rev=344095&view=rev
Log:
[git-llvm] Fix some issues surrouding EOL conversion on Windows.

This patch fixes three issues.

The first is that we didn't consider files which are explicitly
set to eolstyle CRLF in the repo, and there are a handful of
these.

Second is that dos2unix doesn't have a -q option in GnuWin32,
so this codepath wasn't working properly.

Finally with newer versions of Python (or newer versions of Git,
or some combination of the two) patches can't be applied when
we treat stdin as text, because Python silently undoes all the
work we did to convert the newlines to LF using dos2unix by
using universal_newlines=True and then converting them *back*
to CRLF.  So we need to add a way to force stdin to be treated
as binary, and use it when LF-newlines are required.

Differential Revision: https://reviews.llvm.org/D51444

Modified:
    llvm/trunk/utils/git-svn/git-llvm

Modified: llvm/trunk/utils/git-svn/git-llvm
URL: http://llvm.org/viewvc/llvm-project/llvm/trunk/utils/git-svn/git-llvm?rev=344095&r1=344094&r2=344095&view=diff
==============================================================================
--- llvm/trunk/utils/git-svn/git-llvm (original)
+++ llvm/trunk/utils/git-svn/git-llvm Tue Oct  9 16:42:28 2018
@@ -97,7 +97,7 @@ def get_dev_null():
 
 
 def shell(cmd, strip=True, cwd=None, stdin=None, die_on_failure=True,
-          ignore_errors=False):
+          ignore_errors=False, force_binary_stdin=False):
     log_verbose('Running: %s' % ' '.join(cmd))
 
     err_pipe = subprocess.PIPE
@@ -105,14 +105,22 @@ def shell(cmd, strip=True, cwd=None, std
         # Silence errors if requested.
         err_pipe = get_dev_null()
 
+    if force_binary_stdin and stdin:
+        stdin = stdin.encode('utf-8')
+
     start = time.time()
+    text = not force_binary_stdin
     p = subprocess.Popen(cmd, cwd=cwd, stdout=subprocess.PIPE, stderr=err_pipe,
-                         stdin=subprocess.PIPE, universal_newlines=True)
+                         stdin=subprocess.PIPE, universal_newlines=text)
     stdout, stderr = p.communicate(input=stdin)
     elapsed = time.time() - start
 
     log_verbose('Command took %0.1fs' % elapsed)
 
+    if not text:
+        stdout = stdout.decode('utf-8')
+        stderr = stderr.decode('utf-8')
+
     if p.returncode == 0 or ignore_errors:
         if stderr and not ignore_errors:
             eprint('`%s` printed to stderr:' % ' '.join(cmd))
@@ -223,7 +231,7 @@ def fix_eol_style_native(rev, sr, svn_sr
     crlf_files = []
     if len(files) == 1:
         # No need to split propget output on ' - ' when we have one file.
-        if eol_props.strip() == 'native':
+        if eol_props.strip() in ['native', 'CRLF']:
             crlf_files = files
     else:
         for eol_prop in eol_props.split('\n'):
@@ -239,11 +247,11 @@ def fix_eol_style_native(rev, sr, svn_sr
             (f, eol_style) = prop_parts
             if eol_style == 'native':
                 crlf_files.append(f)
-    # Reformat all files with native SVN line endings to Unix format. SVN knows
-    # files with native line endings are text files. It will commit just the
-    # diff, and not a mass line ending change.
-    shell(['dos2unix', '-q'] + crlf_files, cwd=svn_sr_path)
-
+    if crlf_files:
+        # Reformat all files with native SVN line endings to Unix format. SVN knows
+        # files with native line endings are text files. It will commit just the
+        # diff, and not a mass line ending change.
+        shell(['dos2unix'] + crlf_files, ignore_errors=True, cwd=svn_sr_path)
 
 def svn_push_one_rev(svn_repo, rev, dry_run):
     files = git('diff-tree', '--no-commit-id', '--name-only', '-r',
@@ -265,8 +273,10 @@ def svn_push_one_rev(svn_repo, rev, dry_
         # git is the only thing that can handle its own patches...
         log_verbose('Apply patch: %s' % diff)
         try:
+            # If we allow python to apply the diff in text mode, it will silently
+            # convert \n to \r\n which git doesn't like.
             shell(['git', 'apply', '-p2', '-'], cwd=svn_sr_path, stdin=diff,
-                  die_on_failure=False)
+                  die_on_failure=False, force_binary_stdin=True)
         except RuntimeError as e:
             eprint("Patch doesn't apply: maybe you should try `git pull -r` "
                    "first?")