[llvm-dev] 'git llvm push' not working for me on Windows

Sat Feb 2 07:44:04 PST 2019

On 2019-02-02 14:16, James Y Knight via llvm-dev wrote:
> Hm, there's only a few affected files (files that both have
> svn:keywords set, and contain a $keyword$).
> 
> We'll need to remove or replace all of these anyhow, since the
> expansion doesn't actually function with a git checkout.
> 
> clang/lib/Basic/Version.cpp:  StringRef SVNRepository("$URL$");
> 
> clang/tools/scan-build/man/scan-build.1:.\" $Id$
> clang/www/cxx_dr_status.html:<p>Last updated: $Date$</p>
> clang/www/cxx_status.html:<p>Last updated: $Date$</p>
> llvm/utils/vim/syntax/llvm.vim:" Version:      $Revision$
> 
> llvm/utils/vim/syntax/tablegen.vim:" Version:    $Revision$
> llvm/utils/vim/vimrc:" $Revision$

You could add a .gitattributes with content like

   * ident

and then git would expand $Id$ to $Id: sha1$ and back, just like SVN.
I don't think there's an easy way to make this work for anything else,
like $URL$ or $Date$ though.

> 
> On Fri, Feb 1, 2019 at 9:37 PM Petr Hosek <phosek at chromium.org> wrote:
> 
>> I already figured it out. Turned out some sources like
>> clang/lib/Basic/Version.cpp contain Subversion keywords (in this
>> particular case it's $URL$). Subversion would automatically
>> substitute these in the internal checkout that's used by
>> git-llvm-push, but Git wouldn't, so when trying to apply the patch
>> generated by Git you'll get and error because while Git patch has
>> $URL$, the Subversion checkout would have something like $URL:
>> https://llvm.org/svn/llvm-project/cfe/trunk/$. The workaround I came
>> up with is to do the keyword substitution in the generated patch
>> 
> (https://github.com/llvm/llvm-project/blob/master/llvm/utils/git-svn/git-llvm#L321).
>> 
>> On Fri, Feb 1, 2019 at 2:56 PM <paul.robinson at sony.com> wrote:
>> 
>> The usual workaround is to do `rm –rf .git/llvm-upstream-svn` and
>> try again.
>> 
>> You could also patch llvm/utils/git-svn/git-llvm to add the
>> `--verbose` option to the `git apply` command, which would provide
>> better diagnostic output.
>> 
>> HTH,
>> 
>> --paulr
>> 
>> FROM: Petr Hosek [mailto:phosek at chromium.org]
>> SENT: Wednesday, January 30, 2019 10:52 PM
>> TO: Robinson, Paul
>> CC: James Y Knight; llvm-dev; vedant_kumar at apple.com
>> SUBJECT: Re: [llvm-dev] 'git llvm push' not working for me on
>> Windows
>> 
>> I just hit this failure as well. I've uploaded the log to
>> https://reviews.llvm.org/P8126. The error message is:
>> 
>> `git apply -p2 -` returned 1
>> 
>> error: patch failed: lib/Basic/Version.cpp:14
>> 
>> error: lib/Basic/Version.cpp: patch does not apply
>> 
>> Patch doesn't apply: maybe you should try `git pull -r` first?
>> 
>> On Thu, Nov 29, 2018 at 1:18 PM via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>> 
>> Excellent, thanks!
>> 
>> --paulr
>> 
>> FROM: James Y Knight [mailto:jyknight at google.com]
>> SENT: Thursday, November 29, 2018 12:00 PM
>> TO: Robinson, Paul
>> CC: Mikhail Ramalho; vedant_kumar at apple.com; llvm-dev
>> SUBJECT: Re: [llvm-dev] 'git llvm push' not working for me on
>> Windows
>> 
>> Aha! From your output I figured out what I screwed up. Sorry about
>> the trouble...
>> 
>> Illustration of the issue:
>> 
>> svn checkout --depth=empty https://llvm.org/svn/llvm-project
>> 
>> svn update --depth=immediates --parents llvm/trunk/test/DebugInfo
>> 
>> svn update --depth=immediates --parents
>> llvm/trunk/test/DebugInfo/Generic
>> 
>> After these commands, I expected Generic/ to be populated, but it is
>> not, because the "--depth=immediates" *creates* the directory, but
>> then it marks it as depth=empty. And then, later, calling svn update
>> --depth=immediates _doesn't override that_. This is apparently not a
>> bug in subversion, that's is how it is "supposed" to work. It's just
>> super confusing. If I use --depth=files instead of
>> --depth=immediates, that fixes this issue, since in that case, it
>> won't create the subdirs and mark them as depth=empty.
>> 
>> Also, the --parents flag causes the same issue for parent
>> directories. Ugh. After the following, I'd expected llvm/trunk/ to
>> get populated, but it doesn't get populated, because the first 'svn
>> update' created it with sticky depth=empty:
>> 
>> svn checkout --depth=empty https://llvm.org/svn/llvm-project
>> 
>> svn update --depth=files --parents llvm/trunk/test/DebugInfo
>> 
>> svn update --depth=files --parents llvm/trunk/
>> 
>> Subversion has a --set-depth= argument which "fixes" this, but
>> unfortunately, that argument *also* recursively deletes any subdirs
>> which have already been populated. (I just want to _add_ things, not
>> delete already-downloaded data.)
>> 
>> In summary: subversion's sparse-checkout command-line UI is really
>> confusing and annoying! Ugh!
>> 
>> I've pushed a commit (r347883) to hopefully fix this issue. This
>> time for sure.
>> 
>> You will probably need to remove the .git/llvm-upstream-svn
>> directory, one last time, for the fix to work.
>> 
>> On Wed, Nov 28, 2018 at 3:55 PM <paul.robinson at sony.com> wrote:
>> 
>> Poking around in the .git\llvm-upstream-svn tree, I find that
>> llvm\trunk\test\DebugInfo\Generic is empty, as are all the other
>> subdirectories of test\DebugInfo that I tried.  I have other files
>> in the checkin that are in leaf directories but those files all
>> exist.
>> 
>> I hacked git-llvm to add a –verbose option:
>> 
>> `git apply --verbose -p2 -` returned 1
>> 
>> Checking patch include/llvm/IR/DebugInfoFlags.def...
>> 
>> Checking patch include/llvm/IR/DebugInfoMetadata.h...
>> 
>> Checking patch lib/AsmParser/LLLexer.cpp...
>> 
>> Checking patch lib/AsmParser/LLParser.cpp...
>> 
>> Checking patch lib/AsmParser/LLToken.h...
>> 
>> Checking patch lib/Bitcode/Reader/MetadataLoader.cpp...
>> 
>> Checking patch lib/Bitcode/Writer/BitcodeWriter.cpp...
>> 
>> Checking patch lib/IR/AsmWriter.cpp...
>> 
>> Checking patch lib/IR/DebugInfoMetadata.cpp...
>> 
>> Checking patch test/Assembler/disubprogram.ll...
>> 
>> Checking patch
>> test/Assembler/invalid-disubprogram-uniqued-definition.ll...
>> 
>> Checking patch test/Bindings/llvm-c/debug_info.ll...
>> 
>> Checking patch test/Bitcode/DISubprogram-distinct-definitions.ll...
>> 
>> Checking patch test/Bitcode/DISubprogram-v4.ll...
>> 
>> Checking patch test/Bitcode/DISubprogram-v4.ll.bc...
>> 
>> Checking patch test/DebugInfo/Generic/invalid.ll...
>> 
>> error: test/DebugInfo/Generic/invalid.ll: No such file or directory
>> 
>> Checking patch test/DebugInfo/debugify.ll...
>> 
>> Checking patch
>> test/Linker/replaced-function-matches-first-subprogram.ll...
>> 
>> Checking patch
>> test/Transforms/GCOVProfiling/three-element-mdnode.ll...
>> 
>> Patch doesn't apply: maybe you should try `git pull -r` first?
>> 
>> FROM: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] ON BEHALF OF
>> via llvm-dev
>> SENT: Wednesday, November 28, 2018 2:39 PM
>> TO: mikhail.ramalho at gmail.com; jyknight at google.com
>> CC: llvm-dev at lists.llvm.org; vedant_kumar at apple.com
>> SUBJECT: Re: [llvm-dev] 'git llvm push' not working for me on
>> Windows
>> 
>> And now it's failing for me also… complaining about "no such file
>> or directory" on a file that plainly exists.
>> 
>> Verbose log attached.
>> 
>> --paulr
>> 
>> FROM: Mikhail Ramalho [mailto:mikhail.ramalho at gmail.com]
>> SENT: Wednesday, November 28, 2018 12:27 PM
>> TO: jyknight at google.com
>> CC: Robinson, Paul; llvm-dev at lists.llvm.org; vedant_kumar at apple.com
>> SUBJECT: Re: [llvm-dev] 'git llvm push' not working for me on
>> Windows
>> 
>> OK, I've managed to do it:
>> 
>> I was trying to push it from a build/ directory, maybe that's why
>> the git apply was failing.
>> 
>> Pushing the commit from the root of the repo worked.
>> 
>> Em qua, 28 de nov de 2018 às 16:40, Mikhail Ramalho
>> <mikhail.ramalho at gmail.com> escreveu:
>> 
>> Hi,
>> 
>> The patch only changes one file in clang. Here's the patch:
>> https://reviews.llvm.org/D54974
>> 
>> Attached the log. It seems to complain about the git apply -p2.
>> 
>> Maybe it's something related to arc?
>> 
>> Em qua, 28 de nov de 2018 às 15:37, James Y Knight
>> <jyknight at google.com> escreveu:
>> 
>> Can you please run "git llvm --verbose push" and send me the output?
>> 
>> BTW, I've just committed a fix for the handling of binary patch data
>> when you're running it under python3.X. I don't believe that was a
>> regression from my recent changes, but if you're using python3, you
>> could see if that fixes it for you.
>> 
>> On Tue, Nov 27, 2018 at 7:20 PM Mikhail Ramalho via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>> 
>> I'm facing this now on Linux.
>> 
>> Any idea how to fix it?
>> 
>> Em seg, 26 de nov de 2018 às 20:22, via llvm-dev
>> <llvm-dev at lists.llvm.org> escreveu:
>> 
>> I've verified that none of the files I tried to check in had
>> Windows-style line endings. It's something else.
>> 
>> --paulr
>> 
>> FROM: vsk at apple.com [mailto:vsk at apple.com] ON BEHALF OF Vedant Kumar
>> SENT: Thursday, November 22, 2018 9:38 AM
>> TO: Robinson, Paul
>> CC: zturner at google.com; jyknight at google.com; llvm-dev at lists.llvm.org
>> SUBJECT: Re: [llvm-dev] 'git llvm push' not working for me on
>> Windows
>> 
>> As a test case, try committing a change to
>> clang/test/SemaCXX/sourceranges.cpp?
>> 
>> I believe most editors force Windows-style line endings for that
>> file (they show up in my vim as “^M”). I recently tried to
>> commit a change to that file using “git llvm push” on macOS, but
>> hit what I think is the same issue. At least, the error message was
>> the same.
>> 
>> vedant (sent from my iPhone)
>> 
>> On Nov 21, 2018, at 10:06 AM, via llvm-dev <llvm-dev at lists.llvm.org>
>> wrote:
>> 
>> FTR the commit where it failed for me had both llvm and clang
>> changes.  I wonder if that contributed.
>> 
>> --paulr
>> 
>> FROM: llvm-dev [mailto:llvm-dev-bounces at lists.llvm.org] ON BEHALF OF
>> via llvm-dev
>> SENT: Monday, November 19, 2018 6:01 PM
>> TO: zturner at google.com; jyknight at google.com
>> CC: llvm-dev at lists.llvm.org
>> SUBJECT: Re: [llvm-dev] 'git llvm push' not working for me on
>> Windows
>> 
>> Hm.  Just now it worked for me for r347271.  If it happens again
>> I'll try verbose mode and attach a log.
>> 
>> Thanks,
>> 
>> --paulr
>> 
>> FROM: Zachary Turner [mailto:zturner at google.com]
>> SENT: Monday, November 19, 2018 5:26 PM
>> TO: James Y Knight
>> CC: Robinson, Paul; llvm-dev
>> SUBJECT: Re: [llvm-dev] 'git llvm push' not working for me on
>> Windows
>> 
>> Usually every time I've seen that error message, it's been related
>> to line ending normalization.  But James is right, I did use it
>> successfully this morning as well as yesterday.
>> 
>> On Mon, Nov 19, 2018 at 1:03 PM James Y Knight via llvm-dev
>> <llvm-dev at lists.llvm.org> wrote:
>> 
>> It does sound like I must've broken something, but I believe zturner
>> used it on windows successfully. Perhaps you could help me debug the
>> issue? Possibly running it with --verbose would show something
>> useful.
>> 
>> On Mon, Nov 19, 2018 at 1:32 PM <paul.robinson at sony.com> wrote:
>> 
>> I am consistently getting:
>> 
>> `git apply -p2 -` returned 1
>> error: include/llvm/IR/DIBuilder.h: No such file or directory
>> [[ etc ]]
>> Patch doesn't apply; maybe you should try `git pull -r` first?
>> 
>> My usual response to a problem from git-llvm (which is most often
>> an anti-virus issue) is to blow away .git\llvm-upstream-svn but
>> that doesn't help this time.
>> 
>> I see James Knight did a "performance improvement" on Friday,
>> perhaps that doesn't work so well on Windows?  The main
>> performance cost is one-time, and as long as my anti-virus isn't
>> trashing things behind my back the performance is just fine;
>> so it's not clear this is really a necessary improvement.
>> If it's interfering with developing the new git repo, then I'd
>> suggest making it optional or platform-dependent.
>> 
>> As a workaround I fetched the previous version of git-llvm
>> and put it earlier in my PATH, that worked.
>> 
>> --paulr
>> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> --
> 
> Mikhail Ramalho.
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> 
> --
> 
> Mikhail Ramalho.
> 
> --
> 
> Mikhail Ramalho.
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

-- 
whitequark