[llvm-dev] Commit module to Git after each Pass

[Jeremy Lakeman via llvm-dev]

TLDR; fast import reads a sequence of commands from stdin to write whole
files / trees / commits directly into a compressed pack archive.
Primarily designed for bulk importing history from other source control
systems.

If the git commands you are using were a significant performance problem, I
figured this would be quicker.

Have you considered writing each function or even basic block to a separate
file?

On Thu, Mar 8, 2018 at 2:03 PM, Alexandre Isoard <alexandre.isoard at gmail.com
> wrote:

> Maybe. I can't compute the diff/patch for git though, it will have to do
> it itself. Would that still work?
>
> I am not sure I will have time to work on improving that.
> For those who want to try it out I put it there: https://reviews.llvm.
> org/D44244
>
> And yes, this uses all possible forms of ugly. :-)
>
> On Wed, Mar 7, 2018, 18:14 Jeremy Lakeman <Jeremy.Lakeman at gmail.com>
> wrote:
>
>> Could you format the output so that it is compatible with git
>> fast-import?
>> https://www.git-scm.com/docs/git-fast-import
>>
>> On Thu, Mar 8, 2018 at 2:34 AM, Alexandre Isoard via llvm-dev <
>> llvm-dev at lists.llvm.org> wrote:
>>
>>> Today it is entirely in llvm. It is even more costly than
>>> -print-after-all as it:
>>>   - print to file
>>>   - print the entire module (after each basic block, for basic block
>>> passes, it will still print the entire module where only one basic block
>>> changed)
>>>   - call git 2 times (add then commit) and wait for them to finish (I
>>> even save all the empty commits)
>>>
>>> The reason I print the entire module is so that git is able to
>>> show/compress the change properly. Then a simple git log --patch show the
>>> change of each pass.
>>>
>>> Ideally, the -debug output of each pass would be piped as the git commit
>>> message, and the passes name+options would be the commit title. But I
>>> didn't have time to do that.
>>>
>>> My goal was not particularly to be efficient but to be thorough so as to
>>> preserve the maximum of behavioral information of each pass while not
>>> affecting their order and behavior.
>>>
>>> You are right, there could be major speed improvement gained by doing so
>>> as external processing. For instance, the output would be in the format
>>> that git patch takes as input. Could this be a GSoC?
>>>
>>> Today I use git filter-branch + opt as a post processing tool to
>>> generate the reg.dot files of the region info. In any way, the feature is a
>>> breath of fresh air in our development here. That's why I wanted to share.
>>> It will quickly become my reflex to debug with it... probably because I am
>>> at ease with git though.
>>>
>>> On Wed, Mar 7, 2018, 06:19 Alex Bradbury <asb at asbradbury.org> wrote:
>>>
>>>> On 6 March 2018 at 18:43, Alexandre Isoard via llvm-dev
>>>> <llvm-dev at lists.llvm.org> wrote:
>>>> > Hello,
>>>> >
>>>> > I had a stupid idea recently that turned out not so stupid after all.
>>>> I
>>>> > wanted to be able to "see" an entire pass pipeline in action to find
>>>> > unnecessary transformations and/or missed opportunities and generally
>>>> > improve the debug-ability of LLVM.
>>>> >
>>>> > So as the title suggest, I implemented an equivalent of
>>>> "-print-after-all"
>>>> > but instead of printing into stdout I dump into a file that get
>>>> commit into
>>>> > a temporary git. There are some quirks with it but it's working and is
>>>> > actually awesome. For example, at first sight, I see multiple time
>>>> lcssa and
>>>> > instcombine cancelling each other's work.
>>>> >
>>>> > Of course, that has a big impact on compile time when enabled, but
>>>> that's
>>>> > still practical (git being quite good at its job) when debugging.
>>>> >
>>>> > There are improvement I can make, but would you guys be interested in
>>>> such
>>>> > feature?
>>>>
>>>> Hi Alexandre. I can definitely see how it could be useful to track
>>>> changes through git commits, and take advantage of your favourite repo
>>>> history viewer to see changes. How much of your current implementation
>>>> is handled via modifications to LLVM vs an external helper script? For
>>>> instance, I might imagine trying to achieve something similar through
>>>> a script that parses the output of -print-after-all in order to create
>>>> the desired files+commits.
>>>>
>>>> Best,
>>>>
>>>> Alex
>>>>
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180308/0a13146b/attachment-0001.html>