[llvm-dev] Commit module to Git after each Pass

[Fedor Sergeev via llvm-dev]

On 03/15/2018 01:32 PM, Fedor Sergeev via llvm-dev wrote:
> For this to be really usable in this setup we need additionally to:
>   - extend -print-module-scope to cover basic block passes
>   - introduce a clear way to separate module IRs as those are being 
> printed by -print-after-all
>
> But yes, it should work, and a wrapper that pipes to git fast-import 
> seems to be the best way to handle it.
A simple 20-lines perl script does the trick pretty easily: 
https://pastebin.com/4J0b5Tr8

(this assumes my local modification to introduce the *** END OF ** IR 
DUMP marked at the end of -print-module-scope's IR module dump)

] git init
] RA/bin/opt -O3 some-ir.ll -disable-output -print-after-all 
-print-module-scope 2>&1 | filter-LLVM-ir-print.pl | git fast-import 
--done --date-format=now
....

Majority of time is spent to actually print the IR (~2m for my testcase).
Fast-import takes just a second.

regards,
   Fedor.

>
> regards,
>   Fedor.
>
> On 03/15/2018 12:31 AM, Daniel Neilson via llvm-dev wrote:
>> The print-module-after-all type of option exists in upstream:
>>   -print-module-scope                                     - When 
>> printing IR for print-[before|after]{-all} always print a module IR
>>
>> commit 7d160f714357f6784ead669ce516e94991c12e5a
>> Author: Fedor Sergeev <fedor.sergeev at azul.com 
>> <mailto:fedor.sergeev at azul.com>>
>> Date:   Fri Dec 1 17:42:46 2017 +0000
>>
>>     IR printing improvement for function passes - introducing 
>> -print-module-scope
>>
>>
>>     Summary:
>>     When debugging function passes it happens to be rather useful to 
>> dump
>>     the whole module before the transformation and then use this dump
>>     to analyze this single transformation by running it separately
>>     on that particular module state.
>>
>>
>>     Introducing
>>         -print-module-scope
>>     debugging option that forces all the function-level IR dumps
>>     to become whole-module dumps.
>>
>>
>>     This option builds on top of normal dumping controls like
>>        -print-before/after
>>        -filter-print-funcs
>>
>>
>>     The plan is to eventually extend this option to cover other local 
>> passes
>>     (at least loop passes) but that should go as a separate change.
>>
>>
>> Loop passes here:
>> commit 5608259c999fb77c5d6093895696f4daebe6b8cd
>> Author: Fedor Sergeev <fedor.sergeev at azul.com 
>> <mailto:fedor.sergeev at azul.com>>
>> Date:   Fri Dec 1 18:33:58 2017 +0000
>>
>>     IR printing improvement for loop passes - handle -print-module-scope
>>
>>
>> -Daniel
>>
>>
>>> On Mar 14, 2018, at 3:51 PM, Philip Reames via llvm-dev 
>>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote:
>>>
>>> This is interesting, and might be useful.  I don't know that this is 
>>> broadly useful enough for upstream inclusion, but if you could post 
>>> this to github somewhere, I might play with it.
>>>
>>> There might also be room to factor out common functionality. We've 
>>> also run into the need to print whole-module instead of containing 
>>> construct (i.e. this loop).  If we added upstream support for 
>>> something along the lines of -print-module-after-all, building the 
>>> git history could easily be done as a post processing step.
>>>
>>> Philip
>>>
>>>
>>> On 03/06/2018 10:43 AM, Alexandre Isoard via llvm-dev wrote:
>>>> Hello,
>>>>
>>>> I had a stupid idea recently that turned out not so stupid after 
>>>> all. I wanted to be able to "see" an entire pass pipeline in action 
>>>> to find unnecessary transformations and/or missed opportunities and 
>>>> generally improve the debug-ability of LLVM.
>>>>
>>>> So as the title suggest, I implemented an equivalent of 
>>>> "-print-after-all" but instead of printing into stdout I dump into 
>>>> a file that get commit into a temporary git. There are some quirks 
>>>> with it but it's working and is actually awesome. For example, at 
>>>> first sight, I see multiple time lcssa and instcombine cancelling 
>>>> each other's work.
>>>>
>>>> Of course, that has a big impact on compile time when enabled, but 
>>>> that's still practical (git being quite good at its job) when 
>>>> debugging.
>>>>
>>>> There are improvement I can make, but would you guys be interested 
>>>> in such feature?
>>>>
>>>> -- 
>>>> *Alexandre Isoard*
>>>>
>>>>
>>>> _______________________________________________
>>>> LLVM Developers mailing list
>>>> llvm-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>>
>>> _______________________________________________
>>> LLVM Developers mailing list
>>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev