<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
On 03/21/2018 03:23 PM, Jeremy Lakeman wrote:<br>
<blockquote type="cite"
cite="mid:CAAjnzmB0qvQiCHseht+YLUJRXUM8mwDLN-2Nuj+D3ZjRn21ZJA@mail.gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">
<div>Do you really need to write the entire module to a single
file? (Hence my earlier hint...)<br>
</div>
</div>
</blockquote>
First, prime motivation for this particular experiment was to make
Alexandre's implementation<br>
match -print-after-all functionality, which dumps entire set of
modules into a single file.<br>
<br>
Then, even for my own initial implementation of "ala git-commit"
functionality I would<br>
just dump everything into a stream and postprocess it (perhaps
making it a bit easier to postprocess<br>
by properly marking the stream, but still a single stream).<br>
<br>
<blockquote type="cite"
cite="mid:CAAjnzmB0qvQiCHseht+YLUJRXUM8mwDLN-2Nuj+D3ZjRn21ZJA@mail.gmail.com">
<div dir="ltr">Why not write out a separate file for each def, so
you don't need to dump functions that haven't changed?<br>
</div>
</blockquote>
Because I wouldnt want to figure out what has or what has not
changed.<br>
I would rather leave it to git itself :)<br>
<br>
And frankly, the problem I'm seeing right now is not related to
details of a particular dump organization.<br>
I'm seeing a huge performance difference in what I consider to be
very similar implementations<br>
of module IR printing...<br>
<br>
regards,<br>
Fedor.<br>
<br>
<blockquote type="cite"
cite="mid:CAAjnzmB0qvQiCHseht+YLUJRXUM8mwDLN-2Nuj+D3ZjRn21ZJA@mail.gmail.com">
<div class="gmail_extra"><br>
<div class="gmail_quote">On Wed, Mar 21, 2018 at 8:38 PM, Fedor
Sergeev via llvm-dev <span dir="ltr"><<a
href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex"><span
class="">On 03/16/2018 01:21 AM, Fedor Sergeev via
llvm-dev wrote:<br>
> git-commit-after-all solution has one serious issue -
it has a hardcoded git handling which<br>
> makes it look problematic from many angles (picking a
proper git,<br>
> selecting exact way of storing information, creating
repository, replacing the file etc etc).<br>
><br>
> Just dumping information in a way that allows easy
subsequent machine processing<br>
> seems to be a more flexible, less cluttered and
overall clean solution that allows to avoid<br>
> making any of "user interface" decisions mentioned
above.<br>
><br>
> We need to understand why git-commit-after-all works
faster than print-after-all.<br>
</span>
Made an interesting experiment today and extended your
git-commit-after-all to avoid issuing<br>
any git commands if git-repo starts with "/dev/".<br>
<br>
With git-repo=/dev/stderr it becomes functionally equivalent
to print-after-all+print-module-s<wbr>cope,<br>
dumping module into stderr after each pass.<br>
<br>
On my testcase:<br>
<br>
# first normal git-commit-after-all execution<br>
] rm -rf test-git; time $RR/bin/opt -O1 some-ir.ll
-disable-output -git-commit-after-all -git-repo=./test-git<br>
<br>
real 0m7.172s<br>
user 0m6.303s<br>
sys 0m0.902s<br>
# then "printing" git-commit-after-all execution<br>
] time $RR/bin/opt -O1 some-ir.ll -disable-output
-git-commit-after-all -git-repo=/dev/stderr 2>&1 |
grep -c '^; ModuleID'<br>
615<br>
<br>
real 0m2.893s<br>
user 0m2.859s<br>
sys 0m0.356s<br>
# and finally print-after-all<br>
] time $RR/bin/opt -O1 some-ir.ll -disable-output
-print-after-all -print-module-scope 2>&1 | grep -c
"^; ModuleID"<br>
526<br>
<br>
real 2m8.024s<br>
user 0m55.933s<br>
sys 3m19.253s<br>
]<br>
Ugh... 60x???<br>
Now, I'm set to analyze this astonishing difference that
threatens my sanity (while I'm still sane ... hopefully).<br>
<br>
regards,<br>
Fedor.<br>
PS btw, I checked /dev/null - and it works faster than
/dev/stderr as expected :)
<div class="HOEnZb">
<div class="h5"><br>
<br>
> I dont believe in magic... yet :)<br>
><br>
> And, btw, thanks for both the idea and the patch.<br>
><br>
> regards,<br>
> Fedor.<br>
><br>
> On 03/16/2018 12:03 AM, Alexandre Isoard wrote:<br>
>> If this is faster than -print-after-all we may
actually consider pushing that in the code base then?
(after diligent code review of course)<br>
>><br>
>> Note that it uses the same printing method as
-print-after-all:<br>
>> - create a pass of the same pass kind as the
pass we just ran<br>
>> - use Module::print(raw_ostream) to print
(except -print-after-all only print the concerned part
and into stdout)<br>
>><br>
>> If there is improvement to be done to
print-after-all it might also improve
git-commit-after-all. (unless that only improve speed
when printing constructs smaller than module)<br>
>><br>
>> In any case, it is, to me, much more usable
(and extensible) than -print-after-all. But requires git
to be in PATH (I'm curious if that works on Windows).<br>
>><br>
>> On Thu, Mar 15, 2018 at 1:35 PM, Daniel Sanders
<<a href="mailto:daniel_l_sanders@apple.com"
target="_blank" moz-do-not-send="true">daniel_l_sanders@apple.com</a>>
wrote:<br>
>><br>
>> Does <a
href="https://reviews.llvm.org/D44132"
rel="noreferrer" target="_blank"
moz-do-not-send="true">https://reviews.llvm.org/D4413<wbr>2</a>
help at all?<br>
>><br>
>><br>
>>> On 15 Mar 2018, at 09:16, Philip Reames
via llvm-dev <<a
href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a>>
wrote:<br>
>>><br>
>>> The most likely answer is that the
printer used by print-after-all is slow. I know there
were some changes made around passing in some form of
state cache (metadata related?) and that running
printers without doing so work, but are dog slow. I
suspect the print-after-all support was never updated.
Look at what we do for the normal IR emission "-S" and
see if print-after-all is out of sync.<br>
>>><br>
>>> Philip<br>
>>><br>
>>> On 03/15/2018 08:45 AM, Alexandre
Isoard via llvm-dev wrote:<br>
>>>> Huh. Great! 😁<br>
>>>><br>
>>>> I don't believe my poor excuse from
earlier (else we should map all pipes into files!), but
I'm curious why we spend less time in system mode when
going through file than pipe. Maybe /dev/null is not as
efficient as we might think? I can't believe I'm saying
that...<br>
>>>><br>
>>>> On Thu, Mar 15, 2018, 08:25 Fedor
Sergeev <<a href="mailto:fedor.sergeev@azul.com"
target="_blank" moz-do-not-send="true">fedor.sergeev@azul.com</a>>
wrote:<br>
>>>><br>
>>>> Well, git by itself is so
focused on performance, so its not surprising<br>
>>>> to me that even using git
add/git commit does not cause<br>
>>>> performance penalties.<br>
>>>><br>
>>>><br>
>>>> Sure, but still, I write more stuff
(entire module) into a slower destination (file). Even
ignoring git execution time it's counter intuitive.<br>
>>>><br>
>>>> The only difference is that while I
write more, it overwrite itself continuously, instead of
being a long linear steam. I was thinking of mmap the
file instead of going through our raw_stream, but maybe
that's unnecessary then...<br>
>>>><br>
>>>><br>
>>>><br>
>>>> ______________________________<wbr>_________________<br>
>>>> LLVM Developers mailing list<br>
>>>> <a
href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
>>>> <a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
>>><br>
>>> ______________________________<wbr>_________________<br>
>>> LLVM Developers mailing list<br>
>>> <a
href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
>>> <a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
>><br>
>><br>
>><br>
>><br>
>> --<br>
>> Alexandre Isoard<br>
><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> LLVM Developers mailing list<br>
> <a href="mailto:llvm-dev@lists.llvm.org"
target="_blank" moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
> <a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
<br>
<br>
______________________________<wbr>_________________<br>
LLVM Developers mailing list<br>
<a href="mailto:llvm-dev@lists.llvm.org" target="_blank"
moz-do-not-send="true">llvm-dev@lists.llvm.org</a><br>
<a
href="http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev"
rel="noreferrer" target="_blank"
moz-do-not-send="true">http://lists.llvm.org/cgi-bin/<wbr>mailman/listinfo/llvm-dev</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>