<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi David,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Thanks for digging in to thos issue.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Did you ping the llvm list looking for guidance with modules, in particular to report a bug or ask about what has been implemented? My recollection is that Apple has been doing a bit of work with modules and dependencies; I’m not sure
if that work is generally applicable, or if it may be helpful to cmake. I think the people doing the work are Alex Lorenz and Michael Spencer at Apple.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Did you try to pare down an implementation of std::variant to just the functions that required in f18? I wonder if “lots of less” would be fast enough.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I hate to suggest it, but could a member that is a variant be implemented as a pointer to a variant? Would that enable external template declarations?<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Did you profile clang to see where the time is being spent? The perf tool is pretty easy to use to see a gross accounting for time in routines; no special recompilation needed.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">flang-dev <flang-dev-bounces@lists.llvm.org> on behalf of Richard Barton via flang-dev <flang-dev@lists.llvm.org><br>
<b>Reply-To: </b>Richard Barton <Richard.Barton@arm.com><br>
<b>Date: </b>Wednesday, November 27, 2019 at 11:54 AM<br>
<b>To: </b>David Truby <David.Truby@arm.com>, "flang-dev@lists.llvm.org" <flang-dev@lists.llvm.org><br>
<b>Subject: </b>Re: [flang-dev] [F18] Build time investigations<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">Hi David<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Thanks for this great update on the progress so far. I’m obviously privy to most of it already sitting as I do in the same office in Manchester, UK.
<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">I wonder if you could share your strawman proposal of what ideas you will look into next in the absence of any other feedback from the community? Do you intend to prototype the external parser generator idea, or look further into modules
to see if they would be a good fix, even if some years out?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Modules in 10 years seems like a very conservative estimate. What is your rationale here?<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal">Ta<o:p></o:p></p>
<p class="MsoNormal">Rich<o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> flang-dev <flang-dev-bounces@lists.llvm.org> <b>
On Behalf Of </b>David Truby via flang-dev<br>
<b>Sent:</b> 27 November, 2019 16:44<br>
<b>To:</b> flang-dev@lists.llvm.org<br>
<b>Subject:</b> [flang-dev] [F18] Build time investigations<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"> <o:p></o:p></p>
<p class="MsoNormal"><span style="color:black">Hi all,</span> <o:p></o:p></p>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">As I mentioned on the list last week, I’ve been looking into the build time/memory usage of F18 builds and trying to narrow down where the problem lies and what possible solutions there might be. I thought I’d
post an update on here to see if anyone else has any input about my findings so far.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">First I’ll talk about things I tried that I don’t think would help:</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">I looked into using modules to improve build times without significant code changes first. However I quickly hit some stumbling blocks with this with regards to implementation; there does not as yet exist a module
implementation that is compliant with the approved C++20 proposal on modules. Clang 9 has a partial implementation, however there are some gaps in the implementation, particularly as regards module header units (the import <header_file.hpp> mechanism) which
I could not get to work correctly. Furthermore, cmake doesn’t have support for modules in either the current release or in their source trunk; they have some plans for module support but nothing concrete as yet. As such if we went forward with modules we would
be changing our code to introduce a dependency to LLVM on an as-yet-nonexistent compiler and some future version of cmake. So my conclusion here is that this could be a solution 5 or 10 years down the line but doesn’t currently help.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">With regards to writing our own llvm::variant type and using that, I did some investigation into the state-of-the-art of variant implementations to see what the build time characteristics of them are. First, I
found online an incredibly simple implementation that didn’t have all of variants operations and simply used dynamic_cast/rtti everywhere rather than the template trickery found in std::variant (this was just a proof of concept as rtti is not acceptable for
us). I couldn’t get this to compile fully because it didn’t implement all of the operations of variant we actually use, but it did take longer to tell me that it wouldn’t compile than our current implementation takes to compile in total, so this wouldn’t help
even if I went further to get it to compile. I also investigated the variant implementation found here <a href="https://github.com/mpark/variant">https://github.com/mpark/variant</a>. This implementation has had a lot of build time performance work done on
it as you can see from the website (<a href="https://mpark.github.io/variant/">https://mpark.github.io/variant/</a>) however this gives very similar performance to the std::variant currently found in libc++, on which I believe it is based. It seems that the
number/depth of our template instantiations is the problem rather than anything specific to the implementation.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">I also briefly looked at using a custom tagged union implementation rather than std::variant, however I quickly discarded this idea due to the abnormal behaviour of non-trivial types in unions. In short constructors
and destructors are not called automatically, and when changing the active field you need to manually call the destructor of the first type and then placement-new the second type into the same memory space, which is incredibly error prone. As such I don’t
see this as a valid way forward.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">I have also been looking at the parser, since after another look at the trace it seems a large amount of time is spent in the template instantiations of the overloaded operators that are used to generate the parser
as well as the parse_tree. This is something I’m currently looking further into but it’s possible that with some changes to the way the parser is written we could still keep the current variant-based parse tree and see acceptable build times. I feel that it
is the combination of these two things that is causing the build time behaviour we see, and that changing either one could bring us down to acceptable levels. I remember using Boost::Spirit (which the f18 parser seems to have a similar design to) for my dissertation
and seeing very similar build time behaviour to what we see here. In the end I changed to using an external parser generator to avoid this behaviour.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">If anyone has any input on this it’d be very welcome so we can look at a way forward.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Thanks</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">David Truby</span><o:p></o:p></p>
</div>
</div>
</div>
<DIV>
<HR>
</DIV>
<DIV>This email message is for the sole use of the intended recipient(s) and may
contain confidential information. Any unauthorized review, use, disclosure
or distribution is prohibited. If you are not the intended recipient,
please contact the sender by reply email and destroy all copies of the original
message. </DIV>
<DIV>
<HR>
</DIV>
</body>
</html>