<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Mon, Oct 8, 2018 at 10:03 AM Matt Asplund <<a href="mailto:mwasplund@gmail.com">mwasplund@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">It isn't so much that the file format has problems or is mismatched, my main concern is that it has too much information to be used as a interface consumption device. The pcm file has all of the required information to compile the original module source, which seems like over kill when being passed in as a definition for the public exported symbols during compilation of the implementation or other module units that import it. </div></blockquote><div><br>Yeah, some bits might be removable - but a lot of it you probably still want there (& the things in the pcm are only the bits from the module interface file - developers can still separate the implementation of the module out into separate files so they wouldn't be part of the pcm) - for example non-exported things that are used from exported things. They would still be necessary or at least useful for optimization purposes, etc.<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I would think a we could use the same file format but allow for the preprocessor, method pool etc to be optional</div></blockquote><div><br>Not sure the preprocessing information could be optional - macros can be exported from modules (maybe only legacy modules? I forget) and are needed for diagnostic (warnings/errors) accuracy too. (not sure what constitutes the "method pool" - but probably similar issues)<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"> and possibly save a trimmed AST that strips out any implementation details. Has there been a performance comparison for the difference between direct to object compilation and cpp -> pcm -> o?</div></blockquote><div><br>Don't think so.<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">
Would it be worth investigating a second file extension that is a simplified form for a precompiled interface? Possibly (cpp -> pcm + pci) then parallel (pcm -> o) and pci for reference.</div></blockquote><div><br>Maybe - though probably wouldn't need another file extension I guess (well, in the sense that pch and pcm use the same format but different extensions, I guess this other thing could use another extension but the same format)<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Another reason I am hesitant to rely entirely on the pcm file format today is for build tooling in the future. In the past the precompiled header was entirely an implementation detail of the compiler itself, but it is my hope that these module definition files can be directly integrated into build tooling and it would be awesome to work toward a compile agnostic file format here. </div></div></blockquote><div><br>That's a pretty major non-goal for modules at this point. For efficiency (the memory mapping, etc rather than having to fully parse the file, etc) and extensibility (supporting different vendor extensions, quirks, etc) the format of Clang's binary module representations is tightly version locked - trying to standardize even just on Clang's modules would be a pretty big project & drag on future development (having to define how to serialize and deserialize all AST features in an even more deliberate way than it's done now) and standardizing across compiler vendors would be even more awkward.<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>For example, a symbol servers of the future could support a single file type that all compiles use as a in intermediate collection of symbols for speedy intellisense when referencing modules!<br></div><div><br></div><div>As for the cppm file extension, it sounds like we have the same concerns, but different solutions. It was my hope that I could standardize the file format for all of my source files to a single extension (I prefer *.cpp) and not require that a module translation unit also have an "m" at the end. Either way, this sounds like a problem that a build system should be trying to solve, not the compiler.</div></div></blockquote><div><br></div><div>Not sure I follow that last bit. (but yeah, perhaps we don't need a separate extension for modules - if that's the case then there's no real opportunity to standardize on one extension - all the existing/old extensions would continue to be as valid in the future as they are now)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>-Matt</div></div><br><div class="gmail_quote"><div dir="ltr">On Mon, Oct 8, 2018 at 8:55 AM David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Sun, Oct 7, 2018 at 10:23 PM Matt Asplund <<a href="mailto:mwasplund@gmail.com" target="_blank">mwasplund@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I also feel that the precompiled header file format as it is today is not the best fit for the module-ts interface definition.</div></blockquote><div><br></div><div>What particular problems/mismatches do you see/find in this?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"> Would it not be more straightforward to have the compilation of an interface module generate a simplified form of this file that only contains the types and declarations that were exported</div></blockquote><div><br></div><div>By simplified form you mean a valid C++ file, but stripped of all the non-exported features?<br><br>That wouldn't achieve the performance goals/desires of Clang's modules (either Clang Modules, or ModulesTS modules) support - compiler performance. The idea is to avoid having to re-parse the source for every use. The bitcode/pcm/pch format is intended to be efficient to query/lazy load (it contains name lookup tables, structures that can be mapped in from disk and read only when needed, etc).</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"> along with an object file? It feels cumbersome at best to generate this entire file and then turn around and spit out the object file in a second call. </div></blockquote><div><br>It's possible to support .pcm+.o file generation at the same time - and I imagine Clang will support that eventually depending on what sort of build system preferences evolve, and how things align between Clang and GCC ModulesTS support.<br><br>But that said, there's good reason/benefits to doing this as two separate steps - it improves build parallelism, allowing builds that use a module to begin before code generation for that module begins (or in parallel with it). Rather than having to wait until the object file generation is completed before downstream modules or other source files can begin compiling.<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>On a side note, would anyone be opposed to lifting the requirement that a interface module translation unit using the cppm extension? The spec simply says a translation unit will be treated as a module unit if it contains a module declaration and clang seems to already allow this for implementation modules. <br></div></div></blockquote><div><br>Possible, sure - though personally I sort of hope this is an opportunity to reduce the proliferation of different C++ file extensions & just standardize on one.<br><br>- Dave<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div></div></div><br><div class="gmail_quote"><div dir="ltr">On Sun, Oct 7, 2018 at 6:19 PM Matt Asplund <<a href="mailto:mwasplund@gmail.com" target="_blank">mwasplund@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I have not had a chance to pull down and build the branch of gcc that has the modules-ts implementation yet. From reading the spec I did not think that macro definitions within a single translation unit would be affected by it being a module unit at all, much less a previously defined macro in the module interface unit. I was testing some with MSVC (until I hit a not implemented error) and they seem to allow this scenario.<div><br></div><div>-Matt</div></div><br><div class="gmail_quote"><div dir="ltr">On Sun, Oct 7, 2018 at 5:55 PM David Blaikie <<a href="mailto:dblaikie@gmail.com" target="_blank">dblaikie@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Sat, Oct 6, 2018 at 5:46 PM Matt Asplund via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I have been finding a few bugs in the modules-ts implementation and wanted to know if this is still being actively work on</div></blockquote><div><br>Certainly being actively worked on - mostly/entirely by Richard Smith (cc'd) & I think he's prioritizing some of the nastier edge cases as they come up in the standardization process, so "practical" bugs (once where the desired behavior is clear enough & it isn't likely to significantly impact the design/implementation in a significant way) might be a bit lower priority.<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"> or if I should be sending out my own patches.</div></blockquote><div><br>Generally welcome, I should think! :)<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"> I am also finding it hard to track what work is being done with clang modules and what work pertains to ModulesTS (Most notably there is only one bucket for bug reporting under modules). </div></blockquote><div><br>Fair - though they're pretty related - not sure it's worth having a separate bucket. But not a big deal either way.<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><br></div><div>The issue I am stuck on right now is this:</div><div><a href="https://bugs.llvm.org/show_bug.cgi?id=39206" style="color:rgb(102,51,102);font-family:Verdana,sans-serif;font-size:16.25px;font-weight:700" target="_blank"><b>Bug 39206</b></a><span style="color:rgb(0,0,0);font-family:Verdana,sans-serif;font-size:16.25px;font-weight:700;background-color:rgb(208,208,208)"> </span><span id="m_-7573330862027461409m_6665818336532648657m_1697017066970453869m_-7156844679760861721m_-6232986939983077607m_-4467552262429952834gmail-summary_container" style="color:rgb(0,0,0);font-family:Verdana,sans-serif;font-size:16.25px;font-weight:700">- <span id="m_-7573330862027461409m_6665818336532648657m_1697017066970453869m_-7156844679760861721m_-6232986939983077607m_-4467552262429952834gmail-short_desc_nonedit_display">[Modules TS] Macro defined in module interface cannot be set in implementation</span></span> </div></div><div dir="ltr"><div><br></div></div></blockquote><div><br></div><div>Thanks for filing! :) (what does GCC do, I wonder? Not sure if the support over there is near enough to support things)</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><br></div><div>-Matt</div></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, Oct 5, 2018 at 4:32 PM André Jansen Medeiros Villar via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">CC Boris Kolpackov who has the only implamentation of Modules on a build system that i know of.<div><br></div></div><br><div class="gmail_quote"><div dir="ltr">Em qui, 4 de out de 2018 às 15:04, David Blaikie via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> escreveu:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Sun, Sep 30, 2018 at 2:32 AM Whisperity via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">I am just trying to improve CMake a bit to better explain directives<br>
for codes that use and depend on modules, instead of using nasty<br>
shadow targets and hacks. We also need to improve CMake so it knows<br>
dependencies: I ran into the issue that once I build the module PCM<br>
there is no check (neither from CMake's, nor Clang's side) whether the<br>
source file for that module was changed. Sometimes it keeps the PCM<br>
implementation ignoring the original one, sometimes just runs Clang<br>
into a nasty stack trace.<br></blockquote><div><br>I'm super interested in CMake support for modules (both Clang's back-compat modules (with modulemaps) and C++ standard modules) - I'll be talking at the developer meeting about some of the issues around Modules TS build system impact (following on from the talk I gave last year on modular code generation) & honestly would love to have a CMake prototype/demo (for clang back-compat modules and/or modules TS modules), with the intent to get something usable for self-hosting one form of explicit modules or another (modular code generation, etc) with the LLVM build, but not sure I really have the CMake skills to pull that off in time.<br><br>- Dave<br> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
; Whisperity.<br>
Matt Asplund (mwasplund) via cfe-dev <<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a>> ezt írta<br>
(időpont: 2018. szept. 29., Szo, 17:15):<br>
><br>
><br>
> Hello,<br>
><br>
> I am curious if there is any active work being done on the Modules TS implementation. I would like to help fill in the gaps that I am finding while playing around with it and wanted to make sure I wasn’t duplicating work.<br>
><br>
> -Matt<br>
> _______________________________________________<br>
> cfe-dev mailing list<br>
> <a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
> <a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div></div>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div>
_______________________________________________<br>
cfe-dev mailing list<br>
<a href="mailto:cfe-dev@lists.llvm.org" target="_blank">cfe-dev@lists.llvm.org</a><br>
<a href="http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev" rel="noreferrer" target="_blank">http://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev</a><br>
</blockquote></div></div>
</blockquote></div>
</blockquote></div>
</blockquote></div></div>
</blockquote></div>
</blockquote></div></div>