[cfe-dev] [Modules TS] Have the file formats been decided?

[Hamza Sood via cfe-dev]

I’ve been looking into Clang’s implementation of the C++ Modules TS to see if there’s anything I can do to help.

From what I understand, a cppm file is essentially treated as one big header file (with handling for a few extra keywords) which is preprocessed and dumped to disk as a pcm file containing a binary representation of the AST. Consumers of the module will end up importing this pcm file as a precompiled header. (If that summary is incorrect then you can stop reading here...)

There are a few problems I ran into with this, which I think are because of this format:
  - Consumers of the module will import the entire implementation of all of the functions in the cppm, which will lead to a lot of duplicated code between object files (and greatly increased compile times).
  - There’s no way to hide declarations that aren’t exported or that are declared as part of the global module.
  - Library developers will have the ship the entire AST for their project if they want users to be able to import it using Modules.
   - Disk usage for large projects with lots of code will be fairly high (not as big of a problem as the others, but still worth a mention).

Is this format decided on? Or is it just an initial test? If it's not yet concrete, then I'd like to propose a slightly different implementation that could potentially solve these problems. While parsing a cppm file, we could construct two ASTs. One containing the entire file as before, and the other consisting of just exported declarations (without their implementations if they aren't inline or templated). The former AST could be used to generate an object file as usual, while the latter could be dumped to disk as a separate interface file (with some kind of special extension). The interface file would essentially serve as a binary "header", containing only what's needed by consumers of the module.

Has anyone got any thoughts on this?