[cfe-dev] [Modules TS] Have the file formats been decided?

[Stephen Kelly via cfe-dev]

Manuel Klimek via cfe-dev wrote:

>> * We rely on library authors knowing to do that (how would they find out,
>> given that clang documentation doesn't recommend it?)
>>
> 
> This is going to become a priority going forward, but it hasn't been yet.
> Specifically, we'll need to discuss this in more detail, and carefully
> design it, optimally pulling in as many stakeholders as we can.

I very much agree with you there!

However another issue is that the Clang modules implementation is 
'intermediate' in that it doesn't follow the TS syntax. It's not clear to me 
what 'standard modules' in Clang would look like in terms of what files 
users write, and what the buildsystem needs to invoke (The Microsoft 
implementation seems to require the user to write 'foo.ixx' files and 
generate '.ifc' files IIRC).

So there may be two systems to carefully design, or maybe 'standard modules' 
in Clang will also rely on modulemaps (so the user will have to create those 
and .ixx files). It's all part of the design work you mention, so I'll look 
forward to that kickstarting. Please keep me in the loop.

> * Library authors won't automatically name things consistently, so we will
>> also get libraries with <name>lib.modulemap and lib<name>.modulemap and
>> doubtless other variations.
>>
> 
> I'd have hoped most would call them name.modulemap

I see :). Hope! I guess the design you mention above can either rely on that 
or not and make it important or not. It's hard to know whether it is 
important until the design work is done.

> * It would be preferable to have the buildsystem hide that detail, if it
>> must exist. That would be possible with cmake usage requirements, but
>> most libraries don't install cmake config files, and consumers don't
>> necessarily use cmake anyway. I guess other buildsystems would have a
>> similar issue.
>>
> 
> Yep. Modules fundamentally change the C++ build, so I don't see a way that
> wouldn't require putting support into build systems.

Indeed, this is true of both 'clang modules' and 'standard modules'. In the 
case of the Microsoft implementation, an entire new step is required for 
each module (whereas often compile flags for the compile step and link flags 
for the link step are enough to build something, now there will be a 
'generate modules' step which will need its own flags and need to be ordered 
correctly). It is not clear to me how much of that is also the case for the 
Clang implementation, and whether that will change if Clang supports 
'standard modules' in the future.

Given that such fundamental buildsystem support is required, I find it 
interesting that it is not part of the conversation about Modules. That's 
critical to adoption, right?

> Both with autotools and cmake you usually call an installed function to
> detect a library, which will give you the flags you need to compile it.
> Those will need to change for a library wanting to be compiled as a
> module.

Yes, but as I wrote above, there will be an entirely new step (with the 
Microsoft implementation at least - I don't know about Clang) with new 
ordering requirements. Supplying different compile flags is not enough.

> Build system support is needed. This is not very different from the
> current state where many libraries you use require specific flags to
> build.

I don't know that that's true. Perhaps that will be obvious when the design 
work you mentioned above is done. If additional steps to the build are 
required, I might term that 'very different'. 

It's not clear to me how names of things will be relevant or will be 
determined. Will files have to be parsed by the buildsystem to determine the 
name of a module ('module M' syntax)? Is that realistic? From looking at 
lack of Fortan support in Ninja (IIUC Fortran has a module system which 
requires doing that) that would be problematic to the point of preventing 
adoption.

> A final note: without build system integration and changes to your code
> structure, modules are likely to make your incremental builds slower.
> Apple has some experience here, see Daniel Dunbar's talk at the llvm dev
> meeting.

Link me please?

> If you think non zero effort will make people not use them, then they'll
> not use them. I don't think that's bad, it's an effort vs payout
> trade-off.

Oh, I don't know about non-zero effort. I'm interested in what the effort is 
and who will make it. That's not clear to me because as you say, it has not 
been designed yet.

Thanks,

Steve.