[clang] f41f6ea - [C++20] [Modules] Offer -fmodules-embed-all-files option (#107194)

[via cfe-commits]

Author: Chuanqi Xu
Date: 2024-09-20T09:57:46+08:00
New Revision: f41f6ea1f33c4f5e7c94f3d155e44292d1809c50

URL: https://github.com/llvm/llvm-project/commit/f41f6ea1f33c4f5e7c94f3d155e44292d1809c50
DIFF: https://github.com/llvm/llvm-project/commit/f41f6ea1f33c4f5e7c94f3d155e44292d1809c50.diff

LOG: [C++20] [Modules] Offer -fmodules-embed-all-files option (#107194)

See

https://discourse.llvm.org/t/rfc-modules-should-we-embed-sources-to-the-bmi/81029
for details.

Close https://github.com/llvm/llvm-project/issues/72383

Added: 
    clang/test/Driver/fmodules-embed-all-files.cpp

Modified: 
    clang/docs/StandardCPlusPlusModules.rst
    clang/include/clang/Driver/Options.td
    clang/lib/Driver/ToolChains/Clang.cpp

Removed: 
    


################################################################################
diff  --git a/clang/docs/StandardCPlusPlusModules.rst b/clang/docs/StandardCPlusPlusModules.rst
index 2b757a807a0e6e..8e22adad15106e 100644
--- a/clang/docs/StandardCPlusPlusModules.rst
+++ b/clang/docs/StandardCPlusPlusModules.rst
@@ -462,6 +462,37 @@ Currently, Clang accepts the above example, though it may produce surprising
 results if the debugging code depends on consistent use of ``NDEBUG`` in other
 translation units.
 
+Source Files Consistency
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Clang may open the input files\ :sup:`1`` of a BMI during the compilation. This implies that
+when Clang consumes a BMI, all the input files need to be present in the original path
+and with the original contents.
+
+To overcome these requirements and simplify cases like distributed builds and sandboxed
+builds, users can use the ``-fmodules-embed-all-files`` flag to embed all input files
+into the BMI so that Clang does not need to open the corresponding file on disk.
+
+When the ``-fmodules-embed-all-files`` flag are enabled, Clang explicitly emits the source
+code into the BMI file, the contents of the BMI file contain a sufficiently verbose
+representation to reproduce the original source file.
+
+:sup:`1`` Input files: The source files which took part in the compilation of the BMI.
+For example:
+
+.. code-block:: c++
+
+  // M.cppm
+  module;
+  #include "foo.h"
+  export module M;
+
+  // foo.h
+  #pragma once
+  #include "bar.h"
+
+The ``M.cppm``, ``foo.h`` and ``bar.h`` are input files for the BMI of ``M.cppm``.
+
 Object definition consistency
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -484,6 +515,13 @@ fragment is disabled by default. These checks can be enabled by specifying
 and you encounter incorrect or missing diagnostics, please report them via the
 `community issue tracker <https://github.com/llvm/llvm-project/issues/>`_.
 
+Privacy Issue
+-------------
+
+BMIs are not and should not be treated as an information hiding mechanism.
+They should always be assumed to contain all the information that was used to
+create them, in a recoverable form.
+
 ABI Impacts
 -----------
 

diff  --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 021230ceed4cb4..6c6759f19f9b34 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3177,6 +3177,12 @@ def modules_reduced_bmi : Flag<["-"], "fexperimental-modules-reduced-bmi">,
   HelpText<"Generate the reduced BMI">,
   MarshallingInfoFlag<FrontendOpts<"GenReducedBMI">>;
 
+def fmodules_embed_all_files : Joined<["-"], "fmodules-embed-all-files">,
+  Visibility<[ClangOption, CC1Option, CLOption]>,
+  HelpText<"Embed the contents of all files read by this compilation into "
+           "the produced module file.">,
+  MarshallingInfoFlag<FrontendOpts<"ModulesEmbedAllFiles">>;
+
 def fmodules_prune_interval : Joined<["-"], "fmodules-prune-interval=">, Group<i_Group>,
   Visibility<[ClangOption, CC1Option]>, MetaVarName<"<seconds>">,
   HelpText<"Specify the interval (in seconds) between attempts to prune the module cache">,
@@ -7685,10 +7691,6 @@ def fmodules_embed_file_EQ : Joined<["-"], "fmodules-embed-file=">,
   HelpText<"Embed the contents of the specified file into the module file "
            "being compiled.">,
   MarshallingInfoStringVector<FrontendOpts<"ModulesEmbedFiles">>;
-def fmodules_embed_all_files : Joined<["-"], "fmodules-embed-all-files">,
-  HelpText<"Embed the contents of all files read by this compilation into "
-           "the produced module file.">,
-  MarshallingInfoFlag<FrontendOpts<"ModulesEmbedAllFiles">>;
 defm fimplicit_modules_use_lock : BoolOption<"f", "implicit-modules-use-lock",
   FrontendOpts<"BuildingImplicitModuleUsesLock">, DefaultTrue,
   NegFlag<SetFalse>,

diff  --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 7e83ec0e70cbcf..0a2c7b67cc2f0f 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -4196,6 +4196,9 @@ static bool RenderModulesOptions(Compilation &C, const Driver &D,
     Args.ClaimAllArgs(options::OPT_fmodule_output_EQ);
   }
 
+  if (Args.hasArg(options::OPT_fmodules_embed_all_files))
+    CmdArgs.push_back("-fmodules-embed-all-files");
+
   return HaveModules;
 }
 

diff  --git a/clang/test/Driver/fmodules-embed-all-files.cpp b/clang/test/Driver/fmodules-embed-all-files.cpp
new file mode 100644
index 00000000000000..8da836a40446c4
--- /dev/null
+++ b/clang/test/Driver/fmodules-embed-all-files.cpp
@@ -0,0 +1,5 @@
+// RUN: %clang -std=c++20 %s -fmodules-embed-all-files -### 2>&1 | FileCheck %s
+// CHECK: -fmodules-embed-all-files
+
+// RUN: %clang -std=c++20 %s -### 2>&1 | FileCheck %s --check-prefix=NON-EMBED
+// NON-EMBED-NOT: -fmodules-embed-all-files