[clang] [llvm] [clang][MBD] set up module build daemon infrastructure (PR #67562)
Connor Sughrue via llvm-commits
llvm-commits at lists.llvm.org
Sat Apr 20 11:45:30 PDT 2024
https://github.com/cpsughrue updated https://github.com/llvm/llvm-project/pull/67562
>From 8f2e1a2d02227b14dc70fb6898b37b9e0565b296 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sun, 9 Jul 2023 23:19:58 -0400
Subject: [PATCH 01/31] [clang][MBD] set up module build daemon infrastructure
The module build daemon (mbd) will serve as a cc1 tool that helps convert
implicit module command lines to explicit module command lines. A clang
invocation will check to see if a mbd exists, if not the invocation will spawn
one. After the mbd is up and running and a handshake has successfully been
carried out between the mbd and clang invocation the clang invocation will
share it's command line with the mbd. The mbd will then scan the translation
unit and build all modular dependencies before returning a modified cc1 command
line such that all arguments related to modules are converted from the
implicit option to their explicit option.
This commit sets up the basic mbd infrastructure. Including the ability to
spawn a mbd and carry out a handshake between the clang invocation and mbd.
RFC: https://discourse.llvm.org/t/rfc-modules-build-daemon-build-system-agnostic-support-for-explicitly-built-modules/71524
---
.../clang/Basic/DiagnosticFrontendKinds.td | 20 ++
clang/include/clang/Basic/DiagnosticGroups.td | 1 +
clang/include/clang/Driver/Options.td | 13 +
.../include/clang/Frontend/FrontendOptions.h | 8 +
.../Tooling/ModuleBuildDaemon/Frontend.h | 36 +++
.../Tooling/ModuleBuildDaemon/SocketSupport.h | 126 +++++++++
.../clang/Tooling/ModuleBuildDaemon/Utils.h | 58 ++++
clang/lib/Driver/ToolChains/Clang.cpp | 14 +-
clang/lib/Tooling/CMakeLists.txt | 1 +
.../Tooling/ModuleBuildDaemon/CMakeLists.txt | 9 +
.../Tooling/ModuleBuildDaemon/Frontend.cpp | 166 ++++++++++++
.../ModuleBuildDaemon/SocketSupport.cpp | 66 +++++
clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp | 49 ++++
clang/test/Driver/unknown-arg.c | 2 +-
clang/test/Frontend/warning-options.cpp | 2 +-
clang/test/ModuleBuildDaemon/daemon-launch.c | 18 ++
clang/test/ModuleBuildDaemon/handshake.c | 22 ++
clang/test/lit.cfg.py | 11 +
clang/tools/driver/CMakeLists.txt | 3 +
clang/tools/driver/cc1_main.cpp | 7 +
clang/tools/driver/cc1modbuildd_main.cpp | 255 ++++++++++++++++++
clang/tools/driver/driver.cpp | 13 +-
clang/utils/kill_process.py | 86 ++++++
llvm/include/llvm/Support/raw_socket_stream.h | 9 +
llvm/lib/Support/raw_socket_stream.cpp | 24 ++
25 files changed, 1012 insertions(+), 7 deletions(-)
create mode 100644 clang/include/clang/Tooling/ModuleBuildDaemon/Frontend.h
create mode 100644 clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
create mode 100644 clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
create mode 100644 clang/lib/Tooling/ModuleBuildDaemon/CMakeLists.txt
create mode 100644 clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
create mode 100644 clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
create mode 100644 clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
create mode 100644 clang/test/ModuleBuildDaemon/daemon-launch.c
create mode 100644 clang/test/ModuleBuildDaemon/handshake.c
create mode 100644 clang/tools/driver/cc1modbuildd_main.cpp
create mode 100644 clang/utils/kill_process.py
diff --git a/clang/include/clang/Basic/DiagnosticFrontendKinds.td b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
index 14b08d4927ec5e..50e3aab765c7da 100644
--- a/clang/include/clang/Basic/DiagnosticFrontendKinds.td
+++ b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
@@ -264,6 +264,26 @@ def err_test_module_file_extension_version : Error<
"test module file extension '%0' has different version (%1.%2) than expected "
"(%3.%4)">;
+// Module Build Daemon
+def err_basepath_length :
+ Error<"BasePath '%0' is longer then the max length of %1">,
+ DefaultFatal;
+def err_mbd_handshake :
+ Error<"Attempt to handshake with module build daemon has failed: %0">,
+ DefaultFatal;
+def err_mbd_connect :
+ Error<"Attempt to connect to module build daemon has failed: %0">,
+ DefaultFatal;
+def remark_mbd_spawn :
+ Remark<"Successfully spawned module build daemon">,
+ InGroup<ModuleBuildDaemon>;
+def remark_mbd_connection :
+ Remark<"Successfully connected to module build daemon at %0">,
+ InGroup<ModuleBuildDaemon>;
+def remark_mbd_handshake :
+ Remark<"Successfully completed handshake with module build daemon">,
+ InGroup<ModuleBuildDaemon>;
+
def warn_eagerly_load_for_standard_cplusplus_modules : Warning<
"the form '-fmodule-file=<BMI-path>' is deprecated for standard C++ named modules;"
"consider to use '-fmodule-file=<module-name>=<BMI-path>' instead">,
diff --git a/clang/include/clang/Basic/DiagnosticGroups.td b/clang/include/clang/Basic/DiagnosticGroups.td
index 47747d8704b6c8..119b79a0315f7c 100644
--- a/clang/include/clang/Basic/DiagnosticGroups.td
+++ b/clang/include/clang/Basic/DiagnosticGroups.td
@@ -528,6 +528,7 @@ def MissingFieldInitializers : DiagGroup<"missing-field-initializers",
[MissingDesignatedFieldInitializers]>;
def ModuleLock : DiagGroup<"module-lock">;
def ModuleBuild : DiagGroup<"module-build">;
+def ModuleBuildDaemon : DiagGroup<"module-build-daemon">;
def ModuleImport : DiagGroup<"module-import">;
def ModuleConflict : DiagGroup<"module-conflict">;
def ModuleFileExtension : DiagGroup<"module-file-extension">;
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index e24626913add76..80a3e252dce1dc 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -3016,6 +3016,19 @@ defm declspec : BoolOption<"f", "declspec",
NegFlag<SetFalse, [], [ClangOption], "Disallow">,
BothFlags<[], [ClangOption, CC1Option],
" __declspec as a keyword">>, Group<f_clang_Group>;
+
+def fmodule_build_daemon : Flag<["-"], "fmodule-build-daemon">, Group<f_Group>,
+ Flags<[NoXarchOption]>,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Enables module build daemon functionality">,
+ MarshallingInfoFlag<FrontendOpts<"ModuleBuildDaemon">>;
+def fmodule_build_daemon_EQ : Joined<["-"], "fmodule-build-daemon=">, Group<f_Group>,
+ Flags<[NoXarchOption]>,
+ Visibility<[ClangOption, CC1Option]>,
+ HelpText<"Enables module build daemon functionality and defines the path to "
+ "where the module build daemon will write log and socket files">,
+ MarshallingInfoString<FrontendOpts<"ModuleBuildDaemonPath">>;
+
def fmodules_cache_path : Joined<["-"], "fmodules-cache-path=">, Group<i_Group>,
Flags<[]>, Visibility<[ClangOption, CC1Option]>,
MetaVarName<"<directory>">,
diff --git a/clang/include/clang/Frontend/FrontendOptions.h b/clang/include/clang/Frontend/FrontendOptions.h
index a738c1f3757682..70ebe6c443f922 100644
--- a/clang/include/clang/Frontend/FrontendOptions.h
+++ b/clang/include/clang/Frontend/FrontendOptions.h
@@ -404,6 +404,10 @@ class FrontendOptions {
LLVM_PREFERRED_TYPE(bool)
unsigned EmitPrettySymbolGraphs : 1;
+ /// Connect to module build daemon.
+ LLVM_PREFERRED_TYPE(bool)
+ unsigned ModuleBuildDaemon : 1;
+
/// Whether to generate reduced BMI for C++20 named modules.
LLVM_PREFERRED_TYPE(bool)
unsigned GenReducedBMI : 1;
@@ -493,6 +497,10 @@ class FrontendOptions {
/// The output file, if any.
std::string OutputFile;
+ /// If given, the path to the module build daemon's output files and socket
+ /// address
+ std::string ModuleBuildDaemonPath;
+
/// If given, the new suffix for fix-it rewritten files.
std::string FixItSuffix;
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/Frontend.h b/clang/include/clang/Tooling/ModuleBuildDaemon/Frontend.h
new file mode 100644
index 00000000000000..727738ef77b1bd
--- /dev/null
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/Frontend.h
@@ -0,0 +1,36 @@
+//===----------------------------- Frontend.h -----------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_FRONTEND_H
+#define LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_FRONTEND_H
+
+#include "clang/Frontend/CompilerInvocation.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/raw_socket_stream.h"
+
+namespace clang::tooling::cc1modbuildd {
+
+llvm::Error attemptHandshake(llvm::raw_socket_stream &Client,
+ clang::DiagnosticsEngine &Diag);
+
+llvm::Error spawnModuleBuildDaemon(const clang::CompilerInvocation &Clang,
+ const char *Argv0,
+ clang::DiagnosticsEngine &Diag,
+ std::string BasePath);
+
+llvm::Expected<std::unique_ptr<llvm::raw_socket_stream>>
+getModuleBuildDaemon(const clang::CompilerInvocation &Clang, const char *Argv0,
+ clang::DiagnosticsEngine &Diag, llvm::StringRef BasePath);
+
+void spawnModuleBuildDaemonAndHandshake(const clang::CompilerInvocation &Clang,
+ const char *Argv0,
+ clang::DiagnosticsEngine &Diag);
+
+} // namespace clang::tooling::cc1modbuildd
+
+#endif // LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_CLIENT_H
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
new file mode 100644
index 00000000000000..94a10359c61471
--- /dev/null
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
@@ -0,0 +1,126 @@
+//===------------------------- SocketMsgSupport.h -------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_SOCKETMSGSUPPORT_H
+#define LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_SOCKETMSGSUPPORT_H
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/YAMLParser.h"
+#include "llvm/Support/YAMLTraits.h"
+#include "llvm/Support/raw_socket_stream.h"
+
+namespace clang::tooling::cc1modbuildd {
+
+enum class ActionType { HANDSHAKE };
+enum class StatusType { REQUEST, SUCCESS, FAILURE };
+
+struct BaseMsg {
+ ActionType MsgAction;
+ StatusType MsgStatus;
+
+ BaseMsg() = default;
+ BaseMsg(ActionType Action, StatusType Status)
+ : MsgAction(Action), MsgStatus(Status) {}
+};
+
+struct HandshakeMsg : public BaseMsg {
+ HandshakeMsg() = default;
+ HandshakeMsg(ActionType Action, StatusType Status)
+ : BaseMsg(Action, Status) {}
+};
+
+llvm::Expected<std::unique_ptr<llvm::raw_socket_stream>>
+connectToSocket(llvm::StringRef SocketPath);
+llvm::Expected<std::string>
+readBufferFromSocket(llvm::raw_socket_stream &Socket);
+llvm::Error writeBufferToSocket(llvm::raw_socket_stream &Socket,
+ llvm::StringRef Buffer);
+
+template <typename T> std::string convertMsgStructToBuffer(T MsgStruct) {
+ static_assert(std::is_base_of<cc1modbuildd::BaseMsg, T>::value);
+
+ std::string Buffer;
+ llvm::raw_string_ostream OS(Buffer);
+ llvm::yaml::Output YamlOut(OS);
+
+ // TODO confirm yaml::Output does not have any error messages
+ YamlOut << MsgStruct;
+
+ return Buffer;
+}
+
+template <typename T>
+llvm::Expected<T> convertBufferToMsgStruct(llvm::StringRef Buffer) {
+ static_assert(std::is_base_of<cc1modbuildd::BaseMsg, T>::value);
+
+ T MsgStruct;
+ llvm::yaml::Input YamlIn(Buffer);
+ YamlIn >> MsgStruct;
+
+ // YamlIn.error() dumps an error message if there is one
+ if (YamlIn.error())
+ return llvm::make_error<llvm::StringError>(
+ "Syntax or semantic error during YAML parsing",
+ llvm::inconvertibleErrorCode());
+
+ return MsgStruct;
+}
+
+template <typename T>
+llvm::Expected<T> readMsgStructFromSocket(llvm::raw_socket_stream &Socket) {
+ static_assert(std::is_base_of<cc1modbuildd::BaseMsg, T>::value);
+
+ llvm::Expected<std::string> MaybeBuffer = readBufferFromSocket(Socket);
+ if (!MaybeBuffer)
+ return std::move(MaybeBuffer.takeError());
+ std::string Buffer = std::move(*MaybeBuffer);
+
+ llvm::Expected<T> MaybeMsgStruct = convertBufferToMsgStruct<T>(Buffer);
+ if (!MaybeMsgStruct)
+ return std::move(MaybeMsgStruct.takeError());
+ return std::move(*MaybeMsgStruct);
+}
+
+template <typename T>
+llvm::Error writeMsgStructToSocket(llvm::raw_socket_stream &Socket,
+ T MsgStruct) {
+ static_assert(std::is_base_of<cc1modbuildd::BaseMsg, T>::value);
+
+ std::string Buffer = convertMsgStructToBuffer(MsgStruct);
+ if (llvm::Error Err = writeBufferToSocket(Socket, Buffer))
+ return std::move(Err);
+ return llvm::Error::success();
+}
+
+} // namespace clang::tooling::cc1modbuildd
+
+namespace cc1mod = clang::tooling::cc1modbuildd;
+
+template <> struct llvm::yaml::ScalarEnumerationTraits<cc1mod::StatusType> {
+ static void enumeration(IO &Io, cc1mod::StatusType &Value) {
+ Io.enumCase(Value, "REQUEST", cc1mod::StatusType::REQUEST);
+ Io.enumCase(Value, "SUCCESS", cc1mod::StatusType::SUCCESS);
+ Io.enumCase(Value, "FAILURE", cc1mod::StatusType::FAILURE);
+ }
+};
+
+template <> struct llvm::yaml::ScalarEnumerationTraits<cc1mod::ActionType> {
+ static void enumeration(IO &Io, cc1mod::ActionType &Value) {
+ Io.enumCase(Value, "HANDSHAKE", cc1mod::ActionType::HANDSHAKE);
+ }
+};
+
+template <> struct llvm::yaml::MappingTraits<cc1mod::HandshakeMsg> {
+ static void mapping(IO &Io, cc1mod::HandshakeMsg &Info) {
+ Io.mapRequired("Action", Info.MsgAction);
+ Io.mapRequired("Status", Info.MsgStatus);
+ }
+};
+
+#endif // LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_SOCKETMSGSUPPORT_H
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
new file mode 100644
index 00000000000000..77fa4aee1e2ca9
--- /dev/null
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
@@ -0,0 +1,58 @@
+//===------------------------------ Utils.h -------------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+//
+// Functions required by both the frontend and the module build daemon
+//
+//===----------------------------------------------------------------------===//
+
+#ifndef LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_UTILS_H
+#define LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_UTILS_H
+
+#include "llvm/Support/Error.h"
+
+#include <string>
+
+#ifdef _WIN32
+#ifndef NOMINMAX
+#define NOMINMAX
+#endif
+// winsock2.h must be included before afunix.h
+// clang-format off
+#include <winsock2.h>
+#include <afunix.h>
+// clang-format on
+#else
+#include <sys/un.h>
+#endif
+
+namespace clang::tooling::cc1modbuildd {
+
+constexpr std::string_view SOCKET_FILE_NAME = "mbd.sock";
+constexpr std::string_view STDOUT_FILE_NAME = "mbd.out";
+constexpr std::string_view STDERR_FILE_NAME = "mbd.err";
+constexpr std::string_view MODULE_BUILD_DAEMON_FLAG = "-cc1modbuildd";
+
+// A llvm::raw_socket_stream uses sockaddr_un
+constexpr size_t SOCKET_ADDR_MAX_LENGTH = sizeof(sockaddr_un::sun_path);
+
+constexpr size_t BASEPATH_MAX_LENGTH =
+ SOCKET_ADDR_MAX_LENGTH - SOCKET_FILE_NAME.length();
+
+// How long should the module build daemon sit ideal before exiting
+constexpr int TimeoutSec = 15;
+
+// Get a temprary location where the daemon can store log files and a socket
+// address. Of the format /tmp/clang-<BLAKE3HashOfClangFullVersion>/
+std::string getBasePath();
+
+// Check if the user provided BasePath is short enough
+bool validBasePathLength(llvm::StringRef);
+
+} // namespace clang::tooling::cc1modbuildd
+
+#endif
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 096ed14f957046..6bc89a55f1ed74 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -5,7 +5,6 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
-
#include "Clang.h"
#include "AMDGPU.h"
#include "Arch/AArch64.h"
@@ -3868,6 +3867,19 @@ static bool RenderModulesOptions(Compilation &C, const Driver &D,
bool HaveStdCXXModules = IsCXX && HaveStd20;
bool HaveModules = HaveStdCXXModules;
+ // -fmodule-build-daemon enables module build daemon functionality
+ if (Args.hasArg(options::OPT_fmodule_build_daemon))
+ Args.AddLastArg(CmdArgs, options::OPT_fmodule_build_daemon);
+
+ // by default module build daemon socket address and output files are saved
+ // under /tmp/ but that can be overridden by providing the
+ // -fmodule-build-daemon=<path> flag
+ if (Arg *A = Args.getLastArg(options::OPT_fmodule_build_daemon_EQ)) {
+ CmdArgs.push_back(
+ Args.MakeArgString(Twine("-fmodule-build-daemon=") + A->getValue()));
+ CmdArgs.push_back("-fmodule-build-daemon");
+ }
+
// -fmodules enables the use of precompiled modules (off by default).
// Users can pass -fno-cxx-modules to turn off modules support for
// C++/Objective-C++ programs.
diff --git a/clang/lib/Tooling/CMakeLists.txt b/clang/lib/Tooling/CMakeLists.txt
index 91e6cbdcbc44f7..c80b7abd131dac 100644
--- a/clang/lib/Tooling/CMakeLists.txt
+++ b/clang/lib/Tooling/CMakeLists.txt
@@ -13,6 +13,7 @@ add_subdirectory(DumpTool)
add_subdirectory(Syntax)
add_subdirectory(DependencyScanning)
add_subdirectory(Transformer)
+add_subdirectory(ModuleBuildDaemon)
# Replace the last lib component of the current binary directory with include
string(FIND ${CMAKE_CURRENT_BINARY_DIR} "/lib/" PATH_LIB_START REVERSE)
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/CMakeLists.txt b/clang/lib/Tooling/ModuleBuildDaemon/CMakeLists.txt
new file mode 100644
index 00000000000000..c043557206ef9e
--- /dev/null
+++ b/clang/lib/Tooling/ModuleBuildDaemon/CMakeLists.txt
@@ -0,0 +1,9 @@
+set(LLVM_LINK_COMPONENTS
+ Support
+ )
+
+add_clang_library(clangModuleBuildDaemon
+ Frontend.cpp
+ SocketSupport.cpp
+ Utils.cpp
+ )
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
new file mode 100644
index 00000000000000..cf73e27f246bba
--- /dev/null
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -0,0 +1,166 @@
+//===---------------------------- Frontend.cpp ----------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/ModuleBuildDaemon/Frontend.h"
+#include "clang/Frontend/CompilerInstance.h"
+#include "clang/Frontend/FrontendDiagnostic.h"
+#include "clang/Tooling/ModuleBuildDaemon/SocketSupport.h"
+#include "clang/Tooling/ModuleBuildDaemon/Utils.h"
+#include "llvm/ADT/ArrayRef.h"
+#include "llvm/ADT/ScopeExit.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Program.h"
+
+#include <cerrno>
+#include <chrono>
+#include <filesystem>
+#include <fstream>
+#include <optional>
+#include <string>
+#include <thread>
+
+using namespace llvm;
+
+namespace clang::tooling::cc1modbuildd {
+
+llvm::Error attemptHandshake(raw_socket_stream &Client,
+ DiagnosticsEngine &Diag) {
+
+ // Send HandshakeMsg to module build daemon
+ HandshakeMsg Request{ActionType::HANDSHAKE, StatusType::REQUEST};
+ if (llvm::Error Err = writeMsgStructToSocket(Client, Request))
+ return std::move(Err);
+
+ // Read response from module build daemon
+ Expected<HandshakeMsg> MaybeResponse =
+ readMsgStructFromSocket<HandshakeMsg>(Client);
+ if (!MaybeResponse) {
+ return std::move(MaybeResponse.takeError());
+ }
+ HandshakeMsg Response = std::move(*MaybeResponse);
+
+ assert(Response.MsgAction == ActionType::HANDSHAKE &&
+ "The response ActionType should only ever be HANDSHAKE");
+
+ if (Response.MsgStatus == StatusType::SUCCESS) {
+ return llvm::Error::success();
+ }
+
+ return llvm::make_error<StringError>(
+ "Received handshake response 'FAILURE' from module build daemon",
+ std::make_error_code(std::errc::operation_not_permitted));
+}
+
+llvm::Error spawnModuleBuildDaemon(const CompilerInvocation &Clang,
+ const char *Argv0, DiagnosticsEngine &Diag,
+ std::string BasePath) {
+
+ std::vector<StringRef> Args = {Argv0, MODULE_BUILD_DAEMON_FLAG};
+ if (!Clang.getFrontendOpts().ModuleBuildDaemonPath.empty())
+ Args.push_back(BasePath.c_str());
+
+ std::string ErrorBuffer;
+ llvm::sys::ExecuteNoWait(Argv0, Args, std::nullopt, {}, 0, &ErrorBuffer,
+ nullptr, nullptr, /*DetachProcess*/ true);
+
+ // llvm::sys::ExecuteNoWait can fail for a variety of reasons which can't be
+ // generalized to one error code
+ if (!ErrorBuffer.empty())
+ return llvm::make_error<StringError>(ErrorBuffer, inconvertibleErrorCode());
+
+ Diag.Report(diag::remark_mbd_spawn);
+ return llvm::Error::success();
+}
+
+Expected<std::unique_ptr<raw_socket_stream>>
+getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
+ DiagnosticsEngine &Diag, StringRef BasePath) {
+
+ SmallString<128> SocketPath = BasePath;
+ llvm::sys::path::append(SocketPath, SOCKET_FILE_NAME);
+
+ if (llvm::sys::fs::exists(SocketPath)) {
+ Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
+ connectToSocket(SocketPath);
+ if (MaybeClient) {
+ return std::move(*MaybeClient);
+ }
+ consumeError(MaybeClient.takeError());
+ }
+
+ if (llvm::Error Err =
+ spawnModuleBuildDaemon(Clang, Argv0, Diag, BasePath.str()))
+ return std::move(Err);
+
+ constexpr unsigned int MICROSEC_IN_SEC = 1000000;
+ constexpr unsigned int MAX_WAIT_TIME = 30 * MICROSEC_IN_SEC;
+ unsigned int CumulativeTime = 0;
+ unsigned int WaitTime = 10;
+
+ while (CumulativeTime <= MAX_WAIT_TIME) {
+ // Wait a bit then check to see if the module build daemon has initialized
+ std::this_thread::sleep_for(std::chrono::microseconds(WaitTime));
+
+ if (llvm::sys::fs::exists(SocketPath)) {
+ Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
+ connectToSocket(SocketPath);
+ if (MaybeClient) {
+ Diag.Report(diag::remark_mbd_connection) << SocketPath;
+ return std::move(*MaybeClient);
+ }
+ consumeError(MaybeClient.takeError());
+ }
+
+ CumulativeTime += WaitTime;
+ WaitTime = WaitTime * 2;
+ }
+
+ // After waiting around 30 seconds give up and return an error
+ return llvm::make_error<StringError>(
+ "Max wait time exceeded: ",
+ std::make_error_code(std::errc::no_such_process));
+}
+
+void spawnModuleBuildDaemonAndHandshake(const CompilerInvocation &Clang,
+ const char *Argv0,
+ DiagnosticsEngine &Diag) {
+
+ // The module build daemon stores all output files and its socket address
+ // under BasePath. Either set BasePath to a user provided option or create an
+ // appropriate BasePath based on the BLAKE3 hash of the full clang version
+ std::string BasePath;
+ if (Clang.getFrontendOpts().ModuleBuildDaemonPath.empty())
+ BasePath = getBasePath();
+ else {
+ // Get user provided BasePath and confirm it is short enough
+ BasePath = Clang.getFrontendOpts().ModuleBuildDaemonPath;
+ if (!validBasePathLength(BasePath)) {
+ Diag.Report(diag::err_basepath_length) << BasePath << BASEPATH_MAX_LENGTH;
+ return;
+ }
+ }
+
+ // If module build daemon does not exist spawn module build daemon
+ Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
+ getModuleBuildDaemon(Clang, Argv0, Diag, BasePath);
+ if (!MaybeClient) {
+ Diag.Report(diag::err_mbd_connect) << MaybeClient.takeError();
+ return;
+ }
+ raw_socket_stream &Client = **MaybeClient;
+ if (llvm::Error HandshakeErr = attemptHandshake(Client, Diag)) {
+ Diag.Report(diag::err_mbd_handshake) << std::move(HandshakeErr);
+ return;
+ }
+
+ Diag.Report(diag::remark_mbd_handshake);
+ return;
+}
+
+} // namespace clang::tooling::cc1modbuildd
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
new file mode 100644
index 00000000000000..08ddfff8c7f387
--- /dev/null
+++ b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
@@ -0,0 +1,66 @@
+//===------------------------ SocketMsgSupport.cpp ------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/raw_socket_stream.h"
+
+#include <memory>
+#include <string>
+
+using namespace llvm;
+
+namespace clang::tooling::cc1modbuildd {
+
+Expected<std::unique_ptr<raw_socket_stream>>
+connectToSocket(StringRef SocketPath) {
+
+ Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
+ raw_socket_stream::createConnectedUnix(SocketPath);
+ if (!MaybeClient)
+ return std::move(MaybeClient.takeError());
+
+ return std::move(*MaybeClient);
+}
+
+Expected<std::string> readBufferFromSocket(raw_socket_stream &Socket) {
+
+ constexpr unsigned short MAX_BUFFER = 4096;
+ char Buffer[MAX_BUFFER];
+ std::string ReturnBuffer;
+
+ ssize_t n = 0;
+ while ((n = Socket.read(Buffer, MAX_BUFFER)) > 0) {
+ ReturnBuffer.append(Buffer, n);
+ // Read until \n... encountered which is the last line of a YAML document
+ if (ReturnBuffer.find("\n...") != std::string::npos)
+ break;
+ }
+
+ if (Socket.has_error()) {
+ std::error_code EC = Socket.error();
+ Socket.clear_error();
+ return make_error<StringError>("Failed socket read: ", EC);
+ }
+ return ReturnBuffer;
+}
+
+Error writeBufferToSocket(raw_socket_stream &Socket, StringRef Buffer) {
+ Socket << Buffer;
+
+ if (Socket.has_error()) {
+ std::error_code EC = Socket.error();
+ Socket.clear_error();
+ return make_error<StringError>("Failed socket write: ", EC);
+ }
+
+ Socket.flush();
+ return Error::success();
+}
+
+} // namespace clang::tooling::cc1modbuildd
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
new file mode 100644
index 00000000000000..93fc14c4ba23dc
--- /dev/null
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
@@ -0,0 +1,49 @@
+//===------------------------------ Utils.cpp -----------------------------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/ModuleBuildDaemon/Utils.h"
+#include "clang/Basic/Version.h"
+#include "llvm/ADT/SmallString.h"
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/Support/BLAKE3.h"
+#include "llvm/Support/Endian.h"
+#include "llvm/Support/Error.h"
+#include "llvm/Support/Path.h"
+
+#include <string>
+
+using namespace llvm;
+
+namespace clang::tooling::cc1modbuildd {
+
+std::string getBasePath() {
+ llvm::BLAKE3 Hash;
+ Hash.update(clang::getClangFullVersion());
+ auto HashResult = Hash.final<sizeof(uint64_t)>();
+ uint64_t HashValue =
+ llvm::support::endian::read<uint64_t, llvm::endianness::native>(
+ HashResult.data());
+ std::string Key = toString(llvm::APInt(64, HashValue), 36, /*Signed*/ false);
+
+ // Set paths
+ SmallString<128> BasePath;
+ llvm::sys::path::system_temp_directory(/*erasedOnReboot*/ true, BasePath);
+ llvm::sys::path::append(BasePath, "clang-" + Key);
+ return BasePath.c_str();
+}
+
+bool validBasePathLength(llvm::StringRef Address) {
+ // Confirm that the user provided BasePath is short enough to allow the socket
+ // address to fit within the alloted space
+ if (Address.str().length() > BASEPATH_MAX_LENGTH) {
+ return false;
+ }
+ return true;
+}
+
+} // namespace clang::tooling::cc1modbuildd
\ No newline at end of file
diff --git a/clang/test/Driver/unknown-arg.c b/clang/test/Driver/unknown-arg.c
index 26032a407f4131..3a22b824adc5c7 100644
--- a/clang/test/Driver/unknown-arg.c
+++ b/clang/test/Driver/unknown-arg.c
@@ -59,7 +59,7 @@
// SILENT-NOT: warning:
// CC1AS-DID-YOU-MEAN: error: unknown argument '-hell'; did you mean '-help'?
// CC1AS-DID-YOU-MEAN: error: unknown argument '--version'; did you mean '-version'?
-// UNKNOWN-INTEGRATED: error: unknown integrated tool '-cc1asphalt'. Valid tools include '-cc1', '-cc1as' and '-cc1gen-reproducer'.
+// UNKNOWN-INTEGRATED: error: unknown integrated tool '-cc1asphalt'. Valid tools include '-cc1', '-cc1as', '-cc1gen-reproducer', and '-cc1modbuildd'.
// RUN: %clang -S %s -o %t.s -Wunknown-to-clang-option 2>&1 | FileCheck --check-prefix=IGNORED %s
diff --git a/clang/test/Frontend/warning-options.cpp b/clang/test/Frontend/warning-options.cpp
index 444733c8b7f365..42a7109b8d48c5 100644
--- a/clang/test/Frontend/warning-options.cpp
+++ b/clang/test/Frontend/warning-options.cpp
@@ -3,6 +3,6 @@
// CHECK: unknown warning option '-Wmonkey'
// CHECK: unknown warning option '-Wno-monkey'
// CHECK: unknown warning option '-Wno-unused-command-line-arguments'; did you mean '-Wno-unused-command-line-argument'?
-// CHECK: unknown warning option '-Wmodule-build'; did you mean '-Wmodule-conflict'?
+// CHECK: unknown warning option '-Wmodule-build'
// CHECK-NEXT: unknown -Werror warning specifier: '-Werror-vla'
// CHECK: unknown remark option '-Rmodule-built'; did you mean '-Rmodule-build'?
diff --git a/clang/test/ModuleBuildDaemon/daemon-launch.c b/clang/test/ModuleBuildDaemon/daemon-launch.c
new file mode 100644
index 00000000000000..2bfea6aec9dccd
--- /dev/null
+++ b/clang/test/ModuleBuildDaemon/daemon-launch.c
@@ -0,0 +1,18 @@
+// Check that the module build daemon can create a unix socket
+
+// RUN: rm -rf mbd-launch %t
+
+// timeout should exit with status 124 which is treated as a failure by lit on
+// windows. Ideally we would be like to check the exit code and only return true
+// if it equals 124 but lit does not support global bash sysmbols like $?
+
+// RUN: timeout --signal=SIGTERM 2 %clang -cc1modbuildd mbd-launch -v || true
+// RUN: cat mbd-launch/mbd.out | sed 's:\\\\\?:/:g' | FileCheck %s
+
+// CHECK: mbd created and binded to socket at: mbd-launch/mbd.sock
+
+// Make sure mbd.sock does not exist
+// RUN: [ ! -f "mbd-launch/mbd.sock" ] && true || false
+
+// Make sure mbd.err is empty
+// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
diff --git a/clang/test/ModuleBuildDaemon/handshake.c b/clang/test/ModuleBuildDaemon/handshake.c
new file mode 100644
index 00000000000000..059ecf6fae188c
--- /dev/null
+++ b/clang/test/ModuleBuildDaemon/handshake.c
@@ -0,0 +1,22 @@
+// Check that a clang invocation can spawn and handshake with a module build daemon
+
+// RUN: %kill-process "-cc1modbuildd mbd-handshake"
+// RUN: rm -rf mbd-handshake %t
+// RUN: split-file %s %t
+
+//--- main.c
+int main() {return 0;}
+
+// RUN: %clang -fmodule-build-daemon=mbd-handshake -Rmodule-build-daemon %t/main.c &> %t/output-new || true
+// RUN: %clang -fmodule-build-daemon=mbd-handshake -Rmodule-build-daemon %t/main.c &> %t/output-existing || true
+// RUN: %kill-process "-cc1modbuildd mbd-handshake"
+
+// RUN: cat %t/output-new | sed 's:\\\\\?:/:g' | FileCheck %s
+// RUN: cat %t/output-existing | sed 's:\\\\\?:/:g' | FileCheck %s --check-prefix=CHECK-EXIST
+
+// CHECK: remark: Successfully spawned module build daemon [-Rmodule-build-daemon]
+// CHECK: remark: Successfully connected to module build daemon at mbd-handshake/mbd.sock [-Rmodule-build-daemon]
+// CHECK: remark: Successfully completed handshake with module build daemon [-Rmodule-build-daemon]
+
+// Check that a clang invocation can handshake with an existing module build daemon
+// CHECK-EXIST: remark: Successfully completed handshake with module build daemon [-Rmodule-build-daemon]
diff --git a/clang/test/lit.cfg.py b/clang/test/lit.cfg.py
index e5630a07424c7c..19d225ffff82fc 100644
--- a/clang/test/lit.cfg.py
+++ b/clang/test/lit.cfg.py
@@ -205,6 +205,17 @@ def have_host_clang_repl_cuda():
)
)
+config.substitutions.append(
+ (
+ "%kill-process",
+ '"%s" %s'
+ % (
+ config.python_executable,
+ os.path.join(config.clang_src_dir, "utils", "kill_process.py"),
+ ),
+ )
+)
+
config.substitutions.append(("%host_cc", config.host_cc))
config.substitutions.append(("%host_cxx", config.host_cxx))
diff --git a/clang/tools/driver/CMakeLists.txt b/clang/tools/driver/CMakeLists.txt
index d70b92b0984e52..e44475b58b8ac4 100644
--- a/clang/tools/driver/CMakeLists.txt
+++ b/clang/tools/driver/CMakeLists.txt
@@ -28,6 +28,7 @@ add_clang_tool(clang
cc1_main.cpp
cc1as_main.cpp
cc1gen_reproducer_main.cpp
+ cc1modbuildd_main.cpp
DEPENDS
intrinsics_gen
@@ -39,9 +40,11 @@ clang_target_link_libraries(clang
PRIVATE
clangBasic
clangCodeGen
+ clangDependencyScanning
clangDriver
clangFrontend
clangFrontendTool
+ clangModuleBuildDaemon
clangSerialization
)
diff --git a/clang/tools/driver/cc1_main.cpp b/clang/tools/driver/cc1_main.cpp
index b5c6be3c557bb3..b124d09a752f4e 100644
--- a/clang/tools/driver/cc1_main.cpp
+++ b/clang/tools/driver/cc1_main.cpp
@@ -25,6 +25,7 @@
#include "clang/Frontend/TextDiagnosticPrinter.h"
#include "clang/Frontend/Utils.h"
#include "clang/FrontendTool/Utils.h"
+#include "clang/Tooling/ModuleBuildDaemon/Frontend.h"
#include "llvm/ADT/Statistic.h"
#include "llvm/Config/llvm-config.h"
#include "llvm/LinkAllPasses.h"
@@ -226,6 +227,12 @@ int cc1_main(ArrayRef<const char *> Argv, const char *Argv0, void *MainAddr) {
return 1;
}
+ // Handle module build daemon functionality if enabled
+ if (Clang->getFrontendOpts().ModuleBuildDaemon) {
+ clang::tooling::cc1modbuildd::spawnModuleBuildDaemonAndHandshake(
+ Clang->getInvocation(), Argv0, Clang->getDiagnostics());
+ }
+
// Execute the frontend actions.
{
llvm::TimeTraceScope TimeScope("ExecuteCompiler");
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
new file mode 100644
index 00000000000000..e62bdd476abe83
--- /dev/null
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -0,0 +1,255 @@
+//===------- cc1modbuildd_main.cpp - Clang CC1 Module Build Daemon --------===//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===----------------------------------------------------------------------===//
+
+#include "clang/Tooling/ModuleBuildDaemon/SocketSupport.h"
+#include "clang/Tooling/ModuleBuildDaemon/Utils.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/ThreadPool.h"
+
+#include <csignal>
+#include <cstdbool>
+#include <fstream>
+#include <optional>
+#include <string>
+#include <system_error>
+
+#ifdef _WIN32
+#include <windows.h>
+#else
+#include <unistd.h>
+#endif
+
+using namespace llvm;
+using namespace clang::tooling::cc1modbuildd;
+
+// Create unbuffered STDOUT stream so that any logging done by the module build
+// daemon can be viewed without having to terminate the process
+static raw_fd_ostream &unbuff_outs() {
+ static raw_fd_ostream S(fileno(stdout), false, true);
+ return S;
+}
+
+static bool VerboseLog = false;
+static void verboseLog(const llvm::Twine &message) {
+ if (VerboseLog) {
+ unbuff_outs() << message << '\n';
+ }
+}
+
+namespace {
+
+class ModuleBuildDaemonServer {
+public:
+ SmallString<256> SocketPath;
+ SmallString<256> STDERR;
+ SmallString<256> STDOUT;
+
+ ModuleBuildDaemonServer(StringRef Path)
+ : SocketPath(Path), STDERR(Path), STDOUT(Path) {
+ llvm::sys::path::append(SocketPath, SOCKET_FILE_NAME);
+ llvm::sys::path::append(STDOUT, STDOUT_FILE_NAME);
+ llvm::sys::path::append(STDERR, STDERR_FILE_NAME);
+ }
+
+ ~ModuleBuildDaemonServer() { shutdownDaemon(); }
+
+ int setupDaemonEnv();
+ int createDaemonSocket();
+ int listenForClients();
+
+ static void handleConnection(std::shared_ptr<raw_socket_stream> Connection);
+
+ // TODO: modify so when shutdownDaemon is called the daemon stops accepting
+ // new client connections and waits for all existing client connections to
+ // terminate before closing the file descriptor and exiting
+ void shutdownDaemon() {
+ RunServiceLoop = false;
+ if (ServerListener.has_value())
+ ServerListener.value().shutdown();
+ }
+
+private:
+ std::atomic<bool> RunServiceLoop = true;
+ std::optional<llvm::ListeningSocket> ServerListener;
+};
+
+// Used to handle signals
+ModuleBuildDaemonServer *DaemonPtr = nullptr;
+void handleSignal(int) {
+ DaemonPtr->shutdownDaemon();
+ exit(EXIT_SUCCESS);
+}
+} // namespace
+
+// Sets up file descriptors and signals for module build daemon
+int ModuleBuildDaemonServer::setupDaemonEnv() {
+
+#ifdef _WIN32
+ freopen("NUL", "r", stdin);
+#else
+ close(STDIN_FILENO);
+#endif
+
+ freopen(STDOUT.c_str(), "a", stdout);
+ freopen(STDERR.c_str(), "a", stderr);
+
+ if (std::signal(SIGTERM, handleSignal) == SIG_ERR) {
+ errs() << "failed to handle SIGTERM" << '\n';
+ exit(EXIT_FAILURE);
+ }
+
+ if (std::signal(SIGINT, handleSignal) == SIG_ERR) {
+ errs() << "failed to handle SIGINT" << '\n';
+ exit(EXIT_FAILURE);
+ }
+
+// TODO: Figure out how to do this on windows
+#ifdef SIGHUP
+ if (::signal(SIGHUP, SIG_IGN) == SIG_ERR) {
+ errs() << "failed to handle SIGHUP" << '\n';
+ exit(EXIT_FAILURE);
+ }
+#endif
+
+ return EXIT_SUCCESS;
+}
+
+// Creates unix socket for IPC with frontends
+int ModuleBuildDaemonServer::createDaemonSocket() {
+
+ Expected<ListeningSocket> MaybeServerListener =
+ llvm::ListeningSocket::createUnix(SocketPath);
+
+ if (llvm::Error Err = MaybeServerListener.takeError()) {
+ llvm::handleAllErrors(std::move(Err), [&](const llvm::StringError &SE) {
+ std::error_code EC = SE.convertToErrorCode();
+ // Exit successfully if the socket address is already in use. When
+ // translation units are compiled in parallel, until the socket file is
+ // created, all clang invocations will try to spawn a module build daemon.
+#ifdef _WIN32
+ if (EC.value() == WSAEADDRINUSE) {
+#else
+ if (EC == std::errc::address_in_use) {
+#endif
+ exit(EXIT_SUCCESS);
+ } else {
+ llvm::errs() << "MBD failed to create unix socket: " << SE.message()
+ << EC.message() << '\n';
+ exit(EXIT_FAILURE);
+ }
+ });
+ }
+
+ verboseLog("mbd created and binded to socket at: " + SocketPath);
+ ServerListener.emplace(std::move(*MaybeServerListener));
+ return 0;
+}
+
+// Function submitted to thread pool with each frontend connection. Not
+// responsible for closing frontend socket connections
+void ModuleBuildDaemonServer::handleConnection(
+ std::shared_ptr<llvm::raw_socket_stream> MovableConnection) {
+
+ llvm::raw_socket_stream &Connection = *MovableConnection;
+
+ // Read request from frontend
+ Expected<HandshakeMsg> MaybeHandshakeMsg =
+ readMsgStructFromSocket<HandshakeMsg>(Connection);
+ if (!MaybeHandshakeMsg) {
+ errs() << "MBD failed to read frontend request: "
+ << llvm::toString(MaybeHandshakeMsg.takeError()) << '\n';
+ return;
+ }
+
+ // Send response to frontend
+ HandshakeMsg Msg(ActionType::HANDSHAKE, StatusType::SUCCESS);
+ if (llvm::Error WriteErr = writeMsgStructToSocket(Connection, Msg)) {
+ errs() << "MBD failed to respond to frontend request: "
+ << llvm::toString(std::move(WriteErr)) << '\n';
+ return;
+ }
+ return;
+}
+
+int ModuleBuildDaemonServer::listenForClients() {
+
+ llvm::ThreadPool Pool;
+
+ while (RunServiceLoop) {
+ Expected<std::unique_ptr<raw_socket_stream>> MaybeConnection =
+ ServerListener.value().accept({TimeoutSec, 0});
+
+ if (llvm::Error Err = MaybeConnection.takeError()) {
+
+ llvm::handleAllErrors(std::move(Err), [&](const llvm::StringError &SE) {
+ std::error_code EC = SE.convertToErrorCode();
+ if (EC == std::errc::timed_out)
+ RunServiceLoop = false;
+ else
+ errs() << "MBD failed to accept incoming connection: "
+ << SE.getMessage() << ": " << EC.message() << '\n';
+ });
+
+ continue;
+ }
+
+ std::shared_ptr<raw_socket_stream> Connection(std::move(*MaybeConnection));
+ Pool.async(handleConnection, Connection);
+ }
+ return 0;
+}
+
+// Module build daemon is spawned with the following command line:
+//
+// clang -cc1modbuildd [<path>] [-v]
+//
+// OPTIONS
+// <path>
+// Specifies the path to all the files created by the module build daemon.
+// If provided, <path> should immediately follow -cc1modbuildd.
+//
+// -v
+// Provides verbose debug information.
+//
+// NOTES
+// The arguments <path> and -v are optional. If <path> is not provided then
+// BasePath will be /tmp/clang-<BLAKE3HashOfClangFullVersion>
+//
+int cc1modbuildd_main(ArrayRef<const char *> Argv) {
+
+ // -cc1modbuildd is sliced away when Argv is pased to cc1modbuildd_main
+ if (find(Argv, StringRef("-v")) != Argv.end())
+ VerboseLog = true;
+
+ std::string BasePath;
+ // If an argument exists and it is not -v then it must be a BasePath
+ if (!Argv.empty() && strcmp(Argv[0], "-v") != 0)
+ BasePath = Argv[0];
+ else
+ BasePath = getBasePath();
+
+ if (!validBasePathLength(BasePath)) {
+ errs() << "BasePath '" << BasePath << "' is longer then the max length of "
+ << std::to_string(BASEPATH_MAX_LENGTH) << '\n';
+ return 1;
+ }
+
+ llvm::sys::fs::create_directories(BasePath);
+ ModuleBuildDaemonServer Daemon(BasePath);
+
+ // Used to handle signals
+ DaemonPtr = &Daemon;
+
+ Daemon.setupDaemonEnv();
+ Daemon.createDaemonSocket();
+ Daemon.listenForClients();
+
+ return EXIT_SUCCESS;
+}
diff --git a/clang/tools/driver/driver.cpp b/clang/tools/driver/driver.cpp
index 83b5bbb71f5212..54ff34e7426b06 100644
--- a/clang/tools/driver/driver.cpp
+++ b/clang/tools/driver/driver.cpp
@@ -84,6 +84,7 @@ extern int cc1as_main(ArrayRef<const char *> Argv, const char *Argv0,
extern int cc1gen_reproducer_main(ArrayRef<const char *> Argv,
const char *Argv0, void *MainAddr,
const llvm::ToolContext &);
+extern int cc1modbuildd_main(ArrayRef<const char *> Argv);
static void insertTargetAndModeArgs(const ParsedClangName &NameParts,
SmallVectorImpl<const char *> &ArgVector,
@@ -218,10 +219,14 @@ static int ExecuteCC1Tool(SmallVectorImpl<const char *> &ArgV,
if (Tool == "-cc1gen-reproducer")
return cc1gen_reproducer_main(ArrayRef(ArgV).slice(2), ArgV[0],
GetExecutablePathVP, ToolContext);
- // Reject unknown tools.
- llvm::errs()
- << "error: unknown integrated tool '" << Tool << "'. "
- << "Valid tools include '-cc1', '-cc1as' and '-cc1gen-reproducer'.\n";
+ if (Tool == "-cc1modbuildd") {
+ return cc1modbuildd_main(ArrayRef(ArgV).slice(2));
+ }
+
+ // Reject unknown tools
+ llvm::errs() << "error: unknown integrated tool '" << Tool << "'. "
+ << "Valid tools include '-cc1', '-cc1as', '-cc1gen-reproducer', "
+ "and '-cc1modbuildd'.\n";
return 1;
}
diff --git a/clang/utils/kill_process.py b/clang/utils/kill_process.py
new file mode 100644
index 00000000000000..68d0e4ed8dc629
--- /dev/null
+++ b/clang/utils/kill_process.py
@@ -0,0 +1,86 @@
+#!/usr/bin/env python3
+
+# Kills all processes whos command line match provided pattern
+
+import os
+import signal
+import subprocess
+import sys
+import platform
+
+
+def windows_impl(search_pattern):
+ cmd = "wmic process get ProcessId,CommandLine"
+ output = subprocess.check_output(cmd, shell=True, universal_newlines=True)
+ lines = output.split("\n")
+
+ PIDs = []
+
+ for line in lines:
+ if "clang" in line and "kill_process" not in line and "llvm-lit" not in line:
+
+ # creates something like ['..\\llvm-project\\build\\bin\\clang.exe -cc1modbuildd mbd-launch -v', '16260']
+ parts = line.rsplit(None, 1)
+
+ if len(parts) == 2:
+ command_line, pid = parts
+ if search_pattern in command_line:
+ PIDs.append(int(pid))
+
+ return PIDs
+
+
+def unix_impl(search_pattern):
+ ps_output = subprocess.check_output(["ps", "aux"], universal_newlines=True)
+ ps_lines = ps_output.split("\n")
+
+ PIDs = []
+
+ for line in ps_lines:
+ if "clang" in line and "kill_process" not in line and "llvm-lit" not in line:
+ ps_line_components = line.split()
+
+ # output of ps aux
+ # 0 1 2 3 4 5 6 7 8 9 10
+ # USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
+
+ # skip line without COMMAND
+ if len(ps_line_components) < 11:
+ continue
+
+ command_line = " ".join(ps_line_components[10:])
+ if search_pattern in command_line:
+ PIDs.append(int(ps_line_components[1]))
+
+ return PIDs
+
+
+def main():
+ if len(sys.argv) == 1:
+ sys.stderr.write("Error: no search pattern provided\n")
+ sys.exit(1)
+
+ search_pattern = sys.argv[1]
+ sys.stdout.write(
+ f"Searching for process with pattern '{search_pattern}' in command line\n"
+ )
+
+ system_name = platform.system()
+
+ PIDs = []
+ if system_name == "Windows":
+ PIDs = windows_impl(search_pattern)
+ else:
+ PIDs = unix_impl(search_pattern)
+
+ if len(PIDs) == 0:
+ sys.stdout.write("No processes matching search pattern were found\n")
+ return
+
+ sys.stdout.write(f"Killing PIDs {PIDs}\n")
+ for PID in PIDs:
+ os.kill(PID, signal.SIGTERM)
+
+
+if __name__ == "__main__":
+ main()
diff --git a/llvm/include/llvm/Support/raw_socket_stream.h b/llvm/include/llvm/Support/raw_socket_stream.h
index bddd47eb75e1a9..23d557b1470010 100644
--- a/llvm/include/llvm/Support/raw_socket_stream.h
+++ b/llvm/include/llvm/Support/raw_socket_stream.h
@@ -110,6 +110,15 @@ class ListeningSocket {
static Expected<ListeningSocket> createUnix(
StringRef SocketPath,
int MaxBacklog = llvm::hardware_concurrency().compute_thread_count());
+
+ // The use of a default parameter was choosen over std::optional to more
+ // closely resemble the accept system call. A user should be able to call
+ // ListeningSocket.accept() rather then ListeningSocket.accept(std::nullopt)
+ // if they do not want accept to ever timeout
+ Expected<std::unique_ptr<raw_socket_stream>> accept(timeval TV = {-1, -1});
+ ListeningSocket(ListeningSocket &&LS);
+ void shutdown();
+ ~ListeningSocket();
};
//===----------------------------------------------------------------------===//
diff --git a/llvm/lib/Support/raw_socket_stream.cpp b/llvm/lib/Support/raw_socket_stream.cpp
index 14e2308df4d7ed..ef703331a79f5f 100644
--- a/llvm/lib/Support/raw_socket_stream.cpp
+++ b/llvm/lib/Support/raw_socket_stream.cpp
@@ -250,6 +250,7 @@ ListeningSocket::accept(std::chrono::milliseconds Timeout) {
}
void ListeningSocket::shutdown() {
+<<<<<<< HEAD
int ObservedFD = FD.load();
if (ObservedFD == -1)
@@ -259,6 +260,29 @@ void ListeningSocket::shutdown() {
// another thread is responsible for shutdown so return
if (!FD.compare_exchange_strong(ObservedFD, -1))
return;
+=======
+ if (FD == -1)
+ return;
+ ::close(FD);
+ ::unlink(SocketPath.c_str());
+
+ FD = -1;
+}
+
+ListeningSocket::~ListeningSocket() { shutdown(); }
+
+static Expected<int> GetSocketFD(StringRef SocketPath) {
+#ifdef _WIN32
+ SOCKET MaybeWinsocket = socket(AF_UNIX, SOCK_STREAM, 0);
+ if (MaybeWinsocket == INVALID_SOCKET) {
+#else
+ int MaybeWinsocket = socket(AF_UNIX, SOCK_STREAM, 0);
+ if (MaybeWinsocket == -1) {
+#endif // _WIN32
+ return llvm::make_error<StringError>(getLastSocketErrorCode(),
+ "Create socket failed");
+ }
+>>>>>>> [clang][MBD] set up module build daemon infrastructure
::close(ObservedFD);
::unlink(SocketPath.c_str());
>From ced437113bdf962af7c89d72609c212532ae0fd5 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Tue, 6 Feb 2024 16:48:44 -0500
Subject: [PATCH 02/31] Set ModuleBuildDaemonServer member functions to return
void
---
clang/tools/driver/cc1modbuildd_main.cpp | 17 +++++++----------
1 file changed, 7 insertions(+), 10 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index e62bdd476abe83..234f8894bf6d19 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -60,9 +60,9 @@ class ModuleBuildDaemonServer {
~ModuleBuildDaemonServer() { shutdownDaemon(); }
- int setupDaemonEnv();
- int createDaemonSocket();
- int listenForClients();
+ void setupDaemonEnv();
+ void createDaemonSocket();
+ void listenForClients();
static void handleConnection(std::shared_ptr<raw_socket_stream> Connection);
@@ -89,7 +89,7 @@ void handleSignal(int) {
} // namespace
// Sets up file descriptors and signals for module build daemon
-int ModuleBuildDaemonServer::setupDaemonEnv() {
+void ModuleBuildDaemonServer::setupDaemonEnv() {
#ifdef _WIN32
freopen("NUL", "r", stdin);
@@ -117,12 +117,10 @@ int ModuleBuildDaemonServer::setupDaemonEnv() {
exit(EXIT_FAILURE);
}
#endif
-
- return EXIT_SUCCESS;
}
// Creates unix socket for IPC with frontends
-int ModuleBuildDaemonServer::createDaemonSocket() {
+void ModuleBuildDaemonServer::createDaemonSocket() {
Expected<ListeningSocket> MaybeServerListener =
llvm::ListeningSocket::createUnix(SocketPath);
@@ -149,7 +147,6 @@ int ModuleBuildDaemonServer::createDaemonSocket() {
verboseLog("mbd created and binded to socket at: " + SocketPath);
ServerListener.emplace(std::move(*MaybeServerListener));
- return 0;
}
// Function submitted to thread pool with each frontend connection. Not
@@ -178,7 +175,7 @@ void ModuleBuildDaemonServer::handleConnection(
return;
}
-int ModuleBuildDaemonServer::listenForClients() {
+void ModuleBuildDaemonServer::listenForClients() {
llvm::ThreadPool Pool;
@@ -200,10 +197,10 @@ int ModuleBuildDaemonServer::listenForClients() {
continue;
}
+ // Connection must be copy constructable to be passed to Pool.async
std::shared_ptr<raw_socket_stream> Connection(std::move(*MaybeConnection));
Pool.async(handleConnection, Connection);
}
- return 0;
}
// Module build daemon is spawned with the following command line:
>From ae59a58248401bf67104e6b96cb190204f101762 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Tue, 6 Feb 2024 17:35:09 -0500
Subject: [PATCH 03/31] Modify ListeningServer::accept to take
std::optional<std::chrono::microsecond>
---
.../clang/Tooling/ModuleBuildDaemon/Utils.h | 4 ++--
.../Tooling/ModuleBuildDaemon/Frontend.cpp | 9 ++++---
clang/tools/driver/cc1modbuildd_main.cpp | 4 +++-
llvm/include/llvm/Support/raw_socket_stream.h | 3 ++-
llvm/lib/Support/raw_socket_stream.cpp | 24 -------------------
5 files changed, 11 insertions(+), 33 deletions(-)
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
index 77fa4aee1e2ca9..c96a5a9540ad02 100644
--- a/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
@@ -15,6 +15,7 @@
#include "llvm/Support/Error.h"
+#include <chrono>
#include <string>
#ifdef _WIN32
@@ -43,8 +44,7 @@ constexpr size_t SOCKET_ADDR_MAX_LENGTH = sizeof(sockaddr_un::sun_path);
constexpr size_t BASEPATH_MAX_LENGTH =
SOCKET_ADDR_MAX_LENGTH - SOCKET_FILE_NAME.length();
-// How long should the module build daemon sit ideal before exiting
-constexpr int TimeoutSec = 15;
+constexpr std::chrono::microseconds MICROSEC_IN_SEC(1000000);
// Get a temprary location where the daemon can store log files and a socket
// address. Of the format /tmp/clang-<BLAKE3HashOfClangFullVersion>/
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index cf73e27f246bba..dad7d7f3802501 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -98,14 +98,13 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
spawnModuleBuildDaemon(Clang, Argv0, Diag, BasePath.str()))
return std::move(Err);
- constexpr unsigned int MICROSEC_IN_SEC = 1000000;
- constexpr unsigned int MAX_WAIT_TIME = 30 * MICROSEC_IN_SEC;
- unsigned int CumulativeTime = 0;
- unsigned int WaitTime = 10;
+ constexpr std::chrono::microseconds MAX_WAIT_TIME(30 * MICROSEC_IN_SEC);
+ std::chrono::microseconds CumulativeTime(0);
+ std::chrono::microseconds WaitTime(10);
while (CumulativeTime <= MAX_WAIT_TIME) {
// Wait a bit then check to see if the module build daemon has initialized
- std::this_thread::sleep_for(std::chrono::microseconds(WaitTime));
+ std::this_thread::sleep_for(WaitTime);
if (llvm::sys::fs::exists(SocketPath)) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 234f8894bf6d19..4467a73c69a89d 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -178,10 +178,12 @@ void ModuleBuildDaemonServer::handleConnection(
void ModuleBuildDaemonServer::listenForClients() {
llvm::ThreadPool Pool;
+ // How long should the module build daemon sit ideal before exiting
+ std::chrono::microseconds DaemonTimeout(15 * MICROSEC_IN_SEC);
while (RunServiceLoop) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeConnection =
- ServerListener.value().accept({TimeoutSec, 0});
+ ServerListener.value().accept(DaemonTimeout);
if (llvm::Error Err = MaybeConnection.takeError()) {
diff --git a/llvm/include/llvm/Support/raw_socket_stream.h b/llvm/include/llvm/Support/raw_socket_stream.h
index 23d557b1470010..2b54e20e959533 100644
--- a/llvm/include/llvm/Support/raw_socket_stream.h
+++ b/llvm/include/llvm/Support/raw_socket_stream.h
@@ -115,7 +115,8 @@ class ListeningSocket {
// closely resemble the accept system call. A user should be able to call
// ListeningSocket.accept() rather then ListeningSocket.accept(std::nullopt)
// if they do not want accept to ever timeout
- Expected<std::unique_ptr<raw_socket_stream>> accept(timeval TV = {-1, -1});
+ Expected<std::unique_ptr<raw_socket_stream>>
+ accept(std::optional<std::chrono::microseconds> Timeout = std::nullopt);
ListeningSocket(ListeningSocket &&LS);
void shutdown();
~ListeningSocket();
diff --git a/llvm/lib/Support/raw_socket_stream.cpp b/llvm/lib/Support/raw_socket_stream.cpp
index ef703331a79f5f..14e2308df4d7ed 100644
--- a/llvm/lib/Support/raw_socket_stream.cpp
+++ b/llvm/lib/Support/raw_socket_stream.cpp
@@ -250,7 +250,6 @@ ListeningSocket::accept(std::chrono::milliseconds Timeout) {
}
void ListeningSocket::shutdown() {
-<<<<<<< HEAD
int ObservedFD = FD.load();
if (ObservedFD == -1)
@@ -260,29 +259,6 @@ void ListeningSocket::shutdown() {
// another thread is responsible for shutdown so return
if (!FD.compare_exchange_strong(ObservedFD, -1))
return;
-=======
- if (FD == -1)
- return;
- ::close(FD);
- ::unlink(SocketPath.c_str());
-
- FD = -1;
-}
-
-ListeningSocket::~ListeningSocket() { shutdown(); }
-
-static Expected<int> GetSocketFD(StringRef SocketPath) {
-#ifdef _WIN32
- SOCKET MaybeWinsocket = socket(AF_UNIX, SOCK_STREAM, 0);
- if (MaybeWinsocket == INVALID_SOCKET) {
-#else
- int MaybeWinsocket = socket(AF_UNIX, SOCK_STREAM, 0);
- if (MaybeWinsocket == -1) {
-#endif // _WIN32
- return llvm::make_error<StringError>(getLastSocketErrorCode(),
- "Create socket failed");
- }
->>>>>>> [clang][MBD] set up module build daemon infrastructure
::close(ObservedFD);
::unlink(SocketPath.c_str());
>From 2617db8bf000c285b4bc8664d0da705ef1327e88 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Tue, 6 Feb 2024 21:46:02 -0500
Subject: [PATCH 04/31] Make raw_socket_stream FD atomic
---
llvm/include/llvm/Support/raw_socket_stream.h | 11 ++++-------
llvm/lib/Support/raw_socket_stream.cpp | 2 ++
2 files changed, 6 insertions(+), 7 deletions(-)
diff --git a/llvm/include/llvm/Support/raw_socket_stream.h b/llvm/include/llvm/Support/raw_socket_stream.h
index 2b54e20e959533..7c912f376ab906 100644
--- a/llvm/include/llvm/Support/raw_socket_stream.h
+++ b/llvm/include/llvm/Support/raw_socket_stream.h
@@ -111,15 +111,12 @@ class ListeningSocket {
StringRef SocketPath,
int MaxBacklog = llvm::hardware_concurrency().compute_thread_count());
- // The use of a default parameter was choosen over std::optional to more
- // closely resemble the accept system call. A user should be able to call
- // ListeningSocket.accept() rather then ListeningSocket.accept(std::nullopt)
- // if they do not want accept to ever timeout
Expected<std::unique_ptr<raw_socket_stream>>
accept(std::optional<std::chrono::microseconds> Timeout = std::nullopt);
- ListeningSocket(ListeningSocket &&LS);
- void shutdown();
- ~ListeningSocket();
+
+ static Expected<ListeningSocket> createUnix(
+ StringRef SocketPath,
+ int MaxBacklog = llvm::hardware_concurrency().compute_thread_count());
};
//===----------------------------------------------------------------------===//
diff --git a/llvm/lib/Support/raw_socket_stream.cpp b/llvm/lib/Support/raw_socket_stream.cpp
index 14e2308df4d7ed..4908d02d52f364 100644
--- a/llvm/lib/Support/raw_socket_stream.cpp
+++ b/llvm/lib/Support/raw_socket_stream.cpp
@@ -20,6 +20,8 @@
#include <fcntl.h>
#include <thread>
+#include <atomic>
+
#ifndef _WIN32
#include <poll.h>
#include <sys/socket.h>
>From c423b100323076072e005b9705acfaff3efbd945 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Tue, 6 Feb 2024 22:18:34 -0500
Subject: [PATCH 05/31] Remove exit in signal handler
---
clang/test/ModuleBuildDaemon/handshake.c | 6 ++++++
clang/tools/driver/cc1modbuildd_main.cpp | 14 +++++++-------
2 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/clang/test/ModuleBuildDaemon/handshake.c b/clang/test/ModuleBuildDaemon/handshake.c
index 059ecf6fae188c..84bdf9751131e7 100644
--- a/clang/test/ModuleBuildDaemon/handshake.c
+++ b/clang/test/ModuleBuildDaemon/handshake.c
@@ -20,3 +20,9 @@ int main() {return 0;}
// Check that a clang invocation can handshake with an existing module build daemon
// CHECK-EXIST: remark: Successfully completed handshake with module build daemon [-Rmodule-build-daemon]
+
+// Make sure mbd.sock does not exist
+// RUN: [ ! -f "mbd-launch/mbd.sock" ] && true || false
+
+// Make sure mbd.err is empty
+// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 4467a73c69a89d..5253abc7dee94a 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -82,10 +82,7 @@ class ModuleBuildDaemonServer {
// Used to handle signals
ModuleBuildDaemonServer *DaemonPtr = nullptr;
-void handleSignal(int) {
- DaemonPtr->shutdownDaemon();
- exit(EXIT_SUCCESS);
-}
+void handleSignal(int) { DaemonPtr->shutdownDaemon(); }
} // namespace
// Sets up file descriptors and signals for module build daemon
@@ -178,7 +175,6 @@ void ModuleBuildDaemonServer::handleConnection(
void ModuleBuildDaemonServer::listenForClients() {
llvm::ThreadPool Pool;
- // How long should the module build daemon sit ideal before exiting
std::chrono::microseconds DaemonTimeout(15 * MICROSEC_IN_SEC);
while (RunServiceLoop) {
@@ -189,9 +185,13 @@ void ModuleBuildDaemonServer::listenForClients() {
llvm::handleAllErrors(std::move(Err), [&](const llvm::StringError &SE) {
std::error_code EC = SE.convertToErrorCode();
- if (EC == std::errc::timed_out)
+
+ if (EC == std::errc::timed_out) {
RunServiceLoop = false;
- else
+ verboseLog("ListeningServer::accept timed out, shutting down");
+ } else if (EC == std::errc::interrupted && RunServiceLoop == false) {
+ verboseLog("Signal received, shutting down");
+ } else
errs() << "MBD failed to accept incoming connection: "
<< SE.getMessage() << ": " << EC.message() << '\n';
});
>From dd7a0dc5a09a78ad062253edb598cad94408bce8 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 7 Feb 2024 01:24:12 -0500
Subject: [PATCH 06/31] Create setupSignal functionality so I can set signals
back to their default state after ServerListener is destructed
---
clang/tools/driver/cc1modbuildd_main.cpp | 72 +++++++++++++++---------
1 file changed, 46 insertions(+), 26 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 5253abc7dee94a..12475d98ccbd0a 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -43,6 +43,34 @@ static void verboseLog(const llvm::Twine &message) {
}
}
+static void setupSignals(sighandler_t handler) {
+
+ if (std::signal(SIGTERM, handler) == SIG_ERR) {
+ errs() << "failed to handle SIGTERM" << '\n';
+ exit(EXIT_FAILURE);
+ }
+
+ if (std::signal(SIGINT, handler) == SIG_ERR) {
+ errs() << "failed to handle SIGINT" << '\n';
+ exit(EXIT_FAILURE);
+ }
+
+// TODO: Figure out how to do this on windows
+#ifdef SIGHUP
+ if (handler != SIG_DFL) {
+ if (::signal(SIGHUP, SIG_IGN) == SIG_ERR) {
+ errs() << "failed to handle SIGHUP" << '\n';
+ exit(EXIT_FAILURE);
+ }
+ } else {
+ if (::signal(SIGHUP, SIG_DFL) == SIG_ERR) {
+ errs() << "failed to handle SIGHUP" << '\n';
+ exit(EXIT_FAILURE);
+ }
+ }
+#endif
+}
+
namespace {
class ModuleBuildDaemonServer {
@@ -58,8 +86,6 @@ class ModuleBuildDaemonServer {
llvm::sys::path::append(STDERR, STDERR_FILE_NAME);
}
- ~ModuleBuildDaemonServer() { shutdownDaemon(); }
-
void setupDaemonEnv();
void createDaemonSocket();
void listenForClients();
@@ -73,6 +99,10 @@ class ModuleBuildDaemonServer {
RunServiceLoop = false;
if (ServerListener.has_value())
ServerListener.value().shutdown();
+
+ // Prevents the ServerListener destructor being called after
+ // ServerListener::shutdown is run. We do not want to call ::unlink twice
+ exit(0);
}
private:
@@ -97,23 +127,7 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
freopen(STDOUT.c_str(), "a", stdout);
freopen(STDERR.c_str(), "a", stderr);
- if (std::signal(SIGTERM, handleSignal) == SIG_ERR) {
- errs() << "failed to handle SIGTERM" << '\n';
- exit(EXIT_FAILURE);
- }
-
- if (std::signal(SIGINT, handleSignal) == SIG_ERR) {
- errs() << "failed to handle SIGINT" << '\n';
- exit(EXIT_FAILURE);
- }
-
-// TODO: Figure out how to do this on windows
-#ifdef SIGHUP
- if (::signal(SIGHUP, SIG_IGN) == SIG_ERR) {
- errs() << "failed to handle SIGHUP" << '\n';
- exit(EXIT_FAILURE);
- }
-#endif
+ setupSignals(handleSignal);
}
// Creates unix socket for IPC with frontends
@@ -237,18 +251,24 @@ int cc1modbuildd_main(ArrayRef<const char *> Argv) {
if (!validBasePathLength(BasePath)) {
errs() << "BasePath '" << BasePath << "' is longer then the max length of "
<< std::to_string(BASEPATH_MAX_LENGTH) << '\n';
- return 1;
+ return EXIT_FAILURE;
}
llvm::sys::fs::create_directories(BasePath);
- ModuleBuildDaemonServer Daemon(BasePath);
- // Used to handle signals
- DaemonPtr = &Daemon;
+ {
+ ModuleBuildDaemonServer Daemon(BasePath);
+
+ // Used to handle signals
+ DaemonPtr = &Daemon;
+
+ Daemon.setupDaemonEnv();
+ Daemon.createDaemonSocket();
+ Daemon.listenForClients();
+ }
- Daemon.setupDaemonEnv();
- Daemon.createDaemonSocket();
- Daemon.listenForClients();
+ // Prevents handleSignal from being called after ServerListener is destructed
+ setupSignals(SIG_DFL);
return EXIT_SUCCESS;
}
>From d42dc0a4f26509b79472c4317b294e2a77946b76 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 7 Feb 2024 01:24:50 -0500
Subject: [PATCH 07/31] Make blacker happy
---
clang/utils/kill_process.py | 1 -
1 file changed, 1 deletion(-)
diff --git a/clang/utils/kill_process.py b/clang/utils/kill_process.py
index 68d0e4ed8dc629..cf277e43b9d097 100644
--- a/clang/utils/kill_process.py
+++ b/clang/utils/kill_process.py
@@ -18,7 +18,6 @@ def windows_impl(search_pattern):
for line in lines:
if "clang" in line and "kill_process" not in line and "llvm-lit" not in line:
-
# creates something like ['..\\llvm-project\\build\\bin\\clang.exe -cc1modbuildd mbd-launch -v', '16260']
parts = line.rsplit(None, 1)
>From 0704b7a7f80296e452ed254889520843e90c50bf Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 7 Feb 2024 03:02:07 -0500
Subject: [PATCH 08/31] Fix bug where if daemon crashes and does not unlink
socket file the next daemon can remove the file before trying to bind
---
.../Tooling/ModuleBuildDaemon/Frontend.cpp | 3 +-
clang/tools/driver/cc1modbuildd_main.cpp | 51 +++++++++++--------
2 files changed, 32 insertions(+), 22 deletions(-)
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index dad7d7f3802501..8a456ff728fb40 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -88,9 +88,8 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
if (llvm::sys::fs::exists(SocketPath)) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
connectToSocket(SocketPath);
- if (MaybeClient) {
+ if (MaybeClient)
return std::move(*MaybeClient);
- }
consumeError(MaybeClient.takeError());
}
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 12475d98ccbd0a..4bcee57536a9dd 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -133,31 +133,42 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
// Creates unix socket for IPC with frontends
void ModuleBuildDaemonServer::createDaemonSocket() {
- Expected<ListeningSocket> MaybeServerListener =
- llvm::ListeningSocket::createUnix(SocketPath);
-
- if (llvm::Error Err = MaybeServerListener.takeError()) {
- llvm::handleAllErrors(std::move(Err), [&](const llvm::StringError &SE) {
- std::error_code EC = SE.convertToErrorCode();
- // Exit successfully if the socket address is already in use. When
- // translation units are compiled in parallel, until the socket file is
- // created, all clang invocations will try to spawn a module build daemon.
+ bool SocketCreated = false;
+ while (!SocketCreated) {
+
+ Expected<ListeningSocket> MaybeServerListener =
+ llvm::ListeningSocket::createUnix(SocketPath);
+
+ if (llvm::Error Err = MaybeServerListener.takeError()) {
+ llvm::handleAllErrors(std::move(Err), [&](const llvm::StringError &SE) {
+ std::error_code EC = SE.convertToErrorCode();
+
+ // Exit successfully if the socket address is already in use. When
+ // TUs are compiled in parallel, until the socket file is created, all
+ // clang invocations will try to spawn a module build daemon.
#ifdef _WIN32
- if (EC.value() == WSAEADDRINUSE) {
+ if (EC.value() == WSAEADDRINUSE) {
#else
if (EC == std::errc::address_in_use) {
#endif
- exit(EXIT_SUCCESS);
- } else {
- llvm::errs() << "MBD failed to create unix socket: " << SE.message()
- << EC.message() << '\n';
- exit(EXIT_FAILURE);
- }
- });
+ exit(EXIT_SUCCESS);
+ } else if (EC == std::errc::file_exists) {
+ if (std::error_code EC = llvm::sys::fs::remove(SocketPath); EC) {
+ llvm::errs() << "Failed to remove file: " << EC.message() << '\n';
+ exit(EXIT_FAILURE);
+ }
+ } else {
+ llvm::errs() << "MBD failed to create unix socket: "
+ << SE.getMessage() << ": " << EC.message() << '\n';
+ exit(EXIT_FAILURE);
+ }
+ });
+ } else {
+ SocketCreated = true;
+ verboseLog("mbd created and binded to socket at: " + SocketPath);
+ ServerListener.emplace(std::move(*MaybeServerListener));
+ }
}
-
- verboseLog("mbd created and binded to socket at: " + SocketPath);
- ServerListener.emplace(std::move(*MaybeServerListener));
}
// Function submitted to thread pool with each frontend connection. Not
>From 7d99272fcd00a573ba442c8fc5a9c6aa1f2a16a2 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 10 Feb 2024 16:19:21 -0500
Subject: [PATCH 09/31] Remove std::move(s) preventing copy elision
-Wpessimizing-move
---
clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h | 2 +-
clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp | 4 ++--
clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp | 2 +-
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
index 94a10359c61471..88c71be936f5af 100644
--- a/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
@@ -94,7 +94,7 @@ llvm::Error writeMsgStructToSocket(llvm::raw_socket_stream &Socket,
std::string Buffer = convertMsgStructToBuffer(MsgStruct);
if (llvm::Error Err = writeBufferToSocket(Socket, Buffer))
- return std::move(Err);
+ return Err;
return llvm::Error::success();
}
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index 8a456ff728fb40..e7fffa9823568f 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -35,13 +35,13 @@ llvm::Error attemptHandshake(raw_socket_stream &Client,
// Send HandshakeMsg to module build daemon
HandshakeMsg Request{ActionType::HANDSHAKE, StatusType::REQUEST};
if (llvm::Error Err = writeMsgStructToSocket(Client, Request))
- return std::move(Err);
+ return Err;
// Read response from module build daemon
Expected<HandshakeMsg> MaybeResponse =
readMsgStructFromSocket<HandshakeMsg>(Client);
if (!MaybeResponse) {
- return std::move(MaybeResponse.takeError());
+ return MaybeResponse.takeError();
}
HandshakeMsg Response = std::move(*MaybeResponse);
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
index 08ddfff8c7f387..be0665147a0a45 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
@@ -23,7 +23,7 @@ connectToSocket(StringRef SocketPath) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
raw_socket_stream::createConnectedUnix(SocketPath);
if (!MaybeClient)
- return std::move(MaybeClient.takeError());
+ return MaybeClient.takeError();
return std::move(*MaybeClient);
}
>From da46c51e6305e40fe7547054fd744b520d1376b2 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 10 Feb 2024 21:08:01 -0500
Subject: [PATCH 10/31] Remove exit in shutdownDeamon because ListeningSocket
destructor will return if FD == -1
---
clang/tools/driver/cc1modbuildd_main.cpp | 4 ----
1 file changed, 4 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 4bcee57536a9dd..03f6b78590f595 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -99,10 +99,6 @@ class ModuleBuildDaemonServer {
RunServiceLoop = false;
if (ServerListener.has_value())
ServerListener.value().shutdown();
-
- // Prevents the ServerListener destructor being called after
- // ServerListener::shutdown is run. We do not want to call ::unlink twice
- exit(0);
}
private:
>From 3cd0d836ba905ff0f417121b77a7bbb4b70fde7d Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 10 Feb 2024 21:18:28 -0500
Subject: [PATCH 11/31] Take advantage of ability to convert between
std::seconds and std::microseconds
---
clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h | 2 --
clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp | 4 ++--
clang/tools/driver/cc1modbuildd_main.cpp | 2 +-
3 files changed, 3 insertions(+), 5 deletions(-)
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
index c96a5a9540ad02..b37eb18210b344 100644
--- a/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
@@ -44,8 +44,6 @@ constexpr size_t SOCKET_ADDR_MAX_LENGTH = sizeof(sockaddr_un::sun_path);
constexpr size_t BASEPATH_MAX_LENGTH =
SOCKET_ADDR_MAX_LENGTH - SOCKET_FILE_NAME.length();
-constexpr std::chrono::microseconds MICROSEC_IN_SEC(1000000);
-
// Get a temprary location where the daemon can store log files and a socket
// address. Of the format /tmp/clang-<BLAKE3HashOfClangFullVersion>/
std::string getBasePath();
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index e7fffa9823568f..3b2688417cbc70 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -97,11 +97,11 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
spawnModuleBuildDaemon(Clang, Argv0, Diag, BasePath.str()))
return std::move(Err);
- constexpr std::chrono::microseconds MAX_WAIT_TIME(30 * MICROSEC_IN_SEC);
+ std::chrono::seconds MaxWaitTime(30);
std::chrono::microseconds CumulativeTime(0);
std::chrono::microseconds WaitTime(10);
- while (CumulativeTime <= MAX_WAIT_TIME) {
+ while (CumulativeTime <= MaxWaitTime) {
// Wait a bit then check to see if the module build daemon has initialized
std::this_thread::sleep_for(WaitTime);
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 03f6b78590f595..6f1d0d4d3630ec 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -196,7 +196,7 @@ void ModuleBuildDaemonServer::handleConnection(
void ModuleBuildDaemonServer::listenForClients() {
llvm::ThreadPool Pool;
- std::chrono::microseconds DaemonTimeout(15 * MICROSEC_IN_SEC);
+ std::chrono::seconds DaemonTimeout(15);
while (RunServiceLoop) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeConnection =
>From 5bd0c30cb1c97404d96e5fc426d177406393fc40 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 10 Feb 2024 21:22:43 -0500
Subject: [PATCH 12/31] Switch to break statement to get rid of state whn
calling createUnix
---
clang/tools/driver/cc1modbuildd_main.cpp | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 6f1d0d4d3630ec..a52c2294e75a69 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -129,9 +129,7 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
// Creates unix socket for IPC with frontends
void ModuleBuildDaemonServer::createDaemonSocket() {
- bool SocketCreated = false;
- while (!SocketCreated) {
-
+ while (true) {
Expected<ListeningSocket> MaybeServerListener =
llvm::ListeningSocket::createUnix(SocketPath);
@@ -160,9 +158,9 @@ void ModuleBuildDaemonServer::createDaemonSocket() {
}
});
} else {
- SocketCreated = true;
verboseLog("mbd created and binded to socket at: " + SocketPath);
ServerListener.emplace(std::move(*MaybeServerListener));
+ break;
}
}
}
>From 40fb927ab3dd211f7a9a4989e8f29bcc897ede41 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 10 Feb 2024 21:35:01 -0500
Subject: [PATCH 13/31] Fix the style of a few variable names
---
.../clang/Tooling/ModuleBuildDaemon/Utils.h | 14 +++++++-------
.../lib/Tooling/ModuleBuildDaemon/Frontend.cpp | 6 +++---
clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp | 4 ++--
clang/tools/driver/cc1modbuildd_main.cpp | 18 +++++++++---------
4 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
index b37eb18210b344..df721cc8a91861 100644
--- a/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/Utils.h
@@ -33,16 +33,16 @@
namespace clang::tooling::cc1modbuildd {
-constexpr std::string_view SOCKET_FILE_NAME = "mbd.sock";
-constexpr std::string_view STDOUT_FILE_NAME = "mbd.out";
-constexpr std::string_view STDERR_FILE_NAME = "mbd.err";
-constexpr std::string_view MODULE_BUILD_DAEMON_FLAG = "-cc1modbuildd";
+constexpr std::string_view SocketFileName = "mbd.sock";
+constexpr std::string_view StdoutFileName = "mbd.out";
+constexpr std::string_view StderrFileName = "mbd.err";
+constexpr std::string_view ModuleBuildDaemonFlag = "-cc1modbuildd";
// A llvm::raw_socket_stream uses sockaddr_un
-constexpr size_t SOCKET_ADDR_MAX_LENGTH = sizeof(sockaddr_un::sun_path);
+constexpr size_t SocketAddrMaxLength = sizeof(sockaddr_un::sun_path);
-constexpr size_t BASEPATH_MAX_LENGTH =
- SOCKET_ADDR_MAX_LENGTH - SOCKET_FILE_NAME.length();
+constexpr size_t BasePathMaxLength =
+ SocketAddrMaxLength - SocketFileName.length();
// Get a temprary location where the daemon can store log files and a socket
// address. Of the format /tmp/clang-<BLAKE3HashOfClangFullVersion>/
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index 3b2688417cbc70..8c7f440ad79236 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -61,7 +61,7 @@ llvm::Error spawnModuleBuildDaemon(const CompilerInvocation &Clang,
const char *Argv0, DiagnosticsEngine &Diag,
std::string BasePath) {
- std::vector<StringRef> Args = {Argv0, MODULE_BUILD_DAEMON_FLAG};
+ std::vector<StringRef> Args = {Argv0, ModuleBuildDaemonFlag};
if (!Clang.getFrontendOpts().ModuleBuildDaemonPath.empty())
Args.push_back(BasePath.c_str());
@@ -83,7 +83,7 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
DiagnosticsEngine &Diag, StringRef BasePath) {
SmallString<128> SocketPath = BasePath;
- llvm::sys::path::append(SocketPath, SOCKET_FILE_NAME);
+ llvm::sys::path::append(SocketPath, SocketFileName);
if (llvm::sys::fs::exists(SocketPath)) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
@@ -139,7 +139,7 @@ void spawnModuleBuildDaemonAndHandshake(const CompilerInvocation &Clang,
// Get user provided BasePath and confirm it is short enough
BasePath = Clang.getFrontendOpts().ModuleBuildDaemonPath;
if (!validBasePathLength(BasePath)) {
- Diag.Report(diag::err_basepath_length) << BasePath << BASEPATH_MAX_LENGTH;
+ Diag.Report(diag::err_basepath_length) << BasePath << BasePathMaxLength;
return;
}
}
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
index 93fc14c4ba23dc..9ce1ba849bd83c 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
@@ -39,8 +39,8 @@ std::string getBasePath() {
bool validBasePathLength(llvm::StringRef Address) {
// Confirm that the user provided BasePath is short enough to allow the socket
- // address to fit within the alloted space
- if (Address.str().length() > BASEPATH_MAX_LENGTH) {
+ // address to fit within the space alloted to sockaddr_un::sun_path
+ if (Address.str().length() > BasePathMaxLength) {
return false;
}
return true;
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index a52c2294e75a69..49a62006cdcec3 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -76,14 +76,14 @@ namespace {
class ModuleBuildDaemonServer {
public:
SmallString<256> SocketPath;
- SmallString<256> STDERR;
- SmallString<256> STDOUT;
+ SmallString<256> Stderr; // path to stderr
+ SmallString<256> Stdout; // path to stdout
ModuleBuildDaemonServer(StringRef Path)
- : SocketPath(Path), STDERR(Path), STDOUT(Path) {
- llvm::sys::path::append(SocketPath, SOCKET_FILE_NAME);
- llvm::sys::path::append(STDOUT, STDOUT_FILE_NAME);
- llvm::sys::path::append(STDERR, STDERR_FILE_NAME);
+ : SocketPath(Path), Stderr(Path), Stdout(Path) {
+ llvm::sys::path::append(SocketPath, SocketFileName);
+ llvm::sys::path::append(Stdout, StdoutFileName);
+ llvm::sys::path::append(Stderr, StderrFileName);
}
void setupDaemonEnv();
@@ -120,8 +120,8 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
close(STDIN_FILENO);
#endif
- freopen(STDOUT.c_str(), "a", stdout);
- freopen(STDERR.c_str(), "a", stderr);
+ freopen(Stdout.c_str(), "a", stdout);
+ freopen(Stderr.c_str(), "a", stderr);
setupSignals(handleSignal);
}
@@ -255,7 +255,7 @@ int cc1modbuildd_main(ArrayRef<const char *> Argv) {
if (!validBasePathLength(BasePath)) {
errs() << "BasePath '" << BasePath << "' is longer then the max length of "
- << std::to_string(BASEPATH_MAX_LENGTH) << '\n';
+ << std::to_string(BasePathMaxLength) << '\n';
return EXIT_FAILURE;
}
>From 76fb8cfcbaca22003ffcc37beffd2575a1341288 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 10 Feb 2024 21:41:18 -0500
Subject: [PATCH 14/31] After ModuleBuildDaemonServer deconstructs ignore
signals
---
clang/tools/driver/cc1modbuildd_main.cpp | 21 +++++++--------------
1 file changed, 7 insertions(+), 14 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 49a62006cdcec3..32841df1f58e7e 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -43,7 +43,7 @@ static void verboseLog(const llvm::Twine &message) {
}
}
-static void setupSignals(sighandler_t handler) {
+static void modifySignals(sighandler_t handler) {
if (std::signal(SIGTERM, handler) == SIG_ERR) {
errs() << "failed to handle SIGTERM" << '\n';
@@ -55,18 +55,10 @@ static void setupSignals(sighandler_t handler) {
exit(EXIT_FAILURE);
}
-// TODO: Figure out how to do this on windows
#ifdef SIGHUP
- if (handler != SIG_DFL) {
- if (::signal(SIGHUP, SIG_IGN) == SIG_ERR) {
- errs() << "failed to handle SIGHUP" << '\n';
- exit(EXIT_FAILURE);
- }
- } else {
- if (::signal(SIGHUP, SIG_DFL) == SIG_ERR) {
- errs() << "failed to handle SIGHUP" << '\n';
- exit(EXIT_FAILURE);
- }
+ if (::signal(SIGHUP, SIG_IGN) == SIG_ERR) {
+ errs() << "failed to handle SIGHUP" << '\n';
+ exit(EXIT_FAILURE);
}
#endif
}
@@ -123,7 +115,7 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
freopen(Stdout.c_str(), "a", stdout);
freopen(Stderr.c_str(), "a", stderr);
- setupSignals(handleSignal);
+ modifySignals(handleSignal);
}
// Creates unix socket for IPC with frontends
@@ -273,7 +265,8 @@ int cc1modbuildd_main(ArrayRef<const char *> Argv) {
}
// Prevents handleSignal from being called after ServerListener is destructed
- setupSignals(SIG_DFL);
+ // Daemon is shutting down at this point so signals can be ignored
+ modifySignals(SIG_IGN);
return EXIT_SUCCESS;
}
>From 16949971f116096e1a0c019acc0d821b3cbc1dcc Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sun, 11 Feb 2024 00:29:17 -0500
Subject: [PATCH 15/31] Switch over to using non-deprecated version of close
and unlink for windows and improve remove socket file capabilities
---
clang/tools/driver/cc1modbuildd_main.cpp | 7 +++++--
llvm/include/llvm/Support/raw_socket_stream.h | 1 +
2 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 32841df1f58e7e..e2ffc06771ed52 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -15,6 +15,8 @@
#include <csignal>
#include <cstdbool>
+#include <cstdio>
+#include <cstring>
#include <fstream>
#include <optional>
#include <string>
@@ -139,8 +141,9 @@ void ModuleBuildDaemonServer::createDaemonSocket() {
#endif
exit(EXIT_SUCCESS);
} else if (EC == std::errc::file_exists) {
- if (std::error_code EC = llvm::sys::fs::remove(SocketPath); EC) {
- llvm::errs() << "Failed to remove file: " << EC.message() << '\n';
+ if (std::remove(SocketPath.c_str()) != 0) {
+ llvm::errs() << "Failed to remove file: " << strerror(errno)
+ << '\n';
exit(EXIT_FAILURE);
}
} else {
diff --git a/llvm/include/llvm/Support/raw_socket_stream.h b/llvm/include/llvm/Support/raw_socket_stream.h
index 7c912f376ab906..0c966633e38f9c 100644
--- a/llvm/include/llvm/Support/raw_socket_stream.h
+++ b/llvm/include/llvm/Support/raw_socket_stream.h
@@ -130,6 +130,7 @@ class raw_socket_stream : public raw_fd_stream {
#endif // _WIN32
public:
+ // TODO: Should probably be private
raw_socket_stream(int SocketFD);
/// Create a \p raw_socket_stream connected to the UNIX domain socket at \p
/// SocketPath.
>From 785dbb74dcc25adb7c1cee02c0b581dc6db872be Mon Sep 17 00:00:00 2001
From: Connor Sughrue <cpsughrue at gmail.com>
Date: Mon, 12 Feb 2024 23:01:09 -0500
Subject: [PATCH 16/31] fix windows build and formatting issues
---
clang/tools/driver/cc1modbuildd_main.cpp | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index e2ffc06771ed52..3764f4909bc2be 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -45,7 +45,7 @@ static void verboseLog(const llvm::Twine &message) {
}
}
-static void modifySignals(sighandler_t handler) {
+static void modifySignals(decltype(SIG_DFL) handler) {
if (std::signal(SIGTERM, handler) == SIG_ERR) {
errs() << "failed to handle SIGTERM" << '\n';
>From 2afdd942aeb5b6ce9d4135d09eebdc9f2acc37df Mon Sep 17 00:00:00 2001
From: Connor Sughrue <cpsughrue at gmail.com>
Date: Tue, 13 Feb 2024 20:37:15 -0500
Subject: [PATCH 17/31] Remove check that mbd.sock was removed - fails on
windows due to how files are deleted
---
clang/test/ModuleBuildDaemon/daemon-launch.c | 3 ---
clang/test/ModuleBuildDaemon/handshake.c | 3 ---
2 files changed, 6 deletions(-)
diff --git a/clang/test/ModuleBuildDaemon/daemon-launch.c b/clang/test/ModuleBuildDaemon/daemon-launch.c
index 2bfea6aec9dccd..00e94e9ba456df 100644
--- a/clang/test/ModuleBuildDaemon/daemon-launch.c
+++ b/clang/test/ModuleBuildDaemon/daemon-launch.c
@@ -11,8 +11,5 @@
// CHECK: mbd created and binded to socket at: mbd-launch/mbd.sock
-// Make sure mbd.sock does not exist
-// RUN: [ ! -f "mbd-launch/mbd.sock" ] && true || false
-
// Make sure mbd.err is empty
// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
diff --git a/clang/test/ModuleBuildDaemon/handshake.c b/clang/test/ModuleBuildDaemon/handshake.c
index 84bdf9751131e7..af71daa89e90db 100644
--- a/clang/test/ModuleBuildDaemon/handshake.c
+++ b/clang/test/ModuleBuildDaemon/handshake.c
@@ -21,8 +21,5 @@ int main() {return 0;}
// Check that a clang invocation can handshake with an existing module build daemon
// CHECK-EXIST: remark: Successfully completed handshake with module build daemon [-Rmodule-build-daemon]
-// Make sure mbd.sock does not exist
-// RUN: [ ! -f "mbd-launch/mbd.sock" ] && true || false
-
// Make sure mbd.err is empty
// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
>From db966dfaae0590575c673e88a85b3d415eeaa75f Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Tue, 13 Feb 2024 21:52:05 -0500
Subject: [PATCH 18/31] Write test to crash situation where MBD crashes
---
clang/test/ModuleBuildDaemon/daemon-crash.c | 20 ++++++++++++++++++++
clang/test/ModuleBuildDaemon/daemon-launch.c | 3 ++-
clang/tools/driver/cc1modbuildd_main.cpp | 11 +++++++----
3 files changed, 29 insertions(+), 5 deletions(-)
create mode 100644 clang/test/ModuleBuildDaemon/daemon-crash.c
diff --git a/clang/test/ModuleBuildDaemon/daemon-crash.c b/clang/test/ModuleBuildDaemon/daemon-crash.c
new file mode 100644
index 00000000000000..67bccfc3489b5c
--- /dev/null
+++ b/clang/test/ModuleBuildDaemon/daemon-crash.c
@@ -0,0 +1,20 @@
+// Check that the module build daemon can create a unix socket after a crash
+
+// RUN: rm -rf mbd-crash %t
+
+// timeout should exit with status 124 which is treated as a failure by lit on
+// windows. Ideally we would be like to check the exit code and only return true
+// if it equals 124 but lit does not support global bash sysmbols like $?
+
+// RUN: timeout --signal=SIGKILL 2 %clang -cc1modbuildd mbd-crash -v || true
+// RUN: timeout --signal=SIGTERM 2 %clang -cc1modbuildd mbd-crash -v || true
+// RUN: cat mbd-crash/mbd.out | sed 's:\\\\\?:/:g' | FileCheck %s
+
+// There should only be one shutdown log line due to the crash
+// CHECK: MBD created and binded to socket at: mbd-crash/mbd.sock
+// CHECK-NEXT: Removing ineligible file: mbd-crash/mbd.sock
+// CHECK-NEXT: MBD created and binded to socket at: mbd-crash/mbd.sock
+// CHECK-NEXT: Signal received, shutting down
+
+// Make sure mbd.err is empty
+// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
diff --git a/clang/test/ModuleBuildDaemon/daemon-launch.c b/clang/test/ModuleBuildDaemon/daemon-launch.c
index 00e94e9ba456df..9c148ae029cf3c 100644
--- a/clang/test/ModuleBuildDaemon/daemon-launch.c
+++ b/clang/test/ModuleBuildDaemon/daemon-launch.c
@@ -9,7 +9,8 @@
// RUN: timeout --signal=SIGTERM 2 %clang -cc1modbuildd mbd-launch -v || true
// RUN: cat mbd-launch/mbd.out | sed 's:\\\\\?:/:g' | FileCheck %s
-// CHECK: mbd created and binded to socket at: mbd-launch/mbd.sock
+// CHECK: MBD created and binded to socket at: mbd-launch/mbd.sock
+// CHECK-NEXT: Signal received, shutting down
// Make sure mbd.err is empty
// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 3764f4909bc2be..f5c6e4e0ca0791 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -137,15 +137,18 @@ void ModuleBuildDaemonServer::createDaemonSocket() {
#ifdef _WIN32
if (EC.value() == WSAEADDRINUSE) {
#else
- if (EC == std::errc::address_in_use) {
+ if (EC == std::errc::address_in_use) {
#endif
exit(EXIT_SUCCESS);
} else if (EC == std::errc::file_exists) {
if (std::remove(SocketPath.c_str()) != 0) {
- llvm::errs() << "Failed to remove file: " << strerror(errno)
- << '\n';
+ llvm::errs() << "Failed to remove " << SocketPath << ": "
+ << strerror(errno) << '\n';
exit(EXIT_FAILURE);
}
+ // If a previous module build daemon invocation crashes, the socket
+ // file will need to be removed before the address can be binded to
+ verboseLog("Removing ineligible file: " + SocketPath);
} else {
llvm::errs() << "MBD failed to create unix socket: "
<< SE.getMessage() << ": " << EC.message() << '\n';
@@ -153,7 +156,7 @@ void ModuleBuildDaemonServer::createDaemonSocket() {
}
});
} else {
- verboseLog("mbd created and binded to socket at: " + SocketPath);
+ verboseLog("MBD created and binded to socket at: " + SocketPath);
ServerListener.emplace(std::move(*MaybeServerListener));
break;
}
>From a6dcf2f8b9a44c72df2f83c592ff1cb13680738a Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 10 Apr 2024 14:07:10 -0400
Subject: [PATCH 19/31] Build fixes after rebase
---
clang/tools/driver/cc1modbuildd_main.cpp | 2 +-
llvm/include/llvm/Support/raw_socket_stream.h | 7 -------
2 files changed, 1 insertion(+), 8 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index f5c6e4e0ca0791..a99ecb8a3dc1c0 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -191,7 +191,7 @@ void ModuleBuildDaemonServer::handleConnection(
void ModuleBuildDaemonServer::listenForClients() {
- llvm::ThreadPool Pool;
+ llvm::DefaultThreadPool Pool;
std::chrono::seconds DaemonTimeout(15);
while (RunServiceLoop) {
diff --git a/llvm/include/llvm/Support/raw_socket_stream.h b/llvm/include/llvm/Support/raw_socket_stream.h
index 0c966633e38f9c..38c471893624dd 100644
--- a/llvm/include/llvm/Support/raw_socket_stream.h
+++ b/llvm/include/llvm/Support/raw_socket_stream.h
@@ -110,13 +110,6 @@ class ListeningSocket {
static Expected<ListeningSocket> createUnix(
StringRef SocketPath,
int MaxBacklog = llvm::hardware_concurrency().compute_thread_count());
-
- Expected<std::unique_ptr<raw_socket_stream>>
- accept(std::optional<std::chrono::microseconds> Timeout = std::nullopt);
-
- static Expected<ListeningSocket> createUnix(
- StringRef SocketPath,
- int MaxBacklog = llvm::hardware_concurrency().compute_thread_count());
};
//===----------------------------------------------------------------------===//
>From f705e421adba9e9ed3d0c9f7ea56a3d58d66eb83 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 10 Apr 2024 14:16:48 -0400
Subject: [PATCH 20/31] Use llvm support ExponentialBackoff function
---
clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp | 15 ++++-----------
1 file changed, 4 insertions(+), 11 deletions(-)
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index 8c7f440ad79236..40ec386784992a 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -15,6 +15,7 @@
#include "llvm/ADT/ScopeExit.h"
#include "llvm/ADT/SmallString.h"
#include "llvm/ADT/StringRef.h"
+#include "llvm/Support/ExponentialBackoff.h"
#include "llvm/Support/Program.h"
#include <cerrno>
@@ -98,13 +99,8 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
return std::move(Err);
std::chrono::seconds MaxWaitTime(30);
- std::chrono::microseconds CumulativeTime(0);
- std::chrono::microseconds WaitTime(10);
-
- while (CumulativeTime <= MaxWaitTime) {
- // Wait a bit then check to see if the module build daemon has initialized
- std::this_thread::sleep_for(WaitTime);
-
+ ExponentialBackoff Backoff(MaxWaitTime);
+ do {
if (llvm::sys::fs::exists(SocketPath)) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
connectToSocket(SocketPath);
@@ -114,10 +110,7 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
}
consumeError(MaybeClient.takeError());
}
-
- CumulativeTime += WaitTime;
- WaitTime = WaitTime * 2;
- }
+ } while (Backoff.waitForNextAttempt());
// After waiting around 30 seconds give up and return an error
return llvm::make_error<StringError>(
>From 270a30bd334f022ea4984b064301f86290fafa16 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 17 Apr 2024 07:15:03 -0400
Subject: [PATCH 21/31] Remove llvm Support changes from PR
---
llvm/include/llvm/Support/raw_socket_stream.h | 1 -
llvm/lib/Support/raw_socket_stream.cpp | 2 --
2 files changed, 3 deletions(-)
diff --git a/llvm/include/llvm/Support/raw_socket_stream.h b/llvm/include/llvm/Support/raw_socket_stream.h
index 38c471893624dd..bddd47eb75e1a9 100644
--- a/llvm/include/llvm/Support/raw_socket_stream.h
+++ b/llvm/include/llvm/Support/raw_socket_stream.h
@@ -123,7 +123,6 @@ class raw_socket_stream : public raw_fd_stream {
#endif // _WIN32
public:
- // TODO: Should probably be private
raw_socket_stream(int SocketFD);
/// Create a \p raw_socket_stream connected to the UNIX domain socket at \p
/// SocketPath.
diff --git a/llvm/lib/Support/raw_socket_stream.cpp b/llvm/lib/Support/raw_socket_stream.cpp
index 4908d02d52f364..14e2308df4d7ed 100644
--- a/llvm/lib/Support/raw_socket_stream.cpp
+++ b/llvm/lib/Support/raw_socket_stream.cpp
@@ -20,8 +20,6 @@
#include <fcntl.h>
#include <thread>
-#include <atomic>
-
#ifndef _WIN32
#include <poll.h>
#include <sys/socket.h>
>From e6d55f884b086d341a59317862230c2332292c8f Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 17 Apr 2024 07:37:22 -0400
Subject: [PATCH 22/31] Remove OBE socket function that was just serving as a
pass through
---
.../Tooling/ModuleBuildDaemon/SocketSupport.h | 2 --
clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp | 4 ++--
.../Tooling/ModuleBuildDaemon/SocketSupport.cpp | 15 ++-------------
3 files changed, 4 insertions(+), 17 deletions(-)
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
index 88c71be936f5af..854aab2f890aa9 100644
--- a/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
@@ -35,8 +35,6 @@ struct HandshakeMsg : public BaseMsg {
: BaseMsg(Action, Status) {}
};
-llvm::Expected<std::unique_ptr<llvm::raw_socket_stream>>
-connectToSocket(llvm::StringRef SocketPath);
llvm::Expected<std::string>
readBufferFromSocket(llvm::raw_socket_stream &Socket);
llvm::Error writeBufferToSocket(llvm::raw_socket_stream &Socket,
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index 40ec386784992a..71ff6f2fd14507 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -88,7 +88,7 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
if (llvm::sys::fs::exists(SocketPath)) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
- connectToSocket(SocketPath);
+ raw_socket_stream::createConnectedUnix(SocketPath);
if (MaybeClient)
return std::move(*MaybeClient);
consumeError(MaybeClient.takeError());
@@ -103,7 +103,7 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
do {
if (llvm::sys::fs::exists(SocketPath)) {
Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
- connectToSocket(SocketPath);
+ raw_socket_stream::createConnectedUnix(SocketPath);
if (MaybeClient) {
Diag.Report(diag::remark_mbd_connection) << SocketPath;
return std::move(*MaybeClient);
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
index be0665147a0a45..b119a0011dc223 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
@@ -17,17 +17,6 @@ using namespace llvm;
namespace clang::tooling::cc1modbuildd {
-Expected<std::unique_ptr<raw_socket_stream>>
-connectToSocket(StringRef SocketPath) {
-
- Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
- raw_socket_stream::createConnectedUnix(SocketPath);
- if (!MaybeClient)
- return MaybeClient.takeError();
-
- return std::move(*MaybeClient);
-}
-
Expected<std::string> readBufferFromSocket(raw_socket_stream &Socket) {
constexpr unsigned short MAX_BUFFER = 4096;
@@ -45,7 +34,7 @@ Expected<std::string> readBufferFromSocket(raw_socket_stream &Socket) {
if (Socket.has_error()) {
std::error_code EC = Socket.error();
Socket.clear_error();
- return make_error<StringError>("Failed socket read: ", EC);
+ return make_error<StringError>("Failed socket read", EC);
}
return ReturnBuffer;
}
@@ -56,7 +45,7 @@ Error writeBufferToSocket(raw_socket_stream &Socket, StringRef Buffer) {
if (Socket.has_error()) {
std::error_code EC = Socket.error();
Socket.clear_error();
- return make_error<StringError>("Failed socket write: ", EC);
+ return make_error<StringError>("Failed socket write", EC);
}
Socket.flush();
>From cf6e5b1e6606c7b57cdf2db88c84efd7aca0871e Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 17 Apr 2024 08:09:20 -0400
Subject: [PATCH 23/31] Move where signals begin to be ignored
---
clang/tools/driver/cc1modbuildd_main.cpp | 23 ++++++++++++-----------
1 file changed, 12 insertions(+), 11 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index a99ecb8a3dc1c0..74b1d40488ba10 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -38,8 +38,8 @@ static raw_fd_ostream &unbuff_outs() {
return S;
}
-static bool VerboseLog = false;
-static void verboseLog(const llvm::Twine &message) {
+static bool LogVerbose = false;
+static void logVerbose(const llvm::Twine &message) {
if (VerboseLog) {
unbuff_outs() << message << '\n';
}
@@ -148,7 +148,7 @@ void ModuleBuildDaemonServer::createDaemonSocket() {
}
// If a previous module build daemon invocation crashes, the socket
// file will need to be removed before the address can be binded to
- verboseLog("Removing ineligible file: " + SocketPath);
+ logVerbose("Removing ineligible file: " + SocketPath);
} else {
llvm::errs() << "MBD failed to create unix socket: "
<< SE.getMessage() << ": " << EC.message() << '\n';
@@ -156,7 +156,7 @@ void ModuleBuildDaemonServer::createDaemonSocket() {
}
});
} else {
- verboseLog("MBD created and binded to socket at: " + SocketPath);
+ logVerbose("MBD created and binded to socket at: " + SocketPath);
ServerListener.emplace(std::move(*MaybeServerListener));
break;
}
@@ -205,9 +205,9 @@ void ModuleBuildDaemonServer::listenForClients() {
if (EC == std::errc::timed_out) {
RunServiceLoop = false;
- verboseLog("ListeningServer::accept timed out, shutting down");
+ logVerbose("ListeningServer::accept timed out, shutting down");
} else if (EC == std::errc::interrupted && RunServiceLoop == false) {
- verboseLog("Signal received, shutting down");
+ logVerbose("Signal received, shutting down");
} else
errs() << "MBD failed to accept incoming connection: "
<< SE.getMessage() << ": " << EC.message() << '\n';
@@ -242,7 +242,7 @@ int cc1modbuildd_main(ArrayRef<const char *> Argv) {
// -cc1modbuildd is sliced away when Argv is pased to cc1modbuildd_main
if (find(Argv, StringRef("-v")) != Argv.end())
- VerboseLog = true;
+ LogVerbose = true;
std::string BasePath;
// If an argument exists and it is not -v then it must be a BasePath
@@ -268,11 +268,12 @@ int cc1modbuildd_main(ArrayRef<const char *> Argv) {
Daemon.setupDaemonEnv();
Daemon.createDaemonSocket();
Daemon.listenForClients();
- }
- // Prevents handleSignal from being called after ServerListener is destructed
- // Daemon is shutting down at this point so signals can be ignored
- modifySignals(SIG_IGN);
+ // Prevents the signal handler from being called after the
+ // ModuleBuildDaemonServer is destructed. The daemon is shutting down and
+ // the program is about to return so signals can be ignored
+ modifySignals(SIG_IGN);
+ }
return EXIT_SUCCESS;
}
>From b85a651ecaf232da3851726e07fb9b4f79d83a21 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 17 Apr 2024 08:52:19 -0400
Subject: [PATCH 24/31] Update error code handling with new and improved
ListeningSocket::accept
---
clang/tools/driver/cc1modbuildd_main.cpp | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 74b1d40488ba10..a8db35486be654 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -40,7 +40,7 @@ static raw_fd_ostream &unbuff_outs() {
static bool LogVerbose = false;
static void logVerbose(const llvm::Twine &message) {
- if (VerboseLog) {
+ if (LogVerbose) {
unbuff_outs() << message << '\n';
}
}
@@ -206,7 +206,8 @@ void ModuleBuildDaemonServer::listenForClients() {
if (EC == std::errc::timed_out) {
RunServiceLoop = false;
logVerbose("ListeningServer::accept timed out, shutting down");
- } else if (EC == std::errc::interrupted && RunServiceLoop == false) {
+ } else if (EC == std::errc::bad_file_descriptor &&
+ RunServiceLoop == false) {
logVerbose("Signal received, shutting down");
} else
errs() << "MBD failed to accept incoming connection: "
>From 4d4cec9a16804ee187cb3ed26bb8841d7177f52d Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 17 Apr 2024 14:55:36 -0400
Subject: [PATCH 25/31] cppcheck fixes
---
clang/tools/driver/cc1modbuildd_main.cpp | 22 +++++++++++++++++-----
1 file changed, 17 insertions(+), 5 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index a8db35486be654..1ebd91e6161eb0 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -73,7 +73,7 @@ class ModuleBuildDaemonServer {
SmallString<256> Stderr; // path to stderr
SmallString<256> Stdout; // path to stdout
- ModuleBuildDaemonServer(StringRef Path)
+ explicit ModuleBuildDaemonServer(StringRef Path)
: SocketPath(Path), Stderr(Path), Stdout(Path) {
llvm::sys::path::append(SocketPath, SocketFileName);
llvm::sys::path::append(Stdout, StdoutFileName);
@@ -109,13 +109,25 @@ void handleSignal(int) { DaemonPtr->shutdownDaemon(); }
void ModuleBuildDaemonServer::setupDaemonEnv() {
#ifdef _WIN32
- freopen("NUL", "r", stdin);
+ if (!std::freopen("NUL", "r", stdin)) {
+ llvm::errs() << "Failed to close stdin" << '\n';
+ exit(EXIT_FAILURE);
+ }
#else
- close(STDIN_FILENO);
+ if (::close(STDIN_FILENO) == -1) {
+ llvm::errs() << "Failed to close stdin" << '\n';
+ exit(EXIT_FAILURE);
+ }
#endif
- freopen(Stdout.c_str(), "a", stdout);
- freopen(Stderr.c_str(), "a", stderr);
+ if (std::freopen(Stdout.c_str(), "a", stdout) == NULL) {
+ llvm::errs() << "Failed to redirect stdout to " << Stdout << '\n';
+ exit(EXIT_FAILURE);
+ }
+ if (std::freopen(Stderr.c_str(), "a", stderr) == NULL) {
+ llvm::errs() << "Failed to redirect stderr to " << Stderr << '\n';
+ exit(EXIT_FAILURE);
+ }
modifySignals(handleSignal);
}
>From 4ce086014671f91bd0c01a24a9ca59b53bfb901b Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Wed, 17 Apr 2024 16:10:58 -0400
Subject: [PATCH 26/31] Change how we close stdin on linux
---
clang/lib/Driver/ToolChains/Clang.cpp | 1 +
clang/tools/driver/cc1modbuildd_main.cpp | 9 +++------
2 files changed, 4 insertions(+), 6 deletions(-)
diff --git a/clang/lib/Driver/ToolChains/Clang.cpp b/clang/lib/Driver/ToolChains/Clang.cpp
index 6bc89a55f1ed74..1b236019c73544 100644
--- a/clang/lib/Driver/ToolChains/Clang.cpp
+++ b/clang/lib/Driver/ToolChains/Clang.cpp
@@ -5,6 +5,7 @@
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//
+
#include "Clang.h"
#include "AMDGPU.h"
#include "Arch/AArch64.h"
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 1ebd91e6161eb0..aa023ab23ea232 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -109,16 +109,13 @@ void handleSignal(int) { DaemonPtr->shutdownDaemon(); }
void ModuleBuildDaemonServer::setupDaemonEnv() {
#ifdef _WIN32
- if (!std::freopen("NUL", "r", stdin)) {
- llvm::errs() << "Failed to close stdin" << '\n';
- exit(EXIT_FAILURE);
- }
+ if (std::freopen("NUL", "r", stdin) == NULL) {
#else
- if (::close(STDIN_FILENO) == -1) {
+ if (std::freopen("/dev/null", "r", stdin) == NULL) {
+#endif
llvm::errs() << "Failed to close stdin" << '\n';
exit(EXIT_FAILURE);
}
-#endif
if (std::freopen(Stdout.c_str(), "a", stdout) == NULL) {
llvm::errs() << "Failed to redirect stdout to " << Stdout << '\n';
>From a0d066d4fb08f5731b90c89bba53cc172a84daf1 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 20 Apr 2024 10:58:08 -0400
Subject: [PATCH 27/31] Remove blank lines after function declaration and
remove instances of
---
.../Tooling/ModuleBuildDaemon/Frontend.cpp | 33 +++++-----
.../ModuleBuildDaemon/SocketSupport.cpp | 16 +++--
clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp | 4 +-
clang/tools/driver/cc1modbuildd_main.cpp | 63 +++++++++----------
4 files changed, 50 insertions(+), 66 deletions(-)
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index 71ff6f2fd14507..fb64236d60b8aa 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -26,13 +26,10 @@
#include <string>
#include <thread>
-using namespace llvm;
-
namespace clang::tooling::cc1modbuildd {
-llvm::Error attemptHandshake(raw_socket_stream &Client,
+llvm::Error attemptHandshake(llvm::raw_socket_stream &Client,
DiagnosticsEngine &Diag) {
-
// Send HandshakeMsg to module build daemon
HandshakeMsg Request{ActionType::HANDSHAKE, StatusType::REQUEST};
if (llvm::Error Err = writeMsgStructToSocket(Client, Request))
@@ -53,7 +50,7 @@ llvm::Error attemptHandshake(raw_socket_stream &Client,
return llvm::Error::success();
}
- return llvm::make_error<StringError>(
+ return llvm::make_error<llvm::StringError>(
"Received handshake response 'FAILURE' from module build daemon",
std::make_error_code(std::errc::operation_not_permitted));
}
@@ -61,7 +58,6 @@ llvm::Error attemptHandshake(raw_socket_stream &Client,
llvm::Error spawnModuleBuildDaemon(const CompilerInvocation &Clang,
const char *Argv0, DiagnosticsEngine &Diag,
std::string BasePath) {
-
std::vector<StringRef> Args = {Argv0, ModuleBuildDaemonFlag};
if (!Clang.getFrontendOpts().ModuleBuildDaemonPath.empty())
Args.push_back(BasePath.c_str());
@@ -73,22 +69,22 @@ llvm::Error spawnModuleBuildDaemon(const CompilerInvocation &Clang,
// llvm::sys::ExecuteNoWait can fail for a variety of reasons which can't be
// generalized to one error code
if (!ErrorBuffer.empty())
- return llvm::make_error<StringError>(ErrorBuffer, inconvertibleErrorCode());
+ return llvm::make_error<llvm::StringError>(ErrorBuffer,
+ llvm::inconvertibleErrorCode());
Diag.Report(diag::remark_mbd_spawn);
return llvm::Error::success();
}
-Expected<std::unique_ptr<raw_socket_stream>>
+Expected<std::unique_ptr<llvm::raw_socket_stream>>
getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
DiagnosticsEngine &Diag, StringRef BasePath) {
-
SmallString<128> SocketPath = BasePath;
llvm::sys::path::append(SocketPath, SocketFileName);
if (llvm::sys::fs::exists(SocketPath)) {
- Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
- raw_socket_stream::createConnectedUnix(SocketPath);
+ Expected<std::unique_ptr<llvm::raw_socket_stream>> MaybeClient =
+ llvm::raw_socket_stream::createConnectedUnix(SocketPath);
if (MaybeClient)
return std::move(*MaybeClient);
consumeError(MaybeClient.takeError());
@@ -99,11 +95,11 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
return std::move(Err);
std::chrono::seconds MaxWaitTime(30);
- ExponentialBackoff Backoff(MaxWaitTime);
+ llvm::ExponentialBackoff Backoff(MaxWaitTime);
do {
if (llvm::sys::fs::exists(SocketPath)) {
- Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
- raw_socket_stream::createConnectedUnix(SocketPath);
+ Expected<std::unique_ptr<llvm::raw_socket_stream>> MaybeClient =
+ llvm::raw_socket_stream::createConnectedUnix(SocketPath);
if (MaybeClient) {
Diag.Report(diag::remark_mbd_connection) << SocketPath;
return std::move(*MaybeClient);
@@ -113,15 +109,14 @@ getModuleBuildDaemon(const CompilerInvocation &Clang, const char *Argv0,
} while (Backoff.waitForNextAttempt());
// After waiting around 30 seconds give up and return an error
- return llvm::make_error<StringError>(
- "Max wait time exceeded: ",
+ return llvm::make_error<llvm::StringError>(
+ "Max wait time exceeded",
std::make_error_code(std::errc::no_such_process));
}
void spawnModuleBuildDaemonAndHandshake(const CompilerInvocation &Clang,
const char *Argv0,
DiagnosticsEngine &Diag) {
-
// The module build daemon stores all output files and its socket address
// under BasePath. Either set BasePath to a user provided option or create an
// appropriate BasePath based on the BLAKE3 hash of the full clang version
@@ -138,13 +133,13 @@ void spawnModuleBuildDaemonAndHandshake(const CompilerInvocation &Clang,
}
// If module build daemon does not exist spawn module build daemon
- Expected<std::unique_ptr<raw_socket_stream>> MaybeClient =
+ Expected<std::unique_ptr<llvm::raw_socket_stream>> MaybeClient =
getModuleBuildDaemon(Clang, Argv0, Diag, BasePath);
if (!MaybeClient) {
Diag.Report(diag::err_mbd_connect) << MaybeClient.takeError();
return;
}
- raw_socket_stream &Client = **MaybeClient;
+ llvm::raw_socket_stream &Client = **MaybeClient;
if (llvm::Error HandshakeErr = attemptHandshake(Client, Diag)) {
Diag.Report(diag::err_mbd_handshake) << std::move(HandshakeErr);
return;
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
index b119a0011dc223..a0e34fac8d4960 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/SocketSupport.cpp
@@ -13,12 +13,10 @@
#include <memory>
#include <string>
-using namespace llvm;
-
namespace clang::tooling::cc1modbuildd {
-Expected<std::string> readBufferFromSocket(raw_socket_stream &Socket) {
-
+llvm::Expected<std::string>
+readBufferFromSocket(llvm::raw_socket_stream &Socket) {
constexpr unsigned short MAX_BUFFER = 4096;
char Buffer[MAX_BUFFER];
std::string ReturnBuffer;
@@ -34,22 +32,22 @@ Expected<std::string> readBufferFromSocket(raw_socket_stream &Socket) {
if (Socket.has_error()) {
std::error_code EC = Socket.error();
Socket.clear_error();
- return make_error<StringError>("Failed socket read", EC);
+ return llvm::make_error<llvm::StringError>("Failed socket read", EC);
}
return ReturnBuffer;
}
-Error writeBufferToSocket(raw_socket_stream &Socket, StringRef Buffer) {
+llvm::Error writeBufferToSocket(llvm::raw_socket_stream &Socket,
+ llvm::StringRef Buffer) {
Socket << Buffer;
-
if (Socket.has_error()) {
std::error_code EC = Socket.error();
Socket.clear_error();
- return make_error<StringError>("Failed socket write", EC);
+ return llvm::make_error<llvm::StringError>("Failed socket write", EC);
}
Socket.flush();
- return Error::success();
+ return llvm::Error::success();
}
} // namespace clang::tooling::cc1modbuildd
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
index 9ce1ba849bd83c..58736ada624463 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Utils.cpp
@@ -17,8 +17,6 @@
#include <string>
-using namespace llvm;
-
namespace clang::tooling::cc1modbuildd {
std::string getBasePath() {
@@ -31,7 +29,7 @@ std::string getBasePath() {
std::string Key = toString(llvm::APInt(64, HashValue), 36, /*Signed*/ false);
// Set paths
- SmallString<128> BasePath;
+ llvm::SmallString<128> BasePath;
llvm::sys::path::system_temp_directory(/*erasedOnReboot*/ true, BasePath);
llvm::sys::path::append(BasePath, "clang-" + Key);
return BasePath.c_str();
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index aa023ab23ea232..0920963905a54e 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -28,13 +28,12 @@
#include <unistd.h>
#endif
-using namespace llvm;
using namespace clang::tooling::cc1modbuildd;
// Create unbuffered STDOUT stream so that any logging done by the module build
// daemon can be viewed without having to terminate the process
-static raw_fd_ostream &unbuff_outs() {
- static raw_fd_ostream S(fileno(stdout), false, true);
+static llvm::raw_fd_ostream &unbuff_outs() {
+ static llvm::raw_fd_ostream S(fileno(stdout), false, true);
return S;
}
@@ -46,20 +45,17 @@ static void logVerbose(const llvm::Twine &message) {
}
static void modifySignals(decltype(SIG_DFL) handler) {
-
if (std::signal(SIGTERM, handler) == SIG_ERR) {
- errs() << "failed to handle SIGTERM" << '\n';
+ llvm::errs() << "failed to handle SIGTERM" << '\n';
exit(EXIT_FAILURE);
}
-
if (std::signal(SIGINT, handler) == SIG_ERR) {
- errs() << "failed to handle SIGINT" << '\n';
+ llvm::errs() << "failed to handle SIGINT" << '\n';
exit(EXIT_FAILURE);
}
-
#ifdef SIGHUP
if (::signal(SIGHUP, SIG_IGN) == SIG_ERR) {
- errs() << "failed to handle SIGHUP" << '\n';
+ llvm::errs() << "failed to handle SIGHUP" << '\n';
exit(EXIT_FAILURE);
}
#endif
@@ -69,11 +65,11 @@ namespace {
class ModuleBuildDaemonServer {
public:
- SmallString<256> SocketPath;
- SmallString<256> Stderr; // path to stderr
- SmallString<256> Stdout; // path to stdout
+ llvm::SmallString<256> SocketPath;
+ llvm::SmallString<256> Stderr; // path to stderr
+ llvm::SmallString<256> Stdout; // path to stdout
- explicit ModuleBuildDaemonServer(StringRef Path)
+ explicit ModuleBuildDaemonServer(llvm::StringRef Path)
: SocketPath(Path), Stderr(Path), Stdout(Path) {
llvm::sys::path::append(SocketPath, SocketFileName);
llvm::sys::path::append(Stdout, StdoutFileName);
@@ -84,7 +80,8 @@ class ModuleBuildDaemonServer {
void createDaemonSocket();
void listenForClients();
- static void handleConnection(std::shared_ptr<raw_socket_stream> Connection);
+ static void
+ handleConnection(std::shared_ptr<llvm::raw_socket_stream> Connection);
// TODO: modify so when shutdownDaemon is called the daemon stops accepting
// new client connections and waits for all existing client connections to
@@ -97,6 +94,7 @@ class ModuleBuildDaemonServer {
private:
std::atomic<bool> RunServiceLoop = true;
+ std::atomic<llvm::ListeningSocket *> ServerListenerAtomicPtr;
std::optional<llvm::ListeningSocket> ServerListener;
};
@@ -107,7 +105,6 @@ void handleSignal(int) { DaemonPtr->shutdownDaemon(); }
// Sets up file descriptors and signals for module build daemon
void ModuleBuildDaemonServer::setupDaemonEnv() {
-
#ifdef _WIN32
if (std::freopen("NUL", "r", stdin) == NULL) {
#else
@@ -116,7 +113,6 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
llvm::errs() << "Failed to close stdin" << '\n';
exit(EXIT_FAILURE);
}
-
if (std::freopen(Stdout.c_str(), "a", stdout) == NULL) {
llvm::errs() << "Failed to redirect stdout to " << Stdout << '\n';
exit(EXIT_FAILURE);
@@ -131,9 +127,8 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
// Creates unix socket for IPC with frontends
void ModuleBuildDaemonServer::createDaemonSocket() {
-
while (true) {
- Expected<ListeningSocket> MaybeServerListener =
+ llvm::Expected<llvm::ListeningSocket> MaybeServerListener =
llvm::ListeningSocket::createUnix(SocketPath);
if (llvm::Error Err = MaybeServerListener.takeError()) {
@@ -176,39 +171,36 @@ void ModuleBuildDaemonServer::createDaemonSocket() {
// responsible for closing frontend socket connections
void ModuleBuildDaemonServer::handleConnection(
std::shared_ptr<llvm::raw_socket_stream> MovableConnection) {
-
llvm::raw_socket_stream &Connection = *MovableConnection;
// Read request from frontend
- Expected<HandshakeMsg> MaybeHandshakeMsg =
+ llvm::Expected<HandshakeMsg> MaybeHandshakeMsg =
readMsgStructFromSocket<HandshakeMsg>(Connection);
if (!MaybeHandshakeMsg) {
- errs() << "MBD failed to read frontend request: "
- << llvm::toString(MaybeHandshakeMsg.takeError()) << '\n';
+ llvm::errs() << "MBD failed to read frontend request: "
+ << llvm::toString(MaybeHandshakeMsg.takeError()) << '\n';
return;
}
// Send response to frontend
HandshakeMsg Msg(ActionType::HANDSHAKE, StatusType::SUCCESS);
if (llvm::Error WriteErr = writeMsgStructToSocket(Connection, Msg)) {
- errs() << "MBD failed to respond to frontend request: "
- << llvm::toString(std::move(WriteErr)) << '\n';
+ llvm::errs() << "MBD failed to respond to frontend request: "
+ << llvm::toString(std::move(WriteErr)) << '\n';
return;
}
return;
}
void ModuleBuildDaemonServer::listenForClients() {
-
llvm::DefaultThreadPool Pool;
std::chrono::seconds DaemonTimeout(15);
while (RunServiceLoop) {
- Expected<std::unique_ptr<raw_socket_stream>> MaybeConnection =
+ llvm::Expected<std::unique_ptr<llvm::raw_socket_stream>> MaybeConnection =
ServerListener.value().accept(DaemonTimeout);
if (llvm::Error Err = MaybeConnection.takeError()) {
-
llvm::handleAllErrors(std::move(Err), [&](const llvm::StringError &SE) {
std::error_code EC = SE.convertToErrorCode();
@@ -219,15 +211,16 @@ void ModuleBuildDaemonServer::listenForClients() {
RunServiceLoop == false) {
logVerbose("Signal received, shutting down");
} else
- errs() << "MBD failed to accept incoming connection: "
- << SE.getMessage() << ": " << EC.message() << '\n';
+ llvm::errs() << "MBD failed to accept incoming connection: "
+ << SE.getMessage() << ": " << EC.message() << '\n';
});
continue;
}
// Connection must be copy constructable to be passed to Pool.async
- std::shared_ptr<raw_socket_stream> Connection(std::move(*MaybeConnection));
+ std::shared_ptr<llvm::raw_socket_stream> Connection(
+ std::move(*MaybeConnection));
Pool.async(handleConnection, Connection);
}
}
@@ -248,10 +241,9 @@ void ModuleBuildDaemonServer::listenForClients() {
// The arguments <path> and -v are optional. If <path> is not provided then
// BasePath will be /tmp/clang-<BLAKE3HashOfClangFullVersion>
//
-int cc1modbuildd_main(ArrayRef<const char *> Argv) {
-
+int cc1modbuildd_main(llvm::ArrayRef<const char *> Argv) {
// -cc1modbuildd is sliced away when Argv is pased to cc1modbuildd_main
- if (find(Argv, StringRef("-v")) != Argv.end())
+ if (find(Argv, llvm::StringRef("-v")) != Argv.end())
LogVerbose = true;
std::string BasePath;
@@ -262,8 +254,9 @@ int cc1modbuildd_main(ArrayRef<const char *> Argv) {
BasePath = getBasePath();
if (!validBasePathLength(BasePath)) {
- errs() << "BasePath '" << BasePath << "' is longer then the max length of "
- << std::to_string(BasePathMaxLength) << '\n';
+ llvm::errs() << "BasePath '" << BasePath
+ << "' is longer then the max length of "
+ << std::to_string(BasePathMaxLength) << '\n';
return EXIT_FAILURE;
}
>From 233a1b9107fdc0557090e00dc5cc82ef6b18c6dc Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 20 Apr 2024 11:09:43 -0400
Subject: [PATCH 28/31] Remove redundant return
---
clang/tools/driver/cc1modbuildd_main.cpp | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index 0920963905a54e..e59032e8a8f05b 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -184,11 +184,9 @@ void ModuleBuildDaemonServer::handleConnection(
// Send response to frontend
HandshakeMsg Msg(ActionType::HANDSHAKE, StatusType::SUCCESS);
- if (llvm::Error WriteErr = writeMsgStructToSocket(Connection, Msg)) {
+ if (llvm::Error WriteErr = writeMsgStructToSocket(Connection, Msg))
llvm::errs() << "MBD failed to respond to frontend request: "
<< llvm::toString(std::move(WriteErr)) << '\n';
- return;
- }
return;
}
>From fe6fd89e36ef0dfda0766dce96257a57796743f6 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 20 Apr 2024 12:45:57 -0400
Subject: [PATCH 29/31] Move location of signal handler so that I don't have to
wory about syncronization with ServerListener
---
clang/tools/driver/cc1modbuildd_main.cpp | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/clang/tools/driver/cc1modbuildd_main.cpp b/clang/tools/driver/cc1modbuildd_main.cpp
index e59032e8a8f05b..da6f269db65e5e 100644
--- a/clang/tools/driver/cc1modbuildd_main.cpp
+++ b/clang/tools/driver/cc1modbuildd_main.cpp
@@ -86,20 +86,25 @@ class ModuleBuildDaemonServer {
// TODO: modify so when shutdownDaemon is called the daemon stops accepting
// new client connections and waits for all existing client connections to
// terminate before closing the file descriptor and exiting
+ // Meant to be called by signal handler to clean up resources
void shutdownDaemon() {
RunServiceLoop = false;
- if (ServerListener.has_value())
- ServerListener.value().shutdown();
+ // Signal handler is installed after ServerListener is created and emplaced
+ // into the std::optional<llvm::ListeningSocket>
+ ServerListener.value().shutdown();
}
private:
std::atomic<bool> RunServiceLoop = true;
- std::atomic<llvm::ListeningSocket *> ServerListenerAtomicPtr;
+ // llvm::ListeningSocket does not have a default constructor so use
+ // std::optional as storage
std::optional<llvm::ListeningSocket> ServerListener;
};
// Used to handle signals
ModuleBuildDaemonServer *DaemonPtr = nullptr;
+// DaemonPtr is set to a valid ModuleBuildDaemonServer before the signal handler
+// is installed so there is no need to check if DaemonPtr equals nullptr
void handleSignal(int) { DaemonPtr->shutdownDaemon(); }
} // namespace
@@ -121,8 +126,6 @@ void ModuleBuildDaemonServer::setupDaemonEnv() {
llvm::errs() << "Failed to redirect stderr to " << Stderr << '\n';
exit(EXIT_FAILURE);
}
-
- modifySignals(handleSignal);
}
// Creates unix socket for IPC with frontends
@@ -193,6 +196,7 @@ void ModuleBuildDaemonServer::handleConnection(
void ModuleBuildDaemonServer::listenForClients() {
llvm::DefaultThreadPool Pool;
std::chrono::seconds DaemonTimeout(15);
+ modifySignals(handleSignal);
while (RunServiceLoop) {
llvm::Expected<std::unique_ptr<llvm::raw_socket_stream>> MaybeConnection =
@@ -212,7 +216,6 @@ void ModuleBuildDaemonServer::listenForClients() {
llvm::errs() << "MBD failed to accept incoming connection: "
<< SE.getMessage() << ": " << EC.message() << '\n';
});
-
continue;
}
>From f22b21e6121a4de9b2604ae9b2f2c31c1e54a0d1 Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 20 Apr 2024 13:26:16 -0400
Subject: [PATCH 30/31] Remove keyword from header file
---
.../Tooling/ModuleBuildDaemon/SocketSupport.h | 37 +++++++++++--------
1 file changed, 22 insertions(+), 15 deletions(-)
diff --git a/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
index 854aab2f890aa9..0cff3810f04b09 100644
--- a/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
+++ b/clang/include/clang/Tooling/ModuleBuildDaemon/SocketSupport.h
@@ -98,27 +98,34 @@ llvm::Error writeMsgStructToSocket(llvm::raw_socket_stream &Socket,
} // namespace clang::tooling::cc1modbuildd
-namespace cc1mod = clang::tooling::cc1modbuildd;
-
-template <> struct llvm::yaml::ScalarEnumerationTraits<cc1mod::StatusType> {
- static void enumeration(IO &Io, cc1mod::StatusType &Value) {
- Io.enumCase(Value, "REQUEST", cc1mod::StatusType::REQUEST);
- Io.enumCase(Value, "SUCCESS", cc1mod::StatusType::SUCCESS);
- Io.enumCase(Value, "FAILURE", cc1mod::StatusType::FAILURE);
+namespace llvm {
+namespace yaml {
+template <>
+struct ScalarEnumerationTraits<clang::tooling::cc1modbuildd::StatusType> {
+ static void enumeration(IO &Io, clang::tooling::cc1modbuildd::StatusType &S) {
+ Io.enumCase(S, "REQUEST",
+ clang::tooling::cc1modbuildd::StatusType::REQUEST);
+ Io.enumCase(S, "SUCCESS",
+ clang::tooling::cc1modbuildd::StatusType::SUCCESS);
+ Io.enumCase(S, "FAILURE",
+ clang::tooling::cc1modbuildd::StatusType::FAILURE);
}
};
-template <> struct llvm::yaml::ScalarEnumerationTraits<cc1mod::ActionType> {
- static void enumeration(IO &Io, cc1mod::ActionType &Value) {
- Io.enumCase(Value, "HANDSHAKE", cc1mod::ActionType::HANDSHAKE);
+template <>
+struct ScalarEnumerationTraits<clang::tooling::cc1modbuildd::ActionType> {
+ static void enumeration(IO &Io, clang::tooling::cc1modbuildd::ActionType &A) {
+ Io.enumCase(A, "HANDSHAKE",
+ clang::tooling::cc1modbuildd::ActionType::HANDSHAKE);
}
};
-template <> struct llvm::yaml::MappingTraits<cc1mod::HandshakeMsg> {
- static void mapping(IO &Io, cc1mod::HandshakeMsg &Info) {
- Io.mapRequired("Action", Info.MsgAction);
- Io.mapRequired("Status", Info.MsgStatus);
+template <> struct MappingTraits<clang::tooling::cc1modbuildd::HandshakeMsg> {
+ static void mapping(IO &Io, clang::tooling::cc1modbuildd::HandshakeMsg &H) {
+ Io.mapRequired("Action", H.MsgAction);
+ Io.mapRequired("Status", H.MsgStatus);
}
};
-
+} // namespace yaml
+} // namespace llvm
#endif // LLVM_CLANG_TOOLING_MODULEBUILDDAEMON_SOCKETMSGSUPPORT_H
>From ed168c429c4de1a366abc24b110d753bf01d759e Mon Sep 17 00:00:00 2001
From: cpsughrue <cpsughrue at gmail.com>
Date: Sat, 20 Apr 2024 14:37:18 -0400
Subject: [PATCH 31/31] Add test with multiple TUs in one command line
---
.../clang/Basic/DiagnosticFrontendKinds.td | 3 ++-
.../Tooling/ModuleBuildDaemon/Frontend.cpp | 3 ++-
clang/test/ModuleBuildDaemon/daemon-crash.c | 2 +-
clang/test/ModuleBuildDaemon/daemon-launch.c | 2 +-
clang/test/ModuleBuildDaemon/handshake.c | 7 ++---
clang/test/ModuleBuildDaemon/multiple-tu.c | 27 +++++++++++++++++++
6 files changed, 37 insertions(+), 7 deletions(-)
create mode 100644 clang/test/ModuleBuildDaemon/multiple-tu.c
diff --git a/clang/include/clang/Basic/DiagnosticFrontendKinds.td b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
index 50e3aab765c7da..14410f490a069e 100644
--- a/clang/include/clang/Basic/DiagnosticFrontendKinds.td
+++ b/clang/include/clang/Basic/DiagnosticFrontendKinds.td
@@ -281,7 +281,8 @@ def remark_mbd_connection :
Remark<"Successfully connected to module build daemon at %0">,
InGroup<ModuleBuildDaemon>;
def remark_mbd_handshake :
- Remark<"Successfully completed handshake with module build daemon">,
+ Remark<"Clang invocation responsible for %0 successfully completed handshake "
+ "with module build daemon">,
InGroup<ModuleBuildDaemon>;
def warn_eagerly_load_for_standard_cplusplus_modules : Warning<
diff --git a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
index fb64236d60b8aa..8dd4cf81ee3caa 100644
--- a/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
+++ b/clang/lib/Tooling/ModuleBuildDaemon/Frontend.cpp
@@ -145,7 +145,8 @@ void spawnModuleBuildDaemonAndHandshake(const CompilerInvocation &Clang,
return;
}
- Diag.Report(diag::remark_mbd_handshake);
+ Diag.Report(diag::remark_mbd_handshake)
+ << Clang.getFrontendOpts().Inputs[0].getFile();
return;
}
diff --git a/clang/test/ModuleBuildDaemon/daemon-crash.c b/clang/test/ModuleBuildDaemon/daemon-crash.c
index 67bccfc3489b5c..4703d561bf8c00 100644
--- a/clang/test/ModuleBuildDaemon/daemon-crash.c
+++ b/clang/test/ModuleBuildDaemon/daemon-crash.c
@@ -3,7 +3,7 @@
// RUN: rm -rf mbd-crash %t
// timeout should exit with status 124 which is treated as a failure by lit on
-// windows. Ideally we would be like to check the exit code and only return true
+// windows. Ideally we would like to check the exit code and only return true
// if it equals 124 but lit does not support global bash sysmbols like $?
// RUN: timeout --signal=SIGKILL 2 %clang -cc1modbuildd mbd-crash -v || true
diff --git a/clang/test/ModuleBuildDaemon/daemon-launch.c b/clang/test/ModuleBuildDaemon/daemon-launch.c
index 9c148ae029cf3c..c91919b84af963 100644
--- a/clang/test/ModuleBuildDaemon/daemon-launch.c
+++ b/clang/test/ModuleBuildDaemon/daemon-launch.c
@@ -3,7 +3,7 @@
// RUN: rm -rf mbd-launch %t
// timeout should exit with status 124 which is treated as a failure by lit on
-// windows. Ideally we would be like to check the exit code and only return true
+// windows. Ideally we would like to check the exit code and only return true
// if it equals 124 but lit does not support global bash sysmbols like $?
// RUN: timeout --signal=SIGTERM 2 %clang -cc1modbuildd mbd-launch -v || true
diff --git a/clang/test/ModuleBuildDaemon/handshake.c b/clang/test/ModuleBuildDaemon/handshake.c
index af71daa89e90db..cd4d54ffcd6805 100644
--- a/clang/test/ModuleBuildDaemon/handshake.c
+++ b/clang/test/ModuleBuildDaemon/handshake.c
@@ -7,6 +7,7 @@
//--- main.c
int main() {return 0;}
+// Return true regardless so lit test does not exit before daemon is killed
// RUN: %clang -fmodule-build-daemon=mbd-handshake -Rmodule-build-daemon %t/main.c &> %t/output-new || true
// RUN: %clang -fmodule-build-daemon=mbd-handshake -Rmodule-build-daemon %t/main.c &> %t/output-existing || true
// RUN: %kill-process "-cc1modbuildd mbd-handshake"
@@ -15,11 +16,11 @@ int main() {return 0;}
// RUN: cat %t/output-existing | sed 's:\\\\\?:/:g' | FileCheck %s --check-prefix=CHECK-EXIST
// CHECK: remark: Successfully spawned module build daemon [-Rmodule-build-daemon]
-// CHECK: remark: Successfully connected to module build daemon at mbd-handshake/mbd.sock [-Rmodule-build-daemon]
-// CHECK: remark: Successfully completed handshake with module build daemon [-Rmodule-build-daemon]
+// CHECK-NEXT: remark: Successfully connected to module build daemon at mbd-handshake/mbd.sock [-Rmodule-build-daemon]
+// CHECK-NEXT: remark: Clang invocation responsible for {{.*main.c}} successfully completed handshake with module build daemon [-Rmodule-build-daemon]
// Check that a clang invocation can handshake with an existing module build daemon
-// CHECK-EXIST: remark: Successfully completed handshake with module build daemon [-Rmodule-build-daemon]
+// CHECK-EXIST: remark: Clang invocation responsible for {{.*main.c}} successfully completed handshake with module build daemon [-Rmodule-build-daemon]
// Make sure mbd.err is empty
// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
diff --git a/clang/test/ModuleBuildDaemon/multiple-tu.c b/clang/test/ModuleBuildDaemon/multiple-tu.c
new file mode 100644
index 00000000000000..a0ef8a2967f6fd
--- /dev/null
+++ b/clang/test/ModuleBuildDaemon/multiple-tu.c
@@ -0,0 +1,27 @@
+// Check that a driver command line with multiple translation units can create two
+// frontend invocations which can successfully handshake with the module build daemon
+
+// RUN: %kill-process "-cc1modbuildd mbd-multiple-tu"
+// RUN: rm -rf mbd-multiple-tu %t
+// RUN: split-file %s %t
+
+//--- foo.c
+int foo() {return 0;}
+
+//--- main.c
+int foo();
+int main() {return foo();}
+
+// Return true regardless so lit test does not exit before daemon is killed
+// RUN: %clang -fmodule-build-daemon=mbd-multiple-tu -Rmodule-build-daemon %t/main.c %t/foo.c &> %t/output || true
+// RUN: %kill-process "-cc1modbuildd mbd-multiple-tu"
+
+// RUN: cat %t/output | sed 's:\\\\\?:/:g' | FileCheck %s
+
+// CHECK: remark: Successfully spawned module build daemon [-Rmodule-build-daemon]
+// CHECK-NEXT: remark: Successfully connected to module build daemon at mbd-multiple-tu/mbd.sock [-Rmodule-build-daemon]
+// CHECK-DAG: remark: Clang invocation responsible for {{.*main.c}} successfully completed handshake with module build daemon [-Rmodule-build-daemon]
+// CHECK-DAG: remark: Clang invocation responsible for {{.*foo.c}} successfully completed handshake with module build daemon [-Rmodule-build-daemon]
+
+// Make sure mbd.err is empty
+// RUN: [ ! -s "mbd-launch/mbd.err" ] && true || false
More information about the llvm-commits
mailing list