[clang] [Clang][SYCL] Introduce clang-sycl-link-wrapper to link SYCL offloadi… (PR #112245)
Arvind Sudarsanam via cfe-commits
cfe-commits at lists.llvm.org
Mon Oct 14 12:03:16 PDT 2024
https://github.com/asudarsa created https://github.com/llvm/llvm-project/pull/112245
…ng device code
This PR is one of the many PRs in the SYCL upstreaming effort focusing on device code linking during the SYCL offload compilation process. RFC: https://discourse.llvm.org/t/rfc-offloading-design-for-sycl-offload-kind-and-spir-targets/74088
In this PR, we introduce a new tool that will be used to perform device code linking for SYCL offload kind. It accepts SYCL device objects in LLVM IR bitcode format and will generate a fully linked device object that can then be wrapped and linked into the host object.
A primary use case for this tool is to perform device code linking for objects with SYCL offload kind inside the clang-linker-wrapper. It can also be invoked via clang driver as follows:
`clang --target=spirv64 --sycl-link input.bc`
Device code linking for SYCL offloading kind has a number of known quirks that makes it difficult to use in a unified offloading setting. Two of the primary issues are:
1. Several finalization steps are required to be run on the fully-linked LLVM IR bitcode to gaurantee conformance to SYCL standards. This step is unique to SYCL offloading compilation flow.
2. SPIR-V LLVM Translator tool is an extenal tool and hence SPIR-V IR code generation cannot be done as part of LTO. This limitation will be lifted once SPIR-V backend is available as a viable LLVM backend.
Hence, we introduce this new tool to provide a clean wrapper to perform SYCL device linking.
Thanks
>From eff4a0300336c4c106e1d293b19e795f5ccbabc1 Mon Sep 17 00:00:00 2001
From: Arvind Sudarsanam <arvind.sudarsanam at intel.com>
Date: Fri, 27 Sep 2024 13:03:12 -0700
Subject: [PATCH] [Clang][SYCL] Introduce clang-sycl-link-wrapper to link SYCL
offloading device code
This PR is one of the many PRs in the SYCL upstreaming effort focusing on
device code linking during the SYCL offload compilation process.
RFC: https://discourse.llvm.org/t/rfc-offloading-design-for-sycl-offload-kind-and-spir-targets/74088
In this PR, we introduce a new tool that will be used to perform device code
linking for SYCL offload kind. It accepts SYCL device objects in LLVM IR bitcode
format and will generate a fully linked device object that can then be wrapped
and linked into the host object.
A primary use case for this tool is to perform device code linking for objects
with SYCL offload kind inside the clang-linker-wrapper.
It can also be invoked via clang driver as follows:
`clang --target=spirv64 --sycl-link input.bc`
Device code linking for SYCL offloading kind has a number of known quirks that
makes it difficult to use in a unified offloading setting.
Two of the primary issues are:
1. Several finalization steps are required to be run on the fully-linked LLVM
IR bitcode to gaurantee conformance to SYCL standards. This step is unique to
SYCL offloading compilation flow.
2. SPIR-V LLVM Translator tool is an extenal tool and hence SPIR-V IR code
generation cannot be done as part of LTO. This limitation will be lifted once
SPIR-V backend is available as a viable LLVM backend.
Hence, we introduce this new tool to provide a clean wrapper to perform SYCL
device linking.
Thanks
Signed-off-by: Arvind Sudarsanam <arvind.sudarsanam at intel.com>
---
clang/docs/ClangSYCLLinkWrapper.rst | 80 +++
clang/docs/index.rst | 1 +
clang/include/clang/Driver/Options.td | 5 +-
clang/lib/Driver/Driver.cpp | 5 +
clang/lib/Driver/ToolChains/SPIRV.cpp | 12 +
clang/lib/Driver/ToolChains/SPIRV.h | 5 +-
clang/test/Driver/Inputs/libsycl-complex.bc | 0
clang/test/Driver/Inputs/libsycl-crt.bc | 0
.../Driver/clang-sycl-link-wrapper-test.cpp | 9 +
clang/test/Driver/sycl-link-spirv-target.cpp | 7 +
clang/tools/CMakeLists.txt | 1 +
.../clang-sycl-link-wrapper/CMakeLists.txt | 28 +
.../ClangSYCLLinkWrapper.cpp | 530 ++++++++++++++++++
.../clang-sycl-link-wrapper/SYCLLinkOpts.td | 47 ++
14 files changed, 727 insertions(+), 3 deletions(-)
create mode 100644 clang/docs/ClangSYCLLinkWrapper.rst
create mode 100644 clang/test/Driver/Inputs/libsycl-complex.bc
create mode 100644 clang/test/Driver/Inputs/libsycl-crt.bc
create mode 100644 clang/test/Driver/clang-sycl-link-wrapper-test.cpp
create mode 100644 clang/test/Driver/sycl-link-spirv-target.cpp
create mode 100644 clang/tools/clang-sycl-link-wrapper/CMakeLists.txt
create mode 100644 clang/tools/clang-sycl-link-wrapper/ClangSYCLLinkWrapper.cpp
create mode 100644 clang/tools/clang-sycl-link-wrapper/SYCLLinkOpts.td
diff --git a/clang/docs/ClangSYCLLinkWrapper.rst b/clang/docs/ClangSYCLLinkWrapper.rst
new file mode 100644
index 00000000000000..8ceb17f6af9d86
--- /dev/null
+++ b/clang/docs/ClangSYCLLinkWrapper.rst
@@ -0,0 +1,80 @@
+=======================
+Clang SYCL Link Wrapper
+=======================
+
+.. contents::
+ :local:
+
+.. _clang-sycl-link-wrapper:
+
+Introduction
+============
+
+This tool works as a wrapper around the SYCL device code linking process.
+The purpose of this wrapper is to provide an interface to link SYCL device
+bitcode in LLVM IR format, SYCL device bitcode in SPIR-V IR format, and native
+binary objects, and then use the SPIR-V LLVM Translator tool on fully linked
+device objects to produce the final output.
+After the linking stage, the fully linked device code in LLVM IR format may
+undergo several SYCL-specific finalization steps before the SPIR-V code
+generation step.
+The wrapper will also support the Ahead-Of-Time (AOT) compilation flow. AOT
+compilation is the process of invoking the back-end at compile time to produce
+the final binary, as opposed to just-in-time (JIT) compilation when final code
+generation is deferred until application runtime.
+
+Device code linking for SYCL offloading has several known quirks that
+make it difficult to use in a unified offloading setting. Two of the primary
+issues are:
+1. Several finalization steps are required to be run on the fully linked LLVM
+IR bitcode to guarantee conformance to SYCL standards. This step is unique to
+the SYCL offloading compilation flow.
+2. The SPIR-V LLVM Translator tool is an external tool and hence SPIR-V IR code
+generation cannot be done as part of LTO. This limitation can be lifted once
+the SPIR-V backend is available as a viable LLVM backend.
+
+This tool has been proposed to work around these issues.
+
+Usage
+=====
+
+This tool can be used with the following options. Several of these options will
+be passed down to downstream tools like 'llvm-link', 'llvm-spirv', etc.
+
+.. code-block:: console
+
+ OVERVIEW: A utility that wraps around the SYCL device code linking process.
+ This enables linking and code generation for SPIR-V JIT targets and AOT
+ targets.
+
+ USAGE: clang-sycl-link-wrapper [options]
+
+ OPTIONS:
+ --arch <value> Specify the name of the target architecture.
+ --dry-run Print generated commands without running.
+ -g Specify that this was a debug compile.
+ -help-hidden Display all available options
+ -help Display available options (--help-hidden for more)
+ --library-path=<dir> Set the library path for SYCL device libraries
+ -o <path> Path to file to write output
+ --save-temps Save intermediate results
+ --triple <value> Specify the target triple.
+ --version Display the version number and exit
+ -v Print verbose information
+ -spirv-dump-device-code=<dir> Directory to dump SPIR-V IR code into
+ -is-windows-msvc-env Specify if we are compiling under windows environment
+ -llvm-spirv-options=<value> Pass options to llvm-spirv tool
+
+Example
+=======
+
+This tool is intended to be invoked when targeting any of the target offloading
+toolchains. When the --sycl-link option is passed to the clang driver, the
+driver will invoke the linking job of the target offloading toolchain, which in
+turn will invoke this tool. This tool can be used to create one or more fully
+linked device images that are ready to be wrapped and linked with host code to
+generate the final executable.
+
+.. code-block:: console
+
+ clang-sycl-link-wrapper --triple spirv64 --arch native input.bc
diff --git a/clang/docs/index.rst b/clang/docs/index.rst
index 4a497f4d9bcc3c..ccdc16d3e07699 100644
--- a/clang/docs/index.rst
+++ b/clang/docs/index.rst
@@ -97,6 +97,7 @@ Using Clang Tools
ClangOffloadBundler
ClangOffloadPackager
ClangRepl
+ ClangSYCLLinkWrapper
Design Documents
================
diff --git a/clang/include/clang/Driver/Options.td b/clang/include/clang/Driver/Options.td
index 3f4d1a328b4c27..245af5a539a4fa 100644
--- a/clang/include/clang/Driver/Options.td
+++ b/clang/include/clang/Driver/Options.td
@@ -6728,7 +6728,10 @@ def fsycl : Flag<["-"], "fsycl">,
def fno_sycl : Flag<["-"], "fno-sycl">,
Visibility<[ClangOption, CLOption]>,
Group<sycl_Group>, HelpText<"Disables SYCL kernels compilation for device">;
-
+def sycl_link : Flag<["--"], "sycl-link">, Flags<[HelpHidden]>,
+ Visibility<[ClangOption, CLOption]>,
+ Group<sycl_Group>, HelpText<"Perform link through clang-sycl-link-wrapper via the target "
+ "offloading toolchain.">;
// OS-specific options
let Flags = [TargetSpecific] in {
defm android_pad_segment : BooleanFFlag<"android-pad-segment">, Group<f_Group>;
diff --git a/clang/lib/Driver/Driver.cpp b/clang/lib/Driver/Driver.cpp
index d0c8bdba0ede95..184eafa137b178 100644
--- a/clang/lib/Driver/Driver.cpp
+++ b/clang/lib/Driver/Driver.cpp
@@ -4780,6 +4780,11 @@ Action *Driver::ConstructPhaseAction(
if (Phase == phases::Assemble && Input->getType() != types::TY_PP_Asm)
return Input;
+ // Use of --sycl-link will only allow for the link phase to occur. This is
+ // for all input files.
+ if (Args.hasArg(options::OPT_sycl_link) && Phase != phases::Link)
+ return Input;
+
// Build the appropriate action.
switch (Phase) {
case phases::Link:
diff --git a/clang/lib/Driver/ToolChains/SPIRV.cpp b/clang/lib/Driver/ToolChains/SPIRV.cpp
index ce900600cbee51..860fd932a58718 100644
--- a/clang/lib/Driver/ToolChains/SPIRV.cpp
+++ b/clang/lib/Driver/ToolChains/SPIRV.cpp
@@ -95,7 +95,19 @@ void SPIRV::Linker::ConstructJob(Compilation &C, const JobAction &JA,
CmdArgs.push_back("-o");
CmdArgs.push_back(Output.getFilename());
+ // Use of --sycl-link will call the clang-sycl-link-wrapper instead of
+ // the default linker (spirv-link).
+ if (Args.hasArg(options::OPT_sycl_link))
+ Linker = ToolChain.GetProgramPath("clang-sycl-link-wrapper");
C.addCommand(std::make_unique<Command>(JA, *this, ResponseFileSupport::None(),
Args.MakeArgString(Linker), CmdArgs,
Inputs, Output));
}
+
+SPIRVToolChain::SPIRVToolChain(const Driver &D, const llvm::Triple &Triple,
+ const ArgList &Args)
+ : ToolChain(D, Triple, Args) {
+ NativeLLVMSupport = Args.hasArg(options::OPT_sycl_link);
+}
+
+bool SPIRVToolChain::HasNativeLLVMSupport() const { return NativeLLVMSupport; }
diff --git a/clang/lib/Driver/ToolChains/SPIRV.h b/clang/lib/Driver/ToolChains/SPIRV.h
index d4247ee0557f4b..d59a8c76ed4737 100644
--- a/clang/lib/Driver/ToolChains/SPIRV.h
+++ b/clang/lib/Driver/ToolChains/SPIRV.h
@@ -57,8 +57,7 @@ class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain {
public:
SPIRVToolChain(const Driver &D, const llvm::Triple &Triple,
- const llvm::opt::ArgList &Args)
- : ToolChain(D, Triple, Args) {}
+ const llvm::opt::ArgList &Args);
bool useIntegratedAs() const override { return true; }
@@ -72,6 +71,7 @@ class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain {
}
bool isPICDefaultForced() const override { return false; }
bool SupportsProfiling() const override { return false; }
+ bool HasNativeLLVMSupport() const override;
clang::driver::Tool *SelectTool(const JobAction &JA) const override;
@@ -81,6 +81,7 @@ class LLVM_LIBRARY_VISIBILITY SPIRVToolChain final : public ToolChain {
private:
clang::driver::Tool *getTranslator() const;
+ bool NativeLLVMSupport;
};
} // namespace toolchains
diff --git a/clang/test/Driver/Inputs/libsycl-complex.bc b/clang/test/Driver/Inputs/libsycl-complex.bc
new file mode 100644
index 00000000000000..e69de29bb2d1d6
diff --git a/clang/test/Driver/Inputs/libsycl-crt.bc b/clang/test/Driver/Inputs/libsycl-crt.bc
new file mode 100644
index 00000000000000..e69de29bb2d1d6
diff --git a/clang/test/Driver/clang-sycl-link-wrapper-test.cpp b/clang/test/Driver/clang-sycl-link-wrapper-test.cpp
new file mode 100644
index 00000000000000..5004725536e98a
--- /dev/null
+++ b/clang/test/Driver/clang-sycl-link-wrapper-test.cpp
@@ -0,0 +1,9 @@
+// Tests the clang-sycl-link-wrapper tool
+//
+// Test a simple case without arguments
+// RUN: %clangxx -fsycl -emit-llvm -c %s -o %t.bc
+// RUN: clang-sycl-link-wrapper --dry-run -triple spirv64 %t.bc --library-path=%S/Inputs -o a.spv 2>&1 \
+// RUN: | FileCheck %s --check-prefix=CMDS
+// CMDS: "{{.*}}llvm-link{{.*}}" {{.*}}.bc -o [[FIRSTLLVMLINKOUT:.*]].bc --suppress-warnings
+// CMDS-NEXT: "{{.*}}llvm-link{{.*}}" -only-needed [[FIRSTLLVMLINKOUT]].bc {{.*}}libsycl-crt.bc {{.*}}libsycl-complex.bc -o [[SECONDLLVMLINKOUT:.*]].bc --suppress-warnings
+// CMDS-NEXT: "{{.*}}llvm-spirv{{.*}}" {{.*}}-o a.spv [[SECONDLLVMLINKOUT]].bc
diff --git a/clang/test/Driver/sycl-link-spirv-target.cpp b/clang/test/Driver/sycl-link-spirv-target.cpp
new file mode 100644
index 00000000000000..550d40aac5499d
--- /dev/null
+++ b/clang/test/Driver/sycl-link-spirv-target.cpp
@@ -0,0 +1,7 @@
+// Tests the driver when linking LLVM IR bitcode files and targeting SPIR-V
+// architecture.
+//
+// RUN: touch %t.bc
+// RUN: %clangxx --target=spirv64 --sycl-link -### %t.bc 2>&1 \
+// RUN: | FileCheck %s -check-prefix=LINK
+// LINK: "{{.*}}clang-sycl-link-wrapper{{.*}}" "{{.*}}.bc" "-o" "a.out"
diff --git a/clang/tools/CMakeLists.txt b/clang/tools/CMakeLists.txt
index 88e29412e54350..d704ca5c62c97b 100644
--- a/clang/tools/CMakeLists.txt
+++ b/clang/tools/CMakeLists.txt
@@ -12,6 +12,7 @@ add_clang_subdirectory(clang-nvlink-wrapper)
add_clang_subdirectory(clang-offload-packager)
add_clang_subdirectory(clang-offload-bundler)
add_clang_subdirectory(clang-scan-deps)
+add_clang_subdirectory(clang-sycl-link-wrapper)
add_clang_subdirectory(clang-installapi)
if(HAVE_CLANG_REPL_SUPPORT)
add_clang_subdirectory(clang-repl)
diff --git a/clang/tools/clang-sycl-link-wrapper/CMakeLists.txt b/clang/tools/clang-sycl-link-wrapper/CMakeLists.txt
new file mode 100644
index 00000000000000..c51f6f977dddd7
--- /dev/null
+++ b/clang/tools/clang-sycl-link-wrapper/CMakeLists.txt
@@ -0,0 +1,28 @@
+set(LLVM_LINK_COMPONENTS
+ ${LLVM_TARGETS_TO_BUILD}
+ Option
+ )
+
+set(LLVM_TARGET_DEFINITIONS SYCLLinkOpts.td)
+tablegen(LLVM SYCLLinkOpts.inc -gen-opt-parser-defs)
+add_public_tablegen_target(SYCLLinkWrapperOpts)
+
+if(NOT CLANG_BUILT_STANDALONE)
+ set(tablegen_deps intrinsics_gen SYCLLinkWrapperOpts)
+endif()
+
+add_clang_tool(clang-sycl-link-wrapper
+ ClangSYCLLinkWrapper.cpp
+
+ DEPENDS
+ ${tablegen_deps}
+ )
+
+set(CLANG_SYCL_LINK_WRAPPER_LIB_DEPS
+ clangBasic
+ )
+
+target_link_libraries(clang-sycl-link-wrapper
+ PRIVATE
+ ${CLANG_SYCL_LINK_WRAPPER_LIB_DEPS}
+ )
diff --git a/clang/tools/clang-sycl-link-wrapper/ClangSYCLLinkWrapper.cpp b/clang/tools/clang-sycl-link-wrapper/ClangSYCLLinkWrapper.cpp
new file mode 100644
index 00000000000000..31afa26c46518d
--- /dev/null
+++ b/clang/tools/clang-sycl-link-wrapper/ClangSYCLLinkWrapper.cpp
@@ -0,0 +1,530 @@
+//=-- clang-sycl-link-wrapper/ClangSYCLLinkWrapper.cpp - SYCL linker util --=//
+//
+// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
+// See https://llvm.org/LICENSE.txt for license information.
+// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
+//
+//===---------------------------------------------------------------------===//
+//
+// This tool wraps around the sequence of steps required to link device code in
+// SYCL fat objects. SYCL device code linking requires a complex sequence of
+// steps that include linking of llvm bitcode files, linking device library
+// files with the fully linked source bitcode file(s), running several SYCL
+// specific post-link steps on the fully linked bitcode file(s), and finally
+// generating target-specific device code. This tool can be removed once SYCL
+// linking is ported to `ld.lld`.
+//
+//===---------------------------------------------------------------------===//
+
+#include "clang/Basic/Version.h"
+
+#include "llvm/ADT/StringExtras.h"
+#include "llvm/BinaryFormat/Magic.h"
+#include "llvm/Bitcode/BitcodeWriter.h"
+#include "llvm/CodeGen/CommandFlags.h"
+#include "llvm/IR/DiagnosticPrinter.h"
+#include "llvm/IRReader/IRReader.h"
+#include "llvm/LTO/LTO.h"
+#include "llvm/Object/Archive.h"
+#include "llvm/Object/ArchiveWriter.h"
+#include "llvm/Object/Binary.h"
+#include "llvm/Object/ELFObjectFile.h"
+#include "llvm/Object/IRObjectFile.h"
+#include "llvm/Object/ObjectFile.h"
+#include "llvm/Object/OffloadBinary.h"
+#include "llvm/Option/ArgList.h"
+#include "llvm/Option/OptTable.h"
+#include "llvm/Option/Option.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
+#include "llvm/Support/CommandLine.h"
+#include "llvm/Support/FileOutputBuffer.h"
+#include "llvm/Support/FileSystem.h"
+#include "llvm/Support/InitLLVM.h"
+#include "llvm/Support/MemoryBuffer.h"
+#include "llvm/Support/Path.h"
+#include "llvm/Support/Program.h"
+#include "llvm/Support/Signals.h"
+#include "llvm/Support/StringSaver.h"
+#include "llvm/Support/TargetSelect.h"
+#include "llvm/Support/TimeProfiler.h"
+#include "llvm/Support/WithColor.h"
+
+using namespace llvm;
+using namespace llvm::opt;
+using namespace llvm::object;
+
+/// Save intermediary results.
+static bool SaveTemps = false;
+
+/// Print arguments without executing.
+static bool DryRun = false;
+
+/// Print verbose output.
+static bool Verbose = false;
+
+/// Filename of the output being created.
+static StringRef OutputFile;
+
+/// Directory to dump SPIR-V IR if requested by user.
+static SmallString<128> SPIRVDumpDir;
+
+static void printVersion(raw_ostream &OS) {
+ OS << clang::getClangToolFullVersion("clang-sycl-link-wrapper") << '\n';
+}
+
+/// The value of `argv[0]` when run.
+static const char *Executable;
+
+/// Temporary files to be cleaned up.
+static SmallVector<SmallString<128>> TempFiles;
+
+namespace {
+// Must not overlap with llvm::opt::DriverFlag.
+enum WrapperFlags { WrapperOnlyOption = (1 << 4) };
+
+enum ID {
+ OPT_INVALID = 0, // This is not an option ID.
+#define OPTION(...) LLVM_MAKE_OPT_ID(__VA_ARGS__),
+#include "SYCLLinkOpts.inc"
+ LastOption
+#undef OPTION
+};
+
+#define PREFIX(NAME, VALUE) \
+ static constexpr StringLiteral NAME##_init[] = VALUE; \
+ static constexpr ArrayRef<StringLiteral> NAME(NAME##_init, \
+ std::size(NAME##_init) - 1);
+#include "SYCLLinkOpts.inc"
+#undef PREFIX
+
+static constexpr OptTable::Info InfoTable[] = {
+#define OPTION(...) LLVM_CONSTRUCT_OPT_INFO(__VA_ARGS__),
+#include "SYCLLinkOpts.inc"
+#undef OPTION
+};
+
+class WrapperOptTable : public opt::GenericOptTable {
+public:
+ WrapperOptTable() : opt::GenericOptTable(InfoTable) {}
+};
+
+const OptTable &getOptTable() {
+ static const WrapperOptTable *Table = []() {
+ auto Result = std::make_unique<WrapperOptTable>();
+ return Result.release();
+ }();
+ return *Table;
+}
+
+[[noreturn]] void reportError(Error E) {
+ outs().flush();
+ logAllUnhandledErrors(std::move(E), WithColor::error(errs(), Executable));
+ exit(EXIT_FAILURE);
+}
+
+std::string getMainExecutable(const char *Name) {
+ void *Ptr = (void *)(intptr_t)&getMainExecutable;
+ auto COWPath = sys::fs::getMainExecutable(Name, Ptr);
+ return sys::path::parent_path(COWPath).str();
+}
+
+Expected<StringRef> createTempFile(const ArgList &Args, const Twine &Prefix,
+ StringRef Extension) {
+ SmallString<128> OutputFile;
+ if (Args.hasArg(OPT_save_temps)) {
+ // Generate a unique path name without creating a file
+ sys::fs::createUniquePath(Prefix + "-%%%%%%." + Extension, OutputFile,
+ /*MakeAbsolute=*/false);
+ } else {
+ if (std::error_code EC =
+ sys::fs::createTemporaryFile(Prefix, Extension, OutputFile))
+ return createFileError(OutputFile, EC);
+ }
+
+ TempFiles.emplace_back(std::move(OutputFile));
+ return TempFiles.back();
+}
+
+Expected<std::string> findProgram(const ArgList &Args, StringRef Name,
+ ArrayRef<StringRef> Paths) {
+ if (Args.hasArg(OPT_dry_run))
+ return Name.str();
+ ErrorOr<std::string> Path = sys::findProgramByName(Name, Paths);
+ if (!Path)
+ Path = sys::findProgramByName(Name);
+ if (!Path)
+ return createStringError(Path.getError(),
+ "Unable to find '" + Name + "' in path");
+ return *Path;
+}
+
+std::optional<std::string> findFile(StringRef Dir, StringRef Root,
+ const Twine &Name) {
+ SmallString<128> Path;
+ if (Dir.starts_with("="))
+ sys::path::append(Path, Root, Dir.substr(1), Name);
+ else
+ sys::path::append(Path, Dir, Name);
+
+ if (sys::fs::exists(Path))
+ return static_cast<std::string>(Path);
+ return std::nullopt;
+}
+
+void printCommands(ArrayRef<StringRef> CmdArgs) {
+ if (CmdArgs.empty())
+ return;
+
+ llvm::errs() << " \"" << CmdArgs.front() << "\" ";
+ llvm::errs() << llvm::join(std::next(CmdArgs.begin()), CmdArgs.end(), " ")
+ << "\n";
+}
+
+/// Execute the command \p ExecutablePath with the arguments \p Args.
+Error executeCommands(StringRef ExecutablePath, ArrayRef<StringRef> Args) {
+ if (Verbose || DryRun)
+ printCommands(Args);
+
+ if (!DryRun)
+ if (sys::ExecuteAndWait(ExecutablePath, Args))
+ return createStringError(
+ "'%s' failed", sys::path::filename(ExecutablePath).str().c_str());
+ return Error::success();
+}
+
+Expected<SmallVector<std::string>> getInput(const ArgList &Args) {
+ // Collect all input bitcode files to be passed to llvm-link.
+ SmallVector<std::string> BitcodeFiles;
+ for (const opt::Arg *Arg : Args.filtered(OPT_INPUT)) {
+ std::optional<std::string> Filename = std::string(Arg->getValue());
+ if (!Filename || !sys::fs::exists(*Filename) ||
+ sys::fs::is_directory(*Filename))
+ continue;
+ file_magic Magic;
+ if (auto EC = identify_magic(*Filename, Magic))
+ return createStringError("Failed to open file " + *Filename);
+ if (Magic != file_magic::bitcode)
+ return createStringError("Unsupported file type");
+ BitcodeFiles.push_back(*Filename);
+ }
+ return BitcodeFiles;
+}
+
+/// Link all SYCL device input files into one before adding device library
+/// files. Device linking is performed using llvm-link tool.
+/// 'InputFiles' is the list of all LLVM IR device input files.
+/// 'Args' encompasses all arguments required for linking and wrapping device
+/// code and will be parsed to generate options required to be passed into the
+/// llvm-link tool.
+Expected<StringRef> linkDeviceInputFiles(ArrayRef<std::string> InputFiles,
+ const ArgList &Args) {
+ llvm::TimeTraceScope TimeScope("SYCL LinkDeviceInputFiles");
+ Expected<std::string> LLVMLinkPath =
+ findProgram(Args, "llvm-link", {getMainExecutable("llvm-link")});
+ if (!LLVMLinkPath)
+ return LLVMLinkPath.takeError();
+
+ SmallVector<StringRef> CmdArgs;
+ CmdArgs.push_back(*LLVMLinkPath);
+ for (auto &File : InputFiles)
+ CmdArgs.push_back(File);
+ // Create a new file to write the linked device file to.
+ auto OutFileOrErr =
+ createTempFile(Args, sys::path::filename(OutputFile), "bc");
+ if (!OutFileOrErr)
+ return OutFileOrErr.takeError();
+ CmdArgs.push_back("-o");
+ CmdArgs.push_back(*OutFileOrErr);
+ CmdArgs.push_back("--suppress-warnings");
+ if (Error Err = executeCommands(*LLVMLinkPath, CmdArgs))
+ return std::move(Err);
+ return *OutFileOrErr;
+}
+
+const SmallVector<std::string> SYCLDeviceLibNames = {
+ "libsycl-crt.bc",
+ "libsycl-complex.bc",
+ "libsycl-complex-fp64.bc",
+ "libsycl-cmath.bc",
+ "libsycl-cmath-fp64.bc",
+ "libsycl-imf.bc",
+ "libsycl-imf-fp64.bc",
+ "libsycl-imf-bf16.bc",
+ "libsycl-fallback-cassert.bc",
+ "libsycl-fallback-cstring.bc",
+ "libsycl-fallback-complex.bc",
+ "libsycl-fallback-complex-fp64.bc",
+ "libsycl-fallback-cmath.bc",
+ "libsycl-fallback-cmath-fp64.bc",
+ "libsycl-fallback-imf.bc",
+ "libsycl-fallback-imf-fp64.bc",
+ "libsycl-fallback-imf-bf16.bc",
+ "libsycl-fallback-bfloat16.bc",
+ "libsycl-native-bfloat16.bc",
+ "libsycl-itt-user-wrappers.bc",
+ "libsycl-itt-compiler-wrappers.bc",
+ "libsycl-itt-stubs.bc",
+ "libsycl-sanitizer.bc"};
+
+Expected<SmallVector<std::string>> getSYCLDeviceLibFiles(const ArgList &Args) {
+ SmallVector<std::string> DeviceLibFiles;
+ StringRef LibraryPath;
+ if (Arg *A = Args.getLastArg(OPT_library_path_EQ))
+ LibraryPath = A->getValue();
+ if (LibraryPath.empty())
+ return DeviceLibFiles;
+ for (auto &DeviceLibName : SYCLDeviceLibNames) {
+ std::optional<std::string> Filename =
+ findFile(LibraryPath, /*Root=*/"", DeviceLibName);
+ if (Filename)
+ DeviceLibFiles.push_back(*Filename);
+ }
+ return DeviceLibFiles;
+}
+
+/// Link all device library files and input file into one LLVM IR file. This
+/// linking is performed using llvm-link tool.
+/// 'InputFiles' is the list of all LLVM IR device input files.
+/// 'Args' encompasses all arguments required for linking and wrapping device
+/// code and will be parsed to generate options required to be passed into the
+/// llvm-link tool.
+static Expected<StringRef> linkDeviceLibFiles(StringRef InputFile,
+ const ArgList &Args) {
+ llvm::TimeTraceScope TimeScope("LinkDeviceLibraryFiles");
+
+ auto SYCLDeviceLibFiles = getSYCLDeviceLibFiles(Args);
+ if (!SYCLDeviceLibFiles)
+ return SYCLDeviceLibFiles.takeError();
+ if ((*SYCLDeviceLibFiles).empty())
+ return InputFile;
+
+ Expected<std::string> LLVMLinkPath =
+ findProgram(Args, "llvm-link", {getMainExecutable("llvm-link")});
+ if (!LLVMLinkPath)
+ return LLVMLinkPath.takeError();
+
+ // Create a new file to write the linked device file to.
+ auto OutFileOrErr =
+ createTempFile(Args, sys::path::filename(OutputFile), "bc");
+ if (!OutFileOrErr)
+ return OutFileOrErr.takeError();
+
+ SmallVector<StringRef, 8> CmdArgs;
+ CmdArgs.push_back(*LLVMLinkPath);
+ CmdArgs.push_back("-only-needed");
+ CmdArgs.push_back(InputFile);
+ for (auto &File : *SYCLDeviceLibFiles)
+ CmdArgs.push_back(File);
+ CmdArgs.push_back("-o");
+ CmdArgs.push_back(*OutFileOrErr);
+ CmdArgs.push_back("--suppress-warnings");
+ if (Error Err = executeCommands(*LLVMLinkPath, CmdArgs))
+ return std::move(Err);
+ return *OutFileOrErr;
+}
+
+/// Add any llvm-spirv option that relies on a specific Triple in addition
+/// to user supplied options.
+/// NOTE: Any changes made here should be reflected in the similarly named
+/// function in clang/lib/Driver/ToolChains/Clang.cpp.
+static void getSPIRVTransOpts(const ArgList &Args,
+ SmallVector<StringRef, 8> &TranslatorArgs,
+ const llvm::Triple Triple) {
+ // Enable NonSemanticShaderDebugInfo.200 for non-Windows
+ const bool IsWindowsMSVC =
+ Triple.isWindowsMSVCEnvironment() || Args.hasArg(OPT_is_windows_msvc_env);
+ const bool EnableNonSemanticDebug = !IsWindowsMSVC;
+ if (EnableNonSemanticDebug) {
+ TranslatorArgs.push_back(
+ "-spirv-debug-info-version=nonsemantic-shader-200");
+ } else {
+ TranslatorArgs.push_back("-spirv-debug-info-version=ocl-100");
+ // Prevent crash in the translator if input IR contains DIExpression
+ // operations which don't have mapping to OpenCL.DebugInfo.100 spec.
+ TranslatorArgs.push_back("-spirv-allow-extra-diexpressions");
+ }
+ std::string UnknownIntrinsics("-spirv-allow-unknown-intrinsics=llvm.genx.");
+
+ TranslatorArgs.push_back(Args.MakeArgString(UnknownIntrinsics));
+
+ // Disable all the extensions by default
+ std::string ExtArg("-spirv-ext=-all");
+ std::string DefaultExtArg =
+ ",+SPV_EXT_shader_atomic_float_add,+SPV_EXT_shader_atomic_float_min_max"
+ ",+SPV_KHR_no_integer_wrap_decoration,+SPV_KHR_float_controls"
+ ",+SPV_KHR_expect_assume,+SPV_KHR_linkonce_odr";
+ std::string INTELExtArg =
+ ",+SPV_INTEL_subgroups,+SPV_INTEL_media_block_io"
+ ",+SPV_INTEL_device_side_avc_motion_estimation"
+ ",+SPV_INTEL_fpga_loop_controls,+SPV_INTEL_unstructured_loop_controls"
+ ",+SPV_INTEL_fpga_reg,+SPV_INTEL_blocking_pipes"
+ ",+SPV_INTEL_function_pointers,+SPV_INTEL_kernel_attributes"
+ ",+SPV_INTEL_io_pipes,+SPV_INTEL_inline_assembly"
+ ",+SPV_INTEL_arbitrary_precision_integers"
+ ",+SPV_INTEL_float_controls2,+SPV_INTEL_vector_compute"
+ ",+SPV_INTEL_fast_composite"
+ ",+SPV_INTEL_arbitrary_precision_fixed_point"
+ ",+SPV_INTEL_arbitrary_precision_floating_point"
+ ",+SPV_INTEL_variable_length_array,+SPV_INTEL_fp_fast_math_mode"
+ ",+SPV_INTEL_long_constant_composite"
+ ",+SPV_INTEL_arithmetic_fence"
+ ",+SPV_INTEL_global_variable_decorations"
+ ",+SPV_INTEL_cache_controls"
+ ",+SPV_INTEL_fpga_buffer_location"
+ ",+SPV_INTEL_fpga_argument_interfaces"
+ ",+SPV_INTEL_fpga_invocation_pipelining_attributes"
+ ",+SPV_INTEL_fpga_latency_control"
+ ",+SPV_INTEL_task_sequence"
+ ",+SPV_KHR_shader_clock"
+ ",+SPV_INTEL_bindless_images";
+ ExtArg = ExtArg + DefaultExtArg + INTELExtArg;
+ ExtArg += ",+SPV_INTEL_token_type"
+ ",+SPV_INTEL_bfloat16_conversion"
+ ",+SPV_INTEL_joint_matrix"
+ ",+SPV_INTEL_hw_thread_queries"
+ ",+SPV_KHR_uniform_group_instructions"
+ ",+SPV_INTEL_masked_gather_scatter"
+ ",+SPV_INTEL_tensor_float32_conversion"
+ ",+SPV_INTEL_optnone"
+ ",+SPV_KHR_non_semantic_info"
+ ",+SPV_KHR_cooperative_matrix";
+ TranslatorArgs.push_back(Args.MakeArgString(ExtArg));
+}
+
+/// Run LLVM to SPIR-V translation.
+/// Converts 'File' from LLVM bitcode to SPIR-V format using llvm-spirv tool.
+/// 'Args' encompasses all arguments required for linking and wrapping device
+/// code and will be parsed to generate options required to be passed into the
+/// llvm-spirv tool.
+static Expected<StringRef> runLLVMToSPIRVTranslation(StringRef File,
+ const ArgList &Args) {
+ llvm::TimeTraceScope TimeScope("LLVMToSPIRVTranslation");
+ Expected<std::string> LLVMToSPIRVPath =
+ findProgram(Args, "llvm-spirv", {getMainExecutable("llvm-spirv")});
+ if (!LLVMToSPIRVPath)
+ return LLVMToSPIRVPath.takeError();
+
+ SmallVector<StringRef, 8> CmdArgs;
+ CmdArgs.push_back(*LLVMToSPIRVPath);
+ const llvm::Triple Triple(Args.getLastArgValue(OPT_triple));
+ getSPIRVTransOpts(Args, CmdArgs, Triple);
+ StringRef LLVMToSPIRVOptions;
+ if (Arg *A = Args.getLastArg(OPT_llvm_spirv_options_EQ))
+ LLVMToSPIRVOptions = A->getValue();
+ LLVMToSPIRVOptions.split(CmdArgs, " ", /* MaxSplit = */ -1,
+ /* KeepEmpty = */ false);
+ CmdArgs.append({"-o", OutputFile});
+ CmdArgs.push_back(File);
+ if (Error Err = executeCommands(*LLVMToSPIRVPath, CmdArgs))
+ return std::move(Err);
+
+ if (!SPIRVDumpDir.empty()) {
+ std::error_code EC =
+ llvm::sys::fs::create_directory(SPIRVDumpDir, /*IgnoreExisting*/ true);
+ if (EC)
+ return createStringError(
+ EC,
+ formatv("failed to create dump directory. path: {0}, error_code: {1}",
+ SPIRVDumpDir, EC.value()));
+
+ StringRef Path = OutputFile;
+ StringRef Filename = llvm::sys::path::filename(Path);
+ SmallString<128> CopyPath = SPIRVDumpDir;
+ CopyPath.append(Filename);
+ EC = llvm::sys::fs::copy_file(Path, CopyPath);
+ if (EC)
+ return createStringError(
+ EC,
+ formatv(
+ "failed to copy file. original: {0}, copy: {1}, error_code: {2}",
+ Path, CopyPath, EC.value()));
+ }
+
+ return OutputFile;
+}
+
+Error runSYCLLink(ArrayRef<std::string> Files, const ArgList &Args) {
+ llvm::TimeTraceScope TimeScope("SYCLDeviceLink");
+ // First llvm-link step
+ auto LinkedFile = linkDeviceInputFiles(Files, Args);
+ if (!LinkedFile)
+ reportError(LinkedFile.takeError());
+
+ // second llvm-link step
+ auto DeviceLinkedFile = linkDeviceLibFiles(*LinkedFile, Args);
+ if (!DeviceLinkedFile)
+ reportError(DeviceLinkedFile.takeError());
+
+ // LLVM to SPIR-V translation step
+ auto SPVFile = runLLVMToSPIRVTranslation(*DeviceLinkedFile, Args);
+ if (!SPVFile)
+ return SPVFile.takeError();
+ return Error::success();
+}
+
+} // namespace
+
+int main(int argc, char **argv) {
+ InitLLVM X(argc, argv);
+
+ Executable = argv[0];
+ sys::PrintStackTraceOnErrorSignal(argv[0]);
+
+ const OptTable &Tbl = getOptTable();
+ BumpPtrAllocator Alloc;
+ StringSaver Saver(Alloc);
+ auto Args = Tbl.parseArgs(argc, argv, OPT_INVALID, Saver, [&](StringRef Err) {
+ reportError(createStringError(inconvertibleErrorCode(), Err));
+ });
+
+ if (Args.hasArg(OPT_help) || Args.hasArg(OPT_help_hidden)) {
+ Tbl.printHelp(
+ outs(),
+ "clang-sycl-link-wrapper [options] <options to sycl link steps>",
+ "A utility that wraps around several steps required to link SYCL "
+ "device files.\n"
+ "This enables LLVM IR linking, post-linking and code generation for "
+ "SYCL targets.",
+ Args.hasArg(OPT_help_hidden), Args.hasArg(OPT_help_hidden));
+ return EXIT_SUCCESS;
+ }
+
+ if (Args.hasArg(OPT_version))
+ printVersion(outs());
+
+ Verbose = Args.hasArg(OPT_verbose);
+ DryRun = Args.hasArg(OPT_dry_run);
+ SaveTemps = Args.hasArg(OPT_save_temps);
+
+ OutputFile = "a.spv";
+ if (Args.hasArg(OPT_o))
+ OutputFile = Args.getLastArgValue(OPT_o);
+
+ if (Args.hasArg(OPT_spirv_dump_device_code_EQ)) {
+ Arg *A = Args.getLastArg(OPT_spirv_dump_device_code_EQ);
+ SmallString<128> Dir(A->getValue());
+ if (Dir.empty())
+ llvm::sys::path::native(Dir = "./");
+ else
+ Dir.append(llvm::sys::path::get_separator());
+
+ SPIRVDumpDir = Dir;
+ }
+
+ // Get the input files to pass to the linking stage.
+ auto FilesOrErr = getInput(Args);
+ if (!FilesOrErr)
+ reportError(FilesOrErr.takeError());
+
+ // Run SYCL linking process on the generated inputs.
+ if (Error Err = runSYCLLink(*FilesOrErr, Args))
+ reportError(std::move(Err));
+
+ // Remove the temporary files created.
+ if (!Args.hasArg(OPT_save_temps))
+ for (const auto &TempFile : TempFiles)
+ if (std::error_code EC = sys::fs::remove(TempFile))
+ reportError(createFileError(TempFile, EC));
+
+ return EXIT_SUCCESS;
+}
diff --git a/clang/tools/clang-sycl-link-wrapper/SYCLLinkOpts.td b/clang/tools/clang-sycl-link-wrapper/SYCLLinkOpts.td
new file mode 100644
index 00000000000000..23710645016bc0
--- /dev/null
+++ b/clang/tools/clang-sycl-link-wrapper/SYCLLinkOpts.td
@@ -0,0 +1,47 @@
+include "llvm/Option/OptParser.td"
+
+def WrapperOnlyOption : OptionFlag;
+
+def help : Flag<["-", "--"], "help">,
+ HelpText<"Display available options (--help-hidden for more)">;
+
+def help_hidden : Flag<["-", "--"], "help-hidden">,
+ HelpText<"Display all available options">;
+
+def verbose : Flag<["-"], "v">, HelpText<"Print verbose information">;
+def version : Flag<["--"], "version">,
+ HelpText<"Display the version number and exit">;
+
+def o : JoinedOrSeparate<["-"], "o">, MetaVarName<"<path>">,
+ HelpText<"Path to file to write output">;
+def output : Separate<["--"], "output-file">, Alias<o>, Flags<[HelpHidden]>,
+ HelpText<"Alias for -o">;
+
+def library_path_EQ : Joined<["--", "-"], "library-path=">,
+ Flags<[HelpHidden]>, HelpText<"Add <dir> to the library search path">;
+
+def triple : Joined<["--"], "triple">,
+ HelpText<"The device target triple">;
+def arch : Separate<["--", "-"], "arch">,
+ HelpText<"Specify the name of the target architecture.">;
+
+def g : Flag<["-"], "g">, HelpText<"Specify that this was a debug compile.">;
+def debug : Flag<["--"], "debug">, Alias<g>;
+
+def save_temps : Flag<["--", "-"], "save-temps">,
+ Flags<[WrapperOnlyOption]>, HelpText<"Save intermediate results">;
+
+def dry_run : Flag<["--", "-"], "dry-run">, Flags<[WrapperOnlyOption]>,
+ HelpText<"Print generated commands without running.">;
+
+def spirv_dump_device_code_EQ : Joined<["--", "-"], "spirv-dump-device-code=">,
+ Flags<[WrapperOnlyOption]>,
+ HelpText<"Path to the folder where the tool dumps SPIR-V device code. Other formats aren't dumped.">;
+
+def is_windows_msvc_env : Flag<["--", "-"], "is-windows-msvc-env">,
+ Flags<[WrapperOnlyOption, HelpHidden]>;
+
+// Options to pass to llvm-spirv tool
+def llvm_spirv_options_EQ : Joined<["--", "-"], "llvm-spirv-options=">,
+ Flags<[WrapperOnlyOption]>,
+ HelpText<"Options that will control llvm-spirv step">;
More information about the cfe-commits
mailing list