[llvm] [SPIR-V] Add SPIRV-Tools for testing (PR #73044)

[Natalie Chouinard via llvm-commits]

================
@@ -0,0 +1,66 @@
+option(LLVM_INCLUDE_SPIRV_TOOLS_TESTS "Include tests that use SPIRV-Tools" Off)
+mark_as_advanced(LLVM_INCLUDE_SPIRV_TOOLS_TESTS)
+
+if (NOT LLVM_INCLUDE_SPIRV_TOOLS_TESTS)
+  return()
+endif ()
+
+if (NOT "SPIRV" IN_LIST LLVM_TARGETS_TO_BUILD)
+  message(FATAL_ERROR "Building SPIRV-Tools tests is unsupported without the SPIR-V target")
+endif ()
+
+# SPIRV_DIS and SPIRV_VAL variables can be used to provide paths to existing
+# spirv-dis and spirv-val binaries, respectively. Otherwise, build them from
+# SPIRV-Tools source.
+if (NOT SPIRV_DIS OR NOT SPIRV_VAL)
+  include(ExternalProject)
+
+  set(BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/SPIRVTools-bin)
+
+  ExternalProject_Add(SPIRVTools
+    GIT_REPOSITORY https://github.com/KhronosGroup/SPIRV-Tools.git
+    GIT_TAG main
+    BINARY_DIR ${BINARY_DIR}
+    BUILD_COMMAND ${CMAKE_COMMAND} --build ${BINARY_DIR} --target spirv-dis spirv-val
+    BUILD_BYPRODUCTS ${BINARY_DIR}/tools/spirv-dis ${BINARY_DIR}/tools/spirv-val
+    # Don't auto-update on every build.
+    UPDATE_DISCONNECTED 1
+    # Allow manual updating with an explicit SPIRVTools-update target.
+    STEP_TARGETS update
+    INSTALL_COMMAND ""
+    )
+
+  ExternalProject_Add_Step(SPIRVTools get-deps
----------------
sudonatalie wrote:

Hm, I see the issue, it's not related to the `STEP_TARGETS update` bit, it's just that only any second+ builds were succeeding. The first run of the download subcommand was failing because some CMake files were missing, even though they are about to be fetched in the next get-deps step. I was trying to avoid writing a custom download step command, but I think it's necessary.

The download and deps sync should happen at build (not configuration) either way, but I've set it up so they only happen on the first one, or when it's explicitly triggered with `SPIRVTools-update`.

Works with a clean build now.

https://github.com/llvm/llvm-project/pull/73044