[clang] [Clang][Driver][Test] Created test for unsupported driver options (PR #120900)

[Mészáros Gergely via cfe-commits]

================
@@ -0,0 +1,472 @@
+#!/usr/bin/env python3
+
+"""generate_unsupported_in_drivermode.py
+
+This script generates Lit regression test files that validate that options are only exposed to intended driver modes.
+
+The options and driver modes are parsed from Options.td, whose path should be provided on the command line.
+See clang/include/clang/Driver/Options.td
+
+The path to the TableGen executable can optionally be provided. Otherwise, the script will search for it.
+
+The primary maintenance task for this script would be updating the expected return message for a driver mode if
+there are changes over time. See the instantiations of DriverData, specifically the check_str.
+
+Logic:
+1) For each option, (records of class "Option"), and for each driver, (records of class "OptionVisibility")
+    a. if the option's "Visibility" field includes the driver flavour, skip processing this option for this driver
+    b. if the option is part of an option group, (the record has the "Group" property),
+       and the group's "Visibility" field includes the driver flavour, skip processing this option for this driver
+    c. otherwise this option is not supported by this driver flavour, and this pairing is saved for testing
+2) For each unsupported pairing, generate a Lit RUN line, and a CHECK line to parse for expected output. Ex: "error: unknown argument"
+"""
+
+import shutil
+import os
+import json
+import subprocess
+from bisect import bisect_left
+from dataclasses import dataclass
+import argparse
+import dataclasses
+from itertools import batched
+
+# Strings defined in Options.td for the various driver flavours. See "OptionVisibility"
+VISIBILITY_CC1AS = "CC1AsOption"
+VISIBILITY_CC1 = "CC1Option"
+VISIBILITY_CL = "CLOption"
+VISIBILITY_DXC = "DXCOption"
+VISIBILITY_DEFAULT = "DefaultVis"
+VISIBILITY_FC1 = "FC1Option"
+VISIBILITY_FLANG = "FlangOption"
+
+# Lit test prefix strings
+SLASH_SLASH = "// "
+EXCLAMATION = "! "
+
+# Invalid usage of the driver options below causes unique output, so skip testing
+exceptions_sequence = [
+    "cc1",
+    "cc1as",
+]
+
+
+class UnsupportedDriverOption:
+    """Defines an unsupported driver-option combination
+    driver: The driver string as defined by OptionVisibility in Options.td
+    option: The option object from Options.td
+    option_name: Corresponding string for an option. See "Name" for a given option in Options.td
+    prefix: String that precedes the option. Ex. "-"
+    """
+
+    def __init__(self, driver, option, option_name, prefix):
+        self.driver = driver
+        self.option = option
+        self.option_name = option_name
+        self.prefix = prefix
+
+    # For sorting
+    def __len__(self):
+        return len(self.option_name)
+
+    def __lt__(self, other):
+        return len(self.option_name) > len(other.option_name)
+
+
+ at dataclass
+class DriverData:
+    """Dataclass for data specific to each driver
+    lit_cmd_prefix: The beginning string of the Lit command
+    lit_cmd_options: Strings containing additional options for this driver
+    visibility_str: The corresponding visibility string from OptionVisibility in Options.td
+    lit_cmd_end: String at the end of the Lit command
+    check_str: The string or regex to be sent to FileCheck
+    supported_sequence: List of UnsupportedDriverOption objects for supported options
+                        that are Kind *JOINED*, as defined in Options.td
+    test_option_sequence: A list of all the prefix-option pairs that will be tested for this driver
+    """
+
+    lit_cmd_prefix: str
+    lit_cmd_options: str
+    visibility_str: str
+    lit_cmd_end: str = " - < /dev/null 2>&1 | FileCheck -check-prefix=CHECK-COUNT-"
+    check_str: str = "{{(unknown argument|n?N?o such file or directory)}}"
+    supported_sequence: list[UnsupportedDriverOption] = dataclasses.field(
+        default_factory=list
+    )
+    test_option_sequence: list[str] = dataclasses.field(default_factory=list)
+
+
+def find_groups(options_dictionary, option):
----------------
Maetveis wrote:

`option` is only an option in the first call to the function, during recursive calls it will be a group. This is confusing IMO, maybe just naming it a generic `member` or a generic `record`.

https://github.com/llvm/llvm-project/pull/120900