[PATCH] Add new lit argument: --exec-feature=FEATURE

[Daniel Dunbar]

Hi Alexey,

This shouldn't require adding a new lit option.

Ideally, I would propose the right way to handle this would be to have some
way to detect that the compile is done with ASAN support enabled (if the
support is enabled with a configure flag, just set a new build variable).
For example, see how "llvm_use_intel_jit" gets added to the lit features
for LLVM. This way things will "just work" if people build with ASAN
enabled.

Is there a reason that approach doesn't work here? If not, then I still
would prefer using the single existing lit mechanism --param to pass in the
parameter.

 - Daniel


On Fri, Feb 15, 2013 at 6:57 AM, Alexey Samsonov <samsonov at google.com>wrote:

> Hi ddunbar,
>
> This patch adds a new option for lit test runner: --exec-feature=<feature>
> This option allows to specify test execution mode and can be used in
> REQUIRES:
> and XFAIL: lines. Particular use case: we are running LLVM/Clang tests
> under
> AddressSanitizer to find memory errors, and there are some "known"
> buganized
> failures like http://llvm.org/bugs/show_bug.cgi?id=15130. With this patch
> applied,
> we can mark these tests "XFAIL: asan", specify
> LLVM_LIT_ARGS="--exec-feature=asan"
> at configuration time, and keep the build clean.
>
> http://llvm-reviews.chandlerc.com/D413
>
> Files:
>   docs/CommandGuide/lit.rst
>   utils/lit/lit/TestingConfig.py
>   utils/lit/lit/discovery.py
>   utils/lit/lit/LitConfig.py
>   utils/lit/lit/main.py
>
> Index: docs/CommandGuide/lit.rst
> ===================================================================
> --- docs/CommandGuide/lit.rst
> +++ docs/CommandGuide/lit.rst
> @@ -122,6 +122,12 @@
>   suite take the most time to execute.  Note that this option is most
> useful
>   with ``-j 1``.
>
> +.. option:: --exec-feature=FEATURE
> +
> + Specify an additional feature of test execution that can be matched in
> REQUIRES
> + and XFAIL lines. For example, if you run tests under AddressSanitizer,
> you can
> + specify ``--exec-feature=asan``.
> +
>  .. _selection-options:
>
>  SELECTION OPTIONS
> Index: utils/lit/lit/TestingConfig.py
> ===================================================================
> --- utils/lit/lit/TestingConfig.py
> +++ utils/lit/lit/TestingConfig.py
> @@ -29,13 +29,6 @@
>                          'TMP' : os.environ.get('TMP',''),
>                          })
>
> -            # Set the default available features based on the LitConfig.
> -            available_features = []
> -            if litConfig.useValgrind:
> -                available_features.append('valgrind')
> -                if litConfig.valgrindLeakCheck:
> -                    available_features.append('vg_leak')
> -
>              config = TestingConfig(parent,
>                                     name = '<unnamed>',
>                                     suffixes = set(),
> @@ -47,7 +40,7 @@
>                                     test_exec_root = None,
>                                     test_source_root = None,
>                                     excludes = [],
> -                                   available_features =
> available_features)
> +                                   available_features =
> litConfig.execFeatures)
>
>          if os.path.exists(path):
>              # FIXME: Improve detection and error reporting of errors in
> the
> Index: utils/lit/lit/discovery.py
> ===================================================================
> --- utils/lit/lit/discovery.py
> +++ utils/lit/lit/discovery.py
> @@ -222,6 +222,7 @@
>                                      valgrindLeakCheck = False,
>                                      valgrindArgs = [],
>                                      noExecute = False,
> +                                    execFeatures = [],
>                                      ignoreStdErr = False,
>                                      debug = False,
>                                      isWindows =
> (platform.system()=='Windows'),
> Index: utils/lit/lit/LitConfig.py
> ===================================================================
> --- utils/lit/lit/LitConfig.py
> +++ utils/lit/lit/LitConfig.py
> @@ -19,7 +19,7 @@
>
>      def __init__(self, progname, path, quiet,
>                   useValgrind, valgrindLeakCheck, valgrindArgs,
> -                 noExecute, ignoreStdErr, debug, isWindows,
> +                 noExecute, execFeatures, ignoreStdErr, debug, isWindows,
>                   params, config_prefix = None):
>          # The name of the test runner.
>          self.progname = progname
> @@ -30,6 +30,7 @@
>          self.valgrindLeakCheck = bool(valgrindLeakCheck)
>          self.valgrindUserArgs = list(valgrindArgs)
>          self.noExecute = noExecute
> +        self.execFeatures = list(execFeatures)
>          self.ignoreStdErr = ignoreStdErr
>          self.debug = debug
>          self.isWindows = bool(isWindows)
> @@ -47,10 +48,12 @@
>
>          self.valgrindArgs = []
>          if self.useValgrind:
> +            self.execFeatures.append('valgrind')
>              self.valgrindArgs = ['valgrind', '-q',
> '--run-libc-freeres=no',
>                                   '--tool=memcheck',
> '--trace-children=yes',
>                                   '--error-exitcode=123']
>              if self.valgrindLeakCheck:
> +                self.execFeatures.append('vg_leak')
>                  self.valgrindArgs.append('--leak-check=full')
>              else:
>                  # The default is 'summary'.
> Index: utils/lit/lit/main.py
> ===================================================================
> --- utils/lit/lit/main.py
> +++ utils/lit/lit/main.py
> @@ -207,6 +207,10 @@
>      group.add_option("", "--no-execute", dest="noExecute",
>                       help="Don't execute any tests (assume PASS)",
>                       action="store_true", default=False)
> +    group.add_option("", "--exec-feature", dest="execFeatures",
> +                     metavar="FEATURE",
> +                     help="Add user-defined feature of test execution",
> +                     type=str, action="append", default=[])
>      parser.add_option_group(group)
>
>      group = OptionGroup(parser, "Test Selection")
> @@ -271,6 +275,7 @@
>                                      valgrindLeakCheck =
> opts.valgrindLeakCheck,
>                                      valgrindArgs = opts.valgrindArgs,
>                                      noExecute = opts.noExecute,
> +                                    execFeatures = opts.execFeatures,
>                                      ignoreStdErr = False,
>                                      debug = opts.debug,
>                                      isWindows =
> (platform.system()=='Windows'),
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20130221/d4422708/attachment.html>