[PATCH] D56064: More tolerance for flaky tests in libc++ on NetBSD

[Kamil Rytarowski via Phabricator]

krytarowski created this revision.
krytarowski added a reviewer: mgorny.
Herald added a reviewer: EricWF.
Herald added subscribers: llvm-commits, christof.

Tests marked with the flaky attribute ("FLAKY_TEST.")
can still report false positives in local tests and on the
NetBSD buildbot.

Additionally a number of tests (probably all threaded
ones) unmarked with the flaky attribute is flaky on
NetBSD.

An ideal solution on the libcxx side would be to raise
max retries for NetBSD and mark failing tests with
the flaky flag, however this adds more maintenance
burden and constant monitoring of flaky tests.

Reduce the work and handle flaky tests as more flaky
on NetBSD and allow flakiness  of other tests on
NetBSD.


Repository:
  rL LLVM

https://reviews.llvm.org/D56064

Files:
  utils/libcxx/test/format.py


Index: utils/libcxx/test/format.py
===================================================================
--- utils/libcxx/test/format.py
+++ utils/libcxx/test/format.py
@@ -12,6 +12,7 @@
 import os
 import time
 import random
+import platform
 
 import lit.Test        # pylint: disable=import-error
 import lit.TestRunner  # pylint: disable=import-error
@@ -202,6 +203,12 @@
                           for f in os.listdir(local_cwd) if f.endswith('.dat')]
             is_flaky = self._get_parser('FLAKY_TEST.', parsers).getValue()
             max_retry = 3 if is_flaky else 1
+
+            # LIBC++ tests tend to be more flaky on NetBSD, so add more retries.
+            # We don't do this on other platforms because it's slower.
+            if platform.system() in ['NetBSD']:
+                max_retry = max_retry * 3
+
             for retry_count in range(max_retry):
                 cmd, out, err, rc = self.executor.run(exec_path, [exec_path],
                                                       local_cwd, data_files,


-------------- next part --------------
A non-text attachment was scrubbed...
Name: D56064.179468.patch
Type: text/x-patch
Size: 1034 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/libcxx-commits/attachments/20181224/1dcfb6a2/attachment.bin>