[PATCH] D73707: [TSAN] Parameterize the hard-coded threshold of deflake in tsan test

Thu Jan 30 11:35:19 PST 2020

anhtuyen added a comment.

In D73707#1850128 <https://reviews.llvm.org/D73707#1850128>, @rnk wrote:

> More than 10 retries seems excessive. For this particular issue, I think it would be reasonable to mark the test `; UNSUPPORTED: powerpc64`. Adding more retries will make the powerpc64 bots slower and less useful.

Thanks for your opinion Reid @rnk, but I disagree with your comment based on the following points.
(1) This is to make the value configurable in order to enable users who see the needs to adjust the deflake threshold to a value more suitable for their environment. The existing threshold of 10 is the default and thus there should be no impact to other bots.
(2) I have run the entire LIT (check-all) more than 10 times on the same dedicated machine, and the results are not much slower at all. On average, the time for the entire LIT will increase by 0.26 second if the threshold is 100, and  0.48s for 200.

|                                 | Threshold of 200 | Threshold of 100 | Threshold of 10 (Existing Value) |
| 1st                             | 124.04s          | 123.65s          | 122.64s                          |
| 2nd                             | 122.93s          | 123.12s          | 122.49s                          |
| 3rd                             | 123.33s          | 123.26s          | 123s                             |
| 4th                             | 123.21s          | 122.24s          | 121.93s                          |
| 5th                             | 122.6s           | 123.32s          | 122.48s                          |
| 6th                             | 123.29s          | 122.42s          | 123.02s                          |
| 7th                             | 123.01s          | 122.89s          | 122.37s                          |
| 8th                             | 122.57s          | 122.42s          | 122.63s                          |
| 9th                             | 122.61s          | 122.85s          | 122.6s                           |
| 10th                            | 123.34s          | 122.19s          | 122.95s                          |
| 11th                            | 122.98s          | 123.09s          | 122.47s                          |
| Average (in seconds)            | 123.08s          | 122.86s          | 122.6s                           |
| Increase in seconds vs baseline | 0.48s            | 0.26s            | (baseline)                       |
|

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D73707/new/

https://reviews.llvm.org/D73707