[PATCH] D34362: [LNT] Support for different DataSet usage in Polybench for "lnt runtest nt"

[Matthias Braun via Phabricator via llvm-commits]

MatzeB added a comment.

In https://reviews.llvm.org/D34362#790215, @cs14mtech11017 wrote:

> In https://reviews.llvm.org/D34362#789280, @cmatthews wrote:
>
> > It makes more sense to me to be using the --make-param flag to pass a test specific configuration options.  If you want to add all these size classes, all the tests should support them, or have them mapped back to nearest size the tests can handle.
>
>
> I feel having a common flag for test data sizes is better than having to pass them as test specific flag. As you suggested, we can map them back to nearest data sizes for the other tests. I was working with Polybench, so did not look into other tests.  Will proceed with it if everyone agrees on adding a flag like "--testdatasizes=mini|small|medium|large|extra_large" and aliasing --small and --large for backward compatibility.


Look at SPEC for example: It has 3 different sizes: test, train and ref. You can describe them roughly as:

- test: "As fast as posisble, not useful as a benchmark but they touch enough code paths so that they can give you a quick way to test for correctness"
- train: "Somewhat realistic but smaller inputs, useful to produce data for PGO optimizations. The important aspect here is that the data is different enough from ref, so we don't overfit the code because training and reference data were the same"
- ref: "Larger inputs running for a longer time producing stable numbers".

So there is some semantics and intended uses here that is captured fine with "test", "train" and "ref". Mapping this to some generic terms like "small", "medium", "large" that just map to sizes would be a loss IMO.


Repository:
  rL LLVM

https://reviews.llvm.org/D34362