[LNT] r269153 - Do not force the 'test' inputset when running pgo

[Adam Nemet via llvm-commits]

> On May 11, 2016, at 10:40 AM, Matthias Braun via llvm-commits <llvm-commits at lists.llvm.org> wrote:
> 
> Well the important part of the commit is that the user can now choose instead of getting forced.
> 
> As to 'test' vs. 'ref': The 'test' working set is useless for performance measurements as it doesn't really reflect the workload (most SPEC benchmarks spend more time initializing and setting up datastructures than actual computation in the 'test' set). I think --pgo is an indication that the user is interested in performance measurements and therefore wants the full 'ref' set.

Either way, we should document all this.  When I first saw -pgo it wasn’t clear what a test+pgo combination would mean.  Test is usually smaller than train so training for test is sort of silly.

I agree with Matthias that ref makes more sense as a default with pgo but then this should be documented under the -pgo flag.

Adam

> 
> - Matthias
> 
>> On May 11, 2016, at 10:36 AM, Chris Matthews via llvm-commits <llvm-commits at lists.llvm.org> wrote:
>> 
>> I intentionally made test the default.  I don’t think ref should be the default, it takes too long to run.  I want the user to have to elect that, so they know what they are getting into.
>> 
>> 
>>> On May 10, 2016, at 6:31 PM, Matthias Braun via llvm-commits <llvm-commits at lists.llvm.org> wrote:
>>> 
>>> Author: matze
>>> Date: Tue May 10 20:31:54 2016
>>> New Revision: 269153
>>> 
>>> URL: http://llvm.org/viewvc/llvm-project?rev=269153&view=rev
>>> Log:
>>> Do not force the 'test' inputset when running pgo
>>> 
>>> Modified:
>>>  lnt/trunk/lnt/tests/test_suite.py
>>> 
>>> Modified: lnt/trunk/lnt/tests/test_suite.py
>>> URL: http://llvm.org/viewvc/llvm-project/lnt/trunk/lnt/tests/test_suite.py?rev=269153&r1=269152&r2=269153&view=diff
>>> ==============================================================================
>>> --- lnt/trunk/lnt/tests/test_suite.py (original)
>>> +++ lnt/trunk/lnt/tests/test_suite.py Tue May 10 20:31:54 2016
>>> @@ -464,8 +464,8 @@ class TestSuiteTest(BuiltinTest):
>>>       if self.opts.pgo and self.trained:
>>>           defs['TEST_SUITE_PROFILE_USE'] = "On"
>>>           defs['TEST_SUITE_PROFILE_GENERATE'] = "Off"
>>> -            # This could be redefined by user defines.
>>> -            defs['TEST_SUITE_RUN_TYPE'] = "test"
>>> +            if 'TEST_SUITE_RUN_TYPE' not in defs:
>>> +                defs['TEST_SUITE_RUN_TYPE'] = 'ref'
>>> 
>>>       if self.opts.cmake_defines:
>>>           for item in self.opts.cmake_defines:
>>> 
>>> 
>>> _______________________________________________
>>> llvm-commits mailing list
>>> llvm-commits at lists.llvm.org
>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
>> 
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits
> 
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-commits