[llvm-commits] [test-suite] r157636 - in /test-suite/trunk/MultiSource/Benchmarks/tscp181: ./ LICENSE.txt Makefile board.c book.c book.txt data.c data.h defs.h eval.c main.c protos.h readme.txt search.c tscp181.reference_output

[Evan Cheng]

On May 30, 2012, at 8:30 AM, John Criswell wrote:

> On 5/30/12 9:17 AM, Tom Kerrigan wrote:
>> Sorry I haven't had much time to look at this thread today. I understand
>> your concerns. I am not a litigious sort but I want to avoid the
>> situation where TSCP source is released with a confusing/contradictory
>> license, and then somebody takes it and starts selling it, which is
>> something that's happened to me before. So however this is done is fine
>> with me as long as I'm covered.
> 
> First, I'd like to remind people with SVN commit access to think twice 
> before committing third-party code to the SVN repository.  Once code is 
> in the LLVM SVN repository, it is extremely difficult to truly get it 
> out due to the fact that people can just fetch previous revisions.  If 
> we want to truly remove code, we have to run a script that "zeros out" 
> the erroneous commit, or we have to use SVN's access controls to deny 
> access to the offending files.  The former runs the risk of corrupting 
> the repository (which has grown quite large due to its age and the 
> number of LLVM sub-projects using it) and screwing up the revision 
> numbers if not done properly, and the latter decreases performance of 
> the SVN software.  Both approaches waste the time of an llvm.org admin.
> 
> Second, if there's a program that you want to use in the test suite, and 
> it isn't clear whether we want to distribute it as part of test-suite, 
> then you can always make it an External test until you get the licensing 
> details straightened out.
> 
> Third, I'd like to remind everyone that the LLVM test suite is used for 
> more than testing LLVM core.  It's used by researchers for experiments 
> that show up in research papers.  It's used by LLVM-based tools to test 
> their software.  I use it for testing SAFECode (and have found memory 
> safety bugs in various test programs).  IMHO, restrictions on commercial 
> use are fine (examples of what is meant by "commercial use" is helpful 
> also), but I don't really like the idea of restricting a test program 
> for just testing and benchmarking LLVM; that seems way too restrictive.

Really?! If that's the policy, it's news to me. It's certainly not here: http://llvm.org/docs/DeveloperPolicy.html#license
Have we verified all the licenses for all of the benchmarks in LLVM test suite?

> 
>> 
>> I will happily give you guys permission to use TSCP for testing LLVM and
>> doing related activities, which seems like it would cover anything you
>> might want to do and also exclude people from selling it etc. (since how
>> would publishing a game be related to testing LLVM?).
> 
> Disclaimer: I am not a lawyer, and none of this is legal advice.
> 
> Mr. Kerrigan, I really, really appreciate that you want to help the LLVM 
> project by allowing our users to use your code for testing.  However, I 
> fear that even if you get the licensing details straightened out, 
> someone can just grab the code from our repository and use it 
> illegally.  IMHO, a better approach to protecting your investment would 
> be to make an External test(*) for TSCP and then release copies of TSCP 
> to people/organizations in the community that you trust.  That way, you 
> know who has your software and what they are doing with it.

> 
> If you still want TSCP to be included in test-suite, then I suggest 
> something that is liberal enough to allow all the uses of test-suite 
> plus redistribution in source form but that denies commercial use.  I 
> think we have some tests like that in MultiSource/Applications or 
> MultiSource/Benchmarks; you can check their LICENSE.TXT files and use 
> those as examples.
> 
> -- John T.
> 
> (*) An External test in the LLVM test suite is a test in which the test 
> suite has the Makefiles to build a program but does not contain the 
> source code to that program; the source code is *external* to the test 
> suite.  We do this for tests like SPEC for which we do not have 
> redistribution rights.


I asked Tom for permission to include it in our test suite because gcc is generating much better code for TSCP than llvm. Without adding it to the main part of the test suite (external doesn't count because most people won't run it), we won't be tracking its performance. I don't think it's reasonable to ask Tom to change the license just for our sake so I'll just drop it. It seems a very unwelcome and unrealistic requirement to me and it's going to make it difficult to include other benchmarks into our suite.

Evan

> 
>> 
>> What do you think?
>> 
>> On 5/30/12 3:25 PM, Rafael Espíndola wrote:
>>>> - build and run it as part of the benchmark suite
>>>> - publish information about those runs, timing, etc
>>>> - redistribute it as part of the benchmark suite
>>>> - modify it to accomodate platforms, build systems, benchmarking stability
>>>> issues, etc.
>>>> - all of the above w.r.t. to those modifications
>>> Not sure if covered by the second item, but what about using the test
>>> results to improve the compiler, both in tree and proprietary
>>> backends?
>>> 
>>> Cheers,
>>> Rafael
>> _______________________________________________
>> llvm-commits mailing list
>> llvm-commits at cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> 
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits