<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">On Sep 28, 2015, at 4:33 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com" class="">dblaikie@gmail.com</a>> wrote:<br class=""><div><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote">On Mon, Sep 28, 2015 at 4:25 PM, Devin Coughlin via cfe-dev <span dir="ltr" class=""><<a href="mailto:cfe-dev@lists.llvm.org" target="_blank" class="">cfe-dev@lists.llvm.org</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br class="">
We propose taking a “curl + cache” approach to benchmarks. That is, we won’t store the benchmarks themselves in a repository. Instead, the bots will download them from the projects' websites and cache locally.</blockquote><div class=""><br class=""></div><div class="">If we're going to be downloading things from external sources, those sources could be changing, no? Or will we pin to a specific version.</div></div></div></div></div></blockquote><div><br class=""></div><div>We will pin it to a specific version, although we may want to periodically (yearly or even less frequently) update to a newer version of the benchmarks to make sure we get good coverage of newer language features and to keep the benchmarks compiling with ToT clang.</div><br class=""><blockquote type="cite" class=""><div class=""><div dir="ltr" class=""><div class="gmail_extra"><div class="gmail_quote"><div class="">if we're pinning to a specific version, what's the benefit to taking an external dependency like that (untrusted, may be down when we need it, etc), compared to copying the files permanently & checking them in to clang-tests (or clang-tests-external) as I did for GDB?<br class=""></div></div></div></div></div></blockquote><div><br class=""></div><div>The goal here is to avoid unnecessary co-mingling of benchmarks with different licenses in the <a href="http://llvm.org" class="">llvm.org</a> repositories — but as you point out, it comes with significant downsides. What has your experience with gdb and clang-test-external been like? Has dealing with differing licensing w/r/t clang been a challenge?</div><div><br class=""></div><div>Devin</div></div></body></html>