[PATCH] [libcxx] Checking LIT benchmark testing framework.

[Chandler Carruth]

On Fri, Mar 6, 2015 at 11:47 AM, Dmitri Gribenko <gribozavr at gmail.com>
wrote:

> On Fri, Mar 6, 2015 at 11:29 AM, Chandler Carruth <chandlerc at gmail.com>
> wrote:
> >
> > On Fri, Mar 6, 2015 at 10:21 AM, hfinkel at anl.gov <hfinkel at anl.gov>
> wrote:
> >>
> >> Almost none of my benchmarking machines can initiate outgoing
> connections,
> >> so I'd need to download/checkout the Google 'benchmark' library
> manually.
> >> Can you please add some comments about exactly what should be
> retrieved, and
> >> where it should be put.
> >
> >
> > The entire thing seems odd.
> >
> > I would much rather just have the benchmark code live in some utility
> place
> > in one of the repositories. For example, we keep gtest in utils/unittest
> in
> > the LLVM repository. I wonder if we could check the benchmark stuff into
> > utils/benchmark, and add it to CMake normally.
> >
> > Then the benchmarks can be built normally with CMake (or Makefiles) and
> LIT
> > can just be responsible for actually running them.
> >
> > This would make the benchmarks unavailable from a stand-alone build of
> > libc++ but that seems a not-terrible tradeoff.
>
> +1 from me.  It would be great to have a benchmarking framework
> in-tree so that other projects could use it.  For example, we could
> have latency tests for Clang code completion or re-parsing with
> implicit PCH.


Eric reminded me that I may have led to this design. Sorry for that if so...

I think the concern I had was that we don't currently have any unusually
licensed stuff in the libc++ tree, and adding it might be problematic.
Keeping the license of libc++ super clear and explicit is really important
because it is a runtime library and can be used in even more diverse
contexts than the rest of LLVM.

I had originally thought that we might not want to re-use an existing
benchmarking framework, but I can also see lots of good reasons to re-use
an existing framework.

My hope with the suggestion of keeping the benchmarking framework in the
core LLVM repository is that it would make it much more clear that none of
the functionality in libc++ would have anything to do with it, only
benchmarks. The benchmarks themselves of course could always live in the
libc++ repository and be built with an explicitly specified copy of the
benchmark framework if libc++ were being used outside of an LLVM checkout.


Hopefully that clarifies some of why it might be useful to have *some*
levels of separation between the project and the benchmarking stuff...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/cfe-commits/attachments/20150306/204954fa/attachment.html>