[cfe-dev] [LLVMdev] [3.6 Release] RC3 has been tagged

[Hal Finkel]

----- Original Message -----
> From: "Renato Golin" <renato.golin at linaro.org>
> To: "Jack Howarth" <howarth.mailing.lists at gmail.com>
> Cc: "cfe-dev" <cfe-dev at cs.uiuc.edu>, "llvmdev" <llvmdev at cs.uiuc.edu>
> Sent: Wednesday, February 18, 2015 1:55:38 PM
> Subject: Re: [LLVMdev] [cfe-dev]  [3.6 Release] RC3 has been tagged
> 
> On 18 February 2015 at 18:54, Jack Howarth
> <howarth.mailing.lists at gmail.com> wrote:
> > http://www.phoronix.com/scan.php?page=article&item=llvm-clang-3.5-3.6-rc1&num=2
> 
> First, I have to say, I was saddened by that post, since the new
> release regress in a lot of the benchmarks, but not enough to stop
> the
> world and fix all of those regressions.
> 
> I have seen other similar regressions in my internal benchmarks, but
> again, not enough to stop everything. My focus now is to make it work
> well. I'll only focus on performance when the whole toolchain is
> working as I expect, or when I have more people to look at
> performance
> in parallel. I assume other people feel in the same way.
> 
> Second, how many benchmarks are there, and which of them classify as
> "important"? Should we stop for 20% regressions on any benchmark on
> the Internet? SciMark is a well known benchmark, I give you that, but
> to be honest, it means nothing to me. Maybe it will, one day, but not
> today, and probably not for another year or two.
> 
> It seems that others in the community feel in the same way, or we
> would have seen a lot of people jumping up and down until the
> regression got fixed. Hal and others seem to be on top of the issue,

There are a number of us who watch performance numbers regularly, certainly more now then there used to be, but this is still an area where we need improvement.

Also, regarding SciMark, this benchmark is already in our test suite (in MultiSource/Benchmarks/SciMark2-C), but this only gives us the composite over all of the SciMark tests. We should probably split this out so that we get timings for the individual tests (or relevant groups) in LNT. This is what I did for the TSVC benchmark, for example. Jack, would you be interested in working on this?

 -Hal

> and I assume they're the ones most interested in getting that fixed.
> If they're happy with it going to 3.6.1, an no one else is trying to
> get that fixed, I'm happy with that outcome.
> 
> As Hans said, we'll probably have an RC4. If you're so worried with
> this specific regression, can you try to fix it? It seems two commits
> are responsible in equal measure, so it shouldn't be too hard to see
> what they do, produce the code with and without each one of them, and
> see why things go slower. I'm not an expert on x86, and I didn't even
> run that benchmark on ARM/AArch64, so I can't possibly help.
> 
> cheers,
> --renato
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
> 

-- 
Hal Finkel
Assistant Computational Scientist
Leadership Computing Facility
Argonne National Laboratory