[llvm-dev] RFC: Large, especially super-linear, compile time regressions are bugs.

[Renato Golin via llvm-dev]

On 1 April 2016 at 01:09, Mehdi Amini <mehdi.amini at apple.com> wrote:
> What I have in mind is more: if a commit regress the build above a threshold (1% on average for instance), then we should be able to have a discussion about this commit to evaluate if it belongs to O2 or if it should go to O3 for instance.
> Also if the commit is about refactoring, or introducing a new feature, the regression might not be intended at all by the author!

Thresholds as trigger for discussion is exactly what I was looking for.

But Chandler goes further (or so I gathered), that some commits are
really bad and could be candidates for reversion before discussion.
Those, more extreme measures, may be justified if, for example, the
commit is quadratic or more in a core part of the compiler, or double
the testing time, etc.

I agree with both proposals, but we have to make sure what goes where,
to avoid (unintentionally) heavy handing other people's work.


> The metric is "the commit regressed 1%". The natural thing that follows is what happens usually in the community: we look at the data (what is the performance improvement), and decide on a case by case if it is fine as is or not.
> I feel like you're talking about the "metric" like an automatic threshold that triggers an automatic revert and block things, this is not the goal and that is not what I mean when I use of the word metric (but hey, I'm not a native speaker!).

I wasn't talking about automatic reversal, but about pre-discussion
reversal, as I mention above.


> As I said before, I'm mostly chasing *untracked* and *unintentional* compile time regression.

That's is obviously good. :)


> I'm not sure I really totally understand everything you mean.

It's about the threshold between what promotes discussion and what
promotes pre-discussion reverts. This is a hard line to draw with so
many people (and companies) involved.


> Sure, I'm glad you step up to make sure it does not happen. So please continue to voice up in the future as we try to roll thing.
> I hope we're on the same track past the initial misunderstanding we had each other?

Yes. :)

cheers,
--renato