[llvm-dev] [cfe-dev] [Openmp-dev] RFC: End-to-end testing

[David Blaikie via llvm-dev]

On Wed, Oct 16, 2019 at 1:09 PM Roman Lebedev via cfe-dev <
cfe-dev at lists.llvm.org> wrote:

> FWIW I'm personally cautiously non-optimistic about this,
> but maybe i'm just not seeing the whole picture of the proposal.
>
> Both checking final asm, and checking more than one layer of abstraction
> feels overreaching and very prone to breakage/too restrictful.
> Even minimal changes to the scheduling for particular CPU can cause many
> instructions to reorder.
> I'm not sure what effect that will have on middle-end pass development,
> too.
>
> A change affects these end-to-end tests, what then?
> Just blindly regenerate every affected test?
> This will be further complicated once clang isn't the only upstream
> front-end.
>

Agreed that the broader a test is, the more careful one has to be about
making it reliable in spite of other changes - sometimes that's really
difficult (if you're trying to get a particular instruction selection or
register allocation) but in others it can be fairly reliable if done
carefully to sufficiently restrict optimizations, etc. (having function
calls to external functions to act as sinks/sources for values, etc, for
instance - picking places where the output is already "optimal" and
trivially/obviously so (for whatever set of constraints you've provided -
not heroic optimizations, etc) to ensure that it's fairly stable)



- Dave


>
> Roman.
>
> On Wed, Oct 16, 2019 at 11:00 PM David Greene via cfe-dev
> <cfe-dev at lists.llvm.org> wrote:
> >
> > Renato Golin via Openmp-dev <openmp-dev at lists.llvm.org> writes:
> >
> > > We already have tests in clang that check for diagnostics, IR and
> > > other things. Expanding those can handle 99.9% of what Clang could
> > > possibly do without descending into assembly.
> >
> > I agree that for a great many things this is sufficient.
> >
> > > Assembly errors are more complicated than just "not generating VADD",
> > > and that's easier done in the TS than LIT.
> >
> > Can you elaborate?  I'm talking about very small tests targeted to
> > generate a specific instruction or small number of instructions.
> > Vectorization isn't the best example.  Something like verifying FMA
> > generation is a better example.
> >
> >                         -David
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20191016/f195d4fa/attachment.html>