[cfe-dev] Paragraph numbers in C++ tests

[via cfe-dev]

> -----Original Message-----
> From: David Blaikie [mailto:dblaikie at gmail.com]
> Sent: Tuesday, April 30, 2019 12:54 PM
> To: Robinson, Paul
> Cc: Bruno Ricci; Clang Dev
> Subject: Re: [cfe-dev] Paragraph numbers in C++ tests
> 
> On Tue, Apr 30, 2019 at 8:12 AM via cfe-dev <cfe-dev at lists.llvm.org>
> wrote:
> >
> >
> >
> > > -----Original Message-----
> > > From: cfe-dev [mailto:cfe-dev-bounces at lists.llvm.org] On Behalf Of
> Bruno
> > > Ricci via cfe-dev
> > > Sent: Tuesday, April 30, 2019 10:34 AM
> > > To: cfe-dev at lists.llvm.org
> > > Subject: Re: [cfe-dev] Paragraph numbers in C++ tests
> > >
> > >
> > >
> > > On 29/04/2019 19:33, David Blaikie wrote:
> > > > Even if they're updated to reflect current paragraph numbers, they'd
> > > > still fall into disrepair in future standards & be hard to know
> which
> > > > version they applied to. So I'd support adding C++ version numbers
> to
> > > > them. At least on principle - but the reality/situations they're
> used
> > > > in might make that awkward/problematic, I don't know.
> > >
> > > I was thinking about the naming of tests for sections of the C++
> standard.
> > > For example the tests for [namespace.udecl] are:
> > >   p1.cpp
> > >   p3.cpp
> > >   p4.cpp
> > >   p5-cxx0x.cpp
> > >   p6-cxx11.cpp
> > >   p7.cpp
> > >   p8.cpp
> > >   p8-cxx0x.cpp
> > >   p10.cpp
> > >   p11.cpp
> > >   p12.cpp
> > >   p13.cpp
> > >   p15.cpp
> > >   p18.cpp
> > >
> > > I think that naming the tests pn-cxxmm.cpp to mean, "paragraph pn of
> > > cxxmm"
> > > is reasonable, with the understanding that naming a test p3-cxx17.cpp
> does
> > > not
> > > mean that the corresponding rule is specific to C++17, just that it
> was
> > > written
> > > against C++17.
> > >
> > > Bruno
> >
> > The real issue is, "p3" in C++11 might be renumbered "p4" in C++14
> > and "p5" in C++17 and ...  so what does "p3.cpp" actually test?
> > You can't tell just from the name of the file because the name's
> > meaning is context-sensitive.
> >
> > Intuitively I'd think "p3-cxx11.cpp" means "tests p3 as it existed in
> > the C++11 standard" and whether that paragraph is still p3 in later
> > editions is an exercise for the reader.
> 
> I think that^ is what Bruno was saying.
> 
> > I recommend, give up on naming tests after paragraph numbers, because
> > they aren't actually meaningful.
> 
> Eh, I think there's merit to having tests by feature in the standard
> and being able to inspect their coverage (yeah, that gets wonky when
> paragraphs are renumbered - so you might have to update the numbering
> - but the benefit of a correspondence between some part/version of the
> standard even if you have to do some work remapping to a consistent
> version.

Having contended with a test suite that used this naming scheme, I
beg to disagree; using paragraph numbers was not particularly helpful.
I basically ignored them whenever I had to sort out what a given test
was up to.  But it's not my test suite, so I will bow out here.
--paulr

> 
> > --paulr
> >
> > >
> > > >
> > > > On Mon, Apr 29, 2019 at 7:28 AM Bruno Ricci via cfe-dev
> > > > <cfe-dev at lists.llvm.org> wrote:
> > > >>
> > > >> Hi,
> > > >>
> > > >> An annoying part of writing tests for C++ is that the paragraph
> numbers
> > > are not
> > > >> stable between versions of the standard. Is there an accepted way
> to
> > > deal with
> > > >> this issue ? I can imagine several ways to deal with this:
> > > >>   - Rename the tests so that they match the latest standard/draft.
> > > >>   - Add a suffix/prefix with the version for the paragraph number.
> > > >>   - ... ?
> > > >>
> > > >> Bruno
> > > >> _______________________________________________
> > > >> cfe-dev mailing list
> > > >> cfe-dev at lists.llvm.org
> > > >> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> > > _______________________________________________
> > > cfe-dev mailing list
> > > cfe-dev at lists.llvm.org
> > > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> > _______________________________________________
> > cfe-dev mailing list
> > cfe-dev at lists.llvm.org
> > https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev