[cfe-dev] Paragraph numbers in C++ tests

Aaron Ballman via cfe-dev cfe-dev at lists.llvm.org
Wed May 1 12:39:48 PDT 2019


On Wed, May 1, 2019 at 11:58 AM Bruno Ricci via cfe-dev
<cfe-dev at lists.llvm.org> wrote:
>
>
>
> On 30/04/2019 17:53, David Blaikie wrote:
> > On Tue, Apr 30, 2019 at 8:12 AM via cfe-dev <cfe-dev at lists.llvm.org> wrote:
> >>
> >>
> >>
> >>> -----Original Message-----
> >>> From: cfe-dev [mailto:cfe-dev-bounces at lists.llvm.org] On Behalf Of Bruno
> >>> Ricci via cfe-dev
> >>> Sent: Tuesday, April 30, 2019 10:34 AM
> >>> To: cfe-dev at lists.llvm.org
> >>> Subject: Re: [cfe-dev] Paragraph numbers in C++ tests
> >>>
> >>>
> >>>
> >>> On 29/04/2019 19:33, David Blaikie wrote:
> >>>> Even if they're updated to reflect current paragraph numbers, they'd
> >>>> still fall into disrepair in future standards & be hard to know which
> >>>> version they applied to. So I'd support adding C++ version numbers to
> >>>> them. At least on principle - but the reality/situations they're used
> >>>> in might make that awkward/problematic, I don't know.
> >>>
> >>> I was thinking about the naming of tests for sections of the C++ standard.
> >>> For example the tests for [namespace.udecl] are:
> >>>   p1.cpp
> >>>   p3.cpp
> >>>   p4.cpp
> >>>   p5-cxx0x.cpp
> >>>   p6-cxx11.cpp
> >>>   p7.cpp
> >>>   p8.cpp
> >>>   p8-cxx0x.cpp
> >>>   p10.cpp
> >>>   p11.cpp
> >>>   p12.cpp
> >>>   p13.cpp
> >>>   p15.cpp
> >>>   p18.cpp
> >>>
> >>> I think that naming the tests pn-cxxmm.cpp to mean, "paragraph pn of
> >>> cxxmm"
> >>> is reasonable, with the understanding that naming a test p3-cxx17.cpp does
> >>> not
> >>> mean that the corresponding rule is specific to C++17, just that it was
> >>> written
> >>> against C++17.
> >>>
> >>> Bruno
> >>
> >> The real issue is, "p3" in C++11 might be renumbered "p4" in C++14
> >> and "p5" in C++17 and ...  so what does "p3.cpp" actually test?
> >> You can't tell just from the name of the file because the name's
> >> meaning is context-sensitive.
> >>
> >> Intuitively I'd think "p3-cxx11.cpp" means "tests p3 as it existed in
> >> the C++11 standard" and whether that paragraph is still p3 in later
> >> editions is an exercise for the reader.
> >
> > I think that^ is what Bruno was saying.
>
> Yes
>
> >
> >> I recommend, give up on naming tests after paragraph numbers, because
> >> they aren't actually meaningful.
> >
> > Eh, I think there's merit to having tests by feature in the standard
> > and being able to inspect their coverage (yeah, that gets wonky when
> > paragraphs are renumbered - so you might have to update the numbering
> > - but the benefit of a correspondence between some part/version of the
> > standard even if you have to do some work remapping to a consistent
> > version.
> >
>
> I think I agree with the coverage argument. In any case the person writting
> a new test has to consider the previous versions of the standard to see if
> the rule was changed.

Perhaps a better approach is to use a stable name + a counter for the
test (stable_name_1.cpp, stable_name_2.cpp) and the file can have a
list of comments after the RUN lines:

// C++11 [foo.bar]p2
// C++14 [foo.bar]p3
// whatever

This also helps with coverage if we use a consistent convention across
files, I believe. It also means the file names don't have to be as
meaningful, and we can keep interesting historical information rather
than losing it when changing the name of the file.

~Aaron

>
> Bruno
>
> >> --paulr
> >>
> >>>
> >>>>
> >>>> On Mon, Apr 29, 2019 at 7:28 AM Bruno Ricci via cfe-dev
> >>>> <cfe-dev at lists.llvm.org> wrote:
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> An annoying part of writing tests for C++ is that the paragraph numbers
> >>> are not
> >>>>> stable between versions of the standard. Is there an accepted way to
> >>> deal with
> >>>>> this issue ? I can imagine several ways to deal with this:
> >>>>>   - Rename the tests so that they match the latest standard/draft.
> >>>>>   - Add a suffix/prefix with the version for the paragraph number.
> >>>>>   - ... ?
> >>>>>
> >>>>> Bruno
> >>>>> _______________________________________________
> >>>>> cfe-dev mailing list
> >>>>> cfe-dev at lists.llvm.org
> >>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >>> _______________________________________________
> >>> cfe-dev mailing list
> >>> cfe-dev at lists.llvm.org
> >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> >> _______________________________________________
> >> cfe-dev mailing list
> >> cfe-dev at lists.llvm.org
> >> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
> _______________________________________________
> cfe-dev mailing list
> cfe-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev



More information about the cfe-dev mailing list