[cfe-dev] Paragraph numbers in C++ tests

[Bruno Ricci via cfe-dev]

On 30/04/2019 17:53, David Blaikie wrote:
> On Tue, Apr 30, 2019 at 8:12 AM via cfe-dev <cfe-dev at lists.llvm.org> wrote:
>>
>>
>>
>>> -----Original Message-----
>>> From: cfe-dev [mailto:cfe-dev-bounces at lists.llvm.org] On Behalf Of Bruno
>>> Ricci via cfe-dev
>>> Sent: Tuesday, April 30, 2019 10:34 AM
>>> To: cfe-dev at lists.llvm.org
>>> Subject: Re: [cfe-dev] Paragraph numbers in C++ tests
>>>
>>>
>>>
>>> On 29/04/2019 19:33, David Blaikie wrote:
>>>> Even if they're updated to reflect current paragraph numbers, they'd
>>>> still fall into disrepair in future standards & be hard to know which
>>>> version they applied to. So I'd support adding C++ version numbers to
>>>> them. At least on principle - but the reality/situations they're used
>>>> in might make that awkward/problematic, I don't know.
>>>
>>> I was thinking about the naming of tests for sections of the C++ standard.
>>> For example the tests for [namespace.udecl] are:
>>>   p1.cpp
>>>   p3.cpp
>>>   p4.cpp
>>>   p5-cxx0x.cpp
>>>   p6-cxx11.cpp
>>>   p7.cpp
>>>   p8.cpp
>>>   p8-cxx0x.cpp
>>>   p10.cpp
>>>   p11.cpp
>>>   p12.cpp
>>>   p13.cpp
>>>   p15.cpp
>>>   p18.cpp
>>>
>>> I think that naming the tests pn-cxxmm.cpp to mean, "paragraph pn of
>>> cxxmm"
>>> is reasonable, with the understanding that naming a test p3-cxx17.cpp does
>>> not
>>> mean that the corresponding rule is specific to C++17, just that it was
>>> written
>>> against C++17.
>>>
>>> Bruno
>>
>> The real issue is, "p3" in C++11 might be renumbered "p4" in C++14
>> and "p5" in C++17 and ...  so what does "p3.cpp" actually test?
>> You can't tell just from the name of the file because the name's
>> meaning is context-sensitive.
>>
>> Intuitively I'd think "p3-cxx11.cpp" means "tests p3 as it existed in
>> the C++11 standard" and whether that paragraph is still p3 in later
>> editions is an exercise for the reader.
> 
> I think that^ is what Bruno was saying.

Yes

> 
>> I recommend, give up on naming tests after paragraph numbers, because
>> they aren't actually meaningful.
> 
> Eh, I think there's merit to having tests by feature in the standard
> and being able to inspect their coverage (yeah, that gets wonky when
> paragraphs are renumbered - so you might have to update the numbering
> - but the benefit of a correspondence between some part/version of the
> standard even if you have to do some work remapping to a consistent
> version.
> 

I think I agree with the coverage argument. In any case the person writting
a new test has to consider the previous versions of the standard to see if
the rule was changed.

Bruno

>> --paulr
>>
>>>
>>>>
>>>> On Mon, Apr 29, 2019 at 7:28 AM Bruno Ricci via cfe-dev
>>>> <cfe-dev at lists.llvm.org> wrote:
>>>>>
>>>>> Hi,
>>>>>
>>>>> An annoying part of writing tests for C++ is that the paragraph numbers
>>> are not
>>>>> stable between versions of the standard. Is there an accepted way to
>>> deal with
>>>>> this issue ? I can imagine several ways to deal with this:
>>>>>   - Rename the tests so that they match the latest standard/draft.
>>>>>   - Add a suffix/prefix with the version for the paragraph number.
>>>>>   - ... ?
>>>>>
>>>>> Bruno
>>>>> _______________________________________________
>>>>> cfe-dev mailing list
>>>>> cfe-dev at lists.llvm.org
>>>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>>> _______________________________________________
>>> cfe-dev mailing list
>>> cfe-dev at lists.llvm.org
>>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev
>> _______________________________________________
>> cfe-dev mailing list
>> cfe-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-dev