<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Feb 8, 2016 at 4:00 PM, Xinliang David Li <span dir="ltr"><<a href="mailto:davidxl@google.com" target="_blank">davidxl@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Yes, if we talk about all tests in different repos collectively, 0 is<br>
probably covered -- but that is still an assumption from the<br>
unittest's point of view.<br></blockquote><div><br></div><div>Ah, sure - I agree it (along with any other API surface area) should be covered by a unit test, even if the codepath is also covered by a higher level test of some kind.<br><br>But it looks like the 0 padding case is covered by other /unit/ tests in the same file. This seems like a reasonable case of test case reduction by combination (if you have {a, b} x {x, y} to test you can test it in two cases {a, x}, {b, y} rather than running 4 tests, since the features are independent).<br></div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="HOEnZb"><font color="#888888"><br>
David<br>
</font></span><div class="HOEnZb"><div class="h5"><br>
On Mon, Feb 8, 2016 at 3:55 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>> wrote:<br>
><br>
><br>
> On Mon, Feb 8, 2016 at 3:53 PM, Xinliang David Li <<a href="mailto:davidxl@google.com">davidxl@google.com</a>><br>
> wrote:<br>
>><br>
>> On Mon, Feb 8, 2016 at 3:30 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>> wrote:<br>
>> ><br>
>> ><br>
>> > On Mon, Feb 8, 2016 at 3:17 PM, Xinliang David Li <<a href="mailto:davidxl@google.com">davidxl@google.com</a>><br>
>> > wrote:<br>
>> >><br>
>> >> On Mon, Feb 8, 2016 at 3:12 PM, David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>><br>
>> >> wrote:<br>
>> >> ><br>
>> >> ><br>
>> >> > On Mon, Feb 8, 2016 at 3:08 PM, Xinliang David Li<br>
>> >> > <<a href="mailto:davidxl@google.com">davidxl@google.com</a>><br>
>> >> > wrote:<br>
>> >> >><br>
>> >> >> On Mon, Feb 1, 2016 at 10:44 AM, David Blaikie <<a href="mailto:dblaikie@gmail.com">dblaikie@gmail.com</a>><br>
>> >> >> wrote:<br>
>> >> >> ><br>
>> >> >> ><br>
>> >> >> > On Fri, Jan 29, 2016 at 5:58 PM, Xinliang David Li<br>
>> >> >> > <<a href="mailto:davidxl@google.com">davidxl@google.com</a>><br>
>> >> >> > wrote:<br>
>> >> >> >><br>
>> >> >> >> To clarify, it is not 128 iterations, but creating a symbol table<br>
>> >> >> >> with<br>
>> >> >> >> 128 entries -- which is a reasonable size.<br>
>> >> >> ><br>
>> >> >> ><br>
>> >> >> > We don't generally test on "realistic" sized inputs in the<br>
>> >> >> > regression<br>
>> >> >> > suite.<br>
>> >> >> > We write targeted tests for functionality. Broad testing is done<br>
>> >> >> > in<br>
>> >> >> > the<br>
>> >> >> > test-suite and other integration level testing.<br>
>> >> >> ><br>
>> >> >> >><br>
>> >> >> >> Test coverage wise, it is probably the same as a 3-entry symtab.<br>
>> >> >> ><br>
>> >> >> ><br>
>> >> >> > Then let's use a 3-entry symtab.<br>
>> >> >> ><br>
>> >> >> > (why 3? Because it tests the boundaries (first and last) and one<br>
>> >> >> > "normal"<br>
>> >> >> > case of a non-boundary value - while the boundaries probably<br>
>> >> >> > aren't<br>
>> >> >> > interesting in this algorithm, it's cheap enough to just follow<br>
>> >> >> > that<br>
>> >> >> > common<br>
>> >> >> > practice in test case design)<br>
>> >> >><br>
>> >> >> Will update it to 3.<br>
>> >> >><br>
>> >> >> ><br>
>> >> >> > I'm also curious about the padding parameter - what does it do?<br>
>> >> >> > Choose<br>
>> >> >> > how<br>
>> >> >> > many null characters go between each value? What effect does that<br>
>> >> >> > have/why<br>
>> >> >> > is that a tuning parameter? (understanding what it's for can help<br>
>> >> >> > us<br>
>> >> >> > choose<br>
>> >> >> > appropriate test cases/coverage for that functionality)<br>
>> >> >><br>
>> >> >> Internal padding bytes (for alignment to 4 bytes) can be zero to 3.<br>
>> >> ><br>
>> >> ><br>
>> >> > Any idea what's particularly useful to test here? (does it just<br>
>> >> > assert<br>
>> >> > that<br>
>> >> > the parameter is [0,3] ? Or does it have well defined behavior<br>
>> >> > (returning an<br>
>> >> > error code? doing something else?) outside that range? is any case<br>
>> >> > more<br>
>> >> > interesting than any other - or just a simple loop for [0,Padding]<br>
>> >> > done<br>
>> >> > at<br>
>> >> > some point in the algorithm? Does anything test that the algorithm<br>
>> >> > emitted<br>
>> >> > the right padding?)<br>
>> >><br>
>> >> It tests that the reader is (flexible) and capable of handing padding<br>
>> >> bytes not produced by the writer. How many paddings should be emitted<br>
>> >> is not specified. For instance, if some producer forces 8 byte<br>
>> >> alignment, it should be handled too.<br>
>> ><br>
>> ><br>
>> > Ah, OK - perhaps we could just test one pseudo-random (if it's really<br>
>> > just a<br>
>> > "while (null byte)" loop to ignore the padding - I'd probably pick 2<br>
>> > bytes<br>
>> > of padding, but don't mind any small number) amount of padding to test<br>
>> > that<br>
>> > the reader ignores it, rather than testing several amounts of padding?<br>
>> > Alternatively/in addition, might be good to test these features<br>
>> > separately<br>
>> > to make triage easier? Rather than combining compression and padding<br>
>> > together - unless there's an interesting interaction between the two<br>
>> > features in the implementation?<br>
>> ><br>
>><br>
>> I think 0 is more special here, so I would pick 0 and 1 byte.<br>
><br>
><br>
> Is zero bytes of padding not already covered by any other tests? (I assume<br>
> it's covered by most tests as it sounds like it's the common case?)<br>
><br>
>><br>
>> > You say "padding bytes not produced by the writer" - does the writer<br>
>> > produce<br>
>> > zero bytes of padding, or some amount of padding that's just not the<br>
>> > same<br>
>> > amounts as are being tested here?<br>
>><br>
>> The writer can produce 0 or more padding bytes, the assembler and<br>
>> linker may or may not pad more. The purpose of the testing is that the<br>
>> reader does not depend/care about those behavior.<br>
>><br>
>> David<br>
>><br>
>> ><br>
>> > - David<br>
><br>
><br>
</div></div></blockquote></div><br></div></div>