[Lldb-commits] [PATCH] D43686: Add "lldb-test breakpoint" command and convert the case-sensitivity test to use it

[Jim Ingham via lldb-commits]

Sure, as long as we aren't turning the output of the "break set" or other commands into test API - which your version of this lit test does not - then this sort of test seems fine to me.

When we were starting out and building up the set of tests available I at least tended to err on the side of testing the whole lifecycle of the subsystem I was testing.  That way you could get a lot of coverage per test, which we really needed.  So there probably are more tests that run end-to-end than absolutely necessary.  We still should make sure, particularly when we are adding new functionality, that there are enough end-to-end tests before we start testing the peripheries of the feature.  But for things like breakpoint setting, we are well past that point, and writing lots of setting only tests is appropriate, as you say.

BTW, one thing I like about writing dotest.py tests is that it is easy to craft fairly rich failure messages so if you get errors on systems you don't have access to or are dealing with something that fails intermittently on a bot somewhere, you have a hope of figuring out what went wrong.  Is this possible with FileCheck tests?

Jim



> On Feb 23, 2018, at 1:52 PM, Pavel Labath <labath at google.com> wrote:
> 
> On 23 February 2018 at 11:24, Jim Ingham <jingham at apple.com> wrote:
>> To be fair, you could probably have made the dotest.py version of the test close to as fast by not running a process.  The old test was testing that we got a location for the breakpoint AND hit it.  The latter was probably overkill.  But the two tests aren't testing the same thing.
>> 
> 
> Yes, they are not testing the same thing. However, my thought was (and
> it sounds like you agree) that for checking case-sensitivity checking
> we are able to resolve the breakpoint to a file address is enough. We
> have plenty of tests that verify the path from a file address to an
> actual breakpoint hit.
> 
> The reason I went this route (instead of just modifying the original
> test) is that this will make it easy to write more low level
> breakpoint tests. For example, you can write an assembly file with
> amusing .loc directives and check that lldb resolves that reasonably
> (this looks like it could be useful for the ppc patch). Adding llvm-mc
> support to dotest.py would be possible, but more difficult than with a
> test like this.