[Lldb-commits] [PATCH] D40616: ObjectFileELF: Add support for compressed sections

[Jim Ingham via lldb-commits]

I'm a little confused by your response.  

My stated objection to command output dependent tests is and has always been that they make the test dependent on the details of command output.  Over time doing so makes it hard to modify command output.  This is sort of related to interactivity, in the sense that since lldb is an interactive tool with lots of different commands producing different reports of information we can gather, the desire to improve and modify that output is more present than in tools that are less output dependent or whose output is meant to be processed, in which case it really is API.  Command output for lldb is explicitly NOT API, that's why we have a real API for people who want to program lldb...  So the bad effect of the tests in calcifying this output is an issue for lldb where it may not be for other tools.  

But interactivity per se - while it may be a bar to using  the FileCheck method - has never been my objection to it.

Jim


> On Nov 29, 2017, at 12:18 PM, Zachary Turner <zturner at google.com> wrote:
> 
> We’ve had this discussion several times, but at the end of the day nothing has really changed with the larger llvm project having adopted this model and having spent significant effort in moving forward with it.
> 
> Normally, the reason we don't use this model for LLDB tests is because they require interactivity, and if need be I can find several threads where the response has been, specifically "FileCheck tests aren't a good fit for LLDB because of the interactivity and the nature of the test doesn't lend itself well to textual output comparison".
> 
> So here we have a case where there is no interactivity, and in fact the test lends itself perfect to textual comparison.  With all due respect, given that the previously stated reasons for preferring not to use text-scraping tests don't apply in this case, combined with the fact that there is a strong desire by the larger project to use a standard, cross-project testing infrastructure that is understood by several hundred developers instead of several, I don't see a good reason to not at least attempt this route in cases where it makes sense.
> 
> 
> 
> On Wed, Nov 29, 2017 at 12:07 PM Jim Ingham via Phabricator <reviews at reviews.llvm.org> wrote:
> jingham added a comment.
> 
> It does look a little weird as a unit test, but to me this is mostly because it would been much simpler to write it as a regular SB API test.
> 
> Anyway, I really don't want the details of the text output of lldb commands to become API.  Our experience with gdb was that over time as you write more and more tests that scrape text output, you end up not being able to change command output because the burden of fixing up all the tests becomes too onerous.  You can use text scraping in the current lldb testsuite.  We discourage that for the reasons above, and try to isolate the tests that do so by having lldbutils interfaces to do the explicit scraping.  But it is just as easy, and quite often much easier, to examine objects directly in the lldb testsuite, so this mechanism encourages virtue, even though it doesn't enforce it.
> 
> OTOH adding a test mechanism that explicitly relies only on command output scraping leads us down the path that ended up being a real PITN for the gdb testsuite.  So for that reason I am not in favor of going this way.
> 
> 
> https://reviews.llvm.org/D40616
> 
> 
>