[Lldb-commits] [PATCH] D40616: ObjectFileELF: Add support for compressed sections

[Jim Ingham via lldb-commits]

I'm mostly basing this concern on the bad effect this had on gdb all of whose testing was expect based command scraping.  gdb is a tool that's much closer to lldb than any of the compiler tools so that experience seems relevant.  It's been a decade or so since I worked on gdb, but back when I was working on it, you really had to tread very carefully if you wanted to change the output of, say, the break command, or to thread listings, etc, and a bunch of times I just had to bag some cleanup of output I wanted to do because fixing up all the tests was too time consuming.  Because Jason and I had both had this experience when we started working on lldb, we promised ourselves we wouldn't go down this path again...

Jim


> On Nov 29, 2017, at 1:43 PM, Davide Italiano <dccitaliano at gmail.com> wrote:
> 
> On Wed, Nov 29, 2017 at 1:38 PM, Jim Ingham <jingham at apple.com> wrote:
>> I'm a little confused by your response.
>> 
>> My stated objection to command output dependent tests is and has always been that they make the test dependent on the details of command output.  Over time doing so makes it hard to modify command output.  This is sort of related to interactivity, in the sense that since lldb is an interactive tool with lots of different commands producing different reports of information we can gather, the desire to improve and modify that output is more present than in tools that are less output dependent or whose output is meant to be processed, in which case it really is API.  Command output for lldb is explicitly NOT API, that's why we have a real API for people who want to program lldb...  So the bad effect of the tests in calcifying this output is an issue for lldb where it may not be for other tools.
>> 
> 
> Hi Jim,
> in my experience command output changes can be automated via `sed/grep/awk`.
> I'm responsible (and many others are) for fundamental changes in LLVM
> tools output (i.e. typeless pointers changed pretty much every
> load/store/memory_op* in tree) and I found out changing the output of
> tests isn't that troublesome. I'm not yet very familiar with LLDB, so
> the story might be different here.
> I'm personally in favour of this approach because it scaled very well
> in llvm (and lld, FWIW), with many more tests than lldb has.
> 
> --
> Davide