[lldb-dev] LLDB Evolution - Final Form

[Sean Callanan via lldb-dev]

I'll only comment on the stuff that affects me.

> Use llvm streams instead of lldb::StreamString
> Supports output re-targeting (stderr, stdout, std::string, etc), printf style formatting, and type-safe streaming operators.
> Interoperates nicely with many existing llvm utility classes
> Risk: 4
> Impact: 5
> Difficulty / Effort: 7

I don't like that llvm's stringstream needs to be babied to make it produce its string.  You have to wrap it and then flush it and then read the string.  Maybe a subclass could be made that wraps its own string and flushes automatically on read?

> Use llvm::Error instead of lldb::Error
> llvm::Error is an error class that *requires* you to check whether it succeeded or it will assert.  In a way, it's similar to a C++ exception, except that it doesn't come with the performance hit associated with exceptions.  It's extensible, and can be easily extended to support the various ways LLDB needs to construct errors and error messages.
> Would need to first rename lldb::Error to LLDBError so that te conversion from LLDBError to llvm::Error could be done incrementally.
> Risk: 7
> Impact: 7
> Difficulty / Effort: 8

This would help me a lot if the underlying Clang APIs implemented it.  Unfortunately the expression parser mainly just receives diagnostics and has to deal with them.
From the perspective of internal use, I could live with llvm::Error, particularly if it gets us 3f.

> ArrayRef instead of const void *, len everywhere
> Same analysis as StringRef
> MutableArrayRef instead of void *, len everywhere
> Same analysis as StringRef

At one point I made a templated mixin class that would make everything that vended GetNum*/Get*AtIndex functions also vend an iterator.  Hopefully we can get that implemented everywhere.

> Testing - Our testing infrastructure is unstable, and our test coverage is lacking.  We should take steps to improve this.
> Port as much as possible to lit
> Simple tests should be trivial to port to lit today.  If nothing else this serves as a proof of concept while increasing the speed and stability of the test suite, since lit is a more stable harness.
> Separate testing tools
> One question that remains open is how to represent the complicated needs of a debugger in lit tests.  Part a) above covers the trivial cases, but what about the difficult cases?  In https://reviews.llvm.org/D24591 <https://reviews.llvm.org/D24591> a number of ideas were discussed.  We started getting to this idea towards the end, about a separate tool which has an interface independent of the command line interface and which can be used to test.  lldb-mi was mentioned.  While I have serious concerns about lldb-mi due to its poorly written and tested codebase, I do agree in principle with the methodology.  In fact, this is the entire philosophy behind lit as used with LLVM, clang, lld, etc.  
> 
> I don’t take full credit for this idea.  I had been toying with a similar idea for some time, but it was further cemented in an offline discussion with a co-worker.  
> 
> There many small, targeted tools in LLVM (e.g. llc, lli, llvm-objdump, etc) whose purpose are to be chained together to do interesting things.  Instead of a command line api as we think of in LLDB where you type commands from an interactive prompt, they have a command line api as you would expect from any tool which is launched from a shell.
> 
> I can imagine many potential candidates for lldb tools of this nature.  Off the top of my head:
> lldb-unwind - A tool for testing the unwinder.  Accepts byte code as input and passes it through to the unwinder, outputting a compressed summary of the steps taken while unwinding, which could be pattern matched in lit.  The output format is entirely controlled by the tool, and not by the unwinder itself, so it would be stable in the face of changes to the underlying unwinder.  Could have various options to enable or disable features of the unwinder in order to force the unwinder into modes that can be tricky to encounter in the wild.
> lldb-symbol - A tool for testing symbol resolution.  Could have options for testing things like:
> Determining if a symbol matches an executable
> looking up a symbol by name in the debug info, and mapping it to an address in the process.  
> Displaying candidate symbols when doing name lookup in a particular scope (e.g. while stopped at a breakpoint).
> lldb-breakpoint - A tool for testing breakpoints and stepping.  Various options could include:
> Set breakpoints and out addresses and/or symbol names where they were resolved to.
> Trigger commands, so that when a breakpoint is hit the tool could automatically continue and try to run to another breakpoint, etc.
> options to inspect certain useful pieces of state about an inferior, to be matched in lit. 
> lldb-interpreter - tests the jitter etc.  I don’t know much about this, but I don’t see why this couldn’t be tested in a manner similar to how lli is tested.
> lldb-platform - tests lldb local and remote platform interfaces.
> lldb-cli -- lldb interactive command line.
> lldb-format - lldb data formatters etc.

I'm thinking of re-implementing the "inline" tester as a C++ tool that could be driven by lit. 
As far as anything touching the expression parser, I'd prefer just implementing solid testers inside llvm/clang, rather than something on top of LLDB.

> Tests NOW, not later.
> I know we’ve been over this a million times and it’s not worth going over the arguments again.  And I know it’s hard to write tests, often requiring the invention of new SB APIs.  Hopefully those issues will be addressed by above a) and b) above and writing tests will be easier.  Vedant Kumar ran some analytics on the various codebases and found that LLDB has the lowest test / commit ratio of any LLVM project (He didn’t post numbers for lld, so I’m not sure what it is there).
> lldb: 287 of the past 1000 commits
> llvm: 511 of the past 1000 commits
> clang: 622 of the past 1000 commits
> compiler-rt: 543 of the past 1000 commits
> This is an alarming statistic, and I would love to see this number closer to 50%.

I'm definitely a culprit here.  Many fixes I make that don't have tests are for problems we can't reproduce.  In general, LLDB suffers from a lack of "failure tests" which check what happens when something doesn't work.
IMO a very useful avenue for investigation would be to add various selectable "failure generators" into LLDB that simulate the failure of particular operations or subsystems.  Then we could use these as proxies for failures we can't reproduce as integration tests.

> Code style / development conventions - Aside from just the column limitations and bracing styles, there are other areas where LLDB differs from LLVM on code style.  We should continue to adopt more of LLVM's style where it makes sense.  I've identified a couple of areas (incomplete list) which I outline below.  
> Clean up the mess of cyclical dependencies and properly layer the libraries.  This is especially important for things like lldb-server that need to link in as little as possible, but regardless it leads to a more robust architecture, faster build and link times, better testability, and is required if we ever want to do a modules build of LLDB
> Use CMake instead of Xcode project (CMake supports Frameworks).  CMake supports Apple Frameworks, so the main roadblock to getting this working is just someone doing it.  Segmenting the build process by platform doesn't make sense for the upstream, especially when there is a perfectly workable solution.  I have no doubt that the resulting Xcode workspace generated automatically by CMake will not be as "nice" as one that is maintained by hand.  We face this problem with Visual Studio on Windows as well.  The solution that most people have adopted is to continue using the IDE for code editing and debugging, but for actually running the build, use CMake with Ninja.  A similar workflow should still be possible with an OSX CMake build, but as I do not work every day on a Mac, all I can say is that it's possible, I have no idea how impactful it would be on peoples' workflows.

My dev workflow these days is running xcodebuild from vim.  I would love to be running ninja.

> Convert T foo(X, Y, Error &error) functions to Expected<T> foo(X, Y) style (Depends on 1.c)
> llvm::Expected is based on the llvm::Error class described earlier.  It’s used when a function is supposed to return a value, but it could fail.  By packaging the error with the return value, it’s impossible to have a situation where you use the return value even in case of an error, and because llvm::Error has mandatory checking, it’s also impossible to have a sitaution where you don’t check the error.  So it’s very safe.  

Marshaling Error objects around is a regular aggravation and makes my code more messy.  I welcome with open arms anything that reduces that mess.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/lldb-dev/attachments/20160919/dc3d5728/attachment-0001.html>