[llvm-dev] DExTer: Debugging Experience Tester

[David Blaikie via llvm-dev]

Really sounds great & the bugs so far look pretty good.

One possible enhancement/request: It might be worth being able to
prioritize certain kinds of debug info quality problems:

* "incorrect stepping" (especially if it can be shown to step into code
that is /never/ executed) is a pretty high priority for any sample based
profiling
* It's possible that we could prioritize some optimized debug info quality
over others - like "can you print the parameters to a function on
function-entry" (especially without inlining - it'd be good to prioritize
all those "missing parameters on entry to a non-inlined function" perhaps)

- Dave

On Fri, Aug 31, 2018 at 4:26 PM Greg Bedwell via llvm-dev <
llvm-dev at lists.llvm.org> wrote:

> I'm happy to announce that DExTer (Debugging Experience Tester) is now
> available on github at https://github.com/SNSystems/dexter .  To quote
> from the project readme:
>
> "DExTer is a suite of tools used to evaluate the "User Debugging
> Experience". DExTer drives an external debugger, running on small test
> programs, and collects information on the behavior at each debugger step to
> provide quantitative values that indicate the quality of the debugging
> experience."
>
> My hope is that it will be a useful tool to help find both existing cases
> of poor debug experience as well as spot any regressions in that experience
> as they occur.  As I've previously stated, I think that this should be
> nicely complementary to other utlities such as debugify
> <http://llvm.org/docs/SourceLevelDebugging.html#testing-debug-info-preservation-in-optimizations> and
> DIVA <https://github.com/SNSystems/DIVA>. We also intend to contribute
> continuous DExTer testing of LLVM trunk in the future so we'll be able to
> immediately track any differences in the debugging experience score down to
> individual commits as they happen.
>
> For now we've been using it to find small cases where debugging of clang
> compiled programs at "-O2 -g" is significantly worse than with "-O0 -g" in
> ways that can hopefully be resolved through more careful preservation of
> debug data.  We're using the following meta-bug to keep track of the issues
> that we find (and hopefully fix):
> https://bugs.llvm.org/show_bug.cgi?id=38768
>
> The original slides and poster that I presented at EuroLLVM earlier this
> year can be found here:
>
> slides:
> http://llvm.org/devmtg/2018-04/slides/Bedwell-Measuring_the_User_Debugging_Experience.pdf
> poster:
> http://llvm.org/devmtg/2018-04/slides/Bedwell-Measuring_the_User_Debugging_Experience_poster.png
>
> Thanks to everyone who's helped out with the development of DExTer so far,
> especially to Stephen Wilks who worked closely with me on the initial
> development of the project.
>
> Thanks,
> -Greg
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20180904/87ebe7ea/attachment.html>