[PATCH] D144730: [FlowSensitive] Log analysis progress for debugging purposes

Tue Mar 21 15:26:01 PDT 2023

sammccall added a comment.

In D144730#4211222 <https://reviews.llvm.org/D144730#4211222>, @NoQ wrote:

> I love such debugging facilities! They're a massive boost to developer productivity compared to ad-hoc debug prints, and they allow new contributors to study how the tool works. I'm excited to see how this thing turns out.

Thanks! It's great to know this is a problem worth some attention, as any tools here will carry some complexity.

> In the static analyzer's path-sensitive engine we've settled on a Graphviz printout of the analysis graph, with an extra python script to filter and "prettify" it. In our case these graphs are much larger than CFGs (on real-world code they often contain over 200000 nodes) so filtering and searching becomes essential. And we usually don't have an option to debug a reduced toy example, because it's typically very hard to define the notion of "false positive" in a way suitable for a `creduce` predicate, and even when reducing manually it's easy to miss the point entirely and end up with a completely different problem. So we had to learn how to deal with these massive graphs, and these days we're pretty good at that. In your case the graphs probably aren't that massive, but other considerations probably still apply.

Yeah, as I understand the dataflow analysis proper is scoped to a function, so dealing with larger CFGs but not on that scale. I'm hoping that graphviz renderings of the CFG + overlaying it on the code are enough. Text dumps are indeed too difficult to read until you're really zeroed in on the bit you want to look at.

> If you're stuck with real-world code as much as we are in the static analyzer, it might be very convenient to somehow include a graphical representation of the CFG alongside your reports. Given that CFGs are relatively small, you might be able to get away with an ad-hoc graph visualizer scripted into your HTML page, so that to avoid relying on Graphviz.

Here's a prototype I put together: https://htmlpreview.github.io/?https://gist.githubusercontent.com/sam-mccall/c03943495d2ca493be8f3087c22e3b70/raw/986a1d7d9e1ff37e46fc9b6808f0650ae28cc8d3/analyze_optional.html

I tried the ad-hoc method like you suggest to avoid the dependency, but working out how to place the nodes & arcs reasonably quickly looked too much for me. Ended up going back to shelling out to graphviz but embedding the result in the page.

> Then I guess you can try adding your abstract state information directly to the graphical CFG blocks, and build a timeline of such partially analyzed CFGs, from the starting state to the fully converged state, brightly highlighting changes made on every step 🤔Dunno, just dreaming at this point.

Yes! This is very much where I want to go. I'll have an initial version up soon with basic functionality (timeline, CFG, simple textual dump at each point) but showing the most relevant state and changes at each point would be really useful.

Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D144730/new/

https://reviews.llvm.org/D144730