[PATCH] D50441: [XRay] FDR Trace Dump Tool (FDR Trace Loading Refactor)

[Dean Michael Berris via Phabricator via llvm-commits]

dberris added a comment.

In https://reviews.llvm.org/D50441#1194024, @kpw wrote:

> Thought that occurred to me this morning:
>
> This models a sequence of records and not a more complex structure like a tree. Instead of greedily filling a vector and then calling visitors, this could be better done with a streaming model.


Unfortunately, it's not that simple.

Blocks that belong to the same threads/processes have a well-defined order and model a sequence of operations bound to time. This is the reason the indexer is a key visitor implementation, and why we can implement a validator that applies to logical blocks. In the process of re-constituting a single execution trace, we need to model the execution context as individual streams of execution that interlace over time in multiple dimensions (cpu, process, thread).

The complication here is that at runtime, since the implementation can be re-using buffers in a circular buffer, we cannot assume that the blocks which appear at the beginning always come before the blocks that follow in the time dimension. The spatial ordering doesn't imply temporal ordering, so we need to somehow re-construct the temporal order (hence the indexer).

What this patch doesn't show is how we're really supposed to sort the blocks in the indexes first, in temporal order (using the block's wallclock record), before reconstituting a full `Trace` object (which is the ultimate goal anyway) which requires the logical verification first. That's a separate algorithm which we don't implement yet.

There's a bit of foreshadowing going on, which I really should have explained somewhere in the first place. :)

> The "driver" code would maintain a sequence of visitors and call them on each record that it reads.
> 
> It should be a small refactoring and could dramatically improve memory use especially for stateless visitors like the printer.

For the purposes of printing a log, I agree.

That seems like a simple algorithm which applies visitors for every record we encounter as we encounter it, without validating, i.e. a true "dump" tool. The missing abstraction seems to be a pipelining mechanism, or a sequencing visitor that determines the order of the visitors to apply.

That doesn't obviate the need for a logical in-memory structure for the reconstitution algorithm though for other uses. I suppose those can come later when we end up actually moving the current `convert` implementation to using this infrastructure for handling FDR mode logs.


https://reviews.llvm.org/D50441