[llvm-dev] CFG manipulation and !llvm.loop metadata

[Florian Hahn via llvm-dev]

> On Mar 20, 2020, at 17:37, Colin McEwan via llvm-dev <llvm-dev at lists.llvm.org> wrote:
> 
> Hi all,
> 
> I have encountered some issues with the preservation of the location of llvm.loop metadata (containing optimisation hints), and would appreciate some feedback on the issue.
> 
> The IR language description states that llvm.loop metadata is attached to terminator of a loop latch block, and accordingly Loop::getLoopID() searches for it in all loop latches (and only successfully finds it if all latches reference the same metadata)
> 
> However, transforms which modify the CFG, for example using SplitCriticalEdge(), generally don't make any attempt to preserve this property. Some transforms dealing specifically with loops use getLoopID and setLoopID to preserve and reset the metadata after transformations, but function transforms such as GVN and Jump Threading can modify control flow without any attempt to update the location.
> 
> For example:
> 
>        preheader:
>                ...
>        loop.body: ; preds = %preheader, %loop.body
>                ...
>                br i1 %cmp, label %loop.body, %loop.exit, !llvm.loop !123
>        loop.exit:
> 
> If a pass needs to split the critical edge from %loop.body -> %loop.body, it will create something like
> 
>        preheader:
>                ...
>        loop.body: ; preds = %preheader, %loop.body.crit_edge
>                ...
>                br i1 %cmp, label %loop.body.crit_edge, %loop.exit, !llvm.loop !123   // No longer a loop latch block
>        loop.body.crit_edge:
>                ... 
>                br %loop.body                                                         // Now a loop latch, with no !llvm.loop
>        loop.exit:
> 
> 
> Now the loop's latch block is %loop.body.crit_edge, and the !llvm.loop will not be found by Loop::getLoopID(), meaning it is effectively lost. This can happen in many different places.
> 
> 
> I can think of a few approaches to address this, but each has its issues:
> 
>    1. Modify the framework's CFG manipulation tools to maintain the location of the !llvm.loop. A major drawback of this is that LoopInfo is needed to be able to tell whether a block is a loop latch or not (in the example of splitting a latch block's back edge, we need LoopInfo to know whether the edge we're splitting is the latch edge or exit edge) and it's not necessary available or up to date when we manipulate the CFG.
> 
>    2. Have Loop::getLoopID() search other blocks in the loop for metadata. This has potential compile-time implications, and would change the IR language definition of the !llvm.loop as potentially existing (in a valid form) anywhere in the loop.
> 
>    3. Fixup utility functions for function passes to use, to search a loop and move any errant !llvm.loop to the latch block(s) of its loop.
> 
> 
> Additionally, it should probably be explicitly stated in the IR language reference that !llvm.loop preservation is best-effort and may be lost.

The LangRef spells out that transformations are required to drop all metadata they do not know how to preserve (https://llvm.org/docs/LangRef.html#metadata-nodes-and-metadata-strings). As you mentioned, some utilities know how to preserve certain kinds of metadata, but various places conservatively drop metadata. I expect we need to fix a lot of places.

A related issue is preserving debug metadata and a lot of work has gone into that area already.  A good first step might be a verifier that checks if a pass drops !llvm.loop. That could look something like:

1. Create a Loopify pass that adds !llvm.loop for each loop in a function (similar to Debugify https://github.com/llvm/llvm-project/blob/f64e457cb75b61f6566de8327a1bfae498d5a296/llvm/lib/Transforms/Utils/Debugify.cpp)
2. Create a verifier that checks all loops have !llvm.loop metadata
3. Run -loopify before each transformation and the verifier afterwards
4. Fix issues.

Passes that create new loops might need special handling (or an option to also automatically attach llvm.loop metadata to the new loops).

Cheers,
Florian