[llvm] [MachinePipeliner] Fix incorrect handlings of unpipelineable insts (PR #126057)

[Ryotaro Kasuga via llvm-commits]

kasuga-fj wrote:

Below is a portion of dependencies of this case.

```
SU(0):   %27:gpr32 = PHI %21:gpr32all, %bb.3, %28:gpr32all, %bb.4
  # preds left       : 0
  # succs left       : 3
  # rdefs left       : 0
  Latency            : 0
  Depth              : 0
  Height             : 1
  Successors:
    SU(16): Data Latency=0 Reg=%27
    SU(14): Data Latency=0 Reg=%27
    SU(15): Anti Latency=1
SU(1):   %29:ppr_3b = PHI %16:ppr, %bb.3, %30:ppr, %bb.4
  # preds left       : 0
  # succs left       : 5
  # rdefs left       : 0
  Latency            : 0
  Depth              : 0
  Height             : 9
  Successors:
    SU(13): Data Latency=0 Reg=%29
    SU(11): Data Latency=0 Reg=%29
    SU(10): Data Latency=0 Reg=%29
    SU(6): Data Latency=0 Reg=%29
    SU(16): Anti Latency=1

...

SU(14):   %41:gpr32 = ADDWrr %27:gpr32, %12:gpr32common
  # preds left       : 2
  # succs left       : 1
  # rdefs left       : 0
  Latency            : 1
  Depth              : 1
  Height             : 1
  Predecessors:
    SU(0): Data Latency=0 Reg=%27
    SU(16): Ord  Latency=0 Artificial
  Successors:
    SU(15): Data Latency=1 Reg=%41
SU(15):   %28:gpr32all = COPY %41:gpr32
  # preds left       : 2
  # succs left       : 0
  # rdefs left       : 0
  Latency            : 1
  Depth              : 2
  Height             : 0
  Predecessors:
    SU(14): Data Latency=1 Reg=%41
    SU(0): Anti Latency=1
SU(16):   %30:ppr = WHILELO_PWW_S %27:gpr32, %15:gpr32, implicit-def $nzcv
  # preds left       : 2
  # succs left       : 2
  # rdefs left       : 0
  Latency            : 3
  Depth              : 1
  Height             : 1
  Predecessors:
    SU(0): Data Latency=0 Reg=%27
    SU(1): Anti Latency=1
  Successors:
    ExitSU: Data Latency=1 Reg=$nzcv
    SU(14): Ord  Latency=0 Artificial
```

As you can see 

- There is an artificial dependency from `SU(16)` to `SU(14)`, which is created by `CopyToPhiMutation`.
- `SU(14)` and `SU(15)` will be unpipelineable because of the dependency chain `SU(16) -> SU(0) -> SU(15) -> SU(14)`.

With respect to `SU(14)`, `SU(15)`, and `SU(16)`

- The cycles are 19, 28, and 19 respectively, before calling `normalizeNonPipelinedInstructions`.
- In the `normalizeNonPipelinedInstructions`, they are iterated in the order of the instructions, so the new cycles are determined in the order `SU(14)`, `SU(15)`, `SU(16)`.

The new cycle of the `SU(14)` cannot be less than 19, because it is the maximum of all cycles of its predecessors, and `SU(16)` is scheduled at cycle 19 at the moment. But it must be moved to stage 0 (in this case cycle from 0 to 9), which means it is not correctly normalized. The same thing happens with `SU(15)`.

In this case, traversing them in topological order when normalizing may solve the problem, but I'm not sure if it is right for all cases.

https://github.com/llvm/llvm-project/pull/126057