<table border="1" cellspacing="0" cellpadding="8">
<tr>
<th>Issue</th>
<td>
<a href=https://github.com/llvm/llvm-project/issues/75640>75640</a>
</td>
</tr>
<tr>
<th>Summary</th>
<td>
Fine grain resource allocation using resource intervals does not work as expected
</td>
</tr>
<tr>
<th>Labels</th>
<td>
new issue
</td>
</tr>
<tr>
<th>Assignees</th>
<td>
</td>
</tr>
<tr>
<th>Reporter</th>
<td>
michaelmaitland
</td>
</tr>
</table>
<pre>
We have set out to use fine grain resource allocation using resource intervals in the SiFive7 scheduler model:
```
defvar Cycles = defvar Cycles = SiFive7GetCyclesDefault<mx>.c;
let Latency = 4, AcquireAtCycles = [0, 1], ReleaseAtCycles = [1, !add(1, Cycles)] in
defm "" : LMULWriteResMX<"WriteVIALUV", [SiFive7VCQ, SiFive7VA], mx, IsWorstCase>;
```
I am not seeing the scheduling algorithm work as expected with the following example:
```
declare <vscale x 2 x i64> @llvm.riscv.vadd.nxv2i64.nxv2i64(
<vscale x 2 x i64>,
<vscale x 2 x i64>,
<vscale x 2 x i64>,
i64);
define <vscale x 2 x i64> @intrinsic_vadd_vv_nxv2i64_nxv2i64_nxv2i64(<vscale x 2 x i64> %0, <vscale x 2 x i64> %1, <vscale x 2 x i64> %2, <vscale x 2 x i64> %3, i64 %4) nounwind {
entry:
%a = call <vscale x 2 x i64> @llvm.riscv.vadd.nxv2i64.nxv2i64(
<vscale x 2 x i64> undef,
<vscale x 2 x i64> %0,
<vscale x 2 x i64> %1,
i64 %4)
%b = call <vscale x 2 x i64> @llvm.riscv.vadd.nxv2i64.nxv2i64(
<vscale x 2 x i64> undef,
<vscale x 2 x i64> %3,
<vscale x 2 x i64> %2,
i64 %4)
%c = call <vscale x 2 x i64> @llvm.riscv.vadd.nxv2i64.nxv2i64(
<vscale x 2 x i64> undef,
<vscale x 2 x i64> %a,
<vscale x 2 x i64> %b,
i64 %4)
ret <vscale x 2 x i64> %c
}
```
I am debuging scheduling information using the following command:
```
llc the-example-above.ll -mtriple=riscv64 -mcpu=sifive-x280 -misched-dump-schedule-trace -misched-topdown
```
The table I see:
```
*** Final schedule for %bb.0 ***
* Schedule table (TopDown):
i: issue
x: resource booked
Cycle | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 |
SU(0) | i | | | | | | | | | | | | | | | | | | |
SiFive7PipeAB | x | | | | | | | | | | | | | | | | | | |
SU(1) | i | | | | | | | | | | | | | | | | | | |
SiFive7PipeAB | x | | | | | | | | | | | | | | | | | | |
SU(2) | | i | | | | | | | | | | | | | | | | | |
SiFive7PipeAB | | x | | | | | | | | | | | | | | | | | |
SU(3) | | i | | | | | | | | | | | | | | | | | |
SiFive7PipeAB | | x | | | | | | | | | | | | | | | | | |
SU(4) | | | i | | | | | | | | | | | | | | | | |
SiFive7PipeAB | | | x | | | | | | | | | | | | | | | | |
SU(5) | | | i | | | | | | | | | | | | | | | | |
SiFive7PipeA | | | x | | | | | | | | | | | | | | | | |
SiFive7PipeAB | | | x | | | | | | | | | | | | | | | | |
SU(7) | | | | | i | | | | | | | | | | | | | | |
SiFive7VCQ | | | | | x | | | | | | | | | | | | | | |
SiFive7VA | | | | | | x | x | x | x | | | | | | | | | | |
SU(6) | | | | | | | | | | i | | | | | | | | | |
SiFive7VCQ | | | | | | | | | | x | | | | | | | | | |
SiFive7VA | | | | | | | | | | | x | x | x | x | | | | | |
SU(8) | | | | | | | | | | | | | | | i | | | | |
SiFive7VCQ | | | | | | | | | | | | | | | x | | | | |
SiFive7VA | | | | | | | | | | | | | | | | x | x | x | x |
SU(9) | | | | | | | | | | | | | | | | | | | i |
SiFive7PipeAB | | | | | | | | | | | | | | | | | | | x |
SU(0): %4:gprnox0 = COPY $x10
SU(1): %3:vrm2 = COPY $v14m2
SU(2): %2:vrm2 = COPY $v12m2
SU(3): %1:vrm2 = COPY $v10m2
SU(4): %0:vrm2 = COPY $v8m2
SU(5): dead $x0 = PseudoVSETVLI %4:gprnox0, 217, implicit-def $vl, implicit-def $vtype
SU(7): %6:vrm2 = PseudoVADD_VV_M2 undef %6:vrm2(tied-def 0), %3:vrm2, %2:vrm2, $noreg, 6, 0, implicit $vl, implicit $vtype
SU(6): %5:vrm2 = PseudoVADD_VV_M2 undef %5:vrm2(tied-def 0), %0:vrm2, %1:vrm2, $noreg, 6, 0, implicit $vl, implicit $vtype
SU(8): %7:vrm2 = PseudoVADD_VV_M2 undef %7:vrm2(tied-def 0), %5:vrm2, %6:vrm2, $noreg, 6, 0, implicit $vl, implicit $vtype
SU(9): $v8m2 = COPY %7:vrm2
```
It is my intention that SU(6) and SU(8) can be scheduled one cycle earlier in relation to the add that comes before it. The reason I believe this should happen is because (a) the SiFive7VCQ resource is not booked at the cycle before the current issue cycle, (b) the instruction does not use the SiFive7VA on its first cycle since it has AcquireAtCycle = 1 for the SiFive7VA, (c) and because there is no dependency between the instruction and the one before it.
I'm mostly confident that I've described the correct behavior in the scheduler model.
I am looking at the ready cycle for the second add and this is what I see:
```
Scheduling SU(6) %5:vrm2 = PseudoVADD_VV_M2 undef %5:vrm2(tied-def 0), %0:vrm2, %1:vrm2, $noreg, 6, 0, implicit $vl, implicit $vtype
Top Pressure:
VM=4
Ready @9c
SiFive7VA +5x2u
SiFive7VCQ +1x2u
TopQ.A BotLatency SU(6) 11c
TopQ.A @9c
Retired: 8
Executed: 9c
Critical: 8c, 8 SiFive7VA
ExpectedLatency: 3c
- Latency limited.
```
I am surprised to see `Ready @9c`. I feel like it should be ready at cycle 8 based on the fine grained resource intervals.
@fpetrogalli do you have any insights onto what may be going on here?
</pre>
<img width="1px" height="1px" alt="" src="http://email.email.llvm.org/o/eJzUWl9z2rzS_zTKzQ6MLWMgF7kg0LyTmXbe_knTc646srWATmWJI8mEfPszkgw2hJCkJc9zTidlsbQr_fa30tpozawVC4V4RfJrks8uWO2W2lxVolwylBUTTjLFLwrNH69-ICzZGsGiA107cBpqizAXCmFhmFBg0OralAhMSl0yJ7SC2gq1aHuEcmjWTFoQCtwS4Zu4EWscgS2XyGuJBirNUZJsQpIZSbafw6T5C5cc52tmYPpYSrRAshkAPG1rhv4_dLFxhnNWS0eyabUh2Yd-SbLrOJ5EBx-ZQ1U-BssBoVOYlP-uhcGJ6wxJ8uvE96Ukn3n5FSUy-0Qn9X2EpoxzQsfhKioQeknyGQgV54WIuwJCKaEUSDaBj5--f_xhhMOvaD_9g2RTQmm4vr-dfPx-HzSn3tBP1Lh4P_3iG7dXkwZdtfGft_aHNtZNmUWSfdj5fEBp_LwFVoHSDiyij5uPUBMZf8nkQhvhlhU8aPMLmAXcrLB0yOFBuGVQn2sp9YPXxg2rVhJfCmUpmUEg2XRtSyYRNkBhA2I4INkHIINEynXVN8KW6_6acd5XmzUVw8FWEjreknl8DEKn51MIE162JG6Xo98Ez7sglDNCWVH-9B78XK9_NuAPpXfmmVFoHlbe893p6W56ujvz3WI48N-9i6B0rR6E4kBGjbOonHncRRO8JgsrvmRSniWAzwUAasVx3obhNEcvjtaQ1ep1_O44V_w9zp3Enb1Sj7Z6x50r_9six17pWfFS5Ay6E-Zls3FHs5fSIMeiXvg81kmAQs21qbp3tv2UV-qqYoq3KW9_dClLb9BrMmOPFXqNfSmhVzkjQq6cBbqHA-hV5aom2cyKuVhjb0PHCfQqEcD0eF2tettbZs8ZVmLb6fSK6wd1wr-7JYJjhUS49Zn-ObiENn9wIxSTu3s0zLUJoSj6CeyUthGmE_i2VYyTEDq-06uZx-Rz5y6BCH-_E9bWuG3Z-Jbdw0Kh9S_ksS_cP2HvHxlNIYkijYJGkUUxiCKPYhjFKIpxFJfROokijYJGkUUxiCKPYhjFKIqxFxHft--EjhOfOJ9gFFH4_03LO4onLe1e2T4ffBYrnFwHlc25Zvld0SEvPUXe30Dcu_B3btgd_qjnr6Pwjsy9CtQzlDWar2DundB3KMv-AsreAur3KXvfzdGhbHBA2VmZ-w1Qpyk7ytz7cnXAV340q52HvD-OKDxh70Xy3pe4Pw3t-5L2dDeMTkX3vXbJC7N0Q7vj73765cWR_pL8so-uPbJ41SybE-LPV0AnrsNX7Nq3zXSW0DfkvSGmrxB_HvaTMT3DXvzdsO_HdHyemL7NkZfD_obtej68p0J_hrj-CWOvDn0ntJdvS8PvIZ7E_A33sLOz9jKjBz9a_c9uiAcp2WSxMkpvknBANP3_z_8EQgebNDn4qbYzyUg2WZuK7umv00FFD36c7CzocQu6Z5F1LdLjFsmexaBrkRy1GO8Z5DsDjowHN6PXny3WXN9_-3B3__H2gBZCp0DTUTgurVZSlML1OM7D8PJoq3tc4cEzww7msAuzmXYym_28v__5icbDs64aoWMnkIehQ9xCnWEXguaS7l0OlDa48N-H_iPpYnyK-hjiYRdx_jrE-WnEyT7i9MyIxw1iQvPR6_COTuPN9_EOz4z3ssNwWKXdZdtiO3Fi6UBYqB5DgU2F80m3ZA7apxmmOLT3wZIpKHa1HeSgFUIZztiQGSnQQCjoyXjW6XQ46GScx2FLXaGFAufaIAjXh7slgkFmtYJbKFAKXCO4pbBgl7qWHJZstULlQRZYstqGU0HmoXSqgP6215YKbahExUNAYC4oRojNxKGhNgaVi8eIsTsGZVxsBxfKOlOXwRGuMQ7rEXRnnoBWIJyFuTDWNfNYoTwQB0tmD-qBIUBpOAndG6aZvNxSvvXWLdE0PgHHFSoeqo0FugdE9QSnN_VtPiwtzXsRJ3RUQaWtk49QajUX3PMQwuP71ggcbWlEgXGoUhuDpYMCl2wttNlWYA8qr_0nB-FS61-h-hdDYJDxx4agrfsWS614WB8RubDe14cA5tQR87f2bL1dqv8DWeZOr-CzQWtr0_p2_4lks8H2xv818EQGyWW5beo8QdHrfEPrIw_09DptO-BOr770J3Ct3bY-3dKUpuUOjFfam-orOmGQ-6SyK9J82GBZu9jYak6NcKJkMqiW3udxZznvTGOdt0HhdbPdCL1d8VyKSjjk_V2h6dnqiq3Nygjr16b2C8SrdhkbJn24hTmiBCl-hU3YZJJiuwTZdpuOoWA25LBYjtm9kYD8yJsHewucDJL5Cp3RCyalAK7hUdfxPQemfDa1YrF0FrRyOi7nivldCwvt16xW4Pc1yW4u-FXGL7NLdoFX6Sihw2E-ounF8mp-yVmGeTbP8mRcItIERzgsRqzMs9GcFxfiiiY0S2map-MBTdJ-mpTZJZ8Px_NszIY8IYMEKyZkP5TjtFlchGR3NcqHg-RCsgKlDS9wUKrwoSmoUEry2YW58ja9ol5YMkiksM62ozjhJF7d_Ob7G7tMevgqwEVt5NXSuZX1G4PeEHqzEG5ZF_1SV4Te-Pkb0VsZ_S8sHaE3AbUl9CZ49Z8AAAD__1KKoek">