[llvm] [doc][RISCV] Add documentation for customizing VCIX scheduling info (PR #183129)

[Luke Lau via llvm-commits]

================
@@ -0,0 +1,263 @@
+===================================================
+Scheduling Information for RISC-V VCIX Instructions
+===================================================
+
+.. contents::
+   :local:
+
+Summary
+-------
+The purpose of this document is to outline how the scheduling information for RISC-V's ``XSfvcp`` extension -- SiFive Vector Coprocessor Interface (VCIX) -- in LLVM works, why it works the way it does, and how one may modify the code to support their VCIX needs.
+
+SiFive makes no guarantee that modifying the upstream code to describe their VCIX implementations will lead to performance improvements over the default implementation.
+
+Introduction
+------------
+LLVM uses scheduler models to describe the behavior of processor latencies and resources. The scheduler models are attached to a processor definition (i.e. ``-mcpu=``) or tunings (i.e. ``-mtune=``). The challenge with VCIX is that the same processor definition could be used with different coprocessors that have very different latencies or processor resource usage for a given instruction. As a result, a default implementation is provided, and one may use this document to customize the existing implementation to their needs.
+
+Understanding the VCIX Scheduling Model Information
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Supported Scheduling Models
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+VCIX is supported in the SiFive 7 family scheduling models, for instance ``SiFive7VLEN512Model`` and ``SiFive7VLEN1024X300Model``. These models share a large portion of scheduling information, including those for VCIX instructions. Therefore, when it comes to customizing VCIX scheduling info, which we will walk you through in later sections, you only need to modify a single place.
+
+The SiFive 7 scheduling models are used in the following (tuning) processors:
+
+*   ``-mtune=sifive7-series``
+*   ``-mcpu=sifive-x390``
+*   ``-mcpu=sifive-x280``
+*   ``-mcpu=sifive-e76``
+*   ``-mcpu=sifive-s76``
+*   ``-mcpu=sifive-u74``
+
+Understanding the Default Implementation
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To read the default implementation, please open ``llvm/lib/Target/RISCV/RISCVSchedSiFive7.td`` and navigate to the line that says ``// VCIX``. The line can be found on GitHub `here <https://github.com/llvm/llvm-project/blob/a00278632dfed7b856a0ac11a58423cb6b14a8c1/llvm/lib/Target/RISCV/RISCVSchedSiFive7.td#L1161>`__.
+
+To understand the code here, we first provide a brief overview. VCIX Pseudo-instructions are defined in ``llvm/lib/Target/RISCV/RISCVInstrInfoXSf.td`` which can be found on GitHub `here <https://github.com/llvm/llvm-project/blob/main/llvm/lib/Target/RISCV/RISCVInstrInfoXSf.td>`__.
+
+For example:
+
+.. code-block::
+
+  multiclass VPseudoVC_X<LMULInfo m, DAGOperand RS1Class,
+                         Operand OpClass = payload2> {
+    let VLMul = m.value in {
+      let Defs = [SF_VCIX_STATE], Uses = [SF_VCIX_STATE] in {
+        def "PseudoVC_" # NAME # "_SE_" # m.MX
+          : VPseudoVC_X<OpClass, RS1Class>,
+            Sched<[!cast<SchedWrite>("WriteVC_" # NAME # "_" # m.MX)]>;
+        def "PseudoVC_V_" # NAME # "_SE_" # m.MX
+          : VPseudoVC_V_X<OpClass, m.vrclass, RS1Class>,
+            Sched<[!cast<SchedWrite>("WriteVC_V_" # NAME # "_" # m.MX)]>;
+      }
+      def "PseudoVC_V_" # NAME # "_" # m.MX
+        : VPseudoVC_V_X<OpClass, m.vrclass, RS1Class>,
+          Sched<[!cast<SchedWrite>("WriteVC_V_" # NAME # "_" # m.MX)]>;
+    }
+  }
+
+  // snip
+
+  let Predicates = [HasVendorXSfvcp] in {
+    foreach m = MxList in {
+      defm X : VPseudoVC_X<m, GPR>;
+
+  // snip
+
+In this example, for each LMUL ``m.MX``, there are three pseudos defined:
+
+1.  ``PseudoVC_X_SE_ # m.MX``
+2.  ``PseudoVC_V_X_SE_ # m.MX``
+3.  ``PseudoVC_V_X_ # m.MX``
+
+Note that ``#`` concatenates the first string with the LMUL ``m.MX``. When ``m.MX`` is ``M2`` for example, the three pseudos would be defined:
+
+1.  ``PseudoVC_X_SE_M2``
+2.  ``PseudoVC_V_X_SE_M2``
+3.  ``PseudoVC_V_X_M2``
+
+Note that for each of these definitions, there is a ``Sched`` list attached. The ``Sched`` list takes ``SchedWrite`` and ``SchedRead`` objects, which define the behavior of the operands that are written and and read. In the snippet above, the singular write is attached to the pseudo-instruction. It is up to the scheduler model to describe the behavior of each ``SchedWrite``.
+
+Switching back to the scheduling model linked at the start of this section, we explain how behavior is assigned to the VCIX ``SchedWrite`` objects. Let’s take a look at an example:
+
+.. code-block::
+
+  // snip
+
+  defvar Cycles = SiFive7GetCyclesDefault<mx>.c;
+  defvar IsWorstCase = SiFive7IsWorstCaseMX<mx, SchedMxList>.c;
+  let Latency = Cycles,
+      AcquireAtCycles = [0, 1],
+      ReleaseAtCycles = [1, !add(1, Cycles)] in {
+      defm "" : LMULWriteResMX<"WriteVC_V_I",   [VCQ, VA1], mx, IsWorstCase>;
+
+  // snip
+
+A ``WriteRes`` is defined to associate processor resources, processor resource usage, and latency with each ``SchedWrite``. In this example, the ``SchedWrite`` named ``WriteVC_V_I # mx`` is being said to use the ``VCQ`` (vector command queue) and ``VA1`` (vector arithmetic sequencer) processor resources. ``AcquireAtCycles[i]`` defines a cycle, relative to instruction issue, that processor resource ``i`` in the ``WriteRes`` below is acquired at. ``ReleaseAtCycles[i]`` defines a cycle, relative to instruction issue, that processor resource ``i`` in the ``WriteRes`` below is released at. For this ``WriteRes``, we’re saying that the vector command queue is acquired at cycle 0 and released at cycle 1 and the vector arithmetic sequencer is acquired at cycle 1 and released at cycle 1+Cycles. ``Cycles`` gets its value from a function that describes the default behavior. Looking at the entire VCIX default implementation, you can see that all instructions are given this behavior.
----------------
lukel97 wrote:

Nit, might be good to mention that LMULWriteResMX defines multiple WriteRes records?

https://github.com/llvm/llvm-project/pull/183129