[llvm-dev] Enforcing in post-RA scheduling to keep (two) MachineInstrs together

[Alex Susu via llvm-dev]

   Hello.
     I am using the post-RA (Register Allocation) scheduler to avoid data hazards by 
inserting other USEFUL instructions from the program (besides NOPs) and it breaks apart 
some sequences of instructions which should remain "glued" together.
     More exactly, in my [Target]ISelDAGToDAG.cpp it is possible that I replace for 
example a BUILD_VECTOR with a machine SDNode called VLOAD_D_WO_IMM and an INLINEASM, the 
latter having a simple dataflow dependence (black solid edge when outputting the DAG as a 
.DOT after instruction selection) on the result of the former instruction. (I can present 
the .DOT after instruction selection obtained with llc -view-sched-dags).
     When I run the default pre-RA scheduler (which seems to be a "List Scheduling" 
algorithm)  I always obtain the ASM generated code where the string of the INLINEASM 
follows immediately after the associated asm instruction for the VLOAD_D_WO_IMM. But when 
I use also the post-RA scheduler (llc -post-RA-scheduler ...) I get some different 
instructions inserted between the VLOAD_D_WO_IMM and the INLINEASM, which is not correct 
semantically.

     How can I avoid these 2 instructions being separated by the post-RA scheduler? Can I 
customize the behavior of the post-RA scheduler (I found some documentation at 
http://llvm.org/docs/doxygen/html/PostRASchedulerList_8cpp.html)?

     The first natural idea was to use SelectionDAG glue edges, but I noticed that they 
are not very reliable (sometimes I even have difficulties in creating them for example in 
the classes [Target]ISelDAGToDAG, [Target]ISelLowering). Also I understood that anyhow the 
scheduler can disregard the glue edges between SelectionDAG nodes. For example:
         - from http://lists.llvm.org/pipermail/llvm-dev/2014-June/074046.html
             <<You can't Glue the two nodes together forever. All Glue really does is
             keep them together long enough for LLVM to put together a data
             dependency through "Uses" and "Defs" implicit operands. Once the
             MachineInstrs have been created, the two instructions are at the whim
             of the scheduler as much as any others.
             If you really need them to remain together, you have to either create
             a pseudo-instruction and expand it extremely late, or create a bundle
             (depending on what's natural for your target).>>
         - from http://lists.llvm.org/pipermail/llvm-dev/2016-June/100885.html:
             <<If you want to have these nodes stick together, using glue may not be
             sufficient.  After the machine instructions are generated, the scheduler
             may place instructions between the interrupt disable/restore and the
             atomic load itself.  Also, the register allocator may insert some spills
             there---there are ways that this sequence may get separated.
             For this, the best approach may be to define a pseudo-instruction, which
             will be expanded into real instruction in the post-RA expansion pass.>>

     Also, I don't want to use MachineInstr bundles or pseudo-instructions. MachineInstr 
bundles seem to difficult to use and too late in the code generation (I prefer working at 
the level of instruction selection). Also, I found little information about 
pseudo-instructions - there is some API support, namely expandPostRAPseudo() described at 
http://llvm.org/docs/doxygen/html/classllvm_1_1TargetInstrInfo.html. Also, some 
documentation at 
http://llvm.org/devmtg/2014-04/PDFs/Talks/Building%20an%20LLVM%20backend.pdf, slide 55 
(and 53, 54).

    Please let me know if I can customize the post-RA scheduler to avoid scheduling in 
non-consecutive cycles my two SDNodes created "together" or if you recommend a different 
approach.

   Thank you very much,
     Alex