[llvm] [AMDGPU] introduce S_WAITCNT_FENCE_soft emitted by memory legalizer (PR #150167)

[Sameer Sahasrabuddhe via llvm-commits]

ssahasra wrote:

> But there is no hurry in fixing what isn't broken right now.

Despite my rant about moving all waitcnt insertion to the waitcnt inserter pass, this specific thing smells wrong to me. S_WAITCNT_FENCE_soft might be a powerful tool, but I don't see any significant use of it other than fixing this one bug about direct loads to LDS, and that only exists on some specific older architectures. Looks a lot like a solution looking for problems to justify its existence.

I would rather just go back to what @kerbowa had originally proposed, with #138802 and #142018, except with new insight, we can do it in a much more precise surgery.

https://github.com/llvm/llvm-project/pull/150167