[llvm] [InferAS] Infer the address space of inttoptr (PR #173244)

[Luo Yuanke via llvm-commits]

LuoYuanke wrote:

> Would it be OK to revert this until we have a definition for the hook that makes sense?

I prefer not. It is helpful to optimize swizzle cases. For AMDGPU it is not enabled, so I think this patch doesn't affect AMDGPU target.

https://github.com/llvm/llvm-project/pull/173244