[PATCH] D128877: [LoopCacheAnalysis] Fix a type mismatch bug in cost calculation

[Michael Kruse via Phabricator via llvm-commits]

Meinersbur added a comment.

In D128877#3621209 <https://reviews.llvm.org/D128877#3621209>, @congzhe wrote:

> Regarding Michael's question that whether `SE.getNoopOrAnyExtend()` is signed extension or unsigned extension: it is actually well handled in `ScalarEvolution::getAnyExtendExpr()` where it could do either signed or unsigned extension depending on the actual SCEV type of the value we want to extend. I'm wondering if it answers your question? @Meinersbur

Whether to use sext or zext can make a semantic difference. Say we have an element size of `i16 4` and a stride of -1 (going backwards) in 8-bit precision. That represented as `i8 255`. zero-extension of that is `i16 255`, and multiplying both gives `i16 1020` which it should be -4 (`i16 0xFFFC`).

`getNoopOrAnyExtend()` seems to prefer zero-extension over sign-extension.


CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D128877/new/

https://reviews.llvm.org/D128877