<html>
<head>
<meta content="text/html; charset=ISO-8859-1"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<div class="moz-cite-prefix">Hi, Hal:<br>
<br>
Sorry for the disturbing, bunch of dumb newbie questions:<br>
<br>
1) How do you know how many iterations ahead is adequate? Which
VTTI interface tell you the latency of promoting <br>
datum from L2 => L1? If the loop with SW prefetching
instruction inserted are unrolled by some amounter later on, <br>
do you adjust the prefetching code accordingly?<br>
<br>
2) Is "L1" the top most level in the memory hierarchy (i.e. no
L0)? If this is the case (and if my memory serve me right), <br>
I think this change won't help FP programs on Itanium, as FP
datum is loaded from L1. <br>
<br>
3) I saw you almost "hardcode" some u-arch parameters (about
memory hierarchy) at the beginning of<br>
the patch? Is it for the time being? I guess Poly folks may
have some magic interface for that end.<br>
Actually, I'm looking for some interface. If you already know
that, please let me know. Profuse thanks <br>
in advance!<br>
<br>
4). Is this work based on existing published work?<br>
<br>
Have a nice weekend!<br>
Shuxin <br>
<br>
<br>
On 01/02/2013 03:16 PM, Hal Finkel wrote:<br>
</div>
<blockquote
cite="mid:11536641.3039.1357168601317.JavaMail.javamailuser@localhost"
type="cite">
<pre wrap="">Hi again,
I've started working on a transformation pass to add explicit prefetch instructions to prefetch loop data into the L1 cache. The usefulness of this kind of transformation is obviously target dependent (and, moreover, often dependent on how the hardware prefetcher(s) operate). Nevertheless, there are some motivational factors common to many platforms:
1. On many platforms, to prevent L1 cache pollution, the hardware prefetcher does not prefetch into the L1 cache (or, if it does, it is not aggressive enough to achieve maximal performance). This is true on some of my PowerPC hardware, and also true on some modern x86 hardware (such as Nehalem and Barcelona [2]). The hardware prefetchers will prefetch only into L2 (or L3, etc.) but not into L1, and so explicit software prefetching is the only way to pre-load data into the L1 cache. While it is often true that out-of-order cores can hide the latency of L1 misses, explicit prefetching can still sometimes help.
2. Software prefetching is useful for short streams (most prefetchers require at least 2 misses to establish a prefetching stream).
3. Software prefetching is useful for irregular (but predicable) data access patterns.
4. Software prefetching is useful when prefetching all of the necessary data for a loop would exceed the number of streams that the hardware can handle. The number of streams is often fairly limited (~8-32), and the streams are often shared between cores (and hardware threads) for upper-level caches. In cases where a large number of streams would be needed, software prefetching may be the only way to prefetch data for the loop.
5. Hardware prefetching often cannot cross page boundaries [1], and so software prefetching is necessary to prevent misses on page boundaries (and pages can be ~4 kB on many systems).
The initial version of this pass is fairly simple. It uses CodeMetrics to estimate the number of cycles necessary to execute the loop body, and divides that by a heuristic prefetch latency to calculate for how many loop iterations ahead to prefetch data. It then inserts prefetch instructions after every load (but not for loads within one cache line size of some already-prefetched load to avoid double-prefetching cache lines). This is fairly effective on my PowerPC hardware, and (somewhat to my surprise), is sometimes beneficial on my x86 test machine. To be clear, using this pass often produces slowdowns on my Xeon testing system (more often than speedups), so it would certainly need some work to be generally applicable. If anyone is interested in working on this with me, please let me know.
Some future work (in no particular order):
1. Use VTTI instead of (or in addition to) CodeMetrics in order to get a more-accurate estimate of the loop iteration cost.
2. Use loop dependence analysis to inhibit prefetching of loads we've just recently accessed in previous iterations (and maybe nearby data?)
3. Additional heuristics to limit prefetch insertion when we have smarter hardware (like on x86) that needs help only with more-difficult cases
4. node->next prefetching for linked-list iteration
In short, I'm sending this e-mail as (hopefully) a conversation-starter. As is, the pass is quite useful for me, and I'd like to know what kinds of things need to happen to make it useful more generally. I have only a very basic idea of what this means for smarter hardware and ooo cores, so feedback is certainly welcome.
That having been said, I'd like to commit this to trunk (turned off by default). As a side note, gcc has -fprefetch-loop-arrays, and we could similarly add a Clang to enable this pass.
Thanks again,
Hal
Some good references are:
[1] Memory part 5: What programmers can do.
Ulrich Drepper, 2007.
<a class="moz-txt-link-freetext" href="http://lwn.net/Articles/255364/">http://lwn.net/Articles/255364/</a>
[2] When Prefetching Works, When It Doesn’t, and Why
Jaekyu Lee, Hyesoon Kim, and Richard Vuduc, 2012.
<a class="moz-txt-link-freetext" href="http://vuduc.org/pubs/lee2012-taco.pdf">http://vuduc.org/pubs/lee2012-taco.pdf</a>
P.S. It occurs to me that this probably won't apply against today's trunk because of the header renaming, but I can post a rebased patch soon.
</pre>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
llvm-commits mailing list
<a class="moz-txt-link-abbreviated" href="mailto:llvm-commits@cs.uiuc.edu">llvm-commits@cs.uiuc.edu</a>
<a class="moz-txt-link-freetext" href="http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits">http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits</a>
</pre>
</blockquote>
<br>
</body>
</html>