[PATCH] D17694: Enhancing MCStreamer to support referential (rather than copy) emission of bytes and use that to reduce the memory footprint of llvm-dwp
David Blaikie via llvm-commits
llvm-commits at lists.llvm.org
Sun Feb 28 08:45:26 PST 2016
dblaikie created this revision.
dblaikie added reviewers: chandlerc, echristo, grosbach.
dblaikie added a subscriber: llvm-commits.
MCStreamer currently always copies the contents into a temporary buffer before emitting the contents of that buffer to the file. This is problematic when the input is large and already accessible elsewhere.
This change (see the separate sub-diffs/commits within it to follow along more easily) adds support for referential emission of bytes into the object file and uses that to reduce the high water mark in a large dwp run from 9.6 GB to 2.3 GB.
The implementation is totally ad-hoc/prototype code, I don't know MC terribly well enough to know much better and /totally/ looking for input on what this should look like. Should I support referential and non-referential output to the same section? Should it collapse the contents as I've implemented it, or interleave (using the old Contents buffer as a fragment buffer and represent the non-referential chunks?)? Just not support that at all and assert fail if you mix referential and non-referential output in the same section?
Any/all ideas welcome, totally happy to rewrite this in any direction now that I've demonstrated the value in the outcome.
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 25450 bytes
Desc: not available
More information about the llvm-commits