[PATCH] Change MCTextAtom to use std::list instead of std::vector

[David Blaikie]

On Tue, Oct 15, 2013 at 11:27 AM, Stephen Checkoway <s at pahtak.org> wrote:

> Changing MCTextAtom to use a list instead of a vector reduces the memory
> usage of running llvm-objdump -disassemble -symbolize -cfg ninja from 16 GB
> to 9 MB.
>
> I believe the problem is that erasing elements from the vector doesn't
> free up any space so each time an atom is split, we still have all the
> space for those instructions.
>

I'm not sure how relevant this is to you/this issue, but one common idiom
for shrinking vectors capacity is to use copy-and-swap:

std::vector<T> v1 = v;
v.swap(v1);

v1 will generally be initialized with a practical capacity for the current
size of 'v', ignoring v's currently bloated capacity, then swap will just
trivially swap the buffer pointer/size/capacity between the two vectors, v1
dies along with its excess capacity.

This would save the extra couple of pointers per element you'll be paying
to std::list. Though I don't know if there's a logical point in the program
to do such shrinking, nor whether the other savings you describe from using
list dominate the issue anyway.


>
> Using a list also enables us to use std::list::splice() to split the
> instructions in constant time. There is still a linear scan to find the
> split point.
>
> Maybe something could be done using smarter data structures or smarter
> choices of the order atoms are split, but I haven't gone down that route.
>
> One consequence of switching to a list is that the MCTextAtom::at()
> function becomes inefficient since you would need to perform the linear
> scan. However, this is only used in two locations: MCModuleYAML and
> llvm-objdump. In both cases, the code was iterating over every instruction
> so the patch changes the loops to use iterators and MCTextAtom::at() is
> removed.
>
> The patch could be split into two parts, one that changes from vector to
> list and fixes the two at() uses, and one that replaces the copy() +
> erase() with splice().
>
> I still run out of memory trying to run it on my chromium with debug
> symbols but as Ahmed pointed out, there is probably room for improvement
> still. (A singly linked list would save some but C++98 doesn't appear to
> have one and there are probably better optimizations.)
>
> --
> Stephen Checkoway
>
>
>
>
>
>
>
> _______________________________________________
> llvm-commits mailing list
> llvm-commits at cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20131015/7f5749d8/attachment.html>