[LLVMdev] RFC: Reduce the memory footprint of DIEs (and DIEValues)

[David Blaikie]

On Wed, Jun 17, 2015 at 11:26 AM, Duncan P. N. Exon Smith <
dexonsmith at apple.com> wrote:

>
> > On 2015 Jun 16, at 21:40, Yaron Keren <yaron.keren at gmail.com> wrote:
> >
> > Hi Duncan,
> >
> > Singly linked lists are used throught LLVM: ArrayRecycler, Registry,
> LiveInterval, SCEVUnknown, CGBlockInfo, MacroArgCache, ...DIEValueList.
> >
> > What do you think about implementing a singly-linked ADT?
> >
>
> Interesting idea.
>
> I've taken a look at the other singly-linked lists you mentioned.  Most
> of these are some version of a free/recycling list (ArrayRecycler,
> Registry, MacroArgCache, etc.), which need super simple push/pop API and
> no memory management.
>
> LiveInterval and CGBlock also need iteration and a way to conditionally
> remove elements (something like `remove_if()`), and in a few cases it
> would be convenient to have API to destruct and/or destroy nodes.
>
> I think these would all be straightforward to formalize into an ADT.  We
> could even use it to implement a value-based `llvm::forward_list<>` that
> avoided the sorting bug in libstdc++ (which gets "stable" exactly
> backwards) that David hit when refactoring tablegen.
>
> Unfortunately, I think it'd be hard to reuse for DIE::Children and
> DIEValueList.  The latter wants to preserve insertion order and needs
> `push_back()` API (the former can technically get away with a timely
> call to `reverse()`, but it's awkward).  Supporting `push_back()`
> efficiently requires pointing at the last element somehow, either via an
> extra pointer or circular logic (what my latest patch does for
> DIE::Children).
>

One of the other problems I hit when trying to value-ify the DIE children
is that the point of DIE creation is, in some particular cases, divorced
from the knowledge of which container it's going to go in to. Yet pointer
identity is needed before then.

At least that's how I recall it - may be worth going down that path again
just so we can write it up clearly for posterity, I dunno.

If it is possible, then I would love/prefer to use an existing
non-intrusive container (or writing one if we have to). at least could
prototype with std::list, then, as you say, implement a
std::forward_list-like device with the bonus that we could fix sort and do
the circular trick to allow insertion at the end, etc.

(& sorry, will get to proper review of this patch soon... )


>
> We could configure this stuff via templates -- I'd be open to the idea
> -- but I think the intersection between the implementations would be
> practically nil.  Even the iterator implementations need to be
> completely different.  If these are the only `push_back()`-enabled
> slists in tree, is it premature to abstract it?  Would we even want the
> same name as "normal" slists?
>
> (Regardless, I think it would be a great idea for someone to ADT-ify the
> other linked lists!)
>
> >
> > 2015-06-05 3:24 GMT+03:00 Duncan P. N. Exon Smith <dexonsmith at apple.com
> >:
> >
> > > On 2015 May 28, at 14:14, Duncan P. N. Exon Smith <
> dexonsmith at apple.com> wrote:
> > >
> > >
> > >> On 2015-May-26, at 15:30, Duncan P. N. Exon Smith <
> dexonsmith at apple.com> wrote:
> > >>
> > >> David, let me know if you have any comments on the `DIE::Children`
> > >> stuff (patch 0008 in particular, although I think Eric wanted your
> > >> opinion on 0007 as well).
> > >>
> > >>> On 2015-May-26, at 14:41, Eric Christopher <echristo at gmail.com>
> wrote:
> > >>> 0007 - LGTM, might want to have Dave take a look at it as well.
> > >>> 0008 - reverseChildren and finalizeChildren is pretty gross. Also,
> I'm going to punt on the rest of this to Dave.
> > >
> > > I've committed all but these two.  I've attached rebased versions
> > > that incorporate (most of) Fred's feedback.
> > >
> > > David, I found a way to avoid the reverseChildren/finalizeChildren
> > > thing without paying for an extra pointer, but it means that
> > > `DIE` would no longer use `std::unique_ptr<>` internally.  (High
> > > level: change the sibling list to a circular linked list, point at
> > > the last child instead of the first, and store a "this-is-last"
> > > marker on the last child to support iteration.)
> > >
> > > stop-reversing-children.patch implements this on top of 0008;
> > > stop-reversing-children-with-0008.patch has that patch and 0008
> > > merged together.  I think this might be the better way to do it,
> > > but I'm open to other ideas.
> > >
> > > If you're okay with this approach, maybe I'll rewrite 0007 to
> > > save an extra pointer there as well?
> >
> > Ping.  Patches reattached.
> >
> >
> > _______________________________________________
> > llvm-commits mailing list
> > llvm-commits at cs.uiuc.edu
> > http://lists.cs.uiuc.edu/mailman/listinfo/llvm-commits
> >
> >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20150617/2a27ce3c/attachment.html>