[PATCH] D23466: ADT: Remove UB in ilist (and use a circular linked list)

[Duncan P. N. Exon Smith via llvm-commits]

dexonsmith created this revision.
dexonsmith added reviewers: bogner, dblaikie, mehdi_amini.
dexonsmith added a subscriber: llvm-commits.

This removes the undefined behaviour (UB) in ilist/ilist_node/etc.,
mainly by removing (gutting) the ilist_sentinel_traits customization
point and canonicalizing on a single, efficient memory layout.

The new ilist is a doubly-linked circular list.
- ilist_node_base has two ilist_node_base*: Next and Prev.  Size-of: two
  pointers.
- ilist_node<T> (size-of: two pointers) is a type-safe wrapper around
  ilist_node_base.
- ilist_iterator<T> (size-of: two pointers) operates on an
  ilist_node<T>*, and downcasts to T* on dereference.
- ilist_sentinel<T> (size-of: two pointers) is a wrapper around
  ilist_node<T> that has some extra API for list management.
- ilist<T> (size-of: two pointers) has an ilist_sentinel<T>, whose
  address is returned for end().

The new memory layout matches ilist_half_embedded_sentinel_traits<T>
exactly.  The Head pointer that previously lived in ilist<T> is
effectively glued to the ilist_half_node<T> that lived in
ilist_half_embedded_sentinel_traits<T>, becoming the Next and Prev in
the ilist_sentinel_node<T>, respectively.  sizeof(ilist<T>) is now the
size of two pointers, and there is never any additional storage for a
sentinel.

This is a much simpler design for a doubly-linked list, removing most of
the corner cases of list manipulation (add, remove, etc.).  In follow-up
commits, I intend to move as many algorithms as possible into a
non-templated base class (ilist_base) to reduce code size.

Moreover, this fixes the UB in ilist_iterator/getNext/getPrev
operations.  Previously, ilist_iterator<T> operated on a T*, even when
the sentinel was not of type T (i.e., ilist_embedded_sentinel_traits and
ilist_half_embedded_sentinel_traits).  This added UB to all operations
involving end().   Now, ilist_iterator<T> operates on an ilist_node<T>*,
and only downcasts when the full type is guaranteed to be T*.

What did we lose?  There used to be a crash (in some configurations) on
++end().  Curiously (via UB), ++end() would return begin() for users of
ilist_half_embedded_sentinel_traits<T>, but otherwise ++end() would
cause a nice dependable nullptr dereference, crashing instead of a
possible infinite loop.  Options:
 1. Lose that behaviour.
 2. Have a template parameter to opt-in to a crash in asserts builds.
    This requires stealing a bit from Prev.
 3. Have a template parameter to opt-in to a crash in all builds.

IMO #1 is the best option (which is why I implemented it here).  Most
users of ilist effectively had this behaviour anyway.  #2 and #3 are
more complicated than they are worth.

I added only a couple of unit tests to root out specific bugs I hit 
during bring-up, but otherwise this is tested implicitly via the
extensive usage throughout LLVM.

Planned follow-ups:
- Remove ilist_*sentinel_traits<T>.  Here I've just gutted them to
  prevent build failures in sub-projects.  Once I stop referring to them
  in sub-projects, I'll come back and delete them.
- Add ilist_base and move algorithms there.

Eventually, there are other interesting directions:
- Remove ilist_traits::createNode, by deleting the remaining API that
  creates nodes.  Intrusive lists shouldn't be creating nodes
  themselves.
- Remove ilist_traits::deleteNode, by (1) asserting that lists are empty
  on destruction and (2) changing API that calls it to take a Deleter
  functor (intrusive lists shouldn't be in the memory management
  business).
- Reconfigure the remaining callback traits (addNodeToList, etc.) to be
  higher-level, pulling out a simple_ilist<T> that is much easier to
  read and understand.
- Allow tags (e.g., ilist_node<T,tag1> and ilist_node<T,tag2>) so that T
  can be a member of multiple intrusive lists.


https://reviews.llvm.org/D23466

Files:
  include/llvm/ADT/ilist.h
  include/llvm/ADT/ilist_node.h
  unittests/ADT/ilistTest.cpp

-------------- next part --------------
A non-text attachment was scrubbed...
Name: D23466.67904.patch
Type: text/x-patch
Size: 25528 bytes
Desc: not available
URL: <http://lists.llvm.org/pipermail/llvm-commits/attachments/20160812/3bc22ab3/attachment.bin>