[lldb-dev] lldb tests

Fri Mar 30 10:28:55 PDT 2012

Hi Enrico,

Indeed, lldb didn't have much to work with. At least we now know why.
I'll run the tests for my GIL patch, then and ping back when it's ready.

Regards,  

  Filipe

On Friday, March 30, 2012 at 5:23 PM, Enrico Granata wrote:

> Hi Filipe,
> glad to hear it works on a recent clang.
> My bet is that the culprit is actually the compiler, not lldb.
>  
> The way the data formatter works is by accessing the type information for a specific child and extracting type information for the map out of it. This is what the DWARF emitted by old clang has to say about the type we use:
> 0x0000bff4: TAG_class_type [35]  
> AT_name( "__tree_node" )
> AT_declaration( 0x01 )
>  
> 0x0000bffa: TAG_class_type [35]  
> AT_name( "__tree_node_base" )
> AT_declaration( 0x01 )
>  
>  
> Not much. OTOH, the newer clang:
> 0x0000c6cc: TAG_class_type [13] *
> AT_name( "__tree_node<std::__1::pair<int, int>, void *>" )
> AT_byte_size( 0x28 )
> AT_decl_file( "/usr/lib/c++/v1/__tree" )
> AT_decl_line( 595 )
>  
> 0x0000c6d5: TAG_inheritance [31]  
> AT_type( {0x00009b4a} ( __tree_node_base<void *> ) )
> AT_data_member_location( +0 )
> AT_accessibility( DW_ACCESS_public )
>  
> 0x0000c6de: TAG_typedef [14]  
> AT_type( {0x0000d9bd} ( pair<int, int> ) )
> AT_name( "value_type" )
> AT_decl_file( "/usr/lib/c++/v1/__tree" )
> AT_decl_line( 600 )
>  
> 0x0000c6ea: TAG_member [15]  
> AT_name( "__value_" ) <---- this is what we use, and the type information is good enough
> AT_type( {0x0000c6de} ( value_type ) )
> AT_decl_file( "/usr/lib/c++/v1/__tree" )
> AT_decl_line( 602 )
> AT_data_member_location( +28 )
> AT_accessibility( DW_ACCESS_public )
>  
> 0x0000c6fa: TAG_subprogram [17] *
> AT_name( "__tree_node" )
> AT_decl_file( "/usr/lib/c++/v1/__tree" )
> AT_decl_line( 611 )
> AT_declaration( 0x01 )
> AT_external( 0x01 )
> AT_accessibility( DW_ACCESS_public )
> AT_explicit( 0x01 )
>  
> 0x0000c706: TAG_formal_parameter [8]  
> AT_type( {0x00016c30} ( __tree_node<std::__1::pair<int, int>, void *>* ) )
> AT_artificial( 0x01 )
>  
> 0x0000c70c: TAG_formal_parameter [18]  
> AT_type( {0x00016c3a} ( const value_type& ) )
>  
> 0x0000c711: NULL
>  
> 0x0000c712: TAG_template_type_parameter [10]  
> AT_type( {0x0000d9bd} ( pair<int, int> ) )
> AT_name( "_Tp" )
>  
> 0x0000c71b: TAG_template_type_parameter [10]  
> AT_type( {0x0000d646} ( * ) )
> AT_name( "_VoidPtr" )
>  
> 0x0000c724: NULL
>  
> The type information for __value_ refers to:
> 0x0000d9bd: TAG_class_type [6] *
> AT_name( "pair<int, int>" )
> AT_byte_size( 0x08 )
> AT_decl_file( "/usr/lib/c++/v1/utility" )
> AT_decl_line( 212 )
>  
>  
> which is the right type, and hence the test works :-)
>  
> Without type information there is not much that we can do. We could, of course, try and make up the type manually by string concatenation - that is done in the libstdcpp map formatter, but this latter approach is much safer when it works. If you look into the other formatter, you'll see we need to play some tricks to keep things working:
> # we need this function as a temporary workaround for rdar://problem/10801549
> # which prevents us from extracting the std::pair<K,V> SBType out of the template
> # arguments for _Rep_Type _M_t in the map itself - because we have to make up the
> # typename and then find it, we may hit the situation were std::string has multiple
> # names but only one is actually referenced in the debug information. hence, we need
> # to replace the longer versions of std::string with the shorter one in order to be able
> # to find the type name
> def fixup_class_name(self, class_name):
> logger = Logger.Logger()
> if class_name == 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >':
> return 'std::basic_string<char>',True
> if class_name == 'basic_string<char, std::char_traits<char>, std::allocator<char> >':
> return 'std::basic_string<char>',True
> if class_name == 'std::basic_string<char, std::char_traits<char>, std::allocator<char> >':
> return 'std::basic_string<char>',True
> if class_name == 'basic_string<char, std::char_traits<char>, std::allocator<char> >':
> return 'std::basic_string<char>',True
> return class_name,False
>  
> def update(self):
> logger = Logger.Logger()
> try:
> self.count = None
> # we will set this to True if we find out that discovering a node in the map takes more steps than the overall size of the RB tree
> # if this gets set to True, then we will merrily return None for any child from that moment on
> self.garbage = False
> self.Mt = self.valobj.GetChildMemberWithName('_M_t')
> self.Mimpl = self.Mt.GetChildMemberWithName('_M_impl')
> self.Mheader = self.Mimpl.GetChildMemberWithName('_M_header')
>  
> map_type = self.valobj.GetType()
> if map_type.IsReferenceType():
> logger >> "Dereferencing type"
> map_type = map_type.GetDereferencedType()
>  
> map_arg_0 = str(map_type.GetTemplateArgumentType(0).GetName())
> map_arg_1 = str(map_type.GetTemplateArgumentType(1).GetName())
>  
> logger >> "map has args " + str(map_arg_0) + " and " + str(map_arg_1)
>  
> map_arg_0,fixed_0 = self.fixup_class_name(map_arg_0)
> map_arg_1,fixed_1 = self.fixup_class_name(map_arg_1)
>  
> logger >> "arg_0 has become: " + str(map_arg_0) + " (fixed: " + str(fixed_0) + ")"
> logger >> "arg_1 has become: " + str(map_arg_1) + " (fixed: " + str(fixed_1) + ")"
>  
> # HACK: this is related to the above issue with the typename for std::string
> # being shortened by clang - the changes to typename display and searching to honor
> # namespaces make it so that we go looking for std::pair<const std::basic_string<char>, ...>
> # but when we find a type for this, we then compare it against the fully-qualified
> # std::pair<const std::basic_string<char, std::char_traits... and of course fail
> # the way to bypass this problem is to avoid using the std:: prefix in this specific case
> if fixed_0 or fixed_1:
> map_arg_type = "pair<const " + map_arg_0 + ", " + map_arg_1
> else:
> map_arg_type = "std::pair<const " + map_arg_0 + ", " + map_arg_1
>  
> if map_arg_1[-1] == '>':
> map_arg_type = map_arg_type + " >"
> else:
> map_arg_type = map_arg_type + ">"
>  
> logger >> "final contents datatype is: " + str(map_arg_type)
>  
> self.data_type = self.valobj.GetTarget().FindFirstType(map_arg_type)
>  
> logger >> "and the SBType is: " + str(self.data_type)
>  
> # from libstdc++ implementation of _M_root for rbtree
> self.Mroot = self.Mheader.GetChildMemberWithName('_M_parent')
> self.data_size = self.data_type.GetByteSize()
> self.skip_size = self.Mheader.GetType().GetByteSize()
> except:
> pass
>  
>  
> This is way more complicated than desirable. There should be ways to mark the test as an expected failure for older-than-X clang versions. Maybe that is the best way to tackle this issue.
>  
> Thanks.
>  
> Enrico Granata
> ✉ egranata@.com
> ✆ (four oh eight) 862-7683
>  
>  
>  
>  
>  
>  
>  
> On Mar 30, 2012, at 8:23 AM, Filipe Cabecinhas wrote:
> > Hi Enrico,
> >  
> > The old compiler was the culprit, it seems. With clang ToT the test passes.
> > I'm sending the a.out and its dSYM file anyway since this may be a bug in lldb anyway (it doesn't work with the current Mac OS X provided clang).
> >  
> > Thanks,  
> >  
> > Filipe
> >  
> >  
> > On Friday, March 30, 2012 at 3:26 PM, Enrico Granata wrote:
> >  
> > > Hi,
> > > The 'foo' is indeed a left-over debugging printout. I usually remove them before committing the files, but every once in a while one remains. I will replace it with a Logger call later today.
> > > If you actually read the code for libcxx.py:
> > >  
> > > def get_child_at_index(self,index): logger = Logger.Logger() if index < 0: return None if index >= self.num_children(): return None; if self.garbage: return None try: iterator = stdmap_iterator(self.root_node,max_count=self.num_children()) # the debug info for libc++ std::map is such that __begin_node_ has a very nice and useful type # out of which we can grab the information we need - every other node has a less informative # type which omits all value information and only contains housekeeping information for the RB tree # hence, we need to know if we are at a node != 0, so that we can still get at the data need_to_skip = (index > 0) current = iterator.advance(index) if current == None: self.garbage = True return None if self.get_data_type(): if not(need_to_skip): current = current.Dereference() obj = current.GetChildMemberWithName('__value_') obj_data = obj.GetData() self.get_value_offset(current) # make sure we have a valid offset for the next items # we do not return __value_ because then we would end up with a child named # __value_ instead of [0] return self.valobj.CreateValueFromData('[' + str(index) + ']',obj_data,self.data_type) else: # FIXME we need to have accessed item 0 before accessing any other item! if self.skip_size == None: return None return current.CreateChildAtOffset('[' + str(index) + ']',self.skip_size,self.data_type) else: print "foo" return None except Exception as err: print err return None  
> > >  
> > > "foo" is printed when self.get_data_type() does not work. On my side, the right thing to do is add a fair amount of logging to better diagnose this and similar scenarios.
> > >  
> > > On your side, there are several possibilities:
> > > (a) wait for the logging to be in, and then repeat the test case and send the new log files
> > > (b) send the a.out and associated dSYM and let's check if they look correct or the older compiler is doing something wrong with the debug info
> > > (c) compile a new clang from TOT and retry using the updated compiler
> > >  
> > > Thanks,
> > > Enrico Granata
> > > ✉ egranata@.com
> > > ✆ (four oh eight) 862-7683
> > >  
> > >  
> > > On Mar 30, 2012, at 2:54 AM, Filipe Cabecinhas wrote:
> > > > Hi,
> > > >  
> > > > Attahced is the output of the test ran in verbose mode.
> > > > There's a very weird 'foo' that is shown after I enabled logging in the test (I enables verbose formatter logging right before the instruction that fails).
> > > >  
> > > > That print comes from stdmap_SynthProvider.get_child_at_index(), on line 546 of the Python/libcxx.py file. It seems like some kind of error, maybe you know what it's about (Something unfinished in that file?).  
> > > >  
> > > >  
> > > > Filipe
> > > >  
> > > >  
> > > > On Thursday, March 29, 2012 at 6:31 PM, Enrico Granata wrote:
> > > >  
> > > > > Hi,
> > > > >  
> > > > > a good first step is for you to run the test in verbose mode and attach the output.
> > > > > You can also try to manually repeat the test case behavior and seeing what you get.
> > > > >  
> > > > > Moving from there should not be too complicated.
> > > > >  
> > > > > As for tool versions, I am using a previous version of swig because of licensing issues. I also have a more recent clang based off LLVM 3.1 svn. I am not sure why that would be the case but given that the test case at fault here is related to libc++ I would guess that the build of libc++ has something to do with it. However, we are on the same OSX version.
> > > > >  
> > > > > Thanks.
> > > > >  
> > > > > Enrico Granata
> > > > > ✉ egranata@.com
> > > > > ✆ (four oh eight) 862-7683
> > > >  
> > >  
> >  
> >  
> >  
> > <a.out.dSYM.zip><a.out>  