[lldb-dev] Advice on architectures with multiple address spaces

Fri May 25 07:39:47 PDT 2018

As an FYI - this is another way of looking at Address spaces.. I like the word “Route” to the memory space, i think it defines the problem better.

All Armv8 chips effectively have - multiple address available to the debugger

For instance, a JTAG debugger halts an ARMV8 cpu core, and the core halts in user space - the question is what are the various address spaces a developer might want to dump memory for:

A) The current user space view of memory - to print a user space variable.

B) The current Kernel (OS) view of memory - perhaps to print data structure in kernel space

C) At the hypervisor level which is the core’s view of physical memory - perhaps the developer wants to examine a data structure or variable in the hypervisor.

D) The ARM dap - has 3 (sometime 4) memory bus interfaces. They are typically:

Dap Port 0 - is typically the main system bus, often the same as the CPU’s connection to the main system bus but not exactly 

ie: CPU access to a multi-media bus, via a dedicated connection/address range, in contrast the  main system bus has a different connection/route to the multimedia memory

Dap Port 1 - commonly provides access to various Coresight items for most of the cores.

Dap Port 2 - Varies - it could be an embedded Jtag controller - say with an ARM9 or other JTAG only interface)

Dap Port 3 - Varies - But often there are a few CORTEX M series CPUs - and I believe each M3 on the target must have its own dedicated DAP interface.

Assume for a moment, each of these address spaces have a name, there is the default, then there needs to be various override methods

Being able to some how specify these different “access routes” is helpful when debugging hardware at the bare metal level.

In the above, I’m not talking about looking at a complex variable (that would be really nice, ie: cast a memory address to a fancy type)

When debugging a SYSTEM - this becomes very important.  Minimally being able to have a “memory window” that can specify the route is helpful - for example

	Dump Memory at :  HyperVisor address 0x12345678
	Dump Memory at:  Kernel address  0x45678901234
	Dump Memory using the system bus interface

Across multiple cores - you have 

1) Some very common routes - ie: “In the current context” vrs “Kernel context”
These should have commonly defined names.

2) Some platforms may need to add their own “platform defined” items (ie: various armv8 routes would be semi-common)

3) Some VERY specific routes or methods that are developer defined - For example when accessing memory via the DAP MEM_AP, there are special bits in the control register (controlling security and/or cache control). Maybe the developer needs to set those a special way when performing the memory access when using this “route”

The lauterbach jtag debugger can do the above now (it is an address prefix)
The ARM debugger (If I remember correctly) can also do this somewhere in its script language.

But tools like GDB and LLDB can not

Because tools like GDB and LLDB have a scripting language (ie: Python) - being able to write a script in that scripting language and be able to specify the ROUTE is important.

For example, a script that writes to a bunch of hardware locations via the MEMAP (or possibly via different CPU) - to enable a feature so that you can run test/validation code

Examples include write to a hardware register to turn a CLOCK on, or disable/enable a security protection bit so that you can view/write to the memory location.

-Duane.

> On May 25, 2018, at 5:35 AM, Zdenek Prikryl via lldb-dev <lldb-dev at lists.llvm.org> wrote:
> 
> Just a small update. I did create the class AddressBase. Class Address inherits from it. When I compare it with my first implementation, it's cleaner than the additional argument in API. I also implemented all operators to the class AddressBase, so it behaves like addr_t if you're not interested in the address space.
> 
> I thought that I'd rename the class AddressBase to addr_t and see what would happen, but then I realized that LLDB has python support and I'm not sure how this works with classes, so, any advice here?
> 
> Btw. I still don't have a clear answer to my question below...
> 
> Zdenek
> 
>> On 05/17/2018 03:45 PM, Zdenek Prikryl wrote:
>> Greg, Jim, what's your opinion here?
>> 
>> What about having the class Address (pretty much as it is right now) and the
>> 
>> struct AddressBase {
>>   lldb::addr_t m_address;
>>   lldb::as_t m_address_space;
>>   ...
>> }
>> 
>> Another question is, which classes/code should use Address, AddressBase, and addr_t. Do you have any idea here?
>> 
>> Zdenek
>> 
>>> On 05/17/2018 01:38 PM, Pavel Labath wrote:
>>> The Address class may be suitable for the higher layers of lldb, but I
>>> don't think the it can ever be a blanket replacement for lldb::addr_t. It
>>> has way too much smartness built-in. We use addr_t in a lot of places that
>>> don't/shouldn't care about Targets, ExecutionContexts or Sections. All of
>>> lldb-server is one of those places, but this is also true for any low-level
>>> operation which only wants to work with real (virtual) addresses in the
>>> process address space.
>>> 
>>> On the other hand, replacing addr_t with a lighweight struct which is just
>>> adds some sort of an address space identifier seems like a useful thing,
>>> and would go a long way towards bringing Harward architecture support to
>>> lldb-server. (Note that we would still need an addr_t or something of that
>>> form to name the type of the "address" member of the struct, but pretty
>>> much all of the apis that currently take addr_t, could that the new struct
>>> instead).
>>> 
>>> 
>>> On Thu, 17 May 2018 at 12:01, Zdenek Prikryl via lldb-dev <
>>> lldb-dev at lists.llvm.org> wrote:
>>> 
>>> 
>>>> On 04/19/2018 08:22 PM, Jim Ingham wrote:
>>>>>> On Apr 19, 2018, at 10:54 AM, Greg Clayton <clayborg at gmail.com> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>> On Apr 19, 2018, at 10:35 AM, Jim Ingham <jingham at apple.com> wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>>> On Apr 19, 2018, at 9:44 AM, Greg Clayton via lldb-dev <
>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>>> On Apr 19, 2018, at 6:51 AM, Zdenek Prikryl via lldb-dev <
>>> lldb-dev at lists.llvm.org> wrote:
>>>>>>>>> Hi lldb developers,
>>>>>>>>> 
>>>>>>>>> I've been researching using lldb + gdbserver stub that is based on
>>> Harvard architecture with multiple address spaces (one program, multiple
>>> data). The commonly adopted approach is that everything is mapped to a
>>> single "virtual" address space. The stub reads/writes from/to the right
>>> memory based on the "virtual" addresses. But I'd like to use real addresses
>>> with address space id instead. So, I've started looking at what has to be
>>> changed.
>>>>>>>>> I've enhanced read/write commands (e.g. memory read --as <id> ...)
>>> and RSP protocol (new packet) so that the stub can read/write properly.
>>> That wasn't that complicated.
>>>>>>>> It might be nice to add a new RSP protocol packet that asks for the
>>> address space names/values:
>>>>>>>> qGetAddressSpaces
>>>>>>>> 
>>>>>>>> which would return something like:
>>>>>>>> 
>>>>>>>> 1:text;2:data1,3:data2
>>>>>>>> 
>>>>>>>> or it would return not supported. If we get a valid return value
>>> from qGetAddressSpaces, then it enables the use of the new packet you added
>>> above. Else it defaults to using the old memory read functions.
>>>>>>>> 
>>>>>>>>> Now I've hit an issue with expressions (LLVMUserExpression.cpp) and
>>> local variables (DWARFExpressions.cpp). There is a lot of memory read/write
>>> functions that take just an address argument. Is the only way to go to
>>> patch all these calls? Has anybody solved it differently?
>>>>>>>> My quick take is that any APIs that take just a lldb::addr_t would
>>> need to take something like:
>>>>>>>> struct SpaceAddress {
>>>>>>>> static constexpr uint32_t kNoSpace = 0;
>>>>>>>> lldb::addr_t addr;
>>>>>>>> uint32_t space;
>>>>>>>> };
>>>>>>>> 
>>>>>>> I'm curious why you are suggesting another kind of address, rather
>>> than adding this functionality to Address?  When you actually go to resolve
>>> an Address in a target with a process you should have everything you need
>>> to know to give it the proper space.  Then fixing the expression evaluator
>>> (and anything else that needs fixing) would be a matter of consistently
>>> using Address rather than lldb::addr_t.  That seems general goodness, since
>>> converting to an lldb::addr_t loses information.
>>>>>> If we accept lldb_private::Address in all APIs that take a
>>> lldb::addr_t currently, then we need to always be able to get to the target
>>> in case we need to add code to resolve the address everywhere. I am
>>> thinking of SpaceAddress as an augmented lldb::addr_t instead of a section
>>> + offset style address. Also, there will be addresses in the code and data
>>> that do not exist in actual sections. Not saying that you couldn't use
>>> lldb_private::Address. I am open to suggestions though. So your though it
>>> remove all API that take lldb::addr_t and use lldb_private::Address
>>> everywhere all the time?
>>>>> It has always bugged me that we have these two ways of specifying
>>> addresses.  Are there many/any places that have to resolve an Address to a
>>> real address in a process that don't have a Target readily available?  That
>>> would surprise me.  I would much rather centralize on one way than adding a
>>> third.
>>>>> Jim
>>>> So, does it make sense to start with lldb::addr_t replacement? In other
>>>> words, replace all instances of lldb::addr_t with Address. It'd be the
>>>> first step and first patch towards to the ability to extend it in the
>>>> future, right?
>>>> Zdenek
>>> 
>>>>> 
>>>>>>> Jim
>>>>>>> 
>>>>>>> 
>>>>>>>> We would need a default value for "space" (feel free to rename) that
>>> indicates the default address space as most of our architectures would not
>>> need this support. If we added a constructor like:
>>>>>>>> SpaceAddress(lldb::addr_t a) : addr(a), space(kNoSpace) {}
>>>>>>>> 
>>>>>>>> Then all usages of the APIs that used to take just a "lldb::addr_t"
>>> would implicitly call this constructor and continue to act as needed. Then
>>> we would need to allow lldb_private::Address objects to resolve to a
>>> SpaceAddress:
>>>>>>>> SpaceAddress lldb_private::Address::GetSpaceAddress(Target *target)
>>> const;
>>>>>>>> Since each lldb_private::Address has a section and each section
>>> knows its address space. Then the tricky part is finding all locations in
>>> the expression parser and converting those to track and use SpaceAddress.
>>> We would probably need to modify the allocate memory packets in the RSP
>>> protocol to be able to allocate memory in any address space as well.
>>>>>>>> I didn't spend much time think about correct names above, so feel
>>> free to suggest alternate naming.
>>>>>>>> Best advice:
>>>>>>>> - make things "just work" to keep changes to a minimum and allowing
>>> lldb::addr_t to implicitly convert to a SpaceAddress easily
>>>>>>>> - when modifying RSP, make sure to check for existence of new
>>> feature before enabling it
>>>>>>>> - query for address space names so when we dump SpaceAddress we can
>>> show something that means something to the user. This means we would need
>>> to query the address space names from the current lldb_private::Process for
>>> display.
>>>>>>>> Submitting might go easier if we break it down into chunks:
>>>>>>>> 1 - add SpaceAddress and modify all needed APIs to use it
>>>>>>>> 2 - add ProcessGDBRemote changes that enable this support
>>>>>>>> 
>>>>>>>> It will be great to support this as a first class citizen within
>>> LLDB. You might ask the Hexagon folks if they have done anything in case
>>> they already support this is some shape or form.
>>>>>>>> Greg Clayton
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> lldb-dev mailing list
>>>>>>>> lldb-dev at lists.llvm.org
>>>>>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>>>> _______________________________________________
>>>> lldb-dev mailing list
>>>> lldb-dev at lists.llvm.org
>>>> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev
>> 
> 
> _______________________________________________
> lldb-dev mailing list
> lldb-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/lldb-dev