[lldb-dev] Expressions aren't evaluated with respect to target's endianness

[Félix Cloutier]

Hello people again,

Still working on that LLDB stub controlling a big-endian target. I'm working on evaluating expressions. I just realized that lldb doesn't respect the target byte order when it saves expression results.

This expression:

    expr int $x = 4;

causes lldb to allocate memory in the inferior process then write 04 00 00 00 to it, which is little-endian for 4, even though my target is big endian.

The ClangUserExpression was initialized with the correct byte order, but things seem to go wrong with IRInterpreter::ResolveConstant, which calls APInt::getRawData(), returning a pointer to an uint64_t in host byte order. It then calls m_memory_map.WriteMemory, casting the uint64_t* into a uint8_t*, and the rest is history.

I guessed from the previous endianness bugs I reported that it could be fixed with a SetByteOrder call, but turns out that there's no DataExtractor involved. It could be swapped in IRMemoryMap::ResolveConstant, but I'm not sure that's the correct thing to do.

Félix