[llvm-dev] Clang for the PlayStation 2

[Tim Northover via llvm-dev]

Hi Dan,

On Sun, 2 Sep 2018 at 22:14, Dan Ravensloft via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
> I then tried to compile newlib 3.0.0-20180802, as newlib is used as the standard C library for the PS2, and hit multiple asserts, which I have attached below.

After a quick look, three of the issues seem to be related to "long
double", which is translated to fp128:

vfprintf: invalid bitcast from f128 to f64. Part of some call, maybe?
ldtoa: obviously a function taking "long double".
vfscanf: Mishandling return of f128 from a libcall.

Now, mapping long double to fp128 is actually a choice you have as an
ABI creator. It's possible you feel constrained to follow GCC here and
so need that, but if not the simplest fix might well be to just
declare long double is the same as double. PS2 doesn't strike me as a
system where you'd often want a 128-bit float. You'd do that by
modifying tools/clang/lib/Basic/Targets/Mips.h (the setN32N64ABITypes
function, notice FreeBSD has already done this).

If you do have to support f128, you'd probably start by focusing on
MipsISelLowering.cpp. Specifically the functions named things like
LowerFormalArguments, LowerCall, LowerReturn. I'm not sure what the
ABI is likely to be there, but I suspect you'll end up assigning an
fp128 to 2 64-bit integer registers. If needed I'm sure me or a Mips
expert could provide more details.

The other problem (lrintfp) seems to involve an invalid truncation
from f32 to f64, possibly inserted by code unaware of the single-float
option. Nothing looks obviously wrong, so it's the usual debugging
procedure:

1. Try to produce a reduced test-case
2. gdb/lldb it, since not many places produce the Mips-specific
TruncIntFP node that's causing the problem.

For 1, I've actually already got one for you:

void foo(float *in, long long *out) {
  *out = *in;
}

In brief, how I got it was:
1. Run lldb on the crashing Clang command. Go up the backtrace until I
got usable C++ code. DAG.viewGraph() produces a very nice pictorial
representation of the code being selected, it had load(f32) ->
MipsISD::TruncIntFP (f64) -> store. That middle trunc is obviously
really dodgy.
2. From there you could try to write IR that did the same thing.
Instead I printed "BlockName" that was available in one of the frames.
Then I dumped the IR from Clang (-emit-llvm) and just looked at that
IR snippet:

  %23 = load float, float* %x.addr, align 4
  %conv33 = fptosi float %23 to i64
  store i64 %conv33, i64* %retval, align 8
  br label %return

>From there it's a lot easier to write down the C function I gave,
which is what I did.

So the next step is to debug where Mips is producing those TruncIntFP
nodes. There'll be some constraint it's not checking or an unexpected
node type, probably related to -msingle-float. I'm afraid I'm not sure
what yet.

Cheers.

Tim.