[hatari-devel] UAE CPU core based CPU tester/validator

[Toni Wilen <twilen@xxxxxxxxxx>]

Hi

I finally wrote a CPU tester/validator. UAE CPU (gencpu) based as usual :)

It has two projects: cputester generator that runs natively, creates
test data files using cputestgen.ini. m68k native (Amiga currently only
but should be easy to port) simple plain gcc C project that runs on
tested hardware and uses generator created test data to run and validate
instruction(s).

I originally made this to compare and validate UAE CPU against real CPU
but it should be also useful for testing other m68k emulators or FPGA
reimplementations.

It can do following:

- All CPU register contents verified (D0-D7/A0-A7, PC and SR/CCR, MSP)
- Memory writes, including stack modifications verified (if any)
- Loop mode for JIT testing. (generates <test instruction>, dbf dn,loop)
- All 68000 and 68020 instructions supported (even conditional branch
instructions)
- FPU instructions supported, registers verified (this is still WIP)
- Exceptions verified, including 68000 data address errors.

It brute force generates pseudo-randomly single-instruction tests
(millions of tests if you want..) with randomized EA, all addressing
modes, all register combinations, all flag combinations, even all SR bit
combinations (T0/T1/S/M) will be tested if needed.

68020 addressing modes are also tested optionally.

Notes and limitations:

- STOP testing skips all STOP immediate values that would stop the CPU.
- RESET should not be tested in supervisor mode :)
- Single instruction test set will take long time to run on real 68000.
Few minutes to hours..
- Undefined flags (for example DIV and CHK) are also verified. It
probably would be good idea to optionally filter them out. (But this is
not trivial because sometimes they are undefined, sometimes they are not)

TODO:

- Verify exception stack frame contents. Currently only stacked SR and
PC is verified.
- Bus errors?

Here is example of failing loop mode test: (I am not sure if this WinUAE
JIT only problem or generic JIT problem.)

ROR.W:
data/68020/ror.w/0000.dat. 0...

832: e27a 51cf fffc ror.w d1,d2
SR: expected 0000 -> 0008 but got 0009
Registers before:
D0: 00000010 D1: 00000000 D2: ffffffff D3: ffffff00
D4: ffff0000 D5: 80008080 D6: 7fff7fff D7:*00000009
A0: 00000000 A1: 00000080 A2: 00008000 A3: 00007fff
A4: fffffffe A5: ffffff00 A6: 007bff00 A7: 007f7ff8
SR:*0000   PC: 007c0000 ISP: 007fff80 MSP: 00800000
T=0 S=0 X=0 N*0 Z=0 V=0 C*0
Registers after:
D0: 00000010 D1: 00000000 D2: ffffffff D3: ffffff00
D4: ffff0000 D5: 80008080 D6: 7fff7fff D7:*0000ffff
A0: 00000000 A1: 00000080 A2: 00008000 A3: 00007fff
A4: fffffffe A5: ffffff00 A6: 007bff00 A7: 007f7ff8
SR:*0009   PC: 007c0006 ISP: 007fff80 MSP: 00800000
T=0 S=0 X=0 N*1 Z=0 V=0 C*1
OK: No exception generated

N and C flags got set unexpectedly. No errors in non-JIT mode or when
using real 68020 or 68030. (D7 changed from 9 to -1 because it was loop
counter)

68000 emulation is almost fully verified (for example CHK.W, DIV
undefined condition codes are now fully correct, LINK and UNLK order of
operations which makes visible behavior difference if register parameter
was A7)

Only difference that remains is MOVE.W/MOVE.L condition code setting,
different MOVE variants seem to set it in different phases which makes a
difference if instruction causes address error.