Re: [hatari-devel] Basic cpu testsuite

[Eero Tamminen <oak@xxxxxxxxxxxxxx>]

Hi,

On 04/30/2018 12:50 AM, Thorsten Otto wrote:
> On Sonntag, 29. April 2018 20:10:21 CEST Eero Tamminen wrote:
> > What kind of tests should be done for instructions?
> > Is there some way to auto-generate them?
>
> Hm, that will be difficult. Maybe you can come with something that reads the
> table68k, then generates one instruction for each combination of allowed
> addressing modes.

I have some Python code where I've typed down the instruction
bits, which may (or may not) be of little help in test
generation:
	http://eerott.mbnet.fi/hatari/sources/instructions.py

(It's interactive program for getting opcode masks for
Hatari instruction breakpoints.)


> The question is where such code should get its input values
> from, and how to verify the results.

The reason why ask, is that unless such a test(-suite) has good
coverage, it may be more trouble than it's worth.  And you
won't get good coverage for an instruction set without some
kind of test auto-generation.

Without good coverage, it doesn't catch errors that wouldn't be
noticed otherwise, and it doesn't help enough in narrowing down
the issue.


> Also, if you look at the tests, you may notice that they currently also try to
> test some corner cases that might not even be relevant for "real" programs.

If tests are written manually, covering some "relevant"
corner-cases which regressions might not be otherwise
noticed, could be a good strategy. I think that would
need input from Toni & Nicolas on what could be relevant
though.


	- Eero