|Re: [hatari-devel] The new "make test" automatic regression test suite|
[ Thread Index |
| More lists.tuxfamily.org/hatari-devel Archives
On 04/15/2018 01:09 PM, Thomas Huth wrote:
Am Sun, 15 Apr 2018 12:47:43 +0300
schrieb Eero Tamminen <oak@xxxxxxxxxxxxxx>:
On 04/15/2018 12:02 PM, Thomas Huth wrote:
I don't think that this is a good idea for the "built-in" regression
test suite. You want *reproducible* tests there, and if everybody is
using a different TOS version, you can not really achieve this
Of course it's reproducible. Testing just needs to record the used
TOS version, so one knows which versions is needed for reproducing.
There are also bad and patched ROM dumps available in the internet ...
so just recording the TOS version would not be enough. You'd also need
to log/verify the md5sum of such files to make sure that you have the
Good point. I was thinking that we Hatari developers could split
testing of different TOS versions between us, but even for that,
it would be good to make sure that when crosschecking results,
images really are the same.