Re: [hatari-devel] hatari release tester (was: problems in b/w mode)

[Eero Tamminen <oak@xxxxxxxxxxxxxx>]

Hi,

On lauantai 07 tammikuu 2012, Nicolas Pomarède wrote:
> in the specific case of Hatari not working in mono anymore, is it really
> important to know that tos A or tos B works or not ? I have the feeling
> that if a tos works, it's just a side effect due to a different code in
> tos and some different timings.
>
> The mono bug in Hatari can have different effects depending on the TOS /
> machine configuration, but in that case, I don't think knowing all those
> combinations will help ; I think the bug should be hunted around the
> revisions I gave earlier, having just one non working configuration
> should be enough.

I want to make sure there aren't any other trivially detectable
regressions in Hatari v1.6 that would need to be fixed for v1.6.1
release.

The code is meant as a generic Hatari release tester, not as
a test-case for just this particular mono bug.

(I just pushed it to repo)

 
> (on a more general test case, I agree that comparing selected
> screenshots can help verifying regression in an automated way)

So, any comments on what combinations would be good to test? :-)

If e.g. "--fast-boot off" would more robust, the tester could be
used to verify that it works fine with all combinations below.


>> The attached version will go through:
>> * given TOS images

I would propose testing following versions to get good enough
coverage:
  v1.00 de, v1.02 de, v1.04 de, v1.04 us, v1.62 de, v1.62 us,
  v2.06 de, v3.06 us, v4.04, kaostos, etos192k, etos512k[1].

[1] Just the latest version of EmuTOS.


>> * st, ste, tt, falcon machine types

EmuTOS 512k would cover all machine types, EmuTOS 256k and TOS v2
all except Falcon[2], rest of TOSes would be limited to single
machine type.


>> * tv, vga, rgb, mono, 1 plane vdi, 4 planes vdi

ST:
  tv, mono, vdi-1, vdi-4
STE:
  rgb, mono, vdi-1, vdi-4
TT:
  rgb, mono, vdi-1, vdi-4
Falcon:
  rgb, vga, mono

Or does TT use VGA instead of RGB monitors?

VDI mode emulation doesn't work with TOS v4 and with Videl's
programmability it's anyway unnecessary.


>> * 1, 4, 14 MB of memory

ST & STE:
  1 & 4 MB
TT & Falcon:
  1 & 14 MB

Or do you think some other/additional memory configurations
would need to be tested?  Or that one could use some base
config with different amounts of memory instead of testing
them with all of the other combinations?


>> * GEMDOS emulation on/off

Tested for all of above combinations.


I've run all these combinations for Hatari v1.4 and they all
worked fine.

[2]
Testing takes >1/2h and produces well about 300 screenshots,
about half of them for EmuTOS and TOS v2.06 as they work
with so many machine configurations.


	- Eero