| Re: [hatari-devel] The new "make test" automatic regression test suite |
[ Thread Index |
Date Index
| More lists.tuxfamily.org/hatari-devel Archives
]
Hi,
On 04/14/2018 06:24 PM, Thomas Huth wrote:
Am Sat, 14 Apr 2018 17:02:54 +0200
schrieb Thorsten Otto <admin@xxxxxxxxxxx>:
On Samstag, 14. April 2018 14:05:52 CEST Thomas Huth wrote:
The disadvantage here is that you can only use PRG files that use
the supported subset of the GEMDOS emulation layer in Hatari.
But that also means your test environment is quite different from
"normal" usage. Actually, a lot of Hatari cannot be testet this way
(i didn't look at your mini-ROM yet, but i doubt that it tries to do
much hardware initialization, like a real TOS would do). You don't
even know if it is able to boot the ROM, according to the settings.
That's not the purpose of the small test PRGs that we have in the tests
folder: They check special stuff like bus errors, certain CPU opcodes
(natfeats) or emulated OS calls (xbios) - and for this you certainly
don't need a full-blown TOS ROM.
For testing the boot process of real TOS ROMs, we also have got the
"tosboot" tester. But for this, you need to point it to a set of TOS
ROM images that should be tested, so this is not includable in the
automatic test suite.
Maybe Troed's system could run tosboot tester with few TOS ROMs
e.g. once a week or month, and mail the report to Hatari devel
(with link to directory containing the screenshots)?
PS.: once the tests run automatically, it maybe be even possible to
download some "official" ROM from a known location. You will need an
internet connection anyway to clone the repo, and as long as you are
not distributing the ROMs, there should be no legal issues (at least
i think so)
>
First, mercurial can also be used offline as long as you don't want to
push/fetch stuff to/from the repository server. And believe it or not,
sometimes I even still work without internet, e.g. while traveling.
Additionally, adding a hard dependency on a third-party website which
is distributing legally debatable content should be really avoided. So
no, this idea is definitely a no-go.
I don't see why ROM would need to be in repository, or fetched
automatically.
Tests needing real ROM could be executed only if there's a TOS ROM
in some specific locations, otherwise there would be just a warning
about missing the ROM, and a pointer where to get EmuTOS one.
Good point of this scheme is that everybody could be using his
own favorite TOS version for this, so there would be better
coverage. These tests might also use user's own Hatari config
for them, to increase coverage further.
If test requires some specific TOS version, it should check
that correct one is used, and skip otherwise.
Thomas, what do you think?
- Eero