Re: [hatari-devel] Automated testing of Atari software with Hatari - Some tips

[ Thread Index | Date Index | More Archives ]


On 3/8/19 8:48 PM, Christian Zietz wrote:
recently, I was working on a low-level software routine that I wanted to
test for compatibility with as many TOS versions as possible. In order
to automate this task, I put Hatari to use. I want to give a short
summary as a reference in case someone else wants to do the same.

* Writing the test:
I wrote some test cases that exercise my routine and detect whether the
tests pass or fail. To output the status, I considered two options:
- Regular output to the screen via GEMDOS function Cconws.
- Output to the MIDI port via XBIOS function Midiws, as suggested by Eero.
In the end, I decided to keep both in. In case I want to run the test on
actual hardware, I'll still have the screen output.

I forgot to mention that one more option besides midi/serial/printer,
is using NatFeats STDERR:

That would work with Aranym too, but requires on Hatari
"--natfeats on" option.

The only Hatari-specific addition is at the end of my test cases: a call
to XBIOS function #255, passing a pointer to the string
"hatari-debug q\n" on the stack. This will terminate Hatari, when BIOS
intercept is on.

BIOS intercept is deprecated (because it provides too much control
for a thing running under emulation; if it wants, it could use that
to overwrite any file which Hatari process has write access to).

Please consider using NF_SHUTDOWN or NF_EXIT instead:

like Hatari's automated tests do.

FYI: NF_SHUTDOWN is supported by Aranym, but NF_EXIT is better
for automated tests because app can tell what Hatari's return
value should be.

Again, I also call GEMDOS function Pterm afterwards, in
case the test runs on real HW.

* Running the test:
Since my routine does not need any AES functions, I can run it from the
AUTO folder. Hence I created such a directory and copied my test.prg
into it. A GEM program could presumably be run with the "--auto" option

--auto option is needed only if you want to run the program from
something else than GEMDOS HD, otherwise you can just give
the program as an argument to Hatari.

If you want to pass arguments to the Atari program you run under
Hatari, use "hatari-prg-args" helper tool.

I call Hatari from the directory containing my AUTO folder with...

hatari -d . -t TOS --bios-intercept on --midi-out midi.txt --log-level
fatal --alert-level fatal --fast-forward on --run-vbls 7500 --fast-boot on
... where "TOS" is replaced by the image of the TOS version I want to
test with.

If you want more speed, use also:
	--fastfdc on --timer-d on --sound off

(I.e. disable cycle accurate FDC emu, patch timer-D, and don't
process sound with SDL mixer and whatever's behind that on host.)

If you aren't interested in visual output, you can disable
also host output conversions for video frames:
	--disable-video yes

(In reality this runs in a loop over a lot of different TOS
versions.) Further options could be added to specify amount of RAM,
screen type, etc., as needed by the test cases.

The "--run-vbls 7500" option is so that if my test crashes, Hatari does
still eventually terminate. If your test runs for a long time, you need
to increase the number. Most other options are to speed Hatari up and to
remove any clutter/distraction from the output.

If I hadn't been interested in TOS 1.00, I could also have skipped the
MIDI output and redirection and I could have added "--conout 2" instead
to the command line to intercept the VT52 console output. However, due
to limitations of TOS 1.00, this does not work; which is why I used the
MIDI port as suggested by Eero. (He mentioned that for this to work
Hatari needs to be built without portmidi support. Fortunately, this is
the case for the Windows version, anyway.)

Hatari will overwrite the MIDI output file ("midi.txt" in my case), so
after each test run, I append its contents to my test log.

With all that I can automatically verify that my routine passes all test
cases under all considered TOS versions.

Did you consider just forking Hatari's

	- Eero

Mail converted by MHonArc 2.6.19+