Re: [hatari-devel] New option --disable-video and --benchmark

[ Thread Index | Date Index | More lists.tuxfamily.org/hatari-devel Archives ]


Hi,

On 02/26/2017 07:21 PM, Nicolas Pomarède wrote:
Le 26/02/2017 à 13:37, Eero Tamminen a écrit :
see the values I posted earlier, disabling the SDL calls *does* impact
performance I go from 200 VBL/s to 275 VBL/s, that's clear performance
advantage.

I gave numbers that show that there's no real performance
or CPU usage benefit from disabling video output, compared
to using Hatari options which don't have the disadvantage
of losing the output.

Did you try them?


> Also, I don't want to disable accuracy by removing borders
> emulation or disabling spec512 mode, that's not my goal.

I don't understand how that's relevant.  When you disable video,
you're not seeing the border content or spec512 palette changes
anyway, so you don't lose anything extra compared to video disabling.

(Those options don't affect Atari side; music playback or audio quality, they just lower host-side screen conversion overhead.)


I want to compare different rendering method for cpu/video when all
parameters are set to the max (CE mode for CPU, full shifter emulation
and so on). So disabling border or spec512 or using zoom x1 instead of
x2 is not an option.

If you want to benchmark screen conversion routines, you
obviously want to maximize that part's overhead to see
the effect of optimizations better.

But how that's relevant for --disable-video discussion?

"--disable-video" option is last thing you would want to do
for such measurements, as that will minimize[1] the conversions
which you wanted to measure???

[1] = same as frameskip, border & spec512 mode disabling does.


> benchmarking on an idle gem doesn't look like the most accurate way to
> measure all aspects of emulation. I do my tests on some demos where
> screen changes completely on each frame, so all lines will be
> converted/rendered on each frame.

If you tell which music playback program you had in mind,
I can test that also.


>> As to VBL counts with:
>>  --benchmark --run-vbls 2400 --machine st --tos tos104.img --borders
off --fast-forward on
>>
>> I do get measurable differences:
>> * 474 VBL/s --disable-video
>> * 460 VBL/s --disable-video --frameskips 0
>> * 465 VBL/s SDL_VIDEODRIVER=dummy --frameskips 0
>> * 426 VBL/s --frameskips 0 --borders on
>> * 444 VBL/s --frameskips 0
>> * 468 VBL/s --frameskips 4
>> * 474 VBL/s --frameskips 4 -z 1
>> * 477 VBL/s SDL_VIDEODRIVER=dummy --frameskips 4 -z 1
>>
With SDL1 Hatari build (and EmuTOS), the CPU usage results were
quite unexpected:
* 8.5% SDL_VIDEODRIVER=dummy --frameskips 0
* 7.5% --disable-video on --frameskips 0
* 7.0% --frameskips 0
* 6.8% --frameskips 4
....
SDL1 is not my target for benchmark, if performance are worse/counter
intuitive when disabling SDL calls under SDL1, then I don't know, I'm
using SDL2 anyway.

Sure, I just included that just in case (I didn't know which
version of SDL you're using). :-)


Conclusions:

* At least on my setup, --disable-video brings no performance
  advantages, just huge usability disadvantage. Better option is
  to use options already mentioned in Hatari manual performance
  section and use EmuTOS

* Surprisingly, video updates don't seem to have much impact on
  CPU usage, unlike ~10 years ago on Nokia's mobile ARM devices,
  which were the reason why I originally added frameskip support.
  I guess things are better optimized on SDL & HW side nowadays,
  and Hatari's CPU emulation takes larger proportion of CPU than
  decade ago

My metrics is not to measure cpu usage at normal execution speed,
because it's very difficult to tell the difference between 10 or 15% cpu
usage.
That's why I do the benchmark at full CPU speed (no VBL wait on the host
side), then in the end you have a number of VBL/sec that is more
accurate to compare things (ie make the host CPU runs as fast as
possible for emulation, then compare the time it took to generate the
same number of frames)

I provided both.  CPU usage was there as the claim was that
--disable-video helped with that.


Once again, if you run Hatari without any UI, then you loose all
keyboard input. With the UI and --disable-video, you could run sndh
player and change subsong with the corresponding key. Or you can run the
B.I.G. demo, just press up/down/enter and change music.

Please tell why one would want to disable video for
that use-case?

I showed that there's no real performance nor CPU usage advantage
from that, so one can as well run it will frameskip so that one
sees what one's interacting with.


I'd like to have all keys shortcuts while Hatari is running ; having
just the window created at start but not refreshed later gives a quick
solution to this : shortcuts still work, but SDL calls like Copy/Render
don't interfere with the benchmarking.

Having benchmarks that require manual intervention (besides
interrupting them if there's some problem), sounds really
something that should be fixed before doing benchmarking.

I didn't say it required it, just that I want to be able to press F12
during the benchmark if needed or press pause for example.

How long your benchmark runs are?

(I'll normally try to keep them max 1 minute, and then just
repeat them more times to get variance info.)


It can replace the redundant --disable-video. :-)

No, it can't. dummy driver clearly doesn't work in my case. Maybe we
have different HW setup, maybe it depends on the renderer used by SDL2
(GL, SW, ...), but in my case dummy gain is marginal (5-10 %), while
disabling SDL calls gives 35%.

Sorry, I was unclear.  By redundant I didn't that --disable-video
is redundant only if --no-gui is added (dummy video).  I meant that
--disable-video is redundant as-is.

(Whereas --no-gui would genuinely provides something new, but
whether that's needed, is another matter.)


On another more powerful PC I have, running the same demo with
--benchmark and --disable-video 0 or 1 goes from 950 VBL/s to 1600
VBL/s, so nearly 60% increase by disabling SDL calls for Copy/Render.

To summarize :
 --disable-video solve my problem of disabling as much SDL/OS calls as
possible, which driver=dummy doesn't do (at least on several PC I have
acces to)

Please try the options I listed in my previous mail
(and what are documented in Hatari manual).


 --benchmark does the same as fast-forward (ie don't wait for VBL on
host side) *for now*, but considering I might add different behaviour
later, I'd rather have a separate option since the beginning (and this
way, if I break --benchmark, at least it won't break fast-forward)

--benchmark option is OK.

I have issue only with video disabling usefulness claims,
and what you're (not) comparing it with.


	- Eero



Mail converted by MHonArc 2.6.19+ http://listengine.tuxfamily.org/