Hello Eero,
Sunday, December 16, 2012, 3:02:54 PM, you wrote:
Hi,
On sunnuntai 16 joulukuu 2012, George Nakos wrote:
Sunday, December 16, 2012, 12:04:40 PM, you wrote:
In many compression algorithms you only need a small buffer for
decompression.
Don't oversimplify things. Compression is (IMHO) a design decision not
to be taken lightly. For example, what speed is the depacking routine
running at? Does it depack in place or does it need an extra buffer to
store the unpacked data and then copy it in place? What impact does
that have on speed? Is that an acceptable wait for the player? And so
on and so forth.
Naturally one needs to take these into account. But you cannot just
assume things, because assumptions can often be wrong.
Well, I didn't assume things, I know ;). I've been with D-bug since
2006, producing menus and patching stuff for hard drive/68030
(http://dbug.kicks-ass.net/patch.php). I've witnessed Gauntlet 2,
which is about 1.4mb unpacked take 30 seconds to unpack fully. While
unpacking we usually had a black screen, so neither we or the user had
any indication whether something has crashed or not (D-Bug has given
up on depacking flashes from the old Automation years ;). I've also
seen that time halved or maybe less using ARJ mode 7. I've been bitten
colossally in the ass with UPX versions bigger than 3.03 when using
--ultra-brute as they changed that algorithm to something really more
aggressive and made the Prince of Persia cracktro take about 2 minutes
to depack - and we're talking about a program that packed down to 30
meagre kb.
This is more than enough evidence to me to conclude that on Atari
16/32 machines with hard drive, packing is really not worth it. If we
talk about floppy releases though, yes, everything goes there!
(Last ~6 years at work I've been doing system performance/resource
usage analysis, so I do have some clue about this stuff.)
As I stated, package data can also used to speed up things, it doesn't
necessarily slow down things.
If you could use DMA to directly read data to screen, uncompression slows
down things. If your mass media is much slower than your RAM & CPU, using
pre-compressed data can speed up things. So, what are your mass media
and RAM/CPU data throughput speeds?
Using DMA transfers on Falcon using a hard drive is simply not an
option. As for throughput speeds, I'm sure that my STE with hard drive
reaches about 1mb/sec, so it's safe to assume that the falcon is 2x or
3x faster than this (at least on IDE, SCSI should be a bit faster).
So, my ~2mb background file loads at 1 second or under.
But the thing is, you know how many complaints I had about that?
0 (zero).
Out of curiousity, how many user do you have? :-)
atari.org doesn't have any download stats. But, given that I've had
about 10 e-mails of support, and that people organized a compo of this
game on Siliventure last weekend without me knowing that went on for a
few hours, I'd say at least.... 110? :P
When we're doing stuff for the Jaguar, memory and storage space
really precious (2mb of RAM for single load games, or an extra 4mb if
making a cart version).
How fast the loads from the mass media are? Are they faster if you can
do linear loads? How fast memory copies are? Has Jag some SIMD
operations that can do copies with some transformations faster?
Well, off-the-shelf Jaguar doesn't have any mass media support, only
carts, or for people that are willing to mod their machine and upload
code from another machine, just the 2MB. Cartridge space is treated as
RAM for reads, so you get roughly the same speed. For mass media that
you have to buy a JagCD extension, which reads at a whopping 150k/s.
Add to that the bonus bug of the BIOS reads a bit more than you ask it
to (this is random, it's really up to the weather as far as our tests
have shown) and you've got a fine memory management mess in your hands
:). As for the copy/transform bit... Technically the data is read from
the CD into the Jerry chip (which handles sound), and then copied to
main RAM. So if a brave soul is willing to disassemble the bios and
add depacking code it should be possible. Bear in mind though that the
JagCD format has no error correction ("Mr. Tramiel! We found a totally
RAD way to cram even MOAR data than 650MB onto a single CD! We just
have to take out the useless error correction
data!!!!!!!!!!!111111onoenoenoeeoen"), so a wrong read will lead to a
nice depacking crash if things go well :)
(If loading from mass media is really slow, but there isn't enough RAM
for keeping all assets in RAM, one possibility would be to keep them as
compressed in RAM.)
That's what we did in Superfly DX
(http://reboot.atari.org/new-reboot/superflydx.html) - the game is
about 1.2mb big, but it literally uses up the whole RAM all the time,
unpacking data depending on what part is being shown (intro, menu,
main game, credits, etc etc)
And even so, we're using the packer with the fastest observed unpacking
time, Ray's lz77.
How fast that is compared to LZO (which is built with latest GCC from
Vincent at high optimization level)? LZO is supposed to be one of the
fastest decompression algorithms out there, and it offers pretty good
compression rates too. On ARM and Intel hand optimized versions get
fairly close to memcpy speed on decompression.
No idea. At one point where we were happy with our Jag toolset and
helper functions, we just locked them down and started doing stuff
with them! (http://reboot.atari.org/new-reboot/raptor.html). Also
worth mentioning that in Superfly DX the depacking is barely
noticeable throughout the game.