Re: [hatari-devel] memory snapshot format/issues

[David Savinkoff <dsavnkff@xxxxxxxxx>]

----- Eero Tamminen wrote:
> Hi,
> 
> On sunnuntai 10 toukokuu 2015, David Savinkoff wrote:
> > ----- Nicolas Pomarède wrote:
> > > Le 09/05/2015 23:03, Nicolas Pomarède a écrit :
> > > > Le 09/05/2015 21:04, David Savinkoff a écrit :
> > > >>> every component in it's own structure, doing a kind of linked list :
> > > >>>    - component 1 id
> > > >>>    - component 1 size
> > > >>>    - component 1 data
> > > >>> 
> > > >>> ...
> > > >>> 
> > > >>>    - component n id
> > > >>>    - component n size
> > > >>>    - component n data
> 
> Sounds good, but IMHO ID should be user-readable string, see below.
> 
> 
> > > >> Here are a few more suggestions:
> > > >>    - component n id (maybe the variable name as text)
> > > >>    - component n size (size of the data type)
> > > >>    - component n nmemb (number or elements -- see: $man fread)
> > > >>    - component n data
> > > >> 
> > > >> MemorySnapShot_Store(&Control, sizeof( Control ));
> > > >> would become:
> > > >> MemorySnapShot_Store("Control", sizeof( Control ), nmemb, &Control);
> > > >> 
> > > >> This format would allow the format to be endian independent because
> > > >> the file would be written as bytes (byte file access will never go
> > > >> away).
> 
> Not enough, any >1byte values need to have endian conversion
> either with libSDL or netinet.h functions.
> 

I believe it would be best to do your own endian independent conversions.
This can be done purely mathematically by shifting or modulus/dividing
into bytes -- or with toascii() --, then writing the bytes.

> 
> > > >> The snapshot file can even be alphabetically insertion-sorted
> > > >> (quicksort) as it is made so that it can be quickly parsed for
> > > >> collisions, errors etc.
> 
> Would require allocating space for it in RAM first so that the items
> are all present for sorting.  IMHO not necessary in Hatari's own
> snapshot reading code.
> 

Sorting the file makes a rigid unambiguous specification which
prevents abuse. To be robust; Hatari should request the data it
needs, Not relying on the order it was stored.

> 
> > > >> Saving structures with different data types will require the "name"
> > > >> text to identify each element. This doesn't have to cause problems
> > > >> with an alphabetically sorted list, as the full path can be given.
> > > > 
> > > > I thought also about adding the variable's name to each saved
> > > > variable, but I'm not sure it's really necessary, if we save/restore
> > > > variables always in the same order and we add new variables at the
> > > > end of the existing component, then knowing the name of the variable
> > > > is not necessary IMO.
> > > 
> > > I saw that for component n you wrote "maybe the variable name", but in
> > > my idea a component would be the complete set of variables used by an
> > > HW chip for example (video, fdc, cpu, ym, ...)
> 
> If everything is named, it can be easily converted to other, more
> portable, e.g. ASCII based formats, and it's easier to write some
> file format conversion scripts in higher level languages.
> 
> E.g. after release, there could be come a script that converts previous
> version save files to new format; adds default values for new
> memory snapshot items etc.  If field names change, scripts can
> handle that too.
> 
> (In the portable format, data like memory contents would probably
> be separate files.)
> 
> 
> > I wrote "maybe the variable name" because I think the 'id' field should
> > be a descriptive text string associated with the data it represents.
> > 
> > I thought more about floating point variables. Maybe they should be
> > formatted with fprintf and stored as a text string also. Maybe the
> > complete memory snapshot file should be a text file (alphabetically
> > sorted also).
> 
> What items in the snapshot are in floating point?
> 
> 
> > The memory snapshot file should not have an implicit structure where
> > arbitrary structures are used in the current-to-be-obsolete save-order.
> > 
> > The memory snapshot file is compressed, thus it doesn't need to have
> > an old fashioned 'move multiple registers to stack frame' look to it.
> > 
> > A general purpose Memory Snapshot file would have the machine state
> > saved in an insightful manner considering future uses.
> > 
> > There should be discussion on this matter so that a specification is in
> > place before the code.
> 
> I might try coming up with something in next few days.
> 
> 
> 	- Eero
> 
>