Re: [hatari-devel] Things still remaining before v1.8 release?

[Max Böhm <mboehm3@xxxxxxxxx>]

Hello Eero,

Thomas wrote:
> OS X should be fine, and concerning the UTF-8 patch, this change is
> IMHO too big to be included so close to the next release that I'd
> rather postpone it for later. But I'd leave that decision to Eero since
> he was mainly working on the GEMDOS HD emulation (especially the name
> matching) in the past months... Eero, what do you think?

It would be nice if you could review the patch and decide if this can
be included in the 1.8 version.

I have validated the patch on Windows/Linux/OSX as described below.
During the validation I found that still an aspect of the OSX related
UTF-8 conversion was incorrect and therefore have updated again my
patch (sorry for so many updates, this is hopefully the final version now!):

GEMDOS TOS name <-> UTF-8 conversion patch:
https://gist.github.com/2610cd9df21dc827fe45


Details on the validation
-------------------------
I've created a test program to be run in the emulated system which
first creates one file for each character code 32..255 (using Fcreate).
In a second step, after you press a key, it deletes the files (using
Fsfirst and Fdelete). It writes a logfile "atari.log" of the form:

hexcode - 'filename' - <status of Fcreate>
....
hexcode - 'filename' - <status of Fsfirst>, <status of Fdelete>

for example:
....
80 - 'L:\TEST\80_Ç.TXT' - created
81 - 'L:\TEST\81_ü.TXT' - created
82 - 'L:\TEST\82_é.TXT' - created
83 - 'L:\TEST\83_â.TXT' - created
....
80 - 'L:\TEST\80_Ç.TXT' - found, deleted
81 - 'L:\TEST\81_ü.TXT' - found, deleted
82 - 'L:\TEST\82_é.TXT' - found, deleted
83 - 'L:\TEST\83_â.TXT' - found, deleted
....

You find the source code of the test program here:
https://gist.github.com/6c7c4340e2a656b2066b

Then I've run this program on the hg Hatari version (to which I have applied
my patch) on the platforms Windows 7, Ubuntu 14.04, and OSX 10.6.

The atari.log file created by the test program within the emulated
system uses the AtariST character map. To make it readable on the
host I've converted it to utf8 using the recode utility:

recode AtariST..UTF-8 <atari.log >atari.log-utf8

On each platform between the two steps I listed the contents of
the TEST directory on the host.

Windows: dir /B /O:N TEST >windows.log
Linux:   ls TEST >linux.log
OSX:     ls TEST >osx.log

The results of the validation for each platform (atari.log and host.log)
can be found here:

Windows: https://gist.github.com/5b0c1f311860829fb04f
Linux:   https://gist.github.com/dd23685a872ebd1c544b
OSX:     https://gist.github.com/bee6f92b11fe695e9430

This shows that the character mapping works as expected.

===================================

During the testing I noticed a few other things in the GEMDOS layer:

I noticed that the GEMDOS emulation layer can create files on the host
with certain characters in the pathname which cannot be found (by
Fsfirst) or deleted (by Fdelete) thereafter.

Characters on Windows:    " * / : < > ? \ |
Characters on Linux/OSX:  / \

I assume this is intentional.

But one other behaviour of the GEMDOS emulation is not fully clear to
me. The GEMDOS layer replaces certain characters by '+' when returned
by Fsfirst/Fsnext:

' * . : ? { } 0x7F

I don't understand why this is done. As a result those files are not
shown with their original character in the emulated system and you
get an error when you try to drag the file into another folder or into the
Trash icon.

I know the GEMDOS layer had a wildcard '?' inserted to catch such cases,
but this would match on other files too, which is why I commented it out.
In my view with my patch the replacement of special characters by '+'
would no longer be needed at all, what do you think?

Regards,
Max