Re: [hatari-devel] Character conversion for filenames in GEMDOS HD emulation

[Thomas Huth <th.huth@xxxxxx>]

Am Tue, 15 Jul 2014 08:19:49 +0200
schrieb Max Böhm <mboehm3@xxxxxxxxx>:

> Hi Eero,
> 
> >> Here is a patch which integrates these functions into Hatari:
> >> https://gist.github.com/a8b0ba0aa6d3bc8adab5
> >
> > Regarding this part of your patch:
> > * ommented out because this is risky. When deleting files which
> > * contain invalid characters, the inserted wildcard '?' may match
> > * also on other files, and an arbitrary file may be deleted.
> >
> > What kind of use-case you had in mind?
> >
> > The file you get on emulated Atari and select to be deleted,
> > should be the same that gets deleted.  Tools that use wildcards
> > for this kind of things are typically under MiNT, and that
> > bypasses GEMDOS emulation.
> 
> To test the character conversion I have written a small program which
> creates files with special characters:
> 
> #include <stdio.h>
> #include <tos.h>
> 
> void main(void)
> {
>     char name[20];
>     int i, fd, err;
>     Dcreate("C:\\TEST");
>     for (i=32; i<256; i++) {
>         sprintf(name, "C:\\TEST\\%c.TXT", i);
>         fd = Fcreate(name, 0);
>         err = fd<0;
>         if (!err)
>             err = Fclose(fd);
>         printf("%3d - '%s' - %s\n", i, name, err?"err":"ok");
>     }
>     exit(0);
> }
> 
> When I run this on the emulated Atari, I get a file <character>.TXT
> for each character (code 32..255) of the AtariST character set. Files
> which special characters are displayed as "@.TXT" by the GEMDOS HD
> emulation.
> 
> When I now try to delete some of those files by dragging them into
> the trash icon, the GEMDOS emulation replaces "@.TXT" by "?.TXT".
> But this pattern matches on all files in the folder, so arbitrary
> files are deleted.

Eero, could you please have a look at this? This sounds like quite a
severe bug in the GEMDOS emulation...

> With the character conversion patch the files won't be shown as
> "@.TXT" in the first place and therefore I think the replacement by
> "?.TXT" to match invalid characters will no longer be necessary.

Max, I now had a closer look at your patch, and I think it's basically
a good approach, but there are some things that I'd like to discuss:

1) I really dislike this part in gemdos.c:
#ifdef WIN32
 Str_AtariToWindows(pszFileName, pszFileNameHost, INVALID_CHAR);
#else
 Str_AtariToUtf8(pszFileName, pszFileNameHost);
#endif
In the end, there is no need to export both functions to other files,
so I think it would be better to have a "Str_AtariToHost" and a
"Str_HostToAtari" where the implementation in str.c is taking care of
the differences instead.

2) The extra step with mapWindowsToUnicode looks cumbersome ... why
don't you add a proper mapAtariToWindows table directly instead?

3) Str_AtariToUtf8 can create a destination string that is "longer" than
the source, since UTF8 characters can take multiple bytes, right? There
seems to be at least one hunk in your patch where you don't take this
into account so the destination buffer could overflow.

4) What if the (Linux) host sytem does not use a UTF-8 locale? I think
there might still be some people around who use some latin-x locale
instead.

 Thomas