Re: [hatari-devel] GEMDOS filename handling

[Eero Tamminen <oak@xxxxxxxxxxxxxx>]

Hi,

On torstai 31 heinäkuu 2014, Eero Tamminen wrote:
> On torstai 31 heinäkuu 2014, Max Böhm wrote:
> > I found something else which may need attention: When you create a file
> > "atari.log" through the GEMDOC HD layer on the host and then copy
> > another file "atari.lo" into the same directory, the two files seem to
> > be mapped to the same host file by the GEMDOS layer. The emulated
> > system reports that "atari.lo" already exists although this should not
> > be the case.

Nicolas, although you asked not to do commits, this was nasty enough
that I pushed a fix to it.


Max, this was due to faulty logic for matching host names which
might be so long that they get clipped to 8+3 characters.  In
worst case matching pattern is:
   filename*.ext*

Such a pattern is used if part before extension is 8 chars, extension
is 3 chars and there was no match without a pattern.  Otherwise
host filenames clipped to 8+3 couldn't be matched from the clipped
Atari name back to host.  It's possible that it matches multiple
filenames, but that's expected as you will see also on Atari
side multiple filenames (with the same 8+3 name representation).


> > I've repeated the tests (1st and 2nd) under Linux for the TOS versions
> > "latest EmuTOS", 1.04, 1.62, 2.06, 3.06, 4.04 using GEMDOS HD
> > emulation. In addition I did the test (1st and 2nd) on an ACSI
> > harddisk image.
> > 
> > 1st Test: create files for all characters and then delete them by
> > specifying their full name.
> > 2nd Test: create files for all characters and then delete them
> > manually using the GEM Desktop.
> > 
> > The results are identical for all tested TOS versions. All created
> > files can be deleted in both cases.
> > 
> > The GEMDOS HD emulation can't create files with '\' or '/' in their
> > name. The real TOS can't create files with '\' in their name.
> 
> Excellent, thanks!

Actually the test wasn't good enough.  If you have the tested
character as last char before '.', that doesn't excercise all
the potential problems.  You need to have it there twice.


> >>> You find the source code of the test program here:
> >>> https://gist.github.com/6c7c4340e2a656b2066b
> >> 
> >> Good, this really needed a test-case.
> 
> But I just noticed that your tester tests just normal files.
> Could you add directory creation and removal to it too and
> check how those work with real TOS on ACSI or floppy image?

After improving the test (attached), and testing it with
TOS 2.x (results attached), I found out that even native
TOS has problems with files & directories having following
characters in them: ' ', '*', '.', '?', '\'.

While creation of directories with '?' and '*' characters
in them succeeds, one cannot enter them in Desktop nor delete
them.

' ' character is problematic only in files.  I.e. the change
for that in your patch won't work.

It needs still testing with other TOS versions and checking
that file names show up fine in desktop & file selectors,
but I don't currently have time for that unfortunately.
Maybe in a week.

Testing is best done with floppy images.  They work with all
TOS versions without need for HD drivers.  You just need
something like foldr300.prg in the AUTO folder.


> > >> But one other behaviour of the GEMDOS emulation is not fully clear
> > >> to me. The GEMDOS layer replaces certain characters by '+' when
> > >> returned by Fsfirst/Fsnext:
> > >> 
> > >> ' * . : ? { } 0x7F

So far "' { }" and 0x7F don't seem problematic for TOS.

Problem with host files having ':' in them is TOS potentially
interpreting them as virtual files (CON:, PRN: AUX:), but
this could be handled by converting ':' only at position 3.


[...]
> The other alternatives are:
> - instead of '?', using glob() pattern with '[]', which contains all
>   the invalid characters (after your 8-bit char patch, amount of invalid
>   chars is small enough to be handled like that).
> - or first trying with INVALID_CHAR and if that doesn't match, try
>   with pattern


	- Eero

/*
 * tester for what kind of characters are accepted by TOS in file
 * and folder names.
 * 
 * Original file tester written by Max Böhm, Eero Tamminen added
 * testing for folder creation/removal, option for leaving the
 * created files undeleted, fixes for AHCC (int=short) and more
 * error checks & logging.
 */
#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <tos.h>

#define TEST_PATH "\\TEST"
#define TEST_FILE TEST_PATH "\\%02X_%c_%c.TXT"
#define TEST_FILE_LEN 16
#define TEST_DIR_LEN TEST_FILE_LEN-4

static void create_names(int i, char *filename, char *dirname)
{
	sprintf(filename, TEST_FILE, i, i, i);
	memcpy(dirname, filename, TEST_DIR_LEN);
	dirname[TEST_DIR_LEN] = '\0';
}

static void write_log(int logfd, const char *line)
{
	int len;

	Cconws(line);
	len = strlen(line);
	if (Fwrite(logfd, len, line) != len) {
		Cconws("ERROR: can't write to logfile\r\n");
	}
}

static int is_enter(const char *msg)
{
	int scancode;

	/* eat buffered keys */
	while (Cconis()) {
		Cconin();
	}
	Cconws(msg);

	scancode = (Cconin() >> 16) & 0xff;
	if (scancode == 28 || scancode == 114) {
		return 1;
	}
	return 0;
}

static void error_exit(int logfd, const char *msg)
{
	write_log(logfd, msg);
	is_enter("Press key to exit\r\n");
	exit(-1);
}

int main(void)
{
	char line[200];
	char name[TEST_FILE_LEN+2];
	char dirname[TEST_DIR_LEN+1];
	int i, fd, logfd, cerrors, derrors;
	long err1, err2, err3;

	logfd = Fcreate("\\atari.log", 0);
	if (logfd < 0) {
		error_exit(logfd, "ERROR: log creation failed!\r\n");
	}

	err1 = Dcreate(TEST_PATH);
	if (err1) {
		sprintf(line, "Creating '" TEST_PATH "' failed with error %ld!\r\n", err1);
		error_exit(logfd, line);
	}

	cerrors = 0;
	write_log(logfd, "Trying to create 244 files & folders to '" TEST_PATH "'...\r\n");
	for (i = 32; i < 256; i++) {
		create_names(i, name, dirname);

		err1 = Dcreate(dirname);
		err2 = Fcreate(name, 0);
		fd = err2;
		if (err2 >= 0) {
			err2 = Fclose(fd);
		}
		if (err1 || err2) {
			cerrors++;
		}

		sprintf(line, "%02X - %s / %s -> %s / %s (%ld / %ld), fd: %d\r\n",
			i, dirname, name,
			err1 ? "ERROR" : "dir ok",
			err2 ? "ERROR" : "file ok",
			err1, err2, fd);

		write_log(logfd, line);
	}
	write_log(logfd, "Done!\r\n\r\n");

	if (!is_enter("Press Enter to delete them, any other key to exit.\r\n")) {
		Cconws("Leaving files & folders into '" TEST_PATH "\\'!\r\n");
		return 0;
	}

	derrors = 0;
	write_log(logfd, "Trying to delete 244 files & folders from '" TEST_PATH "'...\r\n");
	for (i = 32; i < 256; i++) {
		create_names(i, name, dirname);

		err1 = Ddelete(dirname);
		err2 = Fsfirst(name, 0);
		err3 = Fdelete(name);
		if (err1 || err2 || err3) {
			derrors++;
		}

		sprintf(line, "%02X - %s / %s -> %s / %s / %s (%ld / %ld / %ld)\r\n",
			 i, dirname, name,
			err1 ? "ERROR" : "dir ok",
			err2 ? "NOT FOUND" : "search ok",
			err3 ? "ERROR" : "file ok",
			err1, err2, err3);
		write_log(logfd, line);
	}
	write_log(logfd, "Done!\r\n\r\n");

	if (cerrors || derrors) {
		sprintf(line, "File/folder creation failed for %d and deletion for %d of the characters!\r\n", cerrors, derrors);
		write_log(logfd, line);
	} else {
		write_log(logfd, "Everything succeeded!\r\n");
	}
	Fclose(logfd);

	is_enter("Press key to exit\r\n");
	return 0;
}

Attachment: atari.log.gz
Description: GNU Zip compressed data