Re: [hatari-devel] Too many (4096) active GEMDOS HD DTA entries

[Eero Tamminen <oak@xxxxxxxxxxxxxx>]

Hi Uwe,

On 24.9.2024 20.32, Uwe Seimet wrote:
> While copying some data with KOBOLD3 Hatari displayed this warning:
>
> WARN : Too many (4096) active GEMDOS HD DTA entries, wrapping DTA index

TOS has no "forget DTA" OS call, so Hatari has no way of knowing which 
DTAs' data can be dropped.

Therefore above warning just means that programs running under Hatari 
emulation have called FsFirst() OS-call 4k times with different DTA 
addresses, since emulation was booted.

Did you run Kobold, or other heavy DTA using programs, several times 
before this warning?


> May I assume that the copied data are correct, or are they broken?

If Kobold actually does FsNext() with a DTA on which it did FsFirst() 
over *four thousand* Fsetdta() / FsFirst() calls ago, then the filenames 
associated with DTA would have gone after that warning.

(Hatari searches for all the matching files on FsFirst() call, and 
caches them to internal DTA cache entry for FsNext() calls.)

It seems rather unlikely that Kobold would actively use that many DTAs 
though, but attached is a patch that allows DTA cache to grow unlimited.

You could use that to check how many DTAs Kobold copy operation actually 
consumes...

(DTA cache increments are logged every 256 entries.)


	- Eero

From 6bf72831f6c470474047fa439d06e670422da3d7 Mon Sep 17 00:00:00 2001
From: Eero Tamminen <oak@xxxxxxxxxxxxxx>
Date: Wed, 25 Sep 2024 00:49:00 +0300
Subject: [PATCH] GEMDOS HD: allow unlimited DTA cache growth

There's no "forget DTA" OS call, and no mechanism to detect that
process is not using DTA any more.  If user keeps doing things that
cause more FsFirst() invocations with new DTAs, more DTA (cache)
entries are consumed.  Let it grow, and just warn user when more is
allocated.

PS. Marking current DTAs as free on program termination, and
compacting the cache may not work either, in case DTA has been passed
to another program, or DTA was created by other code (ACC, TOS
desktop, some other resident program) while foreground program was
running.
---
 src/gemdos.c | 26 +++++++++-----------------
 1 file changed, 9 insertions(+), 17 deletions(-)

diff --git a/src/gemdos.c b/src/gemdos.c
index 38db359e..bf5c00dd 100644
--- a/src/gemdos.c
+++ b/src/gemdos.c
@@ -110,7 +110,6 @@ typedef enum {
 
 #define DTA_MAGIC_NUMBER  0x12983476
 #define DTA_CACHE_INC     256      /* DTA cache initial and increment size (grows on demand) */
-#define DTA_CACHE_MAX     4096     /* max DTA cache size (multiple of DTA_CACHE_INC) */
 
 #define  BASE_FILEHANDLE     64    /* Our emulation handles - MUST not be valid TOS ones, but MUST be <256 */
 #define  MAX_FILE_HANDLES    64    /* We can allow 64 files open at once */
@@ -3130,26 +3129,19 @@ static bool GemDOS_SFirst(uint32_t Params)
 
 	if (++DTAIndex >= DTACount)
 	{
-		if (DTACount < DTA_CACHE_MAX)
+		INTERNAL_DTA *pNewIntDTAs;
+		/* increase DTA cache size */
+		pNewIntDTAs = realloc(InternalDTAs, (DTACount + DTA_CACHE_INC) * sizeof(*InternalDTAs));
+		if (pNewIntDTAs)
 		{
-			INTERNAL_DTA *pNewIntDTAs;
-			/* increase DTA cache size */
-			pNewIntDTAs = realloc(InternalDTAs, (DTACount + DTA_CACHE_INC) * sizeof(*InternalDTAs));
-			if (pNewIntDTAs)
-			{
-				InternalDTAs = pNewIntDTAs;
-				memset(InternalDTAs + DTACount, 0, DTA_CACHE_INC * sizeof(*InternalDTAs));
-				DTACount += DTA_CACHE_INC;
-			}
-			else
-			{
-				Log_Printf(LOG_WARN, "Failed to alloc GEMDOS HD DTA entries, wrapping DTA index\n");
-				DTAIndex = 0;
-			}
+			InternalDTAs = pNewIntDTAs;
+			memset(InternalDTAs + DTACount, 0, DTA_CACHE_INC * sizeof(*InternalDTAs));
+			DTACount += DTA_CACHE_INC;
+			Log_Printf(LOG_WARN, "Increased GEMDOS HD DTA cache to %d entries\n", DTACount);
 		}
 		else
 		{
-			Log_Printf(LOG_WARN, "Too many (%d) active GEMDOS HD DTA entries, wrapping DTA index\n", DTAIndex);
+			Log_Printf(LOG_WARN, "Failed to alloc GEMDOS HD DTA entries, wrapping DTA index\n");
 			DTAIndex = 0;
 		}
 	}
-- 
2.39.5