Re: [hatari-devel] Too many (4096) active GEMDOS HD DTA entries

[Uwe Seimet <Uwe.Seimet@xxxxxxxxx>]

Hi,

I would still be interested to learn what the warning means for the result
of the copy operation. Have any files been damaged?

Best regards

Uwe

> Hi Eero,
> 
> Yes, I was copying quite a large number of files from GEMDOS drives to
> various partitions on an image file. Several 1000 files I guess.
> 
> Best regards
> 
> Uwe
> 
> > Hi Uwe,
> > 
> > On 24.9.2024 20.32, Uwe Seimet wrote:
> > > While copying some data with KOBOLD3 Hatari displayed this warning:
> > > 
> > > WARN : Too many (4096) active GEMDOS HD DTA entries, wrapping DTA index
> > 
> > TOS has no "forget DTA" OS call, so Hatari has no way of knowing which 
> > DTAs' data can be dropped.
> > 
> > Therefore above warning just means that programs running under Hatari 
> > emulation have called FsFirst() OS-call 4k times with different DTA 
> > addresses, since emulation was booted.
> > 
> > Did you run Kobold, or other heavy DTA using programs, several times 
> > before this warning?
> > 
> > 
> > > May I assume that the copied data are correct, or are they broken?
> > 
> > If Kobold actually does FsNext() with a DTA on which it did FsFirst() 
> > over *four thousand* Fsetdta() / FsFirst() calls ago, then the filenames 
> > associated with DTA would have gone after that warning.
> > 
> > (Hatari searches for all the matching files on FsFirst() call, and 
> > caches them to internal DTA cache entry for FsNext() calls.)
> > 
> > It seems rather unlikely that Kobold would actively use that many DTAs 
> > though, but attached is a patch that allows DTA cache to grow unlimited.
> > 
> > You could use that to check how many DTAs Kobold copy operation actually 
> > consumes...
> > 
> > (DTA cache increments are logged every 256 entries.)
> > 
> > 
> > 	- Eero
> 
> > From 6bf72831f6c470474047fa439d06e670422da3d7 Mon Sep 17 00:00:00 2001
> > From: Eero Tamminen <oak@xxxxxxxxxxxxxx>
> > Date: Wed, 25 Sep 2024 00:49:00 +0300
> > Subject: [PATCH] GEMDOS HD: allow unlimited DTA cache growth
> > 
> > There's no "forget DTA" OS call, and no mechanism to detect that
> > process is not using DTA any more.  If user keeps doing things that
> > cause more FsFirst() invocations with new DTAs, more DTA (cache)
> > entries are consumed.  Let it grow, and just warn user when more is
> > allocated.
> > 
> > PS. Marking current DTAs as free on program termination, and
> > compacting the cache may not work either, in case DTA has been passed
> > to another program, or DTA was created by other code (ACC, TOS
> > desktop, some other resident program) while foreground program was
> > running.
> > ---
> >  src/gemdos.c | 26 +++++++++-----------------
> >  1 file changed, 9 insertions(+), 17 deletions(-)
> > 
> > diff --git a/src/gemdos.c b/src/gemdos.c
> > index 38db359e..bf5c00dd 100644
> > --- a/src/gemdos.c
> > +++ b/src/gemdos.c
> > @@ -110,7 +110,6 @@ typedef enum {
> >  
> >  #define DTA_MAGIC_NUMBER  0x12983476
> >  #define DTA_CACHE_INC     256      /* DTA cache initial and increment size (grows on demand) */
> > -#define DTA_CACHE_MAX     4096     /* max DTA cache size (multiple of DTA_CACHE_INC) */
> >  
> >  #define  BASE_FILEHANDLE     64    /* Our emulation handles - MUST not be valid TOS ones, but MUST be <256 */
> >  #define  MAX_FILE_HANDLES    64    /* We can allow 64 files open at once */
> > @@ -3130,26 +3129,19 @@ static bool GemDOS_SFirst(uint32_t Params)
> >  
> >  	if (++DTAIndex >= DTACount)
> >  	{
> > -		if (DTACount < DTA_CACHE_MAX)
> > +		INTERNAL_DTA *pNewIntDTAs;
> > +		/* increase DTA cache size */
> > +		pNewIntDTAs = realloc(InternalDTAs, (DTACount + DTA_CACHE_INC) * sizeof(*InternalDTAs));
> > +		if (pNewIntDTAs)
> >  		{
> > -			INTERNAL_DTA *pNewIntDTAs;
> > -			/* increase DTA cache size */
> > -			pNewIntDTAs = realloc(InternalDTAs, (DTACount + DTA_CACHE_INC) * sizeof(*InternalDTAs));
> > -			if (pNewIntDTAs)
> > -			{
> > -				InternalDTAs = pNewIntDTAs;
> > -				memset(InternalDTAs + DTACount, 0, DTA_CACHE_INC * sizeof(*InternalDTAs));
> > -				DTACount += DTA_CACHE_INC;
> > -			}
> > -			else
> > -			{
> > -				Log_Printf(LOG_WARN, "Failed to alloc GEMDOS HD DTA entries, wrapping DTA index\n");
> > -				DTAIndex = 0;
> > -			}
> > +			InternalDTAs = pNewIntDTAs;
> > +			memset(InternalDTAs + DTACount, 0, DTA_CACHE_INC * sizeof(*InternalDTAs));
> > +			DTACount += DTA_CACHE_INC;
> > +			Log_Printf(LOG_WARN, "Increased GEMDOS HD DTA cache to %d entries\n", DTACount);
> >  		}
> >  		else
> >  		{
> > -			Log_Printf(LOG_WARN, "Too many (%d) active GEMDOS HD DTA entries, wrapping DTA index\n", DTAIndex);
> > +			Log_Printf(LOG_WARN, "Failed to alloc GEMDOS HD DTA entries, wrapping DTA index\n");
> >  			DTAIndex = 0;
> >  		}
> >  	}
> > -- 
> > 2.39.5
> > 
> 
> 
>