Re: [AD] unicode proposal

[Peter Wang <novalazy@xxxxxxxxxx>]

On 2009-01-21, Elias Pschernig <elias.pschernig@xxxxxxxxxx> wrote:
> On Wed, 2009-01-21 at 11:43 +1100, Peter Wang wrote:
> > 
> > So the user works with UTF-8 strings and Allegro will automatically
> > convert to and from the "correct" encoding for the filesystem?
> > 
> 
> Yes, that's the idea.
> 
> > I don't think that will be robust on Unix/Linux where there is not
> > necessarily _a_ filesystem encoding.  I have filenames in UTF-8 but
> > occasionally end up with filenames in other (but ASCII-compatible)
> > encodings.  I usually fix them up immediately but it's only possible
> > because, say, mv, works properly even when filenames are not in the
> > expected encoding.  It just treat filenames as raw 8-bit strings.
> 
> Hm. How do KDE or Gnome display those files?

Konqueror displays the unknown character boxes for Latin-1 characters,
but still will open the file when it's clicked.

ROX-Filer is more helpful.  It will display Latin-1 names but also
highlights them in red or tells you in a tooltip so you know something
funny is going on.

> > For Unix, I think that's what we should do as well.  No conversions.
> > All special characters (e.g. path separators, globbing wildcards) are in
> > the ASCII range so any ASCII-compatible encoding will work.  That means
> > we might return filenames to the user which are not UTF-8, and they
> > might try to display them.  I think they will just have to detect if the
> > string is not UTF-8, and if so assume an encoding and convert it before
> > displaying.  We can make that easy for the Latin-1 case.
> 
> Maybe we should just state in the documentation that filenames returned
> by Allegro are "display names", i.e. you can use them for displaying the
> filenames, but you can't expect that creating a new file with the same
> name will overwrite the existing one (if non-standard encodings are
> being used).

If you read a directory, you might be able to display the filenames
but not be able to open them?

> > For Windows, I don't know.  From what I can tell, filenames are stored
> > in UTF-16 so converting to/from UTF-8 is possible, and probably the
> > right thing to do so the user only has to deal with ASCII-compatible
> > filenames everywhere.
> > 
> 
> I assume it's what we also do in A4 if the current encoding is UTF-8.

Peter