[AD] [ alleg-Bugs-3556659 ] VT_GETSTATE invalid argument on embedded system

[SourceForge.net <noreply@xxxxxxxxxx>]

Bugs item #3556659, was opened at 2012-08-12 05:54
Message generated for change (Tracker Item Submitted) made by tobi_v
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105665&aid=3556659&group_id=5665

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Linux
Group: 4.4
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Tobi Vollebregt (tobi_v)
Assigned to: Nobody/Anonymous (nobody)
Summary: VT_GETSTATE invalid argument on embedded system

Initial Comment:
I'm trying to use Allegro 4.4 on an embedded platform (ea3250 development board), using the Linux console and fbcon subsystems.

The Linux console subsystem tries to open /dev/console to call the VT_GETSTATE ioctl on it, but the kernel returns invalid argument when the /dev/console fd is passed to it, thus init_console fails, and hence the console subsystem prevents setting any graphics mode.

Bypassing the /dev/console fixes this, i.e. I hacked Allegro like this:

--- a/src/linux/lconsole.c
+++ b/src/linux/lconsole.c
@@ -118,7 +118,7 @@ static int init_console(void)
        * opening each unused one in turn.
        */
 
-      if ((console_fd = open ("/dev/console", O_WRONLY)) < 0) {
+      if (TRUE || (console_fd = open ("/dev/console", O_WRONLY)) < 0) {
         int n;
         uszprintf (allegro_error, ALLEGRO_ERROR_SIZE, uconvert_ascii("%s /dev/console: %s", tmp),
                    get_config_text("Unable to open"), ustrerror (errno));

Possibly /dev/console no longer has the function it had when this code was written?

A better solution may be desired but this hack at least makes the Linux console subsystem work on devices where the current VT can't be found. (The terminal on which the Allegro program is started is typically /dev/pts/N for ssh connections or /dev/ttySN for serial connections on such a device.)

Related: http://www.allegro.cc/forums/thread/605565/890751

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105665&aid=3556659&group_id=5665