eBook Reader user interface
dhopkins at DonHopkins.com
Tue Jun 5 15:38:02 EDT 2007
print "Hello world!"
I have made a version of the Read activity that supports navigation and
control with the gamepad, and has a few new features and improvements.
The main problem is getting the two gamepads to send unique keycodes
that are not the numeric keypad arrow keys it currently sends (because
those are already bound to functions that don't do the appropriate thing).
The problem right now is that somebody needs to run the command "chmod
a+rwx /dev/input/event1" to give Python the permission to perform the
ioctl that remaps the keys.
The solution to that problem is to decide on a standard set of keycodes
and have the operating system set them when it boots.
I have tried several sets of keycodes, unsuccessfully. The higher
keycodes with numbers >512 never seem to make it to the X server.
I tried using the keycodes for the higher numbered function keys
(KEY_F13  - KEY_F20 ), which made it to the X server, but all
report X keycodes 0.
There may be some magic that needs to be done in the /etc/X11/xorg.conf
file to admit those keycodes, or maybe some other arcane X server
So for now I am mapping to the letters "esdx" and "ijkm" to test out the
gamepad code until the OLPC has a fixed set of distinguishable keypad
It will be very nice to have a gamepad text input method like
triggerfinger or dasher, to input real text in book mode!
The next issue is rotation. I have made the activity respond to top
level window resize events by running "xrandr" to figure out which
direction the screen is rotated.
Then it remaps the behaviors it assigns to the gamepads according to the
I think screen rotation should be handled by the application, not the
operating system, because in some cases the application might want to
associate commands with the LABELS of the gamepad keys instead of the
DIRECTIONS of the keys, so it would not want to rotate their meaning.
For example, a tic-tac-toe game that wanted to use X for X and O for O
would not want to remap the keys on screen rotation! (Admittedly
contrived, but hey it's a popular game!) c(-;
It would be great if there were a way to find out the screen rotation
via dbus. Maybe there is but I just haven't figured it out yet. Could we
add that, if it's not yet supported, please?
Also, I wrote some stuff in the comments of the bug about the screen
rotation button: it's waaaaay too easy to rotate the screen accidentally
(not to mention turn the machine off accidentally, ESPECIALLY when the
screen is upside down and you reach for what you think is the rotation
I propose that both the screen rotation button and the power button
simply pop up big fat obvious dialogs that pop down and do nothing if
you press the button again, but require you to either press and hold the
button, or press another button (like the arrow keys) to actually rotate
the screen or power the system down.
That will let users rotate the screen absolutely in one shot with the
arrow keys without going through all the intermediate rotations.
Rotating is extremely expensive because it laboriously repaints the
entire screen, requiring the application to do all kinds of memory
allocation of giant page sized double buffers. So it should be very hard
to do it accidentally, while right now it's very hard not to do it
Of course the consequences of accidentally shutting the system down with
a mere press of the button are much worse, so the same thing goes
tenfold for the power button!
Kids will naturally go straight for the buttons on the screen and press
them many times in a row, so that should not cause any harm, delay or
See: Rotating in reverse and/or better feedback for the rotate button
Also: Rotate handheld keys with screen
I decided to use the directional arrows for scrolling withing the page
selected, for when the entire page doesn't fit on the screen, or when
you're in continuous mode.
That is because it's possible to press the directional arrows
diagonally, sending two keycodes at once, which makes sense for
directional scrolling commands but not for discrete commands.
I used the other 4 button gamepad for discrete user interface navigation
commands. The left one and the right one move to the previous and next
selected button, in the same direction as they appear on the screen.
The bottom one activates the selected button. The top one goes to the
first button. These button mappings rotate with the screen appropriately.
I added a couple buttons for "first page" and "last page", and I also
added another toolbar of "View" commands, including sizing: fit width,
best fit, zoom in, zoom out; rotating: rotate left, rotate right; and
toggle buttons for display mode: dual page, continuous.
When you use the gamepad to navigate between user interface buttons, it
wraps back and forth from end of one toolbar to the beginning of the other.
Gamepad navigation ignores the "activity" toolbar (for now).
When you navigate with the gamepad, it warps the mouse cursor so it's in
the upper right corner of the button.
I was hoping that warping the mouse into the button would be enough to
trigger the tooltip, but apparently not.
I have been looking into how to pop up and tear down the tooltip under
program control without resorting to ugly hacks like faking events, but
I can't figure out a way to do it short of hacking the C code.
Any suggestions? The other approach is not to use GTK tooltips at all,
and implement some nice Sugar tooltips that do the right thing, and
allow us to add more features like internationalized text (if that's not
already supported -- I dunno) and icons and animation and nice stuff
The tooltips in gamepad mode should be smart enough to pop up when you
navigate to a button, then go away after a few seconds or after you use
If you try to use a button that's "not allowed" (my alternative to gtk's
broken "not sensitive" which unfortunately disallows tooltips and
keyboard navigation), a tooltip should pop up explaining why the
operation is not allowed and what you can do to allow it.
I have written all the code to set the tooltip to the most important
explanation when buttons are not allowed (i.e. the "back page" button
says "you need to load a document first" if no doc is loaded, or "you're
already at the first page" if you do havea doc but you're on the first
It's just that the tooltips don't display when I want them to, and
currently there is no visual indication that the button is not allowed.
I suppose the "not allowed" state should look just like the "not
sensitive" state, except that they let you navigate to the button with
the gamepad and display tooltips.
I made a mixin class for widgets that includes intrinsics for gamepad
navigation, as well as some other useful stuff (like tooltips, and
disabling them without breaking keyboard navigation and tooltips).
It also lets (no, requires!) you to associate an "info" dictionary with
each widget, that tells the system useful stuff about the widget,
including the icon, default tooltip, and whether or not it's accessible
via the game controller keypad.
The toolbars have a method that returns a list of their navigable
widgets, and the application concatenates the widget lists of all
toolbars, handles keyboard events, and manages to navigate between them.
(This gamepad navigation is in place of the standard GTK tab/arrow key
navigation, which I consider flawed and too hard-wired to fix, so I just
programmed around it. For example, on top of the focus and tooltip
problems, when you arrow to a text field, you're stuck and can't arrow
I made the search text field as well as the find next / previous buttons
inaccessibly from the game controllers, because currently there is no
way to input text when the keyboard is hidden.
Once we have a gamepad based text input method we can hook that up to
the search field and enable it for navigation.
I put in placeholder "programmer art" with text labels for the buttons I
added, which are ugly enough to remind everyone that we need real icons,
but I'm not a good enough artist to come up with appropriate icons.
One usability problem I noticed is with the toggle buttons: the "mouse
over" highlighting obscures (overrides, conflates, is the same as) the
highlighting that tells you if the button is selected or not, so once
you're pointing at the button you have no idea about its current state,
and no feedback after you click it to change the state, that the state
was actually changed (modulo the actual effect of the button toggle on
the application of course). The "mouse over" highlighting should be
orthogonal to and composeable with the "checked" highlighting. Moving
over the toggle should activate the "mouse over" highlighting but not
change the "checked" highlighting. Pressing down and holding should keep
the "mouse over" highlighting and toggle the "selected" highlighing to
feedback that it will enter that state iff you release while still in
the button. If you move out of the button while holding down the mouse,
the "mouse over" should de-highlight and the "checked" should go back to
the previous state, to feedback that pressing down and moving out is a
no-op that cancels the toggle action, and nothing should change when you
release the button to toggle (but of course you should be able to slide
back in to resume the "mouse over" and "check toggled" state, then
release to toggle it.
I heard a rumor that somebody had decided on some unique keycodes for
the gamepad. Is that true? What are they? Once I know, I'll put them
into the reader and put up a copy for people to try out. (Right now it
requires opening a shell and running chmod as root to get it to work,
which is kinda clumsy, but a great way to force kids to learn Linux, you
Zvi Devir wrote:
> I have been collecting a few (more than a few) usability bugs and UI
> suggestions, which will hopefully be compiled to a list next month.
> In the meanwhile, I would like to share a couple of ideas regarding the
> mapping of the gamepad keys [which I hope won't state the obvious too much].
> Currently, the UP/DOWN/LEFT/RIGHT keys have a fixed functionality
> regardless of the screen orientation. This isn't a correct logic, as the
> expected functionality changes according to the current orientation. I
> guess the easiest way to implement this kind of behavior would be using
> a smart key mapping mechanism in X.
> Another thought -- the current input method treats those keys as
> direction keys. However, a better input method may be emulating the
> functionality of the mouse (touch pad) using the gamepad keys. In tablet
> mode this can really improve the user experience.
> Just my $0.04.
> Don Hopkins wrote:
>> Goal for improving the eBook reader user interface:
>> I've been doing some exploratory programming with GTK and Sugar,
>> trying to improve the user interface of the eBook reader, and make
>> it useable in book mode with the gamepad.
>> + Support the game keypads in eBook mode.
>> + Low level game keypad support
>> Need to remap low level keyboard scan codes to Linux keyboard codes.
>> The setolpckeys.c program remaps the keys and gamepad buttons.
>> Currently it maps both gamepads to the numeric keypad keys (KEY_KP8, etc),
>> which the X server and GDK translates to directional keys (GDK_Up, etc).
>> I tried to map them to buttons (BTN_A, etc), but the X server seems
>> to ignore keycodes in that range.
>> The xorg.conf file has a keycode mask that looked like it might help,
>> but I couldn't get it to work.
>> Need to have unique keycodes reported for each of the two gamepads,
>> which are not the same as any keyboard keys, without any predefined meanings
>> like arrow keys have.
>> Need to define special purpose keycodes just for the OLPC gamepad,
>> instead of trying to reuse existing but not appropriate keycodes.
>> What is the process for defining new keycodes in <linux/input.h>?
>> Here's my strawman proposal for some new keycodes.
>> Use keys ("KEY_*") instead of buttons ("BTN_*"), since they
>> seem to work better.
>> The 0x1b* range seems to be unused in <linux/input.h>,
>> and it's between other groups of keycodes, so I'll
>> propose using that range for the OLPC.
>> The UP/DOWN/LEFT/RIGHT keys correspond to the directional
>> #define KEY_XO_GAMEPAD_UP 0x1b0
>> #define KEY_XO_GAMEPAD_DOWN 0x1b1
>> #define KEY_XO_GAMEPAD_LEFT 0x1b2
>> #define KEY_XO_GAMEPAD_RIGHT 0x1b3
>> The NORTH/SOUTH/EAST/WEST keys correspond to the other
>> buttons. Those names are agnostic to the button labels,
>> which may change from the current Playstation buttons
>> (X/O/Triangle/Square). Can anyone suggest better names for
>> the four buttons on the right?
>> #define KEY_XO_GAMEPAD_NORTH 0x1b4
>> #define KEY_XO_GAMEPAD_SOUTH 0x1b5
>> #define KEY_XO_GAMEPAD_EAST 0x1b6
>> #define KEY_XO_GAMEPAD_WEST 0x1b7
>> While we're at it, we could define keycodes for the other
>> OLPC buttons and switches on the screen. I think there are
>> some other sensor switches that could generate keycodes,
>> like opening the screen, rotating it around, and putting it
>> into book mode, so I will make some guesses at names for
>> them, just to get the discussion rolling.
>> #define KEY_XO_SCREEN_ROTATE 0x1b8
>> #define KEY_XO_SCREEN_POWER 0x1b9
>> #define KEY_XO_SCREEN_OPEN 0x1ba
>> #define KEY_XO_SCREEN_CLOSE 0x1bb
>> #define KEY_XO_SCREEN_IN 0x1bc
>> #define KEY_XO_SCREEN_OUT 0x1bd
>> Is there an exhaustive list of all buttons and switches and
>> events on the OLPC? Are any more planned? Which ones should
>> be assigned keycodes?
>> Rewrote setolpckeys.c code in Python (just uses ioctl, but needs to know keycodes).
>> Writing utilities like that in Python instead of C makes it easier to
>> reconfigure the keys on the OLPC without a C compiler.
> Devel mailing list
> Devel at lists.laptop.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Devel