eBook Reader user interface

[Don Hopkins]

Goal for improving the eBook reader user interface:

  I've been doing some exploratory programming with GTK and Sugar,
  trying to improve the user interface of the eBook reader, and make
  it useable in book mode with the gamepad.

  + Support the game keypads in eBook mode. 

    + Low level game keypad support

      Need to remap low level keyboard scan codes to Linux keyboard codes. 

      The setolpckeys.c program remaps the keys and gamepad buttons.

        Currently it maps both gamepads to the numeric keypad keys (KEY_KP8, etc),
	which the X server and GDK translates to directional keys (GDK_Up, etc).

	I tried to map them to buttons (BTN_A, etc), but the X server seems 
	to ignore keycodes in that range. 

	The xorg.conf file has a keycode mask that looked like it might help, 
	but I couldn't get it to work. 

	Need to have unique keycodes reported for each of the two gamepads, 
	which are not the same as any keyboard keys, without any predefined meanings
	like arrow keys have. 

	Need to define special purpose keycodes just for the OLPC gamepad,
	instead of trying to reuse existing but not appropriate keycodes. 

	What is the process for defining new keycodes in <linux/input.h>?

	Here's my strawman proposal for some new keycodes. 

	  Use keys ("KEY_*") instead of buttons ("BTN_*"), since they
	  seem to work better.

	  The 0x1b* range seems to be unused in <linux/input.h>, 
	  and it's between other groups of keycodes, so I'll
	  propose using that range for the OLPC. 

	  The UP/DOWN/LEFT/RIGHT keys correspond to the directional
	  keypad.

	  #define KEY_XO_GAMEPAD_UP      0x1b0
	  #define KEY_XO_GAMEPAD_DOWN    0x1b1
	  #define KEY_XO_GAMEPAD_LEFT    0x1b2
	  #define KEY_XO_GAMEPAD_RIGHT   0x1b3

	  The NORTH/SOUTH/EAST/WEST keys correspond to the other
	  buttons. Those names are agnostic to the button labels,
	  which may change from the current Playstation buttons
	  (X/O/Triangle/Square). Can anyone suggest better names for
	  the four buttons on the right?

	  #define KEY_XO_GAMEPAD_NORTH   0x1b4
	  #define KEY_XO_GAMEPAD_SOUTH   0x1b5
	  #define KEY_XO_GAMEPAD_EAST    0x1b6
	  #define KEY_XO_GAMEPAD_WEST    0x1b7

	  While we're at it, we could define keycodes for the other
	  OLPC buttons and switches on the screen. I think there are
	  some other sensor switches that could generate keycodes,
	  like opening the screen, rotating it around, and putting it
	  into book mode, so I will make some guesses at names for
	  them, just to get the discussion rolling. 

	  #define KEY_XO_SCREEN_ROTATE   0x1b8
	  #define KEY_XO_SCREEN_POWER    0x1b9
	  #define KEY_XO_SCREEN_OPEN     0x1ba
	  #define KEY_XO_SCREEN_CLOSE    0x1bb
	  #define KEY_XO_SCREEN_IN       0x1bc
	  #define KEY_XO_SCREEN_OUT      0x1bd

	  Is there an exhaustive list of all buttons and switches and
	  events on the OLPC? Are any more planned? Which ones should
	  be assigned keycodes?

      Rewrote setolpckeys.c code in Python (just uses ioctl, but needs to know keycodes).
        Writing utilities like that in Python instead of C makes it easier to 
	reconfigure the keys on the OLPC without a C compiler. 

    + High level action support.

      GTK uses "Actions" to define the actions available in an
      application, independent of the user interface used to invoke
      them. Actions can be bound to user interface widgets and
      keyboard accelerators, and they can hide, show, enable and
      disable the corresponding parts of the interface. You can
      subclass Action to define custom toolbar buttons and menu items.

      We need to define a generic way of navigating and executing the
      application's actions from the gamepad.

      We can make Sugar specific actions that create the appropriately
      styled and customized Sugar user interface widgets.

      Actions can be used to support navigation and operation of the
      toolbar components from the gamepad:

      Actions have a list of their "proxy" components (toolbar
      buttons, menu items, etc).

      Actions know how to execute a callback function, so the user
      interface components tell the action to activate, instead of
      calling the function themselves.

      The actions also know their labels, icons and tooltips.

      Actions can be shown and hidden, and all their proxies show and
      hide.

      Actions can be made sensitive or not (disabled), and all their
      proxies enable or disable.

      Actions have methods to construct toolbar buttons and menu
      items, which subclasses can override to customize the user
      interface. The higher level GTK UIManager calls these methods in
      actions to create the user interface, although you can still use
      Actions without the UIManager, by creating the components
      yourself.

      There are three standard kinds of actions: Action, which creates
      a normal button or menu item, ToggleAction, which creates a
      toggle button (checkbox), and RadioAction, which creates a radio
      button (multiple choice). 

      The GTK toolbars and menus support keyboard navigation of
      sensitive buttons, as well as showing tooltips on sensitive
      buttons, but it won't show a tooltip on unsensitive (disabled)
      buttons, tabbing skips over disabled buttons, and it kicks you
      out of buttons that get disabled while you're using them, by
      moving the focus into the next button (whatever that happens to
      be).

      All inactive items should show tooltips telling WHY they are
      inactive, and WHAT you have to do before using them. 

      Unfortunately you can't get a tooltip on an inactive item. 
      This needs to be fixed, so we can display helpful tooltips and
      documentation on any item whether active or not. 
 
      Unfortunlatey you can't navigate to an inactive item.  This
      needs to be fixed, so the user can use the gamepad to navigate
      to an inactive item, to find out what it is and why it's
      inactive. (Also, so you can navigate the interface predicably
      with muscle memory, because the number of presses required to
      navigate doesn't change depending on the state of the
      application.)

      A problem with the current GTK keyboard navigation behavior of
      not allowing inactive items to have the input focus, is that it
      violates the principle of least astonishment, makes the
      interface less predictable, and interferes with type-ahead:

        If you're focused on the "back page" button, and select it
        until you get to the first page, then it will become inactive
        (because you can't go back from the first page), which
        results in the input focus being kicked out of the "back
        page" button and throwing you into the "next page" button.

	Imagine how hard this "astonishing" behavior would be to use
	if you were visually impared. It would not be very obvious
	that you had "bounced off" the first page and suddenly were
	moving forward. Other toolbars might arbitrarily relocate the
	input focus to an even more confusing button, when the one
	you're using gets disabled.

        It would be much less astonishing if the input focus simply
        remained in the "back page" button when you hit the first
        page, and a tooltop popped up saying "You can't go to the
        previous page, because you are at the first page."

	I've made a simple API for changing the sensitivity of a
	component, that takes a "reason" string (translated text) to
	show to the user in the tooltip of a disabled control (after
	the normal text). I've changed the code in the eBook reader
	that disables actions to figure out the most important reason
	(there might be several reasons to disable a control, but
	usually one is most important). It passes that reason to the
	API that disables the action, which currently just stores it
	away in a dictionary. Now it needs to be hooked up so it
	actually shows the tooltip with the reason when disabled.

      GTK has an "AcceleratorList" class that keeps a list of keyboard
      accelerators which invoke actions. The UIManager helps manage
      the accelerators for you automatically. 

	I think we should use accelerators for normal keyboard
	acceleration of application actions, but implement the gamepad
	navigation stuff as a higher level framework that uses a more
	semantically meaningful model. (Not just low level tab/backtab
	focus navigation, or simple global keyboard accelerators, but
	an actual framework specifically designed to support browsing
	and executing arbitrary Actions via the game controller, and
	providing feedback about the reasons controls are disabled.)

	The pygtk library has some methods on action_group to add
	actions "action_group.add_actions(action_list)", toggle
	actions "action_group.add_toggle_actions(action_list)", and
	radio actions "action_group.add_radio_actions(action_list,
	cur_value, callback, *args)". These all take an action_list
	that's a list of action specification tuples. That interface
	is more brittle and less extensible than it should be, because
	it takes tuples instead of dictionaries, and it only makes
	stock GTK actions. 

	We need a more flexible API than add_actions and its ilk, which
	supports custom Sugar actions, and uses dictionaries instead
	of tuples, so we can easily pass in additional optional
	parameters without changing the API.

	I have taken a first cut at rewriting pygtk's ActionGroup's
	add_actions, add_toggle_actions and add_radio_actions
	functions in Python, and changing them to use custom
	SugarAction, SugarToggleAction and SugarRadioAction classes.

	But I still don't think the add_actions_esque interface is
	flexible enough. It needs an easy way to add other custom
	widgets (like the search text input field), and it should make
	it easy to customize the user interface classes and configure
	the objects by passing in additional optional parameters and
	configuration dictionaries. (For example, the action
	specification should be a dictionary that has an optional key
	to specify the toolbar button and menu item classes, and it
	should be possible to plug in different kinds of user
	interface controls than toolbars and menus, like pie menus and
	gamepad specific interfaces). 

      GTK has an "atk" module that interfaces to the accessibility toolkit. 

        I'm not clear on just what this does and how it can help us,
        and I would love to hear from somebody who's familiar with it.

        I think we can do what we need to support the gamepad without
        using the atk, but maybe it could be helpful. But my
        impression is that it's more geared towards screen readers, is
        tied closely to the user interface layout (declaring that this
        label is associated with that control, etc), and it looks like
	it takes a lot of lines of code to use (unless you use some
	kind of framework that supports it automatically).

      GTK has a "UIManager" that knows how to build menu bars, menus,
      toolbars and buttons from XML files, and tie together actions,
      accelerators, menus and toolbars. The XML describes the
      structure and layout of the user interface, but refers to the
      actions by name to configure the buttons and menu items with
      labels, icons, tooltips, accelerators, callbacks, and custom
      component classes. There is nothing in the XML about what label,
      icon, tooltip or widget class to use -- just an Action
      name. It's up to the Action to figure out the visual appearance
      and behavior of the gui. That's why it should be easier to use
      custom Action classes.

        I think we need to write our own UIManager and accessibility
        framework that addresses our specific needs (like the
        focus/tooltip/disable reason issues discussed above, custom
        sugar controls, gamepad support, etc), because I don't think
        the UIManager (plus the pygtk utilities that go along with it)
        are flexible enough to support our needs. I think it would be
        easy to implement it in Python, in a way that would be a lot
        more flexible and easier to extend that the current C
        implementation of GTK's GuiManager.

  + Add support for more advanced evince features. 

    Added support for the various properties and functions that evince supports. 

    Need access to the table of contents of the pdf file. 

  + Integrate Poppler renderer for efficiently rendering PDF documents. 

    Make a higher level library independent interface to evince and poppler. 

    Factor out evince and poppler dependencies so you only need to import 
    the library that you need to render the document. 

    The enumerated types that evince uses should be passed as strings instead, 
    so the interface is independent of evince, and you don't have to define all
    the GObject enumerated type classes just to use the interface.