#12278 NORM Future : OSK opened by keyboard frame device does not send key events

[Zarro Boogs per Child]

#12278: OSK opened by keyboard frame device does not send key events
--------------------------------------------+-------------------------------
           Reporter:  dsd                   |       Owner:  dsd           
               Type:  defect                |      Status:  new           
           Priority:  normal                |   Milestone:  Future Release
          Component:  keyboards, on-screen  |     Version:  not specified 
         Resolution:                        |    Keywords:                
        Next_action:  never set             |    Verified:  0             
Deployment_affected:                        |   Blockedby:                
           Blocking:                        |  
--------------------------------------------+-------------------------------
Changes (by dsd):

  * blocking:  12281 =>
  * milestone:  13.1.0 => Future Release


Comment:

 We have examined the use cases and their associated challenges and have
 decided to push out the manually invoked OSK feature for this release.

 We will renew focus on item #2 below to minimize the impact where
 realistically possible.

 Here is the full breakdown:

 === Manual (frame-triggered) OSK use cases ===

 '''1. OSK use in laptop mode.''' You want to type in a language that you
 don't have on your physical keyboard. You have (or recently have had) a
 GTK IM context to work with.

 This might be fixable but it seems like it would need a good few hours of
 work, and it might be painful. There are 4 things to address.

 a) The first critical problem in this case, preventing further
 investigation of this item, is that you currently cannot close the OSK
 while retaining focus to the input widget you were working with. As soon
 as you click the button to close the OSK, the focus goes somewhere else,
 and I'm not sure where or why.

 b) Once that issue is fixed, the real test can be run: what happens when
 you activate a text field (causing the OSK to automatically appear), then
 hide the OSK (the text field should still be focused), then activate the
 OSK via the frame? For this to work we need the text field to retain input
 focus throughout the whole sequence, and for its IM context of the input
 widget not to be deactivated without then being immediately activated
 again. My prediction is that work will be needed to make this happen.

 c) Once that is working, we will need a solution for #12281. This will
 need another few hours work, since a maliit API change is needed, and I
 think a change to the internals too. At a glance, Maliit seems to destroy
 all the OSK widget stuff when the laptop is in tablet mode, rather than
 just "soft disabling" it because the switch is closed, so this will not be
 as easy as just force-activating the OSK.

 d) A potential issue that will affect (b) and (c) is that the way that the
 Sugar frame code brings up the OSK involves creating and activating a new
 IM context. That will deactivate the IM context of the active widget. I'm
 not 100% sure of this point, but if true, this will need to be changed,
 potentially requiring new maliit API and changes to the internals.


 '''2. GTK+ activities where there is no IM context''' because a custom
 input field is used (e.g. Paint, Labyrinth, FotoToon).

 This can be fixed on an activity-by-activity basis. Activities can either
 overlay a standard GTK+ input widget on the right part of the screen, or
 the custom input widgets can be made to interact with the GtkIMContext API
 which will cause the OSK to come up as appropriate.

 If no activity-level fix is made, these cases fall into the following item
 #3 "Non-GTK+ activities."

 Note that the "fix" here is not really a fix - the manual OSK will still
 be broken for these cases unless #1 is fixed too. However, things will be
 "fixed" from the respect that affected activities can be used in tablet
 mode since the OSK will come up automatically, removing the requirement to
 invoke it manually.


 '''3. Non-GTK+ activities''' (etoys, Scratch, pygame-based activities)

 These are out of the question for 13.1.0 (and perhaps indefinitely), since
 we just don't have an appropriate way into the X event stream
 (http://lists.x.org/archives/xorg/2012-November/055033.html)

 -----

 If we do go with solving #1, and given that #3 is impossible, we will be
 faced with a challenge of offering a good manual OSK user experience. This
 is because the manual OSK will not work in the #3 situation, even though
 it will pop up on-demand and give the normal interactive keyboard
 experience, just being unable to actually send keypresses to the active
 app.

 The maliit server could be extended to know when there is no context
 available, and make that state available over dbus. If the frame were to
 send a dbus message every time it were opened (needs new maliit API), it
 could discover this state, and modify the frame icon behaviour accordingly
 (hide it, or make it greyed out). However the idea of sending a dbus
 message at this point sounds bad, and also, would this really provide a
 better user experience? For the #3 case, instead of presenting a keyboard
 that doesn't work, having a hidden or greyed out icon would be just as
 confusing to the user ("where's the friggin OSK gone???").

-- 
Ticket URL: <http://dev.laptop.org/ticket/12278#comment:3>
One Laptop Per Child <http://laptop.org/>
OLPC bug tracking system