Making speech-dispatcher python API support asynchronous socket communication (long)

[Tomeu Vizoso]

Hi Hynek,

this summarizes pretty well my understanding of the issue.

Indeed, OLPC is in the process of updating from F-7 to F-9, and in the
later, python ships with a patch that addresses this issue. Not sure
about pygobject2 and pygtk2, though.

So Hemant, perhaps you would like to try an activity using the
unmodified speech-dispatcher client and monitor with strace -p <pid>
in order to see if it's still waking up periodically? You'll have to
wait a few days until the move to F9 (OLPC-3 builds) is finished,
though.

Thanks all,

Tomeu

On Mon, Jun 23, 2008 at 11:36 AM, Hynek Hanke <hanke at volny.cz> wrote:
>
> Hello all,
>
> it is my impression that we are getting a bit confused here
> about the various ways and their details, so I'll try to sum
> up what I know.
>
> The SSIP protocol client-server TCP protocol is partly
> asynchronous. There is a request-reply scheme, but certain
> data called 'events' can be delivered asynchronously at
> any time. If an event is received, it is desirable to let
> the client application know using some kind of callback.
> Thus it is necessary for the client to monitor the server
> connection at all times so that it is prepared to receive
> these asynchronous events.
>
> The current Python SSIP communication library distributed
> with Speech Dispatcher uses a thread to handle the
> connection. This thread is waiting at the blocking IO
> (socket) for any activity and either handles it as a
> reply on a command previously sent or as an asynchronous
> 'event', in which case it calls a registered callback
> function.
>
> So far we have considered this as the cleanest approach,
> because of the following reasons:
>
> 1) It makes on assumptions on the internals of the
> client application which uses this library other than
> that it must not disturb the functioning of python threads.
>
> 2) There is no unnecessary power consumption, the thread
> in use is waiting at a blocking IO and so there is no
> reason to wake it up unless there is activity on the
> communication channel, in which case it is desirable.
>
> 3) The client application does not need to do anything
> to handle or ``know of'' the additional thread. The programmer
> doesn't have to think of how the mainloop interacts with the
> SSIP events and the whole communication.
>
> 4) The performance (granularity, suspension) is entirely
> handled by the standard mechanism of the Python threading system,
> not for example by hooks to the different mainloops
> of different toolkits. This mechanism exists outside the calling
> program and as such is independent of the code of the calling
> program (larger blocking sections etc.).
>
> As such, this solution seems to be the simplest both
> for the module and client application developer,
> the best controlable in terms of performance and the
> most independent of the code realization of the client
> application. I do not yet really see a good technical reason
> why an application would need to implement a different
> mechanism, although it is very well possible there is one
> and I'd like to have a discussion about this.
>
> Now there is a problem somewhere in PyGTK and Python which
> makes our thread wake up ten times a second without any
> reason. There exists a patch, OLPC is not the only project
> complaining and it will likely be fixed in Python 2.6.
>
> >From what I understand so far, it really seems to me the
> proper solution would be to fix Python instead of creating
> bypasses in Speech Dispatcher communication library. Such
> bypasses would need the library be patched AND all client
> applications which happen to use pygtk+speechd changed.
> For example the Orca screen reader, which is essential for
> accessibility and should eventually be shiped with OLPC,
> would have to be changed. Fixing Python fixes the issue
> for Orca as well automatically, as well as fixing a lot
> of other things.
>
> Please let me know what do you think and most important
> if there are any good technical reasons why the threading
> solution is not desirable apart from the currently discussed
> bug in Python or PyGTK. In such a case we could modify the
> SSIPClient class to not depend on threads and inherit it into
> one class that uses threads and into another one that initializes
> synchronously and then hooks into a toolkit-specific mainloop.
>
> With regards,
> Hynek Hanke
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> Tomeu Vizoso píše v Ne 22. 06. 2008 v 09:39 +0200:
>> On Fri, Jun 20, 2008 at 6:42 PM, Tomas Cerha <cerha at brailcom.org> wrote:
>> > Hynek Hanke wrote:
>> >> It seems to me a clearer solution to just
>> >> let the speechd SSIP bindings do their work in their own thread than
>> >> to extract part of the internals into your program (the socket selects).
>> >
>> > Yes, this would definitely be desirable, if possible.
>>
>> The problem for us is that when pygtk is instructed to activate
>> support for threads, a timer is set up that fires 10x per second. This
>> is pretty bad for us, as we need to suspend as frequently as possible
>> in order to save battery. This issue will be fixed in python 2.6 and
>> we could be patching python 2.5 in order to avoid this, but patching a
>> so basic package as python is very inconvenient for us. For more
>> details, see:
>>
>> http://blogs.gnome.org/johan/2008/01/04/enough-wakeups-in-python-programs/
>>
>> >> This should be considered carefully as not to create problems later
>> >> when for instance we want to switch from a text socket protocol
>> >> to something else.
>> >
>> > Well, the `SSIPClient' will probably always implement the API over
>> > sockets.  Other classes, however, may implement the same API over
>> > different communication channels.  It might be interesting to think, for
>> > example, how DBUS might help with respect to given problems.
>>
>> Hmm, a D-Bus API in speech-dispatcher sounds quite interesting. As an
>> alternative to threads when using a socket, asyncore might be
>> interesting, although haven't looked yet at how to integrate
>> asyncore's loop with pygtk's:
>>
>> http://docs.python.org/lib/module-asyncore.html
>>
>> Regards,
>>
>> Tomeu
>>
>> _______________________________________________
>> Speechd mailing list
>> Speechd at lists.freebsoft.org
>> http://lists.freebsoft.org/mailman/listinfo/speechd
>>
>
>