helping the OLPC project with ejabberd

[Robert McQueen]

Hello Ejabberd Community,

I'm one of the people at Collabora Ltd (www.collabora.co.uk) working on
the collaboration stuff in the OLPC (One Laptop Per Child,
www.laptop.org) project. We're using our Telepathy
(telepathy.freedesktop.org) framework to present the same APIs to the
shared activities whether or not the laptop is using link-local
XMPP/mDNS/multicast or an XMPP server to communicate.

For the XMPP server, we're using ejabberd running on jabber.laptop.org
(and also a test server on olpc.collabora.co.uk), but in a rather
unusual configuration which is turning out to be quite unreliable. As
our expertise is more focussed on the client side software, we'd like
some assistance on how to improve things on the server side,
particularly some input on making a more scalable solution.

We've written up some documentation explaining how we've configured our
ejabberd:
 http://wiki.laptop.org/go/Ejabberd_Configuration

And the server extensions/protocols we make use of:
 http://wiki.laptop.org/go/XMPP_Extensions

For more general documentation about how the shared activity stuff
works, see:
 http://wiki.laptop.org/go/Activity_Sharing
 http://wiki.laptop.org/go/Shared_Sugar_Activities
 http://wiki.laptop.org/go/Shared_Activity_Protocol_1.0

Currently the jabber.laptop.org server is running 1.1.3 with Magnus
Henoch's PEP support patched in, and has about 50 active users and 4000
accounts in total, and we *already* have some reliability problems. We
know that in the very near future we're going to get 1000s (or even
10,000s) more users from the G1G1 (give one get one,
www.laptopgiving.org) programme.

We know that the shared roster stuff is a crazy hack and we very
urgently need to stop relying on it, so we're planning to implement an
XMPP component to use as a registry of OLPC activities and buddies, so
that the client can search for or be notified about smaller, more
relevant sets of information, depending on what's being displayed in the
UI. However, while we're writing that, we need to make sure we can keep
our current configuration stable, and if possible make some server
configuration tweaks to help it scale.

(for example, Aleksey has told me a hack in mod_shared_roster to stop
the actual shared roster group from being included in the initial roster
push, although leaving the presence/PEP stuff behaving correctly, which
will help reduce the bandwidth usage at sign-on - we just present anyone
in the UI we have presence/PEP notifications for, rather than using the
roster for very much at all)

So, I've got a few questions which I'd be very grateful for any input on:

1. Does anyone have any ideas why our ejabberd sometimes exits (leaving
epmd behind, but no ejabberd processes) without logging anything, and
suggestions how to debug it?

2. Sometimes we see this in the log (although not recently on
jabber.laptop.org which runs 1.1.3, just on olpc.collabora.co.uk which
has 1.1.2):
 =ERROR REPORT==== 2007-10-09 20:26:00 ===
 Mnesia(ejabberd at dhansak): ** WARNING ** Mnesia is overloaded:
 {dump_log, time_threshold}
What causes it? Does it result in any degradation of service?

3. The XO machines are intended to use just IPv6 at some point in the
future. Currently jabber.laptop.org is just listening on the machine's
IPv4 address even though it has a routable IPv6 address. How do we make
ejabberd listen on IPv6?

4. Thinking of making interim server-side fixes to aid scalability while
we work on an OLPC directory component, we don't actually need to see
*all* of the contacts in the roster. We actually just need a union of a)
the child's friends, b) people nearby on the network (imagine two
children with laptops next to each other, but not being able to add each
other, and c) some other people in case a and b are small/empty. How
hard would it be to patch mod_shared_roster, or make a new module, so
that you could:
 a) have a group that had users that were e.g logged in on the same /24
(or equivalent /96 or whatever for IPv6) subnet?
 b) have a group that had a random N (say 30) online users in, or
somehow assigned users to groups on the fly when they signed in so that
each user is in a group that has N other online users in?

5. Thinking of our XMPP component, we're no experts in erlang so we're
planning to write it in as an external component, probably prototyping
it Python. This has the benefit that it can be used with other Jabber
servers, reducing the "disruption" to existing server admins who want to
support XO clients. We'd like to know if there are any ways that an
external component can access any of the information in the server, such
as people's presence, rosters, what PEP nodes they have, or even what
MUC rooms they are in. Anything we can find from the server will help us
reduce the need for clients to report redundant information to the OLPC
server component.

Many thanks in advance for any advice you can offer.

Regards,
Rob