[Testing] Tubes and Journal minutes from 8/16/07

[Tomeu Vizoso]

Follow some notes about backups.

On Tue, 2007-10-16 at 14:41 -0400, Kim Quirk wrote:
> JOURNAL
> 
>       * Marco - need to implement something to drop entries after the
>         journal gets too long. If something is starred, it won't be
>         deleted. We were waiting for the school server back up to be
>         available before deleting stuff. 
>       * Eben - there could be some visual design to show people ahead
>         of time what is scheduled for deletion. 
> 
> 
> Plan for laptop backing up to school server
> 
>       * Laptop backing up to the school server; something needs to
>         notify that it is connected to a school server (NM); if it has
>         been more than x hours, it should initiate an rsync. 
>       * Tomeu will talk to Dan Winship to get the dbus info that says
>         the XO is connected to a school server. Then he will ask if it
>         has been >2 hours (with an appropriate amount of random sleep
>         time). Then it will initiate the back to the school server
>         (Dan Winship wrote this rsync script in the past). Need to
>         mark in the datastore that an object has been backed up.

We were thinking of just noting in a file somewhere the modification
timestamp of the newest item that was copied.

>       * Kim will add a Trac item for a future feature: how to do a
>         full restore from school server.
>       * Scott needs to get the datastore info so he can create a
>         program to read the datastore from the school server. Kim will
>         hook up Scott and Ben to see if Ben can do some of the work.
> 
> 
> Three pieces of info for school server backup:'
> 
>         - XO needs to know it is on a school server and initiate a
>         backup
>         - School server needs to display the files on a webserver

I'm afraid we'll need some paging, filtering or searching here. If we
present a simple list of items, and there are thousands, how will the
children be able to find anything? Note that we don't have search in
Browse yet. Also, what will be the memory impact of a so big list in
Browse? Could be quite high, depending on the HTML used.

>         - XO can retreive one file at a time into the journal from the
>         school server webpage.

Could be nice if the server component could bundle using zip the actual
file plus metadata in a .xoc file. After Browse has downloaded it and
put into the Journal like with all downloads, the Journal could then
expand it and put the metadata into xapian.

This would also serve for sharing journal entries, one journal would
send through tubes a bundled entry, and the other side would expand it
and recreate in its own datastore.

>         - For full restore, we will define this as a future feature. 
> 
> 
> Space management - auto delete:
> 
>       * Every day the autodelete job runs: It looks at the total
>         number of items in the journal (if over 500), then looks at
>         only items that have been backed up and are older then 3
>         months, and it will delete those.

Perhaps we can be smarter about which items are we going to delete and
take in account other parameters like:

- number of accesses,
- date of last access,
- date of last resume,
- Keep or Star flag,
- size it takes on disk, etc

So we delete the entries that the kid is less likely to miss.

>       * Eben will send around ideas for more friendly view of items
>         that are going to be deleted.

Tomeu