#4406 HIGH Future : XO leaves trash on USB sticks

[Zarro Boogs per Child]

#4406: XO leaves trash on USB sticks
------------------------+---------------------------------------------------
  Reporter:  gnu        |       Owner:  tomeu                            
      Type:  defect     |      Status:  new                              
  Priority:  high       |   Milestone:  Future Release                   
 Component:  datastore  |     Version:  Development build as of this date
Resolution:             |    Keywords:                                   
  Verified:  0          |    Blocking:                                   
 Blockedby:             |  
------------------------+---------------------------------------------------

Comment(by tomeu):

 Replying to [comment:12 Eben]:
 > Replying to [comment:11 tomeu]:
 > > I would also like to add that the issue in my opinion is not xapian
 storing 19 files and two dirs. I think that writing all that info inside
 only one file is equally good or bad for the user.
 >
 > I think I agree on this point.  It seems that as long as everything is
 tucked within a single hidden top level directory, things can be equally
 as clean.  In either case, we should attempt to minimize the size of
 anything we write here.
 >
 > > The problems with the current approach are:
 > >
 > >  * need to scan the whole drive every time,
 > >
 > >  * the metadata is stored in a binary format that breaks between
 xapian releases.
 > >
 > > The index can be recreated (and cached outside the usb stick), but if
 we store the metadata in the device, using a simpler plain text format
 would be better.
 >
 > I'm not sure if I can be of any help regarding the above, from a user
 experience perspective (apart from, of course, recommending we prevent
 things from breaking when possible, but that goes without saying).  If you
 have more specific questions on experience, let me know.

 Well, the intended user experience impacts directly these questions.

 If we want to be able to do fulltext search inside usb sticks, then we'll
 need to index the names of the files plus all the metadata we want to
 extract from the files, including part or all of the text inside the
 document. In order to make sure we have the fulltext updated, we'll need
 to scan the device after every mount.

 If we want to transfer metadata along with files then we need a way to
 store it. Either we use zip files that encapsulate the file plus the
 metadata or we store it hidden somewhere in the usb stick. The first
 option is less convenient when the files are seen from outside sugar, the
 second is what we have today.

 One more thing: the current DS stores a lot of unneeded info in the index,
 we could greatly reduce the amount of data in the index, the amount of
 time spent indexing and the number of files inside .olpc.store.

-- 
Ticket URL: <http://dev.laptop.org/ticket/4406#comment:13>
One Laptop Per Child <http://dev.laptop.org>
OLPC bug tracking system