#5235 BLOC Never A: Installing WikiBrowse use 99% cpu for several minutes

[Zarro Boogs per Child]

#5235: Installing WikiBrowse use 99% cpu for several minutes
-------------------------------+--------------------------------------------
  Reporter:  marco             |       Owner:  tomeu         
      Type:  defect            |      Status:  new           
  Priority:  blocker           |   Milestone:  Never Assigned
 Component:  journal-activity  |     Version:                
Resolution:                    |    Keywords:  Update.1?     
  Verified:  0                 |  
-------------------------------+--------------------------------------------

Comment(by tomeu):

 Three different issues here:

  1. Journal tries to retrieve the icon of bundle entries, even when
 there's no file yet (downloads).

  2. Browse doesn't write the downloaded file with transfer_ownership=True,
 so we do a copy instead of a move. Takes 14s. of cpu to copy that 15MB
 file to the datastore.

  3. Every time there's progress on a download, the journal extracts the
 icons of the bundles that are being displayed as entries. If one of those
 entries is a big bundle this can take a lot of cpu. 40s. to extract the
 icon from the 15MB bundle. Progress is updated every 5s. or every 10%.

 Fix for 1:

 {{{
 diff --git a/misc.py b/misc.py
 index 5980e83..442b24a 100644
 --- a/misc.py
 +++ b/misc.py
 @@ -47,7 +47,7 @@ def get_icon_name(jobject):

      file_name = None

 -    if jobject.is_activity_bundle():
 +    if jobject.is_activity_bundle() and jobject.file_path:
          try:
              bundle = ActivityBundle(jobject.file_path)
              file_name = bundle.get_icon()
 }}}

 Fix for 2:

 {{{
 diff --git a/downloadmanager.py b/downloadmanager.py
 index f129c0a..07374f1 100644
 --- a/downloadmanager.py
 +++ b/downloadmanager.py
 @@ -167,6 +167,7 @@ class Download:
                  self._dl_jobject.metadata['mime_type'] =
 sniffed_mime_type

              datastore.write(self._dl_jobject,
 +                            transfer_ownership=True,
                              reply_handler=self._internal_save_cb,
                              error_handler=self._internal_save_error_cb,
                              timeout=360 *
 DBUS_PYTHON_TIMEOUT_UNITS_PER_SECOND)
 }}}

 For 3 I don't have a one-liner :/

 We should definitely be caching the extracted icon, perhaps even across
 sessions, by saving the svg icon in a property of the entry metadata.

 Even with caching, we shouldn't block for so long the first time we
 retrieve it. Perhaps we could extract the icon using unzip as an async
 external process instead of zipfile?

-- 
Ticket URL: <http://dev.laptop.org/ticket/5235#comment:7>
One Laptop Per Child <http://dev.laptop.org>
OLPC bug tracking system