#1938 BLOC Trial-2: System lags when switching to Journal

[Zarro Boogs per Child]

#1938: System lags when switching to Journal
-------------------------------+--------------------------------------------
  Reporter:  jfuhrer           |       Owner:  tomeu  
      Type:  defect            |      Status:  new    
  Priority:  blocker           |   Milestone:  Trial-2
 Component:  journal-activity  |     Version:         
Resolution:                    |    Keywords:         
  Verified:  0                 |  
-------------------------------+--------------------------------------------
Comment (by tomeu):

 The following data should give some insights about how are we spending
 time when refreshing the list view in the journal (numbers are on a b3):
 {{{
 0 entries:

 DEBUG - root: datastore.find took 0.167750s and returned 0 entries
 DEBUG - root: ListView._update took 0.000000s for 0 entries

 1 entry:

 DEBUG - root: datastore.find took 0.287241s and returned 1 entries
 DEBUG - root: ListView._update took 0.100000s for 1 entries

 DEBUG - root: datastore.find took 0.042793s and returned 0 entries
 DEBUG - root: ListView._update took 0.000000s for 0 entries

 DEBUG - root: datastore.find took 0.089548s and returned 1 entries
 DEBUG - root: ListView._update took 0.010000s for 1 entries

 DEBUG - root: datastore.find took 0.011119s and returned 0 entries
 DEBUG - root: ListView._update took 0.000000s for 0 entries

 DEBUG - root: datastore.find took 0.089636s and returned 1 entries
 DEBUG - root: ListView._update took 0.010000s for 1 entries

 2 entries:

 DEBUG - root: datastore.find took 0.266975s and returned 2 entries
 DEBUG - root: ListView._update took 0.030000s for 2 entries

 DEBUG - root: datastore.find took 0.217557s and returned 2 entries
 DEBUG - root: ListView._update took 0.030000s for 2 entries

 DEBUG - root: datastore.find took 0.012135s and returned 0 entries
 DEBUG - root: ListView._update took 0.000000s for 0 entries

 DEBUG - root: datastore.find took 0.211665s and returned 2 entries
 DEBUG - root: ListView._update took 0.040000s for 2 entries

 8 entries:

 DEBUG - root: datastore.find took 2.883188s and returned 8 entries
 DEBUG - root: ListView._update took 0.220000s for 8 entries

 DEBUG - root: datastore.find took 0.011771s and returned 0 entries
 DEBUG - root: ListView._update took 0.000000s for 0 entries

 DEBUG - root: datastore.find took 1.602133s and returned 8 entries
 DEBUG - root: ListView._update took 0.150000s for 8 entries

 DEBUG - root: datastore.find took 0.010984s and returned 0 entries
 DEBUG - root: ListView._update took 0.010000s for 0 entries

 DEBUG - root: datastore.find took 1.692017s and returned 8 entries
 DEBUG - root: ListView._update took 0.110000s for 8 entries

 16 entries:

 DEBUG - root: datastore.find took 4.397195s and returned 16 entries
 DEBUG - root: ListView._update took 0.280000s for 16 entries

 DEBUG - root: datastore.find took 0.046173s and returned 0 entries
 DEBUG - root: ListView._update took 0.000000s for 0 entries

 DEBUG - root: datastore.find took 4.338615s and returned 16 entries
 DEBUG - root: ListView._update took 0.270000s for 16 entries

 DEBUG - root: datastore.find took 0.079475s and returned 1 entries
 DEBUG - root: ListView._update took 0.020000s for 1 entries

 DEBUG - root: datastore.find took 1.861575s and returned 8 entries
 DEBUG - root: ListView._update took 0.120000s for 8 entries

 DEBUG - root: datastore.find took 0.205638s and returned 2 entries
 DEBUG - root: ListView._update took 0.030000s for 2 entries
 }}}

 Some comments:

 - Time spent in queries that return 0 results are the same regardless of
 the number of entries in the database. This makes me think the query
 itself is always fast, so no need of optimizing the data access.

 - Time spent in the datastore is proportional to the number of entries
 _returned_, so I suspect of the ORM in sqlalchemy.

 Ben, do you agree with the above? If so, do you think you can reduce
 considerably the amount of time spent there? Or should be better to
 implement paging in the journal for trial2? (a bit too late to add so much
 code, in my opinion)

-- 
Ticket URL: <http://dev.laptop.org/ticket/1938#comment:4>
One Laptop Per Child <http://laptop.org/>