#10722 NORM 11.2.0-: wiki cache invalidation headaches (was: os-builder via bitfrost returns stale data)

[Zarro Boogs per Child]

#10722: wiki cache invalidation headaches
---------------------------------+------------------------------------------
           Reporter:  jvonau     |       Owner:  cjb      
               Type:  defect     |      Status:  reopened 
           Priority:  normal     |   Milestone:  11.2.0-M4
          Component:  website    |     Version:           
         Resolution:             |    Keywords:           
        Next_action:  never set  |    Verified:  0        
Deployment_affected:             |   Blockedby:           
           Blocking:             |  
---------------------------------+------------------------------------------
Changes (by dsd):

  * status:  closed => reopened
  * component:  build-system => website
  * version:  1.5/1.0 Software Build os860 aka 10.1.3 =>
  * milestone:  Not Triaged => 11.2.0-M4
  * resolution:  worksforme =>


Comment:

 Michael, Chris, Scott and myself chatted about this on IRC yesterday
 because I got hit by the same problem (yep its annoying!).

 Key points:
  * wiki.laptop.org runs behind a squid proxy
  * When a page gets edited, MediaWiki asks squid to invalidate its caches
 of that page
  * squid effectively maintains per-user or per-session caches (separating
 sessions through differences in HTTP headers), this is necessary because
 everyone sees a slightly different version of the page (e.g. mine says I'm
 logged in as DanielDrake, nobody else should be served that cached
 version)
   * This explains the case above where firefox shows a different result
 from olpc-os-builder
  * MediaWiki's cache invalidation is limited. If the Activities/11.2 page
 includes the ClockActivity page, when you make an edit to the
 ClockActivity page then only the ClockActivity cache will be purged; 11.2
 caches will remain in use (and now be stale)
   * This is a well-known limitation, unlikely to get fixed easily
   * This explains the original problem seen
  * It's not limited to urllib/osbuilder/sugar-update-control, I saw it a
 few times while using firefox to create the 11.2 activities page last
 week.

 Workarounds/solutions:
  * Wait patiently a number of hours for caches to expire
  * After updating ClockActivity, make a trivial edit to pages such as 11.2
 that include it so that caches get invalidated
   * This worked for me once, but failed the second time: my browser showed
 the new version, but olpc-os-builder continued receiving the old
  * Michael says that using action=purge (e.g. visiting
 http://wiki.laptop.org/index.php?title=Activities/G1G1/11.2&action=purge )
 should cause squid to drop caches
   * I tried this, it didn't work, software updater still got old content
  * Include "no-cache" headers in the HTTP request
   * This worked in Firefox (Ctrl+F5) when I saw the problem
   * It also solves the problem for olpc-os-builder if you run the attached
 python utility first
   * I don't want to make this change in bitfrost itself, because in other
 cases (particularly in deployment contexts), use of caches would be
 desirable
  * Modify squid configuration to skip cache for urllib clients e.g. "cache
 deny browser <regex>"
   * This is the approach I'm going to pursue next.

-- 
Ticket URL: <http://dev.laptop.org/ticket/10722#comment:9>
One Laptop Per Child <http://laptop.org/>
OLPC bug tracking system