[Server-devel] ToDO: Write up Proxy Strategy

[Martin Langhoff]

On Tue, Jan 13, 2009 at 11:42 AM, Caroline Meeks
<caroline at solutiongrove.com> wrote:
> rproxy - write down what needs to be done and how much.

Caroline and others pointed out today at xocamp that the rproxy notes
were never fully documented. There are some emails on this list about
it - but nothing complete. I've now put it in the wiki
http://wiki.laptop.org/go/XS_Blueprints:rproxy - here's the initial
version for discussion...

Notes on an rproxy implenentation for the XS.

We want to have an optional, easy to switch on rproxy setup on the XS.
rproxy can replace or complement the main XS http proxy - if we have a
stable, efficient, high quality http proxy on the XS, pairing it with
rproxy would be a good idea.

As long as we are using Squid, it's not such a good idea :-(

=Rationale=

rproxy is a proxy that expends HTTP 1.1 proxying and provides much
better performance over very bandwidth constrained linkes. It requires
an "upstream" rproxy to match the downstream rproxy. For deployments
where the schools are connected to the internet with very limited
uplinks, rproxy can be a huge win.

Modern internet is made of moderately dynamic content --- the content
is mostly static but un-cacheable due to rotating banners or "latest
news" boxes on the side. Even when content changes, the template for
the content is often 80% of the HTML downloaded. rproxy provides huge
gains with such content.

More about what rproxy is
* http://rproxy.samba.org/
* http://ozlabs.org/~rusty/rproxy.html

=How it would work=

For a deployment with limited connectivity into the schools

 - setup a co-located rproxy server at the ISP
 - enable rproxy on the XSs deployed, pointing to the ISP

Note: the rproxy server at the ISP must only accept requests from
valid clients - either from a defined netblock or whitelisted IP
addresses. An open rproxy is an open proxy!

=Issues with rproxy=

* The working version uses an old version of librsync - which may have
odd bugs and is generally unsupported, untouched, unknown
* There's a half-finished port to the new librsync. Martin Poole
started it a while ago, and abandoned it.
* Does not stream the content through - blocks until the content is
fully fetched before delivery to requester.
* Unknown performance and memory footprint when hit by many users concurrently
* Needs packaging, init scripts, etc...
* Unclear if it behaves reasonably well with ajax "streamed" requests
- it may break or add problematic latencies, etc

=TO DO=

* Test and ensure it works with modern internet usage patterns. Gmail,
youtube, etc...
* package, configuration, init scripts...
* Test under load, paired with a conventional http proxy - may need
performance / memory trimming
* Port to modern librsync
* Make it work with streamed content (possibly the hardest task!)


...

cheers,



martin
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff