[Server-devel] Reverse proxies on the XS

[Martin Langhoff]

2008/3/9 Ivan Krstić <krstic at solarsail.hcs.harvard.edu>:
> On Mar 8, 2008, at 11:02 PM, Martin Langhoff wrote:
>  > brosers misbehave plenty just to survive "in the
>  > real world"
>
>  Can you provide references for this assertion with regard to HTTP-
>  standard cache control and validation headers as processed by non-
>  ancient browsers?

It is quite possible that current browsers are mostly correct
http-wise. If you really need references, I'll be glad to hunt them
down though that will be quite time consuming. Still, from the POV of
our upstream webapp developers, the webapp *must* work correctly in a
world full of ancient browsers (luckily, just a handful) and
misbehaving proxies (unfortunately, thousands of implementations out
there).

So trusting that the proxy+client side will do the right thing hasn't
been a profitable stance so far, and although that's improving
progress is slow. We have a controlled platform, so within the space
we control, we can make some assumptions, meaning that we can tighten
the caching headers in our webapps where it's easy. But you can't
count on the upstream project to be producing good caching headers -
they wouldn't be able to use it on the Internet.

>  > Going back to the memory-heave processes -- if the process can produce
>  > the output quickly but the clients are potentially slow it makes a lot
>  > of sense to offload the serving of bytes to a small-mem-footprint
>  > process, and keep less apache/php processes around and very busy.
>
>  That's not an argument for a reverse proxy; it's an argument for a
>  smarter webserver. Lighty recently grew the ability to do zero-copy
>  FastCGI output, for example.

Agreed. What's needed is good buffering between the memory-heavy
process and the outside world. Which means a 2-tier scheme of
lightweight and heavyweight processes - in that sense, FastCGI and a
reverse proxy are quite similar. If we can get a robust FastCGI setup
- something that has eluded me in the past ;-) - it should be faster
and cheaper in RAM.

cheers,


martin