[Server-devel] BOF on apachecon Amsterdam

[Martin Langhoff]

On Tue, Apr 1, 2008 at 12:03 PM, Aaron Huslage <huslage at gmail.com> wrote:
> >  - We are running memory-hog-webapps based on mod_php and mod_python
> > in prefork mode... but we have very strict memory constraints. Any
> > hints on how to compile apache (on Fedora and Debian) so that the
> > memory is released to the OS pool rather than to the child process'
> > private 'free' pool?
>
> I don't think this is currently possible.

With alternative mem handling libs it has *always* been possible, but
often unstable/unreliable on linux. Current apache programmers can -
hopefully - shed some light into the current situation as now we have
more alternatives. It may be that php and python are even ready for
threaded worker model. Or that dietlibc has a different malloc. Or
that someone has a special custom malloc we can use.

Or perhaps FastCGI is the way -- hopefully not ;-)

Note - I hate to speculate ahead. Hopefully the apache crowd will tell
us what the state of things is.

> Another web server can probably do
> this (Lighttpd?), but that comes with its own issues.

Exactly. And we lose what apache brings to the party.
> One way to be to run the CoDeeN code. I'm sure they would be willing to work
> with us, since I know they want to open source the code.

ISTR jg telling me they are using a licensed proprietary proxy but I
could be wrong. Are you in contact with them? I would love to see them
around here :-)

> Otherwise, running
> mod_proxy on the XS with no caching

The majority of XSs will have a horrible connection - we _need_ very
smart caching working together with the upstream proxy as smartly as
humanely possible.

> What needs to be configurable from mod_perl? What custom behaviours are we
> talking about, or is this just a general ask for future needs?

We want

 - The caching and handling of 1.1 cache-headers to be solid.

 - The upstream proxy to be able to pass hints to the xs proxy of
files to prefetch. These hints would initially be of popular resouces
across schools and content we intend to "push" out, for example, for
sw or content updates.

 - For some content, we may even add a SHA1 as a "local etag" to stuff
that looks unchanging but doesn't carry etags, the upstream proxy can
then buffer the response and save retransmission if the SHA1s match.
This would allow us to workaround web apps that aren't cache-smart. As
long as we can do this quickly - before http times out - we can save a
ton of traffic. HTTP timeouts and other issues limit how much we can
do here, but even within those constraints, I think we can do a lot.

mod_proxy circa 1.3.x was completely closed to mod_perl and didn't
even play well with other modules. It was excellent as a "standalone"
module but it did not respect the request phases, so you could not
combine it with authen/authz handlers from other modules.

cheers,



m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff