[Server-devel] Apache proxy CRCsync & mozilla gsoc project?

[Martin Langhoff]

On Wed, Apr 1, 2009 at 8:17 PM, Toby Collett <thjc at plan9.net.nz> wrote:
> So a quick question, what sort of http transfers are chunking most often
> used for? I believe we will get poor results with the method for most types
> of binary data, which tend to be the larger files. In the web context these
> will generally have not changed at all (in which case traditional caching
> will help) or will have changed completely in which case the hashing is just
> overhead. Happy to be corrected on this point.

I agree. We can apply this method exclusively to */text and */xml mimetypes.

And something I forgot on the earlier pro-streaming notes: the memory
model of apache doesn't really release memory back to the kernel -- it
keeps it in the per-process memory pool. This means that if we have
unbound memory allocations (such as buffering whole requests), then
our memory usage will be terrible.

It's a bit less horrid with worker-threads, but in general, apache
modules usually strive to maintained fixed-sized buffers.

So it's something to keep in mind :-)

cheers,


m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff  - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff