[Http-crcsync] crccache ready for some testing I think

[Alex Wulms]

A cool. That makes sense. So we should also foresee something in the protocol 
that the client and server can negotiate the delta type (block versus byte 
level). I guess that the server could specify it's http-sync awareness with 
some response header when a client asks for a page for the first time.

Op zondag 5 april 2009, schreef Toby Collett:
> The complete hash scenario (as long as we are hashing the trailing block
> then we dont need the strong hash) is not relevant in the two proxy case
> that we are working on at the moment, but could be used by http-sync aware
> servers who know how their own content has changed. For a news site that
> has just updated a story, or any site which embeds rotating ads in their
> content, it would allow them to return on the changes at a byte resolution
> rather than at a block resolution.
>
> The classic approach provides a means to determine whether a new page needs
> to be fetched, I was hoping to provide a means that an update can be sent
> as the response instead of requiring a full page fetch. This does require
> the server to know exactly which page you have locally though. Actually
> thinking about it sending the etag of the cached page would actually be
> pretty good for this, as it is up to the server to decide what goes into
> that field so they could use it quiet effeciently.
>
> Toby
>
> 2009/4/6 Alex Wulms <alex.wulms at scarlet.be>
>
> > Op zondag 5 april 2009, schreef Rusty Russell:
> > > On Sunday 05 April 2009 07:33:06 Alex Wulms wrote:
> > >
> > > I think Toby's draft, which suggested just leaving the trailing block
> > > as a literal, is an interesting possibility.  Worst case, it's 19
> > > bytes.
> >
> > Actually, in worst case scenario it is (blocksize-1) = (pagesize/20 - 1).
> > But that is not the main reason the code had to be adapted. Main reason
> > is that the automatic tail detection in the previous version of crcsync
> > library
> > did give complications with keeping track of what was already written
> > back to
> > the client and what was still pending (especially when read_block
> > returned a 'result==0'  response), which had led to a subtle bug in
> > crccache. That is
> > the problem I wanted to get fixed, which is meanwhile done. The fact that
> > the
> > tailblock is now also encoded is just a nice added bonus of a problem
> > investigation and it's associated bug fix :-)
> >
> > > I also think we should go to 60-bit csums (but needs new CRC algo).
> >
> > I agree that that would indeed be better.
> >
> > > Finally, I think the strong hash sent in request in Toby's draft is not
> > > useful as the block signature + size can be used for caching in exactly
> >
> > the
> >
> > > same way.
> >
> > I was already wondering about the value of the strong hash in the request
> > in
> > Toby's draft.
> > What kind of scenario could that be used for? Would the purpose be that
> > the server caches complete pages? Can't that be handled with the
> > classical 'last-modified/modified-since/304-not-modified' protocol? The
> > purpose of crccache/crcsync is to cache the 'static' part of dynamic
> > pages. So I do not directly see the added value to use a strong hash or a
> > block-signature+size to detect 'unmodified' pages at the server level,
> > when a
> > simpler and faster protocol already exists.
> >
> > Kind regards,
> > Alex