[Http-crcsync] http-sync standard

[Alex Wulms]

Hi,

There is one more open point that almost slipped my mind:

- How should crcsync aware web/application servers indicate which other pages  
are similar to the currently returned page, so that a crcsync client can use 
the current page as template to fetch other pages from the same site that are 
not yet in the clients cache. I was thinking about some header with a URL 
pattern with some wildcards or so. But I'm pretty sure there are some caveats 
if we oversimplify the pattern syntax.

Kind regards,
Alex
 

Op maandag 8 juni 2009, schreef Alex Wulms:
> Hi all,
>
> The updated version of the http-crcsync standard/discussion paper is in the
> git repository at:
> http://repo.or.cz/w/httpd-crcsyncproxy.git?a=blob;f=crccache/doc/http_crcsy
>nc_protocol.odt;h=26c746a1a97fb4edeea37ed1d6ce8bb19638a03e;hb=49dd514544163b
>ccc8be33bdee61603ea5664cbd
>
> There are still a few open points:
> - Number of hashes
> - Hash size
> - Re-apply original content-encoding or not on the re-constructed entity?
> - How to indicate an error in the re-constructed entity to an HTTP 1.0
> client.
>
> I would appreciate your input (especially of Toby, Tridge and Rusty) on
> these open points. How about discussing them next weekend on some irc
> channel, to get them closed?
>
> Kind regards,
> Alex Wulms
>
> Op donderdag 4 juni 2009, schreef Alex Wulms:
> > Hi everybody,
> >
> > Toby and I have recently discussed below idea of making http-crcsync an
> > extension to RFC3229. We both believe it makes a lot of sense. If nobody
> > objects, I'll start updating the protocol description document this
> > Sunday accordingly after which Toby and I can fully align the Apache
> > based code as well. Please note that Toby already implemented a first
> > step in this direction, in order to experiment returning a 226 in stead
> > of 200 response code with Apache.
> >
> > Pedro,
> >
> > Please don't let this protocol finetuning stop you from starting your
> > work on the firefox client. The core of the protocol (e.g. the
> > http-crcsync response body) will remain unchanged.
> >
> > Cheers,
> > Alex
> >
> > Op woensdag 13 mei 2009, schreef Toby Collett:
> > > Hi Alex,
> > > Finally got a chance to read this. Is a good description of the
> > > problem/scope.
> > >
> > > I have a question for all with regard to RFC3229. It would be entirely
> > > possible for us to design the http-sync to be an extension of RFC3229,
> > > basically add a new conditional request if-blocks-match or something of
> > > the like, and expect a response with 226 instance coding applied.
> > >
> > > If we take this approach then our block encoding just becomes a
> > > standard instance coding. The presence of a if-blocks-match header can
> > > be taken to imply an accepts instance coding 'blocks' or whatever we
> > > call it to save sending redundant information.
> > >
> > > The question then becomes, is it worth the effort, the last lot of
> > > traffic I saw in a quick google search on rfc3229 was in 2004 with
> > > regard to blogs and so on. So it is not widely deployed. On the other
> > > hand with careful design of our protocol we can be compatible, so is
> > > there a reason not to do it?
> > >
> > > Also a minor note, rather than having a small trailing block, why dont
> > > we just add the trailing data to the last large block, this way we dont
> > > end up with a hash thats almost the size of the block it represents (or
> > > in some cases bigger)
> > >
> > > Toby
> > >
> > > 2009/4/29 Alex Wulms <alex.wulms at scarlet.be>
> > >
> > > > Hi,
> > > >
> > > > Please find in attachment my current ideas on the http-crcsync
> > > > protocol.
> > > >
> > > > I hope you don't mind but I have drafted the paper in open office and
> > > > not as a
> > > > plain text file. An open office version makes it easier to visibly
> > > > divide the
> > > > document in logical sections in order to enhance the readability and
> > > > the structure. Though, I have added a text export for convenience, in
> > > > case somenone might not have OO available.
> > > >
> > > > As I mentioned yesterday in this list, the paper is based on the
> > > > insights that
> > > > have grown while working on the crccache proxy modules and after
> > > > studying the
> > > > various papers on delta-encoding (the RFC, Toby's draft and the
> > > > protocol proposal on the original rproxy site) and everything related
> > > > to it.
> > > >
> > > > Like Toby proposed, maybe it would make sense to convert this into a
> > > > wiki and
> > > > continue the work there. What do you think?
> > > >
> > > > Cheers,
> > > > Alex
> > > >
> > > > Op maandag 6 april 2009, schreef tridge at samba.org:
> > > > > Hi Toby,
> > > > >
> > > > >  > This is transmitted in two forms. One is an SHA1 hash of the
> > > > >  > complete chached file, and the other is a set of block hashes.
> > > > >
> > > > > ...
> > > > >
> > > > >  > Content-Hash:
> > > > >  > This will be an sha1 hash of the entire cached body, and will
> > > > >  > allow
> > > >
> > > > the
> > > >
> > > > >  > server to transmit deltas based on its knowledge of past
> > > > >  > versions of
> > > >
> > > > the
> > > >
> > > > >  > page.
> > > > >
> > > > > How do you imagine that this hash would be used? I don't think it
> > > > > is practical to think that servers would keep a record of all the
> > > > > dynamic pages they have been served out, and for static pages I
> > > > > think the normal cache tag mechanisms already work well.
> > > > >
> > > > > I think a SHA1 or similar of the servers generated page is really
> > > > > worthwhile, but a whole file hash of what the client has in cache
> > > > > doesn't gain anything that I can see.
> > > > >
> > > > > btw, I also wonder if you've seen this:
> > > > >
> > > > >   http://www.ietf.org/rfc/rfc3229.txt
> > > > >
> > > > > That is the result of earlier efforts to standardise delta encoding
> > > > > in HTTP. I was a little bit involved in that effort, although it
> > > > > concentrated on storing old copies on the server, which I wasn't
> > > > > interested in.
> > > > >
> > > > > Still, it should provide some very useful background to the current
> > > > > effort.
> > > > >
> > > > > Cheers, Tridge
> > > > > _______________________________________________
> > > > > Http-crcsync mailing list
> > > > > Http-crcsync at lists.laptop.org
> > > > > http://lists.laptop.org/listinfo/http-crcsync
> >
> > _______________________________________________
> > Http-crcsync mailing list
> > Http-crcsync at lists.laptop.org
> > http://lists.laptop.org/listinfo/http-crcsync
>
> _______________________________________________
> Http-crcsync mailing list
> Http-crcsync at lists.laptop.org
> http://lists.laptop.org/listinfo/http-crcsync