One thing we need to do is think about the headers carefully, as this is the aspect of the project we could promote as a web standard. There is a large amount of flexibility we could put in to this, but as Rusty has said, if there is a way someone can implement a protocol wrong they will. So we need to keep it as simple as possible.<br>
<br>At the moment we append the block size and hashes for the blocks to the request. The response has a content encoding set, and will need a strong hash added. The number of blocks is fixed at 20 for the moment, with a hash size of 30 bits, which felt like a nice balance between overhead and performance. This keeps our header at around the 128 byte mark when you have base64 encoded the hashes (we dont pad the base64 encoding, so 30bits->5bytes).<br>
<br>The other aspect we need to standardise is the encoding of the response. Again at the moment this is a very simplistic binary encoding. The response is encoded in sections, each beginning with either a 'L' to indicate a literal section or a 'B' to indicate a matched block (actually we could make one a default and save a few bytes here). A literal section then has a 4 byte int in network byte order for the size of the literal section, followed by the data. a block section has a single byte indicating the block number.<br>
<br>There is no error checking in the encoding itself, this is assumed to be taken care in other layers, and we through in a strong hash on the whole file to make sure this is correct. There is a risk if we get a corruption of the literal length byte that we could try read a very large amount of data, not sure if this is acceptable.<br>
<br>Toby<br><br><br><br><div class="gmail_quote">2009/3/31 Gervase Markham <span dir="ltr"><<a href="mailto:gerv@mozilla.org">gerv@mozilla.org</a>></span><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">On 25/03/09 18:20, Toby Collett wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Not a GSoC project, just a project(crcsync is the name at the moment).<br>
Initial target is a double proxy server, one each end of the slow link,<br>
with dreams of web standards and browser integration following.<br>
<br>
Seems to me that both projects need the same upstream server extension<br>
to be able to send the deltas down. Current state of the apache modules<br>
is that all the major pieces are in place but not a lot testing and no<br>
optimisation has been carried out yet.<br>
</blockquote>
<br></div>
OK. So maybe the browser integration for this, or at least the groundwork for it, is what our SoC project should be. Particularly if you have Apache modules that work already.<br>
<br>
See<br>
<a href="https://wiki.mozilla.org/Community:SummerOfCode09:WebPagesOverRsync" target="_blank">https://wiki.mozilla.org/Community:SummerOfCode09:WebPagesOverRsync</a><br>
for where we are at the moment. We are getting incredible amounts of interest in this project - more than all the others combined. It seems like an idea whose time has come.<br>
<br>
Gerv<br>
</blockquote></div><br><br clear="all"><br>-- <br>This email is intended for the addressee only and may contain privileged and/or confidential information<br>