[Http-crcsync] General comments on crcsync document

[Patrick McManus]

Hi Toby, et al!

On Fri, 2009-07-17 at 08:00 +0200, Toby Collett wrote:

> 
> The issue with the numbers is that in the case of say a credit card
> transaction, if we have a strong hash failure we cannot simply
> retransmit the data, which means the user will see an error page.

this is true with the failure of any non-idempotent http transaction. A
meaningful evaluation would need to take into account existing failure
rates from other sources of this kind of transaction (e.g. network,
server, firewall failures) to see it in context instead of just as a raw
number. (i.e. if we've already got a 1 in a million failure rate
baseline - adding a 1 in 100 million crc failure rate doesn't matter
much. these numbers are made up, but personal experience tells me that
natural failure rates are a lot more common than 1 in a million.)
Because of this inherent unreliability, applications have provisions for
resubmit built in - though I agree it is nice not to use them.

>  I will sit down later today and see if I can come up with some
> numbers and then run them past the list...

one thing you need to take into account, and I'm not really sure how to
do this, is to consider that a crc is not a uniform hash. It is designed
to catch small differences with a probability better than 1 - 1/2^32 ..
but large differences with a probability worse than that.. the question
is where does a false positive delta fall on that scale - being a delta
it is not independent data. My gut says this works in crc's favor.

finally, it is not purely a matter of optimal sizing. if crc32c is "good
enough" we should be biased in its favor because of the ubiquity of
quality implementations (software and hardware) available for it.


>         > Or am I now overcomplicating things?

algorithm negotiation is not a big deal to me, but doing it in the way
proposed is consistent with basic http mechanisms, which is a good thing
in an http extension imho. Minimally I think it would be good to state
the algorithm in the if-block header, giving room for a v2, while
stating in the spec that XXX is the only currently defined value and
negotiation is currently unspecified.

-Patrick