[Http-crcsync] Progress on implementing CRCSync client logic to base encoding on 'similar' pages in the cache

[Alex Wulms]

Op 02-08-10 17:52, Martin Langhoff schreef:
> Hi Alex,
>
> good progress!
>
> On Sat, Jul 31, 2010 at 7:18 AM, Alex Wulms <alex.wulms at scarlet.be> wrote:
>   
>> Next step is that I must implement the logic to maintain the memory
>> structure for each new page retrieved (like said above, at the moment it is
>> only initialized/loaded on startup). Once that is done I'll check-in the
>> changes into the GIT repository.
>>     
> One tricky aspect is that (AFAIK) apache processes don't share memory
> (with few exceptions, and those are very expensive
> performance-wise...).
>
> My humble understanding is that apache is fast because it avoids
> shared-memory segments that would need locking semantics to update --
> contention around the locks is usually the problem.
>
> Can we design a scheme to store that info in the filesystem, in a way
> that doesn't require locking (using posix 'mv' atomicity...)? It could
> be on a designated directory that we can then setup as a tmpfs...
>
> cheers,
>
>
> m
>   
APR (apache portable runtime) supports SHM (shared memory)
functionality, including a global lock function. I'm using it. Using SHM
and especially the global lock might be relatively slow compared to
using synchronization and data sharing features in a
single-process/multi-threaded application but my gut feeling tells me
that the time spent on the locking and SHM access is neglect-able
compared to the time required to read the page-body from the
file-system-cache (in order to calculate the CRC blocks) and the time
required to write the updated response back to the file-system-cache.
Obviously I keep the critical section as small/short as possible, to
reduce the chance on contention and scalability issues related to
serialization of the calls.

The APR based SHM code is actually also used by mod_ssl and mod_ldap.
That is where I found my inspiration ;-)

Cheers,
Alex