[Code-review] [PATCH] Maintain a metadata copy outside the index.

[Michael Stone]

On Wed, Jun 18, 2008 at 09:12:31PM +0200, Tomeu Vizoso wrote:
> +try:
> +    import cjson
> +    has_cjson = True
> +except ImportError:
> +    import simplejson
> +    has_cjson = False
> +

May I assume that at least one of the packages providing these python
libraries is listed as a dependency of the sugar-datastore RPM?

Also, do we need to test this code against both libraries?

> +    def _export_metadata(self, uids_to_export):
> +        uid = uids_to_export.pop()
> +        props = self.indexmanager.get(uid).properties
> +        self._store_metadata(uid, props)
> +        return len(uids_to_export) > 0

I take it that we're not worried about the effect of concurrent writers
on our stored metadata?

What happens if our _store_metadata() function dies with something like
ESPACE?

> +    def _encode_json(self, metadata, file_path):
> +        if has_cjson:
> +            f = open(file_path, 'w')
> +            f.write(cjson.encode(metadata))
> +            f.close()
> +        else:
> +            simplejson.dump(metadata, open(file_path, 'w'))

Hard power failure during this write could generate a corrupted
.metadata file. We should write the metadata to a tmpfile in the same
directory, then rename() the tmpfile into place. That way the update
will be (closer to) atomic.

> +    def _delete_metadata(self, uid, props):
> +        path = os.path.join(self.base, uid + '.metadata')
> +        if os.path.exists(path):
> +            os.unlink(path)

The props argument seems to be unused.

----

General question: what code reads the .metadata files?

Thanks,

Michael