[Code-review] [PATCH] Maintain a metadata copy outside the index.

[Martin Langhoff]

On Fri, Jun 20, 2008 at 6:00 AM, Tomeu Vizoso <tomeu at tomeuvizoso.net> wrote:
>>  b) to inform an in-flight backup process that we're out of space so
>>  that it can make its own decision about whether to continue.
>
> Martin, any ideas?

I don't think this is a concern. With this patch in, backup does not
need to create *any* files outside of tmpdirs. Of course I want to
test it once we are done (and fix any problem), but the current design
seems to be sane in that regard.

NOTE NOTE NOTE: the backup process does not make too many promises
about the internal consistency of the data being backed up. Downstream
users of the data will have to fend for themselves - spot inconsistent
data. On the upside, if backups have been running, there'll be several
versions to pick from. We are no different from Amanda, for example,
in that regard.

(Solutions? No practical ones: Ask activity writers to aim for atomic
updates, lock-the-whole-system-on-backup, use a snapshot-capable FS.)

>> The thing I'm getting at is that we may well be invoking the backup
>> software _because_ the system is in a crazy state and we want to try to
>> save our data. It would be really good if this were feasible to achieve.
>> (I'm not quite sure whether it is.)

BTW, here I do agree w Michael - it is an important use case.

>> Incidentally, why do we hardcode saving the .metadata files into the
>> 'base' DS directory? Sure, it's a sensible default setting, but if I had
> Well, metadata is being put next to the files they relate to. I'm
> supposing here that we only are interested in the metadata if we have
> the files and vice versa.

+1 with Tomeu. Metadata should sit with the data - not elsewhere.

>> Also, what permissions are going to be assigned to the .metadata files?
>> Are these permissions correct?
>
> 644, looks like. Which permissions should it have? Same as the data file (604)?

Same as data file I say.

cheers,



m
-- 
 martin.langhoff at gmail.com
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff