MANIFEST and .xo bundles

[Jameson "Chema" Quinn]

Since Develop now relies on bundlebuilder to create its journal entries (.xo
files), I have become familiar with the fragility of the activity bundle
format. The original sin is that, if you have a bundle of files, you
shouln't need a separate list of those files in MANIFEST. The various
symptoms all flow from this: mistyped files (doesn't save), including the
root directory in the paths (ditto), adding a file with cp but forgetting to
add it to MANIFEST (deleted on next activity update), including MANIFEST
itself in the manifest (no effect, a random source of variation)

What is the MANIFEST supposed to accomplish in the first place?

1. Something like .gitignore
problems: because it's a whitelist instead of a blacklist, there is always
the possibility that, even if Develop carefully maintains the integrity,
someone will add an important file behind Develop's back. This file would
tend to be deleted on the next update.
suggested solution: just use a file named .gitignore - follow the standard
(though for simplicity's sake it would be OK to only obey the top-level
gitignore). Moreover, we could have an implicit global .gitignore which
would ignore (at least) .pyc files.

2. Give a clear ordering of the files for cryptographic signing purposes.
problems: obvious
solution: obviously just use a well-defined and documented ordering
algorithm

So, my suggestion would be to abandon the MANIFEST and use .gitignore, with
an implicit global default.

If not, I will have to rewrite Develop to take care of the MANIFEST
automatically, and not show it for manual editing. Problems ensue: if the
MANIFEST is absent or missing files, Develop will have to use a modal dialog
box to ask what to do. This solution seems ugly to me.

.........................

While I'm writing an email on this stuff, I should include a brief
discussion of activity auto-updates. Currently, an activity is reinstalled
iff it is currently installed in Activities/ and the user runs a bundle with
a different version in activity/activity.info . This means that in Develop,
you have to change the version for every debugging cycle - pretty annoying.
It also means that downgrades are treated identically to upgrades, and of
course there is no security.

I suggest that:
-when running the same or previous versions (or, in the future, versions
signed by someone you don't recognize as a developer for that activity), a
modal dialog box should come up for a few seconds with the default option to
not reinstall (which, if attempting to join a shared activity with a newer
version, would mean not joining. Backwards compatibility, on the other hand,
would be assumed, and graceful failure would be the activity's
responsibility)

-develop a two level system of signing activity bundles: "maintainers" would
have to sign/revoke "maintainers" and "developers", and "developers" would
sign specific versions of the activity bundle; credentials would be chained
and kept, so that missing intermediate versions would not be a problem.
Ideally the signature would be valid for the unpacked directory with the
same ignores as .gitignore, not just for the specific compressed version of
it, so that activity bundles could be recreated on the fly for sharing
purposes. (I forget who's idea this is based on, apologies). (This may be
overdesign, but the good part is that it keeps simple "I sign this bundle"
separate from the whole maintainer/developer chain, so the simple part could
be implemented first and remain unchanged later.)

-Any activity signed by the current user is reinstalled always (allows easy
debug cycle and reverts)

-For now, develop would create its own key pair. When bitfrost's P_IDENT is
implemented, it would use that.

(it may be that even when P_IDENT is implemented, it requires activities to
include their own hash in all signed data: "signed by me using Develop
activity" instead of just "signed by me". I suspect that this would not
require any backwards-incompatible change to the signature format used)

Jameson
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.laptop.org/pipermail/devel/attachments/20080330/4801e614/attachment.html>