[Server-devel] [PATCH] postprocess.py gets _actually_ fleshed out- and incrontab tweaks

[martin.langhoff at gmail.com]

From: Martin Langhoff <martin at laptop.org>

Take two on the patch posted earlier. 2 of the 3 files got dropped.

To recap

postprocess.py now provides hardlined, timestamped
directories, and maintains a symlink pointing to
the latest transferred directory.

the timestamped directories are maintainted with a
resolution of the nearest minute - we don't expect to
have more than one per hour.

The incrond crontab gets updated to deal with rsync
clients too. To work well with rsync, the rsync client
must use the -T option to avoid creating tempfiles in the
monitored directory.

Strangely, incrond cannot handle comments on its tab file
so we put those comments in a separate file (which got committed
on its own, accidentally, earlier).
---
 server/incron-ds-backup.conf |    2 +-
 server/postprocess.py        |   39 +++++++++++++++++++++++++++++++++------
 2 files changed, 34 insertions(+), 7 deletions(-)

diff --git a/server/incron-ds-backup.conf b/server/incron-ds-backup.conf
index 5bb4dff..23a554d 100644
--- a/server/incron-ds-backup.conf
+++ b/server/incron-ds-backup.conf
@@ -1 +1 @@
-/var/lib/ds-backup/completion IN_CREATE /usr/local/ds-backup/server/postprocess.py $@ $#
\ No newline at end of file
+/var/lib/ds-backup/completion IN_CREATE,IN_MOVED_TO /usr/local/ds-backup/server/postprocess.py $@ $#
\ No newline at end of file
diff --git a/server/postprocess.py b/server/postprocess.py
index 82a2418..b6d91bc 100755
--- a/server/postprocess.py
+++ b/server/postprocess.py
@@ -14,6 +14,8 @@ import sys
 import os
 import re
 import pwd
+import subprocess
+from subprocess import PIPE
 
 homebasepath = '/library/users'
 dirpath      = sys.argv[1]
@@ -57,17 +59,42 @@ try:
     os.setgid(user[3])
     os.setuid(user[2])
 except OSError, e:
-    sys.stderr.write('Could not set gid %s uid %s' % user[3], user[2])
+    sys.stderr.write('Could not set gid %s uid %s' % (user[3], user[2]))
 
 # rm the flagfile
 os.unlink(fpath)
 
-# os.system() rsync 
-#print 
-
-
-
+#
+# UTC timestamp to the minute
+# clients are not expected to retry often if they succeed.
+# and there is no point in us trying to keep any better
+# granularity than this.
+#
+# Popen seems to be the verbose pythonic way
+# of replacing backticks. ".communicate()[0]"!
+# 
+datestamp = subprocess.Popen(['date', '-u', '+%Y-%m-%d_%H:%M'],stdout=PIPE
+                             ).communicate()[0]
+# comes with newline - rstrip() will chomp it
+datestamp = datestamp.rstrip()
 
+sys.stdout.write(datestamp)
+exitcode = subprocess.call(['cp', '-al',
+                            user[5] + '/datastore-current',
+                            user[5] + '/datastore-' + datestamp])
+if (exitcode != 0):
+    sys.stderr.write('Cannot cp -al')
+    exit(1)
 
+# Note the -n parameter here. Without it
+# the symlink lands inside the previous
+# target of datastore-last. Oops!
+exitcode = subprocess.call(['ln', '--force', '-sn',
+                 user[5] + '/datastore-' + datestamp,
+                 user[5] + '/datastore-last'])
+if (exitcode != 0):
+    sys.stderr.write('Cannot ln')
+    exit(1)
 
 
+# done
-- 
1.5.6.dirty