[Server-devel] [PATCH] Introducing: server side traffic control

[martin.langhoff at gmail.com]

From: Martin Langhoff <martin at laptop.org>

Before attempting to initiate a backup, well behaved
clients will contact the XS to check if backup services
are available.

This small script checks that

 - the server is not too busy (loadavg)
 - it does have at least a few blocks and inodes to spare
 - less than 10 clients have been OK'd to backup in the last 5 minutes
 - the SN provided makes sense, and is registered

and then gives a go-ahead.

Misbehaved clients will naturally ignore this - this is merely
traffic control for good clients.
---
 server/backup-available.py |   69 ++++++++++++++++++++++++++++++++++++++++++++
 server/ds-backup.conf      |   11 +++++++
 2 files changed, 80 insertions(+), 0 deletions(-)
 create mode 100644 server/backup-available.py
 create mode 100644 server/ds-backup.conf

diff --git a/server/backup-available.py b/server/backup-available.py
new file mode 100644
index 0000000..71c944e
--- /dev/null
+++ b/server/backup-available.py
@@ -0,0 +1,69 @@
+# this is not a cgi script
+# will only work as a mod_python handler
+
+from mod_python import apache
+import os
+import re
+import pwd
+import random
+
+def handler(req):
+
+    recentclientsdir = '/var/lib/ds-backup/recentclients'
+    basehomedir = '/library/users'
+
+    req.content_type= 'text/plain'
+
+    # max 5% loadavg
+    if (os.getloadavg()[0] > 5):
+        return apache.HTTP_SERVICE_UNAVAILABLE
+
+    # we need at least a few blocks...
+    libstat = os.statvfs(basehomedir);
+    if (libstat[3] < 100 or libstat[6] < 100):
+        return apache.HTTP_SERVICE_UNAVAILABLE
+
+    # Limit concurrent rsync clients
+    # We touch a file with the client identifier
+    # every time we reply with a 200 OK. So
+    # we can check for recent "OKs".
+    # (clients that retry the rsync transfer won't
+    #  re-request this url, anyway.)
+    clientcount = os.system('find ' + recentclientsdir + 
+                            ' -mmin -5 -type f | wc -l');
+    if (clientcount > 10 ):
+        return apache.HTTP_SERVICE_UNAVAILABLE
+
+    # Read the XO SN
+    req.add_common_vars()
+    pathinfo = req.subprocess_env['PATH_INFO']
+    m = re.match('/available/(\w+)$', pathinfo)
+    if (m):
+        # req.log_error(clientid)
+        clientid = m.group(1)
+    else:
+        # We don't like your SN
+        return apache.HTTP_FORBIDDEN
+    
+    # Have we got a user acct for the user?
+    try:
+        homedir = pwd.getpwnam(clientid)[5]
+    except KeyError:
+        return apache.HTTP_FORBIDDEN
+
+    # check the homedir is in the right place
+    m = re.match(basehomedir, homedir)
+    if (not m):
+        return apache.HTTP_FORBIDDEN
+
+    #return apache.HTTP_UNAUTHORIZED
+    #return apache.HTTP_FORBIDDEN
+    #return apache.HTTP_VERSION_NOT_SUPPORTED
+    #
+    req.write('')
+
+    os.system('touch ' + recentclientsdir + '/' + clientid)
+    # TODO: 1 in 10, cleanup recentclients dir
+    if (random.randint(0,10) == 1):
+        os.system('find ' + recentclientsdir + ' -type f -mmin +10 -print0 | xargs -0 -n 100 --no-run-if-empty rm' )
+    return apache.OK
diff --git a/server/ds-backup.conf b/server/ds-backup.conf
new file mode 100644
index 0000000..8ef5b84
--- /dev/null
+++ b/server/ds-backup.conf
@@ -0,0 +1,11 @@
+# This is the configuration file for apache
+# amd should be included within the main part
+# of the apache config 
+Alias /backup/1 /usr/local/ds-backup/server/backup-available.py
+
+<Directory /usr/local/ds-backup/server/ >
+    AddHandler mod_python .py
+    PythonHandler backup-available
+    PythonDebug On
+    #
+</Directory>
-- 
1.5.6.dirty