[Server-devel] [PATCH] ds_backup.py: Implement server availability check

[Martin Langhoff]

On Wed, Jun 25, 2008 at 12:43 AM, Michael Stone <michael at laptop.org> wrote:
> On Tue, Jun 24, 2008 at 08:18:22PM -0400, martin.langhoff at gmail.com wrote:
>> +def check_server_available(server, xo_serial):
>> +
>>      try:
> ...
>> +        ret = urllib2.urlopen(server + '/available/%s' % xo_serial).read()
>> +        return 200
>> +    except HTTPError, e:
>> +        # server is there, did not fullfull req
>> +        #  expect 404, 403, 503 as e[1]
>> +        return e.code
>> +    except URLError, e:
>> +        # log it?
>> +        # print e.reason
>> +        return -1
>
> Why not just propagate the exceptions?

Because different libs use different exceptions and conventions as to
how to report the errorcode, and the decision of what lib to use is
local to check_server_available(). If urllib2 gets replaced with
urloncarrierpigeons, callers won't want to know, much less implement
the (arguably dirtier) exceptions that come with it.

>>  def rsync_to_xs(from_path, to_path, keyfile, user):
>>
>> @@ -71,7 +61,7 @@ def rsync_to_xs(from_path, to_path, keyfile, user):
>>      if not re.compile('/$').search(from_path):
>>          from_path = from_path + '/'
>>
>> -    ssh = '/usr/bin/ssh -F /dev/null -o "PasswordAuthentication no" -i "%s" -l "%s"' \
>> +    ssh = '/usr/bin/ssh -F /dev/null -o "PasswordAuthentication no" -o "StrictHostKeyChecking no" -i "%s" -l "%s"' \
>
> What's the justification for turning off host key checking?

Well, it's not pretty

 - We are not using the conventional SSH ".ssh" infrastructure. And we
are doing odd things with it. So we end up with keys called
"owner.key" and "owner.key.pub" which unfortunately is a regression on
SSH's natural abilities (which include keeping different kinds of keys
around), and sometimes we set odd filemodes those files. I cannot
assume known_hosts is maintained or sane. The ssh mgmt is a mistery to
me atm.

 - We are not getting the schoolserver host key as part of the
registration process.

Both are things that need quite a bit of spelunking through the Sugar
code - once both are resolved (the first one might just mean hitting
me with a cluebat) - we can return to your regularly scheduled
program...

>> +    for n in range(1,7):
>> +        sstatus = check_server_available(backup_url, sn)
>> +        if (sstatus == 200):
>> +            # cleared to run
>> +            rsync_to_xs(ds_path, 'schoolserver:datastore', pk_path, sn)
>> +            # this marks success to the controlling script...
>> +            os.system('touch ~/.sugar/default/ds_backup-done')
>> +            exit(0)
>> +        elif (sstatus == 503):
>> +            # exponenxtial backoff
>> +            time.sleep(60 * 2**n)
>> +        elif (sstatus == -1):
>> +            # could not connect - XS is not there
>> +            exit(1)
>> +        else:
>> +            # 500, 404, 403, or other unexpected value
>> +            exit(1)
>
> Perhaps we should return either appropriate errno values or the HTTP
> error numbers on error conditions?

Sounds like a good idea.



m
-- 
 martin at laptop.org -- School Server Architect
 - ask interesting questions
 - don't get distracted with shiny stuff - working code first
 - http://wiki.laptop.org/go/User:Martinlanghoff