[Server-devel] [PATCH] Touch a ".transfer_complete" to mark completion, minor cleanups

[Michael Stone]

On Mon, Jun 16, 2008 at 05:01:32PM -0400, Martin Langhoff wrote:
> On Mon, Jun 16, 2008 at 4:42 PM, Michael Stone <michael at laptop.org> wrote:
> >> -def new_backup_notify(server, nonce, xo_serial):
> >> -    try:
> >> -        auth = sha.sha(nonce + xo_serial)
> >> -        # TODO: add auth header
> >> -        ret = urllib.urlopen(server + '/new/%s' % xo_serial).read()
> >> -    except IOError, e:
> >> -        if e[1] == 403:
> >> -            # Auth not accepted. Shouldn't normally happen.
> >> -            raise BackupError(server)
> >> -
> >
> > Why are we taking out this part of Ivan's protocol? As I recall, this
> > part of the protocol is important for mitigating DoS attacks on the
> > backup server...
> 
> In any case, all of this can merely prevent a "friendly fire" DoS. 

I misremembered the protocol. I had thought that the server might demand
some hashcash in response to our very first "initiate a backup" HTTP
transaction in order to throttle client connections.

As it turns out, the GET to the /new/* URL serves as a "commit message"
to inform the server that the client is satisfied that it has
transmitted all necessary backup information. It's arguable whether
there's any security impact because the user's SSH key and their UUID
are so poorly protected on current systems. However, I think that it
does have substantial reliability impact: if the client powercycles
while transmitting, in your current protocol description at 

  http://wiki.laptop.org/go/XS_backup_restore

the server's most recent backup would be left in an inconsistent state.
Would the client receive broken data on the next restore? If not, why
not?

> true DoS protection from clients we need an rsync wrapper 

Throttling login rate would take care of the problem. Hashcash at the
initial GET stage can be used to implement more sophisticated
throttling. We should be very wary of deploying a known-vulnerable
protocol which backward-compatibility might force us to keep.

> easy to do, a bit harder to secure. We'll need rssh to lock down
> things in this area, and as far as I can see, rssh ain't too friendly
> with wrappers in high-level languages.

It's not really that we need to control what happens once the user has
logged in; rather we need to control the rate at which logins occur,
yes? 

> Is subprocess.check_call() a better idiom? Where can I find good examples?
> At times I am guilty of cargo-cultism in my Python.Show me the good
> gospel, and I'll be a convert.

Start with 

  pydoc subprocess.check_call

Interesting example:

  from subprocess import check_call, CalledProcessError
  try: check_call(['/bin/ls', '/path/to/dir'], stdout=PIPE)
  except CalledProcessError,e: print "%s died with error %d" % (e.cmd, e.returncode)

Second example:

  from subprocess import Popen, PIPE
  cmd = ['/bin/sed', '-e', 's/1/2/g']
  proc = Popen(cmd, stdin=PIPE, stdout=PIPE) # inherit parent's stderr
  (out, err) = proc.communicate('1')
  print out, proc.poll()

> You want to close it? Ok, makes sense.

It doesn't matter so much here since we're short-lived, but generally,
it's good to close unneeded FDs as soon as they become unneeded. :)