[Testing] Buildbot Questions

[Michael Stone]

Titus, Grig,

I have some questions about the proper usage of Buildbot and about how
the current design for pybots.org originated. The design questions
should definitely be considered to be in dialogue with Titus'
two-month-old post:

  http://ivory.idyll.org/blog/oct-07/pull-vs-push-continuous-integration.html

------------------------------------------------------------------------

First, I've made a buildmaster and I've equipped it with a
PilgrimChangeSource which polls the website where new builds appear and
which creates Changes with appropriate hyperlinks when they show up.

Implementation Question 1: How (or maybe "where") should I calculate the
URLs for the build tarball from the knowledge that, say, build
'joyride-1384' is available?

  At present, I'm performing this calculation in my PilgrimChangeSource
  and I'm encoding the information in two new attributes in the Change
  object I create in that ChangeSource.

  Perhaps this logic would be more appropriately placed in a
  PilgrimChange object? Or somewhere farther "downstream" from the
  ChangeSource?

Implementation Question 2: Given the Design Questions which follow, what
are some decent ways to proceed in implementing my link-checker and
permissions-checker bots?

  In particular, should I concentrate on writing specific
  BuildSteps/RemoteCommands that describe the appropriate operations or
  should I work on writing a single 'ChrootTest' buildstep that my
  test-bots can implement, each in their own way?

------------------------------------------------------------------------

Design Question 1: Why does buildbot use a master-slave architecture?

  Our build process has several steps that must be run as root on fairly
  carefully configured F-7 machines. Since buildbot seems to be set up
  to operate by granting remote shell access to the build master, we
  would effectively be granting the buildmaster root access to every
  build-slave on which we actually try to build images. This is not so
  happy, particularly given the fact that BuildBot transmits plaintext
  authentication credentials.

Design Question 2: Why is the information about how to perform builds
and about how to interpret the results centralized on the buildmaster?

  We have had serious problems with human bottlenecks and with
  machines/systems that only one person knows how to administer. Are
  there examples of how to use buildbot a fashion that distributes
  maintenance reponsibility, for example, by distributing the action of
  adding and configuring new buildslaves?

Design Question 3: Why does the pybots design drop the problems of
test-setup and test-dependency-satisfaction into the 'object-language'
of shell commands and BuildStep-ordering instead of placing it in the
'meta-language' of buildbot abstractions (like BuildSteps, Builds, ...)?

  We have some wildly different kinds of tests that we might want to
  collectively control and from which we might want to merge status info.
  These include chroot-only tests, inside-QEMU tests, on-several-laptop
  mesh tests, and on-physically-modified-laptop power-management stuff.
  What are some of the tradeoffs between an on-slave opaque
  'prepare-pkgs.sh' vs. an on-slave 'standard Makefile'-like solution vs.
  a meta-language 'hierarchy-of-builders' solution?

Michael

P.S. - Thanks for having helped to make such a thought-provoking tool!
P.P.S. - My work so far is available at 

  http://dev.laptop.org/git/users/mstone/xobots/     and
  http://teach.laptop.org:8010/

Contributions and improvements gratefully accepted!