Formal release methodolology

[Mitch Bradley]

Kim Quirk wrote:
> Ok... I think what we haven't defined at this point that is probably 
> worthy of discussion is:
>
> If or when we will need formalized testing before release.
>
> If we decide that there is a good reason for formalized testing; then 
> we can put in place the process that ensures we know exactly what is 
> being 'released' to formal testing; and as we get close to a formal 
> public upgrade, we know and cherry pick exactly what fixes we need in 
> that release.
>
> No aspect of the code base from EC to Wireless Firmware to kernel to 
> OS to shipping activities would be outside of the formal release process.
>
> To support that, we absolutely need the branch or stream or mainline 
> (whatever we want to call it) that allows developers to continue to 
> work, fix bugs, etc, while a smaller triage team is fine-tuning a release.
>
> I would argue that if we agree on some formal testing and release then 
> we also have to identify owners of the subsystems who are willing to 
> do the mundane tasks of cherry picking and quality control that will 
> be needed to get the pieces into a build repeatable, testable, 
> supportable release.
>
> Kim

FYI, here is how the firmware team (i.e. Richard and I) implement the 
release-stabilization parts of the above.  I do not address the formal 
testing here, just the special disciplines during run-up-to-release.

a) When a major system release is impending, we build a firmware release 
and designate it as a "test candidate" for that release.  For example, 
here are the tag lines from  http://wiki.laptop.org/go/Firmware  
relating to Update.1.

    * q2d12 - 2008-02-02 OLPC Firmware q2d12
      <http://wiki.laptop.org/go/OLPC_Firmware_q2d12> Update.1 test
      candidate 5
    * q2d11 - 2008-01-30 OLPC Firmware q2d11
      <http://wiki.laptop.org/go/OLPC_Firmware_q2d11> Update.1 test
      candidate 4
    * q2d10 - 2008-01-25 OLPC Firmware q2d10
      <http://wiki.laptop.org/go/OLPC_Firmware_q2d10> Update.1 test
      candidate 3
    * q2d09 - 2008-01-18 OLPC Firmware q2d09
      <http://wiki.laptop.org/go/OLPC_Firmware_q2d09> Update.1 test
      candidate 2
    * q2d08 - 2008-01-05 OLPC Firmware q2d08
      <http://wiki.laptop.org/go/OLPC_Firmware_q2d08> Update.1 test
      candidate

The individual changes can be found by clicking the links - typically 
the change list for a refresh version is short, and we try our best to 
isolate the changes so as not to invalidate prior testing of unrelated 
aspects.

b) Subsequent candidates, until the final release, are built by 
cherry-picking.  We have a script in our build system that lets us merge 
specific commits on top of the base revision.  We only do "important" 
things.

c) As always, when we make a new firmware, it is documented via a wiki 
page and announced to the devel list.

d) As of the last couple of spins, our release procedure includes 
building an RPM for submission to Dennis via the "joyride" injection path.

e) Development continues during this phase, but we are careful to 
package our commits so that isolated critical bug fixes apply cleanly on 
top of the base candidate.  (Branching would be another way to do it, 
but at present, the "be careful" method works well for the  small OFW 
code base managed by a small team.)