last day for another stable build
Jim Gettys
jg at laptop.org
Mon Mar 5 21:34:50 EST 2007
Ronak,
"Bugs are good" - Doug Clark
I'd like to encourage a bit different view of the situation, and that
the what your propose below become the norm, rather than the exception.
Rather than there being one set of firmware which has seen a full QA
cycle as the "norm", I urge that there also be an experimental version
of firmware also available that has not necessarily completed the full
formal QA cycle.
Here's my reasoning:
Formal QA testing has some *really* good characteristics: it is much
more likely to uncover certain classes of bugs (e.g. memory leaks that
occur over time, as an explicit test that is left running for days can
uncover such bugs better than the much more random testing done by
developers or experimenters). It's also good at preventing previously
seen bugs from reappearing (regression testing), and scaling may also be
systematically tested.
But it is very much less effective at uncovering new bugs: formal
testing is always done the same way, and only after you've released such
QA'ed software would you even start using the firmware in other
environments and circumstances not expected by the formal QA process
that might discov new bugs.
If all firmware goes through an extended QA cycle before it sees the
light of day, then you have in effect serialized the discovery of new
bugs that formal QA can not find, as you will generally not have the
right hardware/software to trigger the bugs in your test environment,
until *after* you've seen the bug, reproduced it, and added explicit
tests to your test environment.
A good example of this was recently provided by the problems associating
with the WRT54GL access points; until this problem was noted, the
process to isolate the bug, and then add the hardware to a QA testing
environment doesn't even begin.
If all testing in the field is left until after an extended formal QA
cycle, then those discoveries and fixes will not start for an extended
period; putting both a QA'ed and an experimental set of firmware in
parallel will speed the process of bug discovery and bug killing.
Certainly, when we get to the BTest-3 and CTest builds, and for FRS for
sure, we'll likely want to choose to use a set of firmware that has
completed a full QA cycle; but I believe we should be planning on a two
track development path on the firmware, to speed the bug discovery
process as much as possible.
Best regards,
- Jim Gettys
.
On Mon, 2007-03-05 at 17:04 -0800, Ronak Chokshi wrote:
> Marcelo,
>
> We feel that whenever OLPC schedules a build of laptops, the wireless
> firmware to be used in them should be the last stable version which
> has been released through Marvell website and has most of the bug
> fixes from the Marvell engineering team as well as the required
> feature set targeted for that build of laptops. Also, more
> importantly, we would also like to make sure that the wireless
> firmware has passed most of the tests done by the Marvell QA team. All
> this takes time and in this build’s case, since the multicast support
> is required and has been very newly added, there was not enough time
> to rigorously test the development firmware released on the TechWiki
> by Cozybit or even time to merge the recent bug fixes to this version
> of the firmware. Hence, I said that “for the purpose of this build, we
> will have to use the development build released by Cozybit on the
> TechWiki". So, I would look at this more as an exception than a
> regular behavior that we would like to follow going forward.
>
>
>
> Thanks
>
> Ronak
>
>
>
> -----Original Message-----
> From: Marcelo Tosatti [mailto:marcelo at kvack.org]
> Sent: Monday, March 05, 2007 4:47 PM
> To: Ronak Chokshi
> Cc: Marcelo Tosatti; Christopher Blizzard; sugar; Devel at laptop.org;
> David Woodhouse
> Subject: Re: last day for another stable build
>
>
>
> Ronak,
>
>
>
> On Fri, Mar 02, 2007 at 02:21:25PM -0800, Ronak Chokshi wrote:
>
> > Hi Marcelo,
>
> > For the purpose of the build, we will have to use the development
> build
>
> > released by Cozybit on the TechWiki
>
> > (http://laptop.org/teamwiki/index.php/Image:Libertas-firmware.tgz)
> since
>
> > as highlighted in the Release Notes of these release, Cozybit is
> still
>
> > to adopt our bug fixes.
>
> >
>
> > We are testing this build (v5.220.9.p10) in our lab over a network
> of
>
> > approx. 15 days. We will be doing this through the weekend and will
> post
>
> > our test result details by end of the day on Monday. This will give
>
> > everyone confidence in the firmware build that will go in the next
> build
>
> > of laptops.
>
>
>
> I just noticed that v5.220.9.p10 has been released.
>
>
>
> What is not clear to me is whether we can include such release in our
> OS
>
> build or not?
>
>
>
> It appears that we can, given your comment above saying "For the
> purpose
>
> of the build, we will have to use the development build released by
>
> Cozybit on the TechWiki". Is that right?
>
>
>
> Thanks
>
>
>
> >
>
> > Thanks
>
> > Ronak
>
> >
>
> > -----Original Message-----
>
> > From: Marcelo Tosatti [mailto:marcelo at kvack.org]
>
> > Sent: Friday, March 02, 2007 10:30 AM
>
> > To: Ronak Chokshi
>
> > Cc: Marcelo Tosatti; Christopher Blizzard; sugar; Devel at laptop.org;
>
> > David Woodhouse
>
> > Subject: Re: last day for another stable build
>
> >
>
> > On Fri, Mar 02, 2007 at 10:01:13AM -0800, Ronak Chokshi wrote:
>
> > > Hi Marcelo,
>
> > > I would suggest to include the firmware v5.110.12.p0. This is the
> most
>
> > > stable build. Every other build on the TechWiki is a development
> build
>
> > > and work in progress.
>
> > >
>
> > > Whoever will include this build, here are the instructions on
> getting
>
> > > the binary from our website:
>
> > >
>
> > > - Go to http://www.marvell.com/drivers/search.do
>
> > > - Type "OLPC" in the Product category and hit Submit.
>
> > > - The first version in the search results that pops up should be
>
> > > v5.110.12.p0 and should say " USB 8388 OLPC Firmware for MPP (Mesh
>
> > > Portal Point)"
>
> > >
>
> > > This has a zip file and a .bin in it. This is the firmware build
> that
>
> > > you would need.
>
> >
>
> > Ronak,
>
> >
>
> > We need multicast support. What is the plan regarding a Marvell
> release
>
> > of such feature?
>
>
> _______________________________________________
> Devel mailing list
> Devel at laptop.org
> http://mailman.laptop.org/mailman/listinfo/devel
--
Jim Gettys
One Laptop Per Child
More information about the Devel
mailing list