[support-gang] Real time clock battery during XO storage

[James Cameron]

Really need some essential data before diagnosing and testing this,
because the ambiguity leads to an explosion of combinations that make
it impractical, so please answer these questions:

1.  what is a serial number from the batch?  (I don't have access to
records that identify Elaine, but I do have a list of serial numbers
and associated data).

2.  what version of OLPC OS was installed?  (You didn't say, and it is
critical).

3.  what was the symptom that led you to determine the laptops needed
debricking?

4.  what was done with serial adapters to debrick them?

5.  did you install the latest firmware at the same time as debricking
them, or did you leave the laptops with the same problematic firmware?

6.  if you left the laptops with the same problematic firmware, can
you please explain why?

Do ask your contacts in the deployment to get these answers if you
can't supply them, thanks.

It is my hope that I can tell you how to approach the situation now,
and in the future.

On Mon, Dec 09, 2013 at 08:19:32PM -0800, Nancie Severs wrote:
> Sorry, I'm still traveling and had not checked these posts lately.  I hope my
> explanation here is helpful.
> 
> I refer to the 1000 plus XO-1's that Elaine N/Cambodia received (she says from
> Walter) in 2008. My observation comes from these facts:
> 
> The Cambodia deployment has kept many of these original XO-1s in their original
> shopping boxes, unopened, unbooted and to date has perhaps 200+- still
> undistributed to projects or children. When the boxes are opened to for example
> distribute 20 of these XOs to a new project, as they did in March of 2013, then
> the XOs charge, and boot fine to the originally installed Sugar OS. The system
> date can be set & the laptops can be reflashed to the desired build.  (I don't
> know what date shows up when they boot. I have not been present at the opening
> of the stored laptops.)
> 
> These laptops will work if used continuously. But this is what has happened to
> laptops from this original 1000+ group (I don't know the stats of what was
> originally received.) Like a G1G1 XO in the US that was booted, used a bit and
> then stored, the Cambodia XOs from this group will brick if stored long enough
> for the clock battery to completely discharge. In Cambodia's case, some XOs
> distributed to a project that were unused after some time, were returned to the
> project village teacher/director. They were bricked. Other Cambodia projects
> also report having some that were put away for whatever reason and are
> bricked.  Adam & I taught the repair & I left a serial adapter in Cambodia for
> sharing among the several and distant locations (but had only one I could
> spare).
> 
> (By the way, when unpacking brad new stored XO-1s they are finding the blotchy
> screen problem in about 1 in 5 of these long stored XO-1. There is another
> thread on this conversation.)
> 
> For whatever reason, the bricking occurs when the laptops are stored AFTER
> being booted. I don't htink it is practical to remove the clock bateery for
> storage. For large #'s of XOs, the labor involved removing clock batteries from
> the motherboard  & the risk of popping the battery holder off the motherboard
> and sometimes breaking it or otherwise damaging the motherboard makes this
> impractical.
> 
> Note, that once debricked, the same XOs will not brick again.
> Nancie:)
> Nancie Severs
> OLPC Support Volunteer
> 
> 
> On Tuesday, December 10, 2013 8:08 AM, James Cameron <quozl at laptop.org> wrote:
> Summary: a method for storing an XO-1 by turning off the RTC; remove
> the main battery, and the RTC battery for five seconds and then
> replace it.  Store without turning on.
> 
> On Mon, Dec 09, 2013 at 09:00:20AM -0500, Nathan C. Riddle wrote:
> > On Sun, Dec 8, 2013 at 10:44 PM, James Cameron wrote:
> > > On Fri, Dec 06, 2013 at 08:52:34PM -0500, Nathan C. Riddle wrote:
> > > > I noticed this in Nancie's blog on Planet about her current trip
> > > > to Cambodia:
> > > >
> > > > "And because the Cambodia XO-1 laptops are susceptible to
> > > > bricking if stored AFTER THE FIRST BOOT, it is important that
> > > > everyone know those laptops are not broken and should not be
> > > > scrapped."
> > >
> > > It would be interesting to see which particular type of bricking
> > > Nancie is talking about there.  I have no further details from
> > > her.  There are many causes leading to bricking, and they aren't
> > > all related to the RTC battery that you focus on in the rest of
> > > your post:
> > >
> > > > This brings to mind a puzzlement :
> > > >
> > > > XO's recently received in unopened boxes and manufactured in
> > > > late 2008 or early 2009 had RTC batteries sufficiently charged
> > > > to properly boot the XO-1 without any date / time error.
> > >
> > > No, it would be more correct to say that those XOs were
> > > manufactured with a version of the firmware that does not fail in
> > > the same way as earlier firmware.
> >
> > The RTC batteries in the received (unopened boxes) XO-1's appeared
> > not discharged after more than 4 years, since the XO turned on and
> > booted with no "system date error".
> 
> I'll rephrase; an XO turned on and booted without "system date error"
> is not _proof_ that the RTC battery was not discharged.  It depends on
> the firmware version, the manufacturing tags, and whether it was
> powered up since the RTC battery was inserted.
> 
> > I think a manufacturer would expect more than 2.5 months for
> > distribution of a laptop.
> 
> Not this one.  We're different.  ;-)
> 
> The 2.5 months is for the XO-1.5, the XO-1 is 4 months, and the
> XO-1.75 is 12 months ... you had not been specific about model, so I
> chose the minimum.
> 
> Rather than use a primary battery that would require replacing, OLPC
> used a rechargeable battery.  The laptops were intended to stay with
> the learners.
> 
> Also, laptops were intended to be manufactured immediately prior to
> deployment.
> 
> > Yes, these would have had the later firmware allowing boot with
> > discharged RTC battery, but all XO-1's that I have seen would have
> > displayed an system date error message. So, the observed was
> > perplexing.
> >
> > >
> > > Get me a serial number, and I can read from our database what
> > > version of firmware was flashed by the factory.
> > >
> > > Given the time since manufacture, the RTC batteries will be
> > > entirely discharged, and the newer firmware on those laptops
> > > doesn't consider that to be a problem, especially as all the
> > > registers of the RTC will be invalid rather than just one or two.
> > >
> > > > Yet, RTC batteries charged for 30 hours (XO on and not asleep)
> > > > at the end of a school year discharged by the start of school in
> > > > the fall (main battery stored separately).
> > >
> > > How long is that?  I don't know the length of the school year you
> > > refer to, nor which hemisphere it is in.  (fall = autumn).
> >
> > June 1 to September 1 is storage time.
> 
> Three months.
> 
> Every three month storage time will degrade the RTC battery more.  If
> you have been in the habit of doing this over several years, then the
> battery will have aged more rapidly.
> 
> Stopping the oscillator will reduce the problem.
> 
> > > In the scenario you describe (main battery stored separately), 2.5
> > > months is the expected minimum operating time, at the start of the
> > > product life.  This will degrade as the product ages.  It will
> > > more quickly degrade if the RTC battery is ever fully discharged,
> > > or stored at temperatures outside the specifications for the
> > > laptop.
> > >
> > > Five years after manufacture, I'd expect a minimum operating time
> > > of about three to five weeks.  Yet another reason not to bless the
> > > learners with XO-1s.
> > >
> > > Once the RTC voltage falls sufficiently, bit rot occurs in the RTC
> > > registers, and this gives a completely different error to a fully
> > > discharged battery.
> > >
> > > > This suggests some sort of "unpacking" similar to the first boot
> > > > on a PC or an XO tablet.
> > >
> > > I disagree, there are other explanations for what you describe,
> > > which I have detailed above.
> > >
> > > There is yet another explanation; that the RTC batteries vary in
> > > how much they exceed the battery manufacturer's capacity
> > > specification.
> > >
> > > (Yes, there is a first-boot expansion of the filesystem on XO-1.5,
> > > XO-1.75, and XO-4, but this has nothing to do with the operating
> > > time of the RTC in the absence of the main battery.)
> > >
> > > > If so, is there a way to set parameter(s) in firmware back to
> > > > the factory value so the RTC battery does not discharge during
> > > > storage ?
> > >
> > > No, there are no firmware parameters that control the RTC battery
> > > discharge rate, and the discharge rate is not under the control of
> > > the firmware or the software.
> 
> I take that back.  There is no _easy_ way.
> 
> Thanks to Paul Fox for the idea.
> 
> Looking at the datasheet for the CS5536 chip (AMDG_CS5536.pdf), page
> 475 shows the RTC_CRA register has a divider chain control bit mask,
> which defaults to "Oscillator Disabled" on initial power up via the
> RTC battery.  The datasheet on page 154 says there is a feature
> "Oscillator disabling to save battery during storage".
> 
> Looking at Open Firmware, the file dev/ds1385r.fth, during power up
> the Divider Chain Control is set to "Normal Operation", which starts
> the oscillator.
> 
> So between the time the RTC battery is inserted and before the laptop
> is turned on, the oscillator is not running, so the battery drain will
> be minimal.
> 
> (You might prove it is happening by checking the date in the clock of
> those unopened units.  Although I can't see how they could have a
> valid date from 2008 or 2009 given they won't have been turned on to
> set that date.)
> 
> However, it turns out that simply turning off the oscillator "bricks"
> the XO-1 that I tested on.
> 
> The symptom is that pressing the power button gives about a one second
> pulse on the power indicator, then it goes blank.  There is no serial
> cable output from the processor.
> 
> It is fixed by removing the RTC battery, the main battery, the power
> cable, and waiting for 5 seconds.  This same fix can be used before
> storing an XO-1, but it requires disassembly and the battery holder
> may fail.
> 
> I tried as many combinations of commands that I could think of, but
> none solved the problem.  I was able to scope the RTC oscillator to
> prove whether it was running or not.
> 
> The laptop starts fine with the RTC oscillator stopped, provided it
> stopped because of removal of the RTC battery.  It just won't start if
> the RTC oscillator is stopped by writing to RTC control register.
> 
> Speculation: the XO-1 embedded controller is not detecting a 32 KHz
> signal from ball C8, GPIO27, or is not detecting some other normal
> response of the processor, and is abandoning the power up.
> 
> Speculation: extreme low storage temperature or movement of the
> battery in the holder may have stopped the oscillator in the unopened
> batch you mentioned.
> 
> > > Please make sure you upgrade to the latest firmware.  I spent
> > > weeks on this problem between March and May this year, and it
> > > would piss me off to find that this was wasted effort and cost.
> 
> CC: devel@ as it is of interest to developers.
> 
> 
> --
> James Cameron
> http://quozl.linux.org.au/
> _______________________________________________
> support-gang mailing list
> support-gang at lists.laptop.org
> http://lists.laptop.org/listinfo/support-gang
> 
> 

> _______________________________________________
> support-gang mailing list
> support-gang at lists.laptop.org
> http://lists.laptop.org/listinfo/support-gang


-- 
James Cameron
http://quozl.linux.org.au/