Yum random crashes in XO4 f20 images

[James Cameron]

G'day Peter,

Thanks for any ideas you may have.

The problem also reproduces on OLPC Fedora 20 image for XO-4:

http://build.laptop.org/14.1.0/os1/xo-4/41001o4.zd (552 MB)

*** Error in `/usr/bin/python': free(): invalid pointer: 0x047c79ae ***
======= Backtrace: =========
/lib/libc.so.6(+0x6c8b4)[0xb6c828b4]
/lib/libc.so.6(+0x754e8)[0xb6c8b4e8]
======= Memory map: ========
[...]

The error varies in detail, but always suggests corruption of heap or
pointers to heap.

The triggering conditions are interactive use of yum, yum update, or
yum used by olpc-os-builder.  The latter is a simple reproducer for me.

I'm reproducing it on an XO-4, with 2GB of RAM, no swap, 8 GB eMMC, 8
GB USB flash drive.

While memory demand by yum is large by comparison to other programs,
the available memory at the time of failure is ample.  There are no
kernel out of memory (OOM) events.  It seems more likely to occur when
the filesystem cache is under heavy demand.

The method to recreate the problem was:

1.  install the system image 41001o4.zd using fs-update and then boot,

2.  configure wireless network,

3.  "yum install -y git olpc-os-builder"

4.  clone the master branch of
git://dev.laptop.org/projects/olpc-os-builder
(last verified with b87e6ee)

5.  run "./osbuilder.py examples/olpc-os-14.1.0-xo4.ini" repeatedly
until the error occurs (usually within about five attempts),


I've also tried running under valgrind, but that causes illegal
instruction.  It is quite likely I'm not using valgrind correctly.
http://dev.laptop.org/~quozl/z/1XRYtO.txt

The workaround at the moment is to build our Fedora 20 images on
Fedora 18.  Fedora 18 shows no sign of the problem.  I'm worried that
a low probability heap corruptor may cause instability of applications
in the field.

The exact same kernel is being used for Fedora 18 and Fedora 20.

On Tue, Sep 09, 2014 at 03:55:24PM +0100, Peter Robinson wrote:
> What version of OOB are you using, and what config files? I can try
> and recreate the problem here on other devices.

-- 
James Cameron
http://quozl.linux.org.au/