[PATCH] cafe_nand: remove busy-wait loop

[John Gilmore]

> So, the transfer rate dropped from about 9.6mb/sec to about 5.3mb/sec.

I was afraid of this.  Flash is pretty fast, and there's almost no
parallelism available in it (e.g. you can't queue up a bunch of reads
to occur "when the data rotates under the heads", or start a "seek"
early).  Thus it's mostly worth sitting and waiting for the chip to
finish, rather than making the CPU try to do something else.  It all
happens at chip speeds -- and OLPC designed the CaFE chip to run as
fast as the flash chips and memory bus can run (because the Geode's
native flash support, for some reason, didn't go that fast).  [Note,
however, that the Marvell spec for the chip doesn't give ANY speed
specs.  It does say it might be possible to run its PCI bus at 66 MHz
rather than 33 -- which do we do?]

I really think we need to separate the effects of filesystem
compression from the effects of Flash storage.  Other reports show
systems being able to do many thousands of transactions per second to
SSD (flash-based "disks"), but we don't come anywhere close to that.
Why don't we?  Nobody knows.

There is a trivial way to make an uncompressed NAND image.  Take an
existing build's tar image, extract it to a directory, use the same
mkfs.jffs2 command from the standard build log, but add
"--compression-mode none" to build an uncompressed image.  Burning
that into NAND will let you test uncompressed reads.  That will give
you best-case reads, too, since all the data for a given file will be
contiguous.  

Perhaps there is some sysfs knob that will also let you turn off
compressed writes.

It's unfortunate that there's no "block device" way to access Flash
chips in Linux without using a filesystem at all.  It's much easier to
measure and tune I/O performance without a filesystem first, then see
what (filesystem or driver) optimizations are required to make a
particular filesystem fast on that device.

	John

PS: During the long delay while the CaFe is erasing a block, it can do
reads from other blocks (but not this one); do we do these in parallel?