#10911 NORM 11.2.0-: investigate XO-1 build growth

[Zarro Boogs per Child]

#10911: investigate XO-1 build growth
---------------------------------+------------------------------------------
           Reporter:  dsd        |       Owner:  dsd          
               Type:  task       |      Status:  new          
           Priority:  normal     |   Milestone:  11.2.0-final 
          Component:  distro     |     Version:  not specified
         Resolution:             |    Keywords:               
        Next_action:  never set  |    Verified:  0            
Deployment_affected:             |   Blockedby:               
           Blocking:             |  
---------------------------------+------------------------------------------

Comment(by dsd):

 IRC conversation suggests that a ubifs space inefficiency (compared to
 jffs2) is to be expected, but there is room for improvement.

 {{{
 12:16 < dedekind_> Only UBI wastes ~15MiB
 12:17 < dedekind_> because it takes 2 KiB from each eraseblock
 12:17 < dedekind_> because AFAIR cafe controller does not support sub-
 pages
 12:18 < dedekind_> dsd, with a fair amount of work it is possible to
 optimize
                    things
 12:18 < dedekind_> dsd, space efficiency has not been our concern
 12:18 < dsd> ah yes, our wiki notes "    If we can enable sub-page writes
 on
              our flash device, we can recover some extra space from UBI. "
 12:19 < dedekind_> yes, around 7-8 MiB
 12:19 < dsd> i'll look into that
 12:19 < dedekind_> but AFAIR this is a controller limitation, dwmw2 should
 know
 12:19 < dedekind_> dsd, of course things like using zlib could help
 12:19 < dedekind_> dsd, but they have a side effect
 12:19 < dwmw2> er
 12:20 < dwmw2> why wouldn't we support sub-pages? Just because our HW ECC
                doesn't cope?
 12:20 < dedekind_> dwmw2, yes
 12:20  * dwmw2 tries to remember the café hw ecc
 12:20 < dedekind_> dwmw2, but if you could hack the driver and use SW ecc
 for
                    the first page ...
 12:20 < dwmw2> I thought it supported 512-byte pages, which would have
 meant it
                was *likely* to support subpages too (although not
 guaranteed)
 12:20 < dwmw2> oh, wait.
 12:20 < dedekind_> dsd, that may give even more that 7-8 MiB because
 you'll fit
                    more nodes and spend less
 12:21 < dsd> ok, that would be 1 step
 12:21 < dwmw2> I think it hard-codes the layout, with the ecc in last
 bytes of
                the 2KiB page
 12:21 < dsd> i'll look into it
 12:21 < dwmw2> I don't think it can support subpages with hardware ecc
 12:21 < dwmw2> you'd have to do it in software
 12:22 < dedekind_> dwmw2, could there be a hack - use sw ECC for the first
 page
                    ?
 12:22 < dwmw2> 'first page'?
 12:22 < dedekind_> firs NAND page of every eraseblock
 12:22 < dwmw2> oh
 12:22 < dwmw2> ew :)
 12:22 < dedekind_> where we store headers and need sub-pages
 12:22 < dedekind_> we do not need them for the rest
 12:24 < dedekind_> dsd, the other easy thing to do could be to just make
 UBIFS
                    on-flash data structures smaller, but this would make
 olpc
                    incompatible with stock kernels
 12:24 < dedekind_> dsd, currently we have plenty of "reserved for future"
 bytes
                    in node headers
 12:24 < dedekind_> see ubifs-media.h
 12:25 < dsd> mm ok, i'll try that at least as an experiment to see how
 much
              difference it makes
 12:25 < dedekind_> dsd, and also, there is a possiblity to make UBIFS
 reserve
                    less for internal needs, but I would not suggest to
 touch
                    that - it can open a can of worms
 12:25 < dedekind_> unless you have plenty of time, of course
 12:27 < dedekind_> dsd, we tried to make things correctly, but of course
 there
                    is no guarantee that everything will work well if you
 just
                    change size of UBIFS node headers
 12:27 < dedekind_> dsd, but in theory - it should work
 12:27 < dsd> i dont have "plenty" but i certainly have time to run
 experiments
              such as these and post numbers to your mailing list, be
              responsive, and hope that i can spur some developer interest
 12:28 < dsd> dilinger will also be interested in this, he is also playing
 with
              ubifs on debian on XO
 12:31 < dedekind_> dsd, unfortunately I have no time to really look at
 this
 12:32 < dedekind_> dsd, but there is for sure enough room to optimize
 space
                    overhead
 12:32 < dedekind_> dsd, but you cannot make it as good as in JFFS2 for
 sure
 }}}

-- 
Ticket URL: <http://dev.laptop.org/ticket/10911#comment:4>
One Laptop Per Child <http://laptop.org/>
OLPC bug tracking system