[Server-devel] [XSCE] Steps involved in generating OSM maps for IIAB

[Tim Moody]

I assume that with new geo data the index will also need to be regenerated.  Whoosh 2.6 is the last version compatible with the current index.  Hopefully the new index will support later versions of Whoosh.
My other request is that we make sure we can accommodate different levels of zoom for platforms with different amounts of storage when tiles are generated (vertical partitioning).  I also wonder if different levels of zoom for different geographical areas is possible.  A school's home country might have more zoom than other places I study in geography class but don't visit.  (horizontal partitioning).  I know that OSM provides subsets of data for different regions.

Date: Wed, 13 May 2015 22:26:05 +0530
Subject: Re: [XSCE] Steps involved in generating OSM maps for IIAB
From: anishmg at umich.edu
To: xsce-devel at googlegroups.com
CC: server-devel at lists.laptop.org

Hi Adam,

So I have gotten past step one of the map generation process which is importing planet data in a postgres db (which took a few tries due to a lack of understanding of the system setup). Next, I need to generate the tiles. The process will probably take a couple of weeks and it may be a few more days when I can get around to doing this (but should be doable without problems)

Many thanks to Braddock for helping out with this and making it possible. :-)

Cheers,
Anish


On Wed, May 13, 2015 at 4:16 PM, Adam Holt <holt at laptop.org> wrote:
On Fri, Apr 24, 2015 at 12:19 PM, Anish Mangal <anishmg at umich.edu> wrote:
Hi Braddock,

Am willing to give this a shot. The map data on IIAB could do with an update. Let me know how/where I should access the machine.

Anish,

How can others assist you here?

PS Tim Moody miraculously fixed OpenStreetMap city/town search on a Rwanda-bound IIAB/XSCE server on CentOS "7.1", by downgrading Whoosh from 2.7 to 2.6 overnight, just last night!
 

Will share my public ssh key with you separately.

Best,
Anish


On Fri, Apr 24, 2015 at 9:12 PM, Braddock Gaskill <braddock at gmail.com> wrote:
Hi Tim,The first step of the OSM processing is to import it into a huge PostGreSQL database, and is what is used in tile rendering.  That database has to reside on an SSD or the later rendering of tiles is too slow.  So no, you can't really reduce the SSD storage requirement.
I have a machine available that I built for this task if anyone wants to use it remotely.
-braddock
On Thu, Apr 23, 2015 at 11:13 PM, Tim Moody <tim at timmoody.com> wrote:
Any way to do the calcs in smaller chunks to cut down on the storage requirements, or even get to ramdisk?

Sent from my BlackBerry® PlayBook™
www.blackberry.com
From: "Braddock Gaskill" <braddock at gmail.com>
To: "xsce-devel at googlegroups.com" <xsce-devel at googlegroups.com>, "Braddock Gaskill" <braddock at braddock.com>
Sent: 24 April, 2015 1:45 AM
Subject: Re: [XSCE] Steps involved in generating OSM maps for IIAB

IIAB Open Street Map generation...
Note:-generating the OSM tiles requires a machine with at least a 500GB SSD (conventional HD is too slow) and 16 GB RAM, preferably with 4 or 8 cores. 
-It will take about a week of processing.
-It was last done in 2012 on an Ubuntu 12.04 machine, and the OSM tools have probably changed a lot
-I have a machine that should do the job if you want to do it remotely
-Documentation on the procedure is available in the IIAB "Maintainers Guide"https://github.com/braddockcg/internet-in-a-box/blob/master/doc/Maintainers_Guide.txt


-Further raw notes on what I did are in:https://github.com/braddockcg/internet-in-a-box/blob/master/doc/NOTES.txtSee both sections:"OpenStreetMap - 8/23/12"
and further down
OSM Planet Import
-If you are lucky, the OSM project documentation and tools have improved since 2012.  There was a LOT of trial and error back then.

-braddock



On Thu, Apr 23, 2015 at 10:24 PM, Anish Mangal <anishmg at umich.edu> wrote:
Yes. A listing of steps involved would be useful.

On Fri, Apr 24, 2015 at 10:53 AM, Tony Anderson <tony_anderson at usa.net> wrote:
Hi,



My recollection is that was done by Braddock. It involved rendering the tiles downloaded from OSM.



Tony



On 04/24/2015 06:23 AM, Anish Mangal wrote:


Hi,



Are the steps to generate OSM data for use with IIAB documented somewhere. Am willing to give it a try to download the map-data and generate the required images/files.



Best,

Anish


















--
-- 
Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !




-- 



 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.laptop.org/pipermail/server-devel/attachments/20150514/a0d397a8/attachment-0001.html>