[Server-devel] [UKids] Re: [XSCE] Tilestache on XSCE stack (OSM offline)
Jérôme Gagnon-Voyer
gagnonje at gmail.com
Tue Aug 11 13:18:06 EDT 2015
I see i see, so the disk space can't be exactly compared to what we see in
.mbtiles, since if you had to transfer that to another server and did not
want to do rendering on fly, this would be a different file size (much
bigger).
Thanks
On Tue, Aug 11, 2015 at 12:49 PM, Anish Mangal <anishmangal2002 at gmail.com>
wrote:
> I just used the default settings in renderd. As far as I know, I didn't
> generate any png or jpg tiles, only metatiles were generated, which are
> renderd as png on the fly.
>
> On Tue, Aug 11, 2015 at 8:10 PM, Jérôme Gagnon-Voyer <gagnonje at gmail.com>
> wrote:
>
>> Hi Anish
>>
>> Regarding file size
>>
>> Would you mind sharing how you generated those tiles? Like the settings
>> used, if its PNG 8 bit, full png or JPG, metatiles size, style used, etc...
>>
>> I'll be sending you and Tim in separate email the link to download the
>> .mbtiles file
>>
>> On Tue, Aug 11, 2015 at 9:35 AM, Anish Mangal <anishmg at umich.edu> wrote:
>>
>>>
>>> Hi
>>>
>>> Here is the data
>>> size zoom
>>> 28K 0
>>> 48K 1
>>> 80K 2
>>> 184K 3
>>> 340K 4
>>> 788K 5
>>> 2.1M 6
>>> 5.8M 7
>>> 19M 8
>>> 58M 9
>>> 188M 10
>>> 673M 11
>>> 17M 12
>>> 54M 13
>>> 134M 14
>>> 375M 15
>>> 1.1G 16
>>> 3.0G 17
>>> 9.6G 18
>>>
>>> As is obvious, for zoom levels 0-11 I had generated tiles for the entire
>>> world, which was essentially empty tiles except the shape of continents
>>> since the only data source I used was the nepal osm data
>>>
>>> Zoom level 12 and onwards, tiles were generated only for nepal.
>>>
>>> The bigger question i'd be asking is the setup required to generate
>>> tiles, and what it would take to do for the entire planet. I guess Braddock
>>> would be happy to give access to Jerome on hadron if he wants to give it a
>>> try. It is a machine with a 500G SSD and 32G RAM and an 8 core processor.
>>>
>>>
>>>
>>> On Tue, Aug 11, 2015 at 6:38 PM, Tim Moody <tim at timmoody.com> wrote:
>>>
>>>> Anish, you generated tiles for Nepal. How does the size compare to
>>>> what you got?
>>>>
>>>>
>>>>
>>>> *From:* unleashkids at googlegroups.com [mailto:
>>>> unleashkids at googlegroups.com] *On Behalf Of *Tim Moody
>>>> *Sent:* Tuesday, August 11, 2015 9:04 AM
>>>> *To:* 'Jérôme Gagnon-Voyer' <gagnonje at gmail.com>
>>>> *Cc:* 'Adam Holt' <holt at laptop.org>; 'xsce-devel' <
>>>> xsce-devel at googlegroups.com>; 'Unleash Kids!' <
>>>> unleashkids at googlegroups.com>; 'server-devel' <
>>>> server-devel at lists.laptop.org>
>>>> *Subject:* RE: [UKids] Re: [XSCE] Tilestache on XSCE stack (OSM
>>>> offline)
>>>>
>>>>
>>>>
>>>> Can I get the data, maybe a dropbox link? not sure of a place to ftp
>>>> unless you have one.
>>>>
>>>>
>>>>
>>>> *From:* unleashkids at googlegroups.com [
>>>> mailto:unleashkids at googlegroups.com <unleashkids at googlegroups.com>] *On
>>>> Behalf Of *Jérôme Gagnon-Voyer
>>>> *Sent:* Tuesday, August 11, 2015 9:01 AM
>>>> *To:* Tim Moody <tim at timmoody.com>
>>>> *Cc:* Adam Holt <holt at laptop.org>; xsce-devel <
>>>> xsce-devel at googlegroups.com>; Unleash Kids! <
>>>> unleashkids at googlegroups.com>; server-devel <
>>>> server-devel at lists.laptop.org>
>>>> *Subject:* Re: [UKids] Re: [XSCE] Tilestache on XSCE stack (OSM
>>>> offline)
>>>>
>>>>
>>>>
>>>> You define the path to the mbtiles in tilestache.cfg, so yes, they
>>>> could be in any location (as long as the access rights / permissions are
>>>> right)
>>>>
>>>>
>>>>
>>>> On Tue, Aug 11, 2015 at 8:28 AM, Tim Moody <tim at timmoody.com> wrote:
>>>>
>>>> This looks great and not overly complex. XSCE uses one convention that
>>>> you are probably not aware of and that is that we put data in /library, in
>>>> this case probably /library/osm or tilestache or some combination. Can
>>>> tilestache.cfg point to a location different than the html and config files?
>>>>
>>>>
>>>>
>>>> *From:* unleashkids at googlegroups.com [mailto:
>>>> unleashkids at googlegroups.com] *On Behalf Of *Adam Holt
>>>> *Sent:* Tuesday, August 11, 2015 12:19 AM
>>>> *To:* xsce-devel <xsce-devel at googlegroups.com>; Unleash Kids! <
>>>> unleashkids at googlegroups.com>; server-devel <
>>>> server-devel at lists.laptop.org>
>>>> *Subject:* [UKids] Re: [XSCE] Tilestache on XSCE stack (OSM offline)
>>>>
>>>>
>>>>
>>>> Don't stop pushing Jerome- I hope you can demonstrate Nepal-or-larger
>>>> working models during our Toronto summit Oct 10-12.
>>>>
>>>> See you 7AM EDT Thursday on our weekly call, if you have time for our
>>>> unusually early meeting this week: http://tinyurl.com/xsceminutes
>>>>
>>>>
>>>> (Several of our early design discussions on offline mapping began on
>>>> unleashkids at googlegroups.com, so am bringing that thread full circle.)
>>>>
>>>>
>>>>
>>>> On Mon, Aug 10, 2015 at 11:01 PM, Anish Mangal <anishmg at umich.edu>
>>>> wrote:
>>>>
>>>> adding server-devel (the publicly archived mailing list) as well.
>>>>
>>>> Great work. Thanks!
>>>>
>>>> Happy to help with the testing. I have an intel NUC running an atom
>>>> processor 2 Ghz, and 2GB RAM - not exactly low end, but still a viable
>>>> appliance to deploy the xsce.
>>>>
>>>> Could you tell me how to get the mbtiles from your ftp. If this works,
>>>> I can help with creating a playbook - and possibly admin console
>>>> integration.
>>>>
>>>> Thanks,
>>>>
>>>> Anish
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, Aug 11, 2015 at 8:19 AM, Jérôme Gagnon-Voyer <
>>>> gagnonje at gmail.com> wrote:
>>>>
>>>> Hi all
>>>>
>>>>
>>>>
>>>> This is about the project to bring a new way of storing OpenStreetMaps
>>>> offline for internet in a box (schoolserver).
>>>>
>>>>
>>>>
>>>> (Apologies if I'm using the wrong mailing list, feel free to add more)
>>>>
>>>>
>>>>
>>>> I wanted to test what it would look like to run Tilestache to serve
>>>> .mbtiles using the schoolserver. So I installed XSCE on a local VM with
>>>> Fedora 21 here, installed Tilestache, moved the .mbtiles in it, and did
>>>> some testing.
>>>>
>>>> So far I only tested with Nepal zoom 0-13 and Nepal zoom 14-15 (to see
>>>> how combining multiple files was working)
>>>>
>>>> I reduced the VM to use only 1 core at 50% (I'm on MBP Intel i5 2.6Ghz,
>>>> 2 cores). Speed was decent when viewing with leaflet in my browser from my
>>>> laptop.
>>>>
>>>>
>>>>
>>>> I've created a gist that explains all the steps required to do it
>>>>
>>>> https://gist.github.com/jeromegv/cb64bc2c9777794e80af
>>>>
>>>>
>>>>
>>>> The reason I wanted to test it was also to see if the integration would
>>>> be relatively straightforward. I assume you would eventually want to do the
>>>> install through the schoolserver admin console. When comes the time to do
>>>> that, I might not be the person to do this integration, but I think that
>>>> the steps I've done so far should at least prove that the XSCE stack should
>>>> be compatible.
>>>>
>>>>
>>>>
>>>> - I might not have the best testing setup or what is similar to a
>>>> typical schoolserver, if you have the proper hardware to test, let me know,
>>>> I can send you the .mbtiles on a FTP for nepal and you can test it (about
>>>> 1Gb)
>>>> - Feel free to provide feedback on the GIST and if the integration
>>>> seems to make sense
>>>> - There's an extra optimization that is possible for tilestache, to
>>>> cache the tiles to the HD once they were retrieved and served from the
>>>> .mbtiles once. It's currently disabled. Not so sure if it's worth it and
>>>> how much we would gain considering MBtiles are already pre-rendered files,
>>>> perhaps better to test once we have a full planet. The downside is that it
>>>> stores on HD a lot of PNG files and this might eventually baloon if you get
>>>> tons of people to use those maps. Benefit of sticking entirely to mbtiles
>>>> is we know exactly how much disk space. Might be optional setting from
>>>> admin console when there's plenty of HD available.
>>>> - Next step for me will be to test generation of MBtiles for bigger
>>>> country and eventually get to the Planet. Will need some type of cloud
>>>> setup to manage/store all that.
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Jérôme Gagnon-Voyer
>>>>
>>>> --
>>>>
>>>> Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
>>>>
>>>> --
>>>> Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Unleash Kids" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to unleashkids+unsubscribe at googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>> --
>>>> Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Unleash Kids" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to unleashkids+unsubscribe at googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>
>>>> Jérôme Gagnon-Voyer
>>>>
>>>> --
>>>> Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Unleash Kids" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to unleashkids+unsubscribe at googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>> --
>>>> Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
>>>> ---
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Unleash Kids" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to unleashkids+unsubscribe at googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>
>>
>> --
>> Jérôme Gagnon-Voyer
>>
>
>
>
> --
> Anish
>
> --
> Unsung Heroes of OLPC, interviewed live @ http://unleashkids.org !
> ---
> You received this message because you are subscribed to the Google Groups
> "Unleash Kids" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to unleashkids+unsubscribe at googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.
>
--
Jérôme Gagnon-Voyer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.laptop.org/pipermail/server-devel/attachments/20150811/a4818bb8/attachment-0001.html>
More information about the Server-devel
mailing list