[Marble-devel] Re: Natural Earth / Vector Files [Was: Marble Sprint]
Torsten Rahn
tackat at t-online.de
Sun Oct 10 08:54:05 CEST 2010
Hi John,
Am Sonntag, 10. Oktober 2010 02:08:48 schrieb John Layt:
> The main benefit for the lightweight requirement switching to Natural Earth
> (NE) would be updated borders, a separate internal border dataset, and a
> few more physical features, but this should be worth it.
Yes, it definetely would be worth it. And we'd have the advantage that we
could even switch to a higher resolution for vectors easily.
> The main benefit for the second case is the embedded data attributes such
> as country code, groupings, relative magnitude, etc, which will allow easy
> linking and manipulation. We could do something similar using PNT, but we
> would have to create our own attribute storage to link things together,
> and then maintain the attribute data, so sticking with the shapefiles
> would seem a better solution there.
ok.
> a) Convert just the required NE datasets to pnt format, either merging the
> 3 scale levels into a single file with just 3 detail levels, or use
> Douglas- Peucker on the 1:10m files to create the required detail levels
We could also do the Douglas-Peuker (?) based level creation dynamically
inside our GeoDataLineString class (and in the deprecated GeoPolygon class):
Everytime the GeoDataLineString is modified (by adding or removing
coordinates) a dirty flag would be set. This dirty flag would would trigger
the determination of the detail level for each node once the nodes get
accessed for drawing (and the dirty flag would get unset).
We already do something like this for a similar expensive filter process
inside GeoDataLineString so the dirty flag which indicates modification is
already there. :-)
This would move the detail-evaluation from the converter into our linestring
class and it would have the benefit that it would automatically speed up KML
and GPX rendering since they are all using GeoDataLineString :-)
So with this approach we could even save storing the detail level in the PNT
file. We'd still have to modify the PNT format a bit since in the third
parameter it also indicates the start/end of a linestring and its type.
Another incentive for modifying the PNT format a bit according to our needs
would be accuracy: PNT stores data like this:
1st short int (16 bit) : Header (provides type of polygon/start or detail
2nd short int (16 bit): latitude in arcminutes
3rd short int (16 bit): longitude in arcminutes
We could change this to something like this:
1st integer (32 bit): Latitude in arcseconds + highest bit indicates new
polygon starts: header information has to be read from 3rd integer
2nd integer (32 bit): Longitude in arcseconds
optional 3rd integer: would store Geonames.org ID of the feature + highest
bit indicates whether it's a linestring or linearring.
> Overall, it seems the best approach for the updating the lightweight layer
> is indeed to convert the shapefiles to PNT format, provided the D-H
> algorithm can be deployed in a way to mark each point with a detail level
> rather than just throwing the points away.
>
> For the heavier vector layer with attributes, rather than just implementing
> a shapefile importer based on shapelib (a C library), it might be better
> to implement an GDAL/OGR importer which would import any vector format and
> is a C++ library. That would depend on packaging weight and difficulty,
> so needs more investigation, but that's for another day.
Thanks for the analysis. Sounds all good. :-)
Another thing:
Right now there are two different approaches for rendering vector data:
1.) Via Vectorcomposer/Vectormap and GeoPolygon: This is old old code and very
much deprecated but still used to display our PNT based vector maps.
2.) Via GeoPainter and GeoDataLineString ("libgeodata"): This is the new
"clean" approach. But it has got a feature regression: LinearRings (=closed
polygons) which contain one of the poles inside don't get rendered properly
yet (Think of Antarctica).
We are using this to render KML and GPX already. Thanks to Thibault Gridel it
can also render PNT, but we aren't using it yet due to the feature regression.
Shape file support should be based on this approach.
Ideally we'd get rid of 1.) altogether really soon ;-)
Best Regards,
Torsten
> Cheers!
>
> John.
>
>
> A lot of the NE data is duplicated / re-mixed, the core shapefiles sizes
> are:
>
> 1:110m 1:50m 1:10m
> ------ ----- -----
> Admin level 0 countries 172 KB 1.36 MB 6.55 MB
> Admin level 0 land borders 39 KB 301 KB 896 KB
> Admin level 0 sea borders 12 KB 40 KB 79 KB
> Admin level 0 disputed 40 KB 157 KB
> Admin level 1 regions 39 KB 339 MB 13.9 MB *
> Admin level 1 land borders 16 KB 60 KB 4.82 MB
> Coastlines 79 KB 883 KB 2.15 MB
> Rivers 19 KB 420 KB 3.29 MB
> Lakes 10 KB 286 KB 786 MB
> Glaciers 13 KB 208 KB 1.23 MB
> Dateline 18 KB 18 KB 18 KB
> Playas 18 KB 106 KB
> Ice Shelves 105 KB 211 KB
> Minor Islands 449 KB
> Reefs 171 KB
> ------- ------- ---------
> 417 KB 4.08 MB 34.03 MB
>
> * level 1 regions are USA/Canada only at 110m and 50m, but whole world at
> 10m, perfect for KGeography use :-)
>
> Other useful files:
> Physical Features Land 146 KB 1.50 MB 692 KB
> Physical Features Sea 348 KB 836 KB 836 MB
> Populated Places 347 KB 1.48 MB
> Urban Areas 439 KB 3.48 MB
> Bathmetry 11.64 MB
>
> Not sure how useful the 1:110m really is, but could be reduced for use in a
> country picker in kdelibs?
> _______________________________________________
> Marble-devel mailing list
> Marble-devel at kde.org
> https://mail.kde.org/mailman/listinfo/marble-devel
More information about the Marble-devel
mailing list