[Marble-devel] [RFC] How to specify download policies

[Jens-Michael Hoffmann]

Hi,

we want to specify download policies.

The first use case is OpenStreetMap. Here we want to specify that in browsing mode we may have up to 20 open connections and in bulk downloading mode (for "Download region") we must 
not have more than 2 concurrent connections.

So, how does it look like now in openstreetmap.dgml?

<layer name="openstreetmap" backend="texture" >
    <!-- OpenStreetMap mapnik rendered tiles expire after seven days or 604800 seconds -->
    <texture name="mapnik_data" expire="604800" >
        <sourcedir format="PNG"> earth/openstreetmap </sourcedir>
        <storageLayout levelZeroColumns="1" levelZeroRows="1" mode="OpenStreetMap" />
        <projection name="Mercator" />
        <downloadUrl protocol="http" host="a.tile.openstreetmap.org" path="/" />
        <downloadUrl protocol="http" host="b.tile.openstreetmap.org" path="/" />
        <downloadUrl protocol="http" host="c.tile.openstreetmap.org" path="/" />
    </texture>
</layer>

Here we can see that for OpenStreetMap there are 3 download hosts defined which get used in a round robin fashion.

For defining our above mentioned policies there are different possibilities:

1) simple and at the moment sufficient, policies apply to all specified hosts

<layer name="openstreetmap" backend="texture" >
    <!-- OpenStreetMap mapnik rendered tiles expire after seven days or 604800 seconds -->
    <texture name="mapnik_data" expire="604800" >
        <sourcedir format="PNG"> earth/openstreetmap </sourcedir>
        <storageLayout levelZeroColumns="1" levelZeroRows="1" mode="OpenStreetMap" />
        <projection name="Mercator" />
        <downloadUrl protocol="http" host="a.tile.openstreetmap.org" path="/" />
        <downloadUrl protocol="http" host="b.tile.openstreetmap.org" path="/" />
        <downloadUrl protocol="http" host="c.tile.openstreetmap.org" path="/" />

+       <downloadPolicy downloadType="Browse" maxConnections="20" />
+       <downloadPolicy downloadType="Bulk" maxConnections="2" />

    </texture>
</layer>


2) complicated but powerful, policies can apply to subsets of hosts, for example we could say that
a.tile.openstreetmap.org may have 10 connections and b.tile.openstreetmap.org
and c.tile.openstreetmap.org may have 10 connections together, etc.

<layer name="openstreetmap" backend="texture" >
    <!-- OpenStreetMap mapnik rendered tiles expire after seven days or 604800 seconds -->
    <texture name="mapnik_data" expire="604800" >
        <sourcedir format="PNG"> earth/openstreetmap </sourcedir>
        <storageLayout levelZeroColumns="1" levelZeroRows="1" mode="OpenStreetMap" />
        <projection name="Mercator" />
        <downloadUrl protocol="http" host="a.tile.openstreetmap.org" path="/" />
        <downloadUrl protocol="http" host="b.tile.openstreetmap.org" path="/" />
        <downloadUrl protocol="http" host="c.tile.openstreetmap.org" path="/" />

+        <downloadPolicy downloadType="Browse" maxConnections="20">
+            <host>a.tile.openstreetmap.org</host>
+            <host>b.tile.openstreetmap.org</host>
+            <host>c.tile.openstreetmap.org</host>
+        </downloadPolicy>
+        <downloadPolicy downloadType="Bulk" maxConnections="2">
+            <host>a.tile.openstreetmap.org</host>
+            <host>b.tile.openstreetmap.org</host>
+            <host>c.tile.openstreetmap.org</host>
+        </downloadPolicy>

    </texture>
</layer>


What do you think?
I'm sure there are other possibilities as well, please don't hesitate to propose alternative solutions.


Best regards,

Jens-Michael