Simple Mapnik XML StyleSheet Generation

Mapnik XML Stylesheets

The normal choice for generating maps from OpenStreetMap data is Mapnik which is a very flexible image generator for maps.

To use Mapnik an XML based stylesheet needs to be generated which Mapnik will use to draw the parts of the map using information taken from a PostgreSQL database. The data is normally inserted into the database using osm2pgsql

The creation of the stylesheet is a complex task. The main OpenStreetMap web site originally used a hand-created stylesheet but has recently moved to a stylesheet that is generated from CSS type rules using the CartoCSS processor.

There are other methods of creating Mapnik XML stylesheets and in this context the Spreadnik pre-processor is relevant. This attempts to create a simple spreadsheet based way of defining what is to be drawn which is then converted into a Mapnik XML stylesheet.

This web page describes a new method of creating Mapnik XML stylesheets that is similar in some ways to Spreadnik in that it uses a spreadsheet, but simpler because it gives the user fewer choices although this does reduce the number of lines required.

Advantages / Disadvantages

There are a number of key advantages of using this spreadsheet method of generating the Mapnik stylesheet compared to manually generating one or using a tool that requires manual generation of the SQL statements.

Automatic generation of SQL statements: There is no need to understand the SQL syntax since the Perl script automatically generates an SQL statement that includes all columns that are required and tries to minimise the number of objects that are fetched.
Consistency of formatting of an object between zoom levels: Since the formatting uses only a small number of parameters the Perl script can ensure that the colours and formatting are consistent across zoom levels. This means that a particular road type will always be the same colour and have a width that scales across zoom levels.
Consistency between similar objects: With all of the objects of a similar type listed in a simple spreadsheet it is easy to ensure that there is consistency between them. This means that similar types of objects can appear in the same format at the same zoom level
Simplicity: By reducing the number of configurable options the whole process of customising the style sheet is simplified. This should make map generation available to more people.

There are also disadvantages of the spreadsheet method. Primarily because it is not possible to customise the SQL statements or to have drawing styles that are different to the ones that the spreadsheet allows or particularly complex.

Stylesheet Changes

While converting the CartoCSS stylesheet to a spreadsheet (in around October 2013) a number of small changes were made. The CartoCSS stylesheet has changed since then so some of these comments no longer apply.

The first set of changes are related to the limited options in the spreadsheet which has also imposed a consistency in formatting so that the following has changed (only visible at some zoom levels):

Roads are always drawn with fill and with casing. The original OpenStreetMap style had some minor road types drawn only with casing and no fill at some mid-range zoom levels.
Roads are generally narrower at higher zoom levels than in the standard rendering, this is due to the automatic scaling of roads to get a similar width at lower zoom levels.
Roads are drawn with a wider line for more major roads and narrower line for less major roads (according to the highway tag) which the OpenStreetMap style doesn't quite do at some zoom levels.
Some road casing colours are slightly different to make them more consistent between road types and similar to the fill colours.
The bridge and tunnel styles are slightly different for some highway or railway types.
The railway=miniature type is drawn with a symbol rather than a complicated multi-layered line style.
Similar types of objects are drawn at the same zoom level, for example amenity=bar is now drawn at zoom 16 where amenity=pub and amenity=biergarten are also drawn rather than at zoom 17.
Turning circles are not drawn. These use a very complex SQL statement to determine the road type which does not fit with the spreadsheet method.
There are some small changes in line-cap, line-join, text spacing and other Mapnk symboliser details since these are defined for all object types in the Perl script rather than for each object.

The second set of changes are stylistic changes to improve clarity (only visible at some zoom levels):

The road references are rendered in text along the highway rather than in shields on the highway. This is my personal preference and switching back should be easy since the spreadsheet contains the required information.
The path, footway, bridleway and cycleway highway types are all rendered in the same type of dashed lines using the same colours as the original but not using different widths and dash types.
The track highway type is rendered with different dash lengths depending on the tracktype tag (shorter dashes for less distinct tracks and longer dashes for more distinct tracks) rather than different colours, widths and dash patterns.
Residential roads are rendered in light grey and white is reserved for unclassified roads, this is to show the difference between them.
Residential roads are not drawn until a higher zoom level (zoom 12) this has the side-effect of making residential areas less visible at lower zooms. (This could be fixed by having a darker residential fill at low zooms).
Aerodromes are drawn only at lower zoom levels otherwise there are many of them drawn at zoom 10 even if they are grass landing strips.
All roads of type highway=service are drawn from zoom 13, not just the more important types (this could also be changed if required).
Private access highways are shown with narrow red lines across the road rather than a wide red dashed shading.
No green dashed shading is used on permissive access highways.
Designated access highways are drawn normally and then overdrawn (semi-opaque) with the appropriate footway, bridleway or cycleway styles
The administrative boundaries have been drawn less opaque and narrower so that they do not obscure other map objects.
Shops that do not have a specific symbol defined have the type of shop printed instead (in a distinct font).

There are some parts of the implementation of the standard stylesheet that have not been debugged so there may be parts of the map that do not quite render properly. There are probably also some errors in the conversion that might not render correctly.

Spreadsheet Description

Each sheet contains a different type of data (for example: areas, lines, points) that has different columns to define what is to be drawn.

Each row within a sheet contains the information required to draw one type of object for a range of zoom levels.

Some information is common to all objects:

LAYER - The name of the layer which corresponds to one database query and can contain many styles.
STYLE - The name of the style which corresponds to drawing one group of things for each matching object.
FILTER - The primary database comparison which limits what is drawn for the style and also is used to minimise the amount selected from the database. Only keys matching the values in the filter column will be selected from the database.
SUB-FILTER - An additional filter that can be applied which does not influence the data selected from the database like it does in the 'filter' column.
MIN_ZOOM - The first zoom level (lowest numbered) that matches this style; additional 'MIN-ZOOM-*' columns are used for extra features.
MAX-ZOOM - The last zoom level (highest numbered) that matches this style; additional 'MAX-ZOOM-*' columns are used for extra features.

When drawing the objects they are drawn in the following order:

Sheet order - Shapefiles are drawn before areas which are drawn before lines which are drawn before points.
Layer order - The first layer on a sheet is drawn completely before the second layer before the third layer ...
Reverse style order - The first style within a layer is drawn on top of the second on top of the third ... (or you can consider the last style is drawn first, then the second last, ...).
Line order - Within a style only one line will be drawn for each object, if the first one matches (zoom and filters) then it is drawn, else if the second matches it is drawn, else if the third matches ... ; therefore the most restrictive filter should be placed before the least restrictive filters.

Objects are only drawn for the zoom levels which match the specified range, by using more than one line with the same LAYER, STYLE and FILTER for different zoom ranges the styling of an object can be changed at each zoom level.

In the object definitions in the spreadsheets the database columns must be enclosed in square brackets where they are used (for example: [highway]) and the text values that they are compared against should use single or double quotes (for example: "motorway" or 'motorway').

In the object definitions the additional styles defined by the "*-STYLE" column will override any automatically generated styles (for example: line widths can be fixed not automatically scale in the "lines" sheet). The styles should be appropriate for the symboliser type and consist of a name, an equals sign and a value; if there are multiple style options then they should be separated by a semi-colon (for example: "stroke-width=1 ; opacity=0.5").

Shapefiles Sheet

These are for reading data from a file, generally used for low zoom (to avoid database accesses) or for coastlines.

The columns for the shapefile sheet are the simplest possible to allow implementation of the standard OSM shapefiles.

Each row is implemented as a separate layer so the rules for layer names described above does not apply here.

Areas Sheet

These are areas or polygons in the data (for example: landuse, natural, buildings) and they can be filled with a plain colour or a symbol and/or have a line drawn round them and/or have a label placed in them.

There are four sections for each area object drawn in this order (bottom to top as seen in the final image):

A plain colour fill controlled by the MIN-ZOOM-FILL, MAX-ZOOM-FILL, FILL-COLOUR and FILL-STYLE columns.
A pattern fill controlled by the MIN-ZOOM-SYMBOL, MAX-ZOOM-SYMBOL and FILL-SYMBOL columns.
A border around the object controlled by the MIN-ZOOM-LINE, MAX-ZOOM-LINE, LINE-COLOUR and LINE-STYLE columns.
A label for the area controlled by the MIN-ZOOM-LABEL, MAX-ZOOM-LABEL, LABEL-COLOUR, LABEL-SIZE, LABEL-STYLE and LABEL columns.

A label is only drawn on objects that are "large enough" which is automatically controlled by additional filtering on the 'way_area' database column. The labels are drawn after the lines so that they are not obscured.

Lines Sheet

These are linear objects (for example: highway, railway, boundaries) and they can be drawn with a casing (normally used on roads) and/or an underlay (a solid colour for under a dashed line) and/or a fill colour or pattern, and can optionally have tunnel and/or bridge markings and/or oneway markings and/or one or two labels along them.

There are six sections for each line object which will be drawn in order (bottom to top as seen in the final image):

A casing controlled by the MIN-ZOOM, MAX-ZOOM, MIN-ZOOM-CASING and CASING-COLOUR columns and with a width controlled by the WIDTH-Z18 column (see below).
- If the line is on a bridge and the zoom is more than MIN-ZOOM-TUNNEL-BRIDGE then the casing colour is changed to BRIDGE-CASING-COLOUR.
- If the line is in a tunnel and the zoom is more than MIN-ZOOM-TUNNEL-BRIDGE then the casing colour is changed to TUNNEL-CASING-COLOUR and drawn as a dashed line.
A fill controlled by the MIN-ZOOM, MAX-ZOOM, FILL-COLOUR, FILL-STYLE, FILL-PATTERN and WIDTH-Z18 (see below) columns.
- If there is an UNDERLAY-COLOUR then this is drawn before the fill to allow a dashed line to be drawn above it.
- If the line is in a tunnel and the zoom is more than MIN-ZOOM-TUNNEL-BRIDGE then the fill colour is changed to TUNNEL-FILL-COLOUR and drawn as a dashed line.
- If there is a FILL-PATTERN selected then this is used instead of the FILL-COLOUR.
Direction arrows (using a specific arrow pattern) controlled by the MIN-ZOOM, MAX-ZOOM and MIN-ZOOM-ONEWAY columns.
A label drawn along the line controlled by the MIN-ZOOM, MAX-ZOOM, MIN-ZOOM-LABEL, LABEL-COLOUR, LABEL-SIZE, LABEL-STYLE and LABEL columns (if SHIELD-SYMBOL is not set).
A shield drawn on the line controlled by the MIN-ZOOM, MAX-ZOOM, MIN-ZOOM-LABEL, LABEL-COLOUR, LABEL-SIZE, LABEL-STYLE and LABEL columns (if SHIELD-SYMBOL is set).
A second label drawn on the line controlled by the MIN-ZOOM, MAX-ZOOM, MIN-ZOOM-LABEL2, LABEL2-COLOUR, LABEL2-SIZE, LABEL2-STYLE and LABEL2 columns.

The width of the line fill is set by the value of WIDTH-Z18 when the zoom is 18; for smaller zooms the width halves for each change of 2 in the zoom level and for larger zooms the width doubles for each change of 2 in the zoom level. The width of the casing is generally 1.5 times the width of the filled section but always between 1 and 3 pixels wider.

Points Sheet

These are point objects (for example: barriers, amenities) which are drawn with a symbol and/or one or two labels.

There are three sections for each point:

A symbol for the object controlled by the MIN-ZOOM-SYMBOL, MAX-ZOOM-SYMBOL and SYMBOL columns.
A label for the object controlled by the MIN-ZOOM-LABEL, MAX-ZOOM-LABEL, LABEL-COLOUR, LABEL-SIZE, LABEL-STYLE and LABEL columns.
A second label for the object controlled by the MIN-ZOOM-LABEL2, MAX-ZOOM-LABEL2, LABEL2-COLOUR, LABEL2-SIZE, LABEL2-STYLE and LABEL2 columns.

In addition the same symbol and label(s) can be used for polygons if the POLYGON column specifies this is to happen. This allows the same map marking for an object (e.g. A shop) that has been marked with a point and one that has been marked with a polygon.

Generating The Stylesheet

The sheets must be exported from the spreadsheet in TAB separated format without quotes to files named after each of the sheet names (for example: shapefiles.txt, areas.txt ...).

The Mapnik stylesheets (one for each zoom level and one containing all information) are generated by running the Perl script called "mapnik.pl".

Generated Mapnik Stylesheet

The Mapnik stylesheet that is generated is split up into sections for each zoom level of the final map and each zoom level has its own set of Layers and Styles. This is in contrast to the OpenStreetMap Mapnik stylesheet that has traditionally mixed zoom levels within a single Layer.

The advantage of generating a new Layer for each zoom level for the same set of objects is that an optimised SQL statement can be used which is automatically generated by the Perl script. This means that the minimum set of data can be extracted from the database since all rules will match the current zoom which is not always the case if one Layer is used for multiple zoom levels. This also means that the minzoom and maxzoom attributes on the Layer can be used instead of in each rule which simplifies the file.

Each Layer also uses multiple styles (in general) and the cache-features attribute on the Layer to optimise the performance with re-using the same object information in each layer.

When generating the map tiles Mapnik will process the objects read from the database in the following order:

For each Layer in the stylesheet (in order):
- Read the objects from the database using an SQL statement to select the ones of interest.
- For each Style in the Layer (in order):
  - For each object read from the database (in order):
    - For each Rule in the Style (in order):
      - If the object matches the rule then draw the symbols defined in the rule and go to the next object.

By using multiple Styles within the Layer the rules for drawing lines can ensure that road casings (borders) are drawn before the line fill which is drawn before the oneway arrows which are drawn before the names and/or references. This avoids needing to select the data from the database multiple times for casing, fill, oneway and then text which has traditionally been done by the OpenStreetMap Mapnik stylesheet.

For the Layers that are used for lines the group-by attribute is used and the layer tag of the object in the database is used to order the objects. In this case the processing of objects is slightly different as shown below:

For each Layer in the stylesheet (in order):
- Read the objects from the database using an SQL statement to select the ones of interest and sort them by "layer" tag.
- For each "layer" tag read from the database (in order):
  - For each Style in the Layer (in order):
    - For each object read from the database with this "layer" tag (in order):
      - For each Rule in the Style (in order):
        
        If the object matches the rule then draw the symbols defined in the rule and go to the next object.

This means that if a single Layer is used for roads and railways then they will be drawn in the correct order with the lower layers completely drawn first before the upper layers. The OpenStreetMap Mapnik stylesheet has for a long time used more than 20 Layers to draw roads and bridges in the correct order (but not tunnels on different layers) which means searching the database more times.

Status

This software is no longer maintained (since August 2014).

This method of creating a Mapnik stylesheet is easier than the Carto based stylesheet used on the main OSM map. Maintaining feature parity with that stylesheet is very complex so this stylesheet is no longer updated.

There are still, in my opinion, advantages of this custom style which is why it is still used. These advantages are: replacement of shields on roads with text and drawing of rights of ways over other highways.

Download

The complete source code to create your own styles can be browsed in the Subversion viewer which also has a list of the latest changes.

The source code can be downloaded from the Subversion repository with a command like the following:

svn co http://gedanken.org.uk/svn/spreadsheet-mapnik/trunk spreadsheet-mapnik