Imagico.de

blog

Florence in Spring 2022

August 11, 2022
by chris
13 Comments

State of the Map 2022

In about a week the OSMFs State of the Map conference is going to start and to avoid anyone looking for me in vain there (i have been at the last four State of the Map conferences – the last two of which were virtual events) – i am not going to be there this year.

This has multiple reasons, one of the most relevant being that this year’s event is – more than in any of the last four years – an event quite ostentatiously targeted at wealthy people and people whose visit is paid for by a third party. It is perfectly fine to have such an event, and i wish everyone at the conference an enjoyable and successful time, but this makes it much less attractive for me to visit.

With the strong limitations of the last two years to meet in larger groups, i would this year have liked to meet in person with various people involved in OpenStreetMap whom i could not meet in the past years, including quite a few who are likely going to be in Florence despite the fairly steep costs of a visit there during the height of holiday season with hotel and travel prices at their peak. But to be frank – what i enjoyed most in previous years was in particular meeting and talking to economically more marginalized people.

I played a bit with the idea of visiting Florence but not going to the commercialized conference and instead just meeting with people outside the organized event and visiting and getting to know the city otherwise. But as said there were other reasons speaking against it and also, while i could have afforded a visit to Florence in August, that is not equally the case for many other people active in OpenStreetMap. And choosing Florence in August as the place and time where people from the OSM community have the chance to meet me in person would also have made a statement as to whom i am mostly interested in meeting, even when done detached from the SotM conference.

In past years i repeatedly have criticized that affordability of the visit is not a criterion in the planning of the conference, not even on paper, and neither for picking the place nor for picking the date of the conference. This fact is of course perfectly in line with the OSMFs diversity statement – which disallows discrimination by almost anything, but quite explicitly not by wealth.

Anyway – for those in the OSM community who would like to meet me in person – i plan to be at the Karlsruhe Hack Weekend in September, after also having been absent there for the past two years. Despite the limited number of participants allowed, there is still room for people to come there, Karlsruhe is just a two hour train ride from Frankfurt Airport and there is a good selection of decent and affordable places to stay in Karlsruhe.

I also generally look with interest at the announcements of any other events in the OSM community that have been planned with the visible intend and desire to be affordable and accessible also for less wealthy people and being truly welcoming for a diverse audience in the original sense of the word. For the past two years the options to visit such events have been rather limited, but i hope this will get better in the future again.

Florence in Spring 2022

Florence in Spring 2022

Concluding remarks on tree rendering

June 5, 2022
by chris
1 Comment

Concluding remarks on tree rendering

This is part four of a four part series of blog posts on the depiction of trees in maps – see part 1 for a discussion of tree rendering in traditional maps and plans, part 2 for mapping of trees in OpenStreetMap and rendering in OSM based maps and part 3 for the discussion of a new concept of tree rendering.

The tree rendering implementation i showed and explained in the previous part is essentially just a sketch demonstrating a few of the design concepts you can explore once you truly disregard the technical constraints of common automated map rendering tools. Is this too much for a general purpose map style? Maybe.

As i mentioned recently again the whole world of automated rule-based map design is, once you leave the pre-made paths scouted already by tech companies and tool developers working for them, largely unexplored territory. Working on the tree rendering once more made me realize how much that is true especially for large map scales and high zoom levels. Once you start truly designing maps specifically for z19/z20 you are well into the traditional domain of architectural drawings and site plans – which is a whole new field of considerations and design ideas, as i tried hinting at in the first part with some examples. Most digital interactive map designers so far simply treat this scale range as an extrapolation of the smaller scales. This even more applies to all the modern vector tile maps where everything above z16/z17 is literally just the same rendering blown up to larger scales plus more POI symbols and labels.

And i think this series of blog posts also once more demonstrates that the inherent, high level capabilities of currently available map rendering tools are very limited compared to what in principle would be possible and desirable to have to produce well readable maps. The CartoCSS + Mapnik toolchain i use here has the advantage that it provides the means (via PostGIS code) to implement this on a low level by hand but practically such low level techniques are probably beyond reach for many map designers. And using this method has its own limitations – apart from the obvious efficiency issues you also cannot combine this kind of manual symbol processing with use of symbol/label collision detection and prevention for example.

So once again my strong suggestion to the OSM and the FOSS community that i have already stated on various occasions: You need to strategically invest in advanced map design tools for automated rule-based map rendering. And if you do so while looking at things purely from a software developer perspective or following the footsteps of big tech companies you will fail. The proprietary software producers have learned this lesson. They actively involve and actively listen to map designers in their strategic planning and research. Their target users so far are predominantly still the traditional map creators to whom map design means manual processing of concrete data and not the development of automated rules to process generic data like i discuss it here. But this is going to change. So do not miss this window of opportunity.

Flowering tree

A new design for tree rendering in digital maps

June 3, 2022
by chris
5 Comments

A new design for tree rendering in digital maps

This is part three of a four part series of blog posts on the depiction of trees in maps – see part one for a discussion of tree rendering in traditional maps and plans and part two for mapping of trees in OpenStreetMap and rendering in OSM based maps.

Based on what i discussed in the first two parts of this series of blog posts, i have contemplated and experimented with new ideas for rendering trees in digital maps for quite some time. In the alternative-colors style i had already reverted the removal of tree depiction from z16 and z17 earlier. Since trees are important for many map users and mappers evidently invest a lot of time and energy in mapping them, degrading them to a POI of local importance is not a good idea in my opinion.

But the problems with the existing rendering in OSM-Carto otherwise remain:

  • The transparency in rendering is not working well on the higher zoom levels (z18 and z19) where it applies to a larger area.
  • The symbols use a lot of space and affect readability of the map, in particular in areas with dense presence of trees and this could even discourage mappers from mapping all trees in some cases. At the same time it is wasteful because it transports no information beyond the simple here is a tree while – as discussed in the previous post – mappers use all kinds of secondary tags to further characterize trees which could well be used for more differentiated rendering, in particular at the higher zoom levels.
  • These problems will further aggravate if you’d want to extend the style to higher zoom levels.

I contemplated the idea of using profile symbols, because – as discussed – this is very common in traditional maps and it allows transporting differentiated additional information on the type of tree. However, this approach would not work too well at larger scales/higher zoom levels. And it does not provide very precise information on the position of the tree. As i showed, large scale plans typically use from-the-top view depictions and have developed a lot of useful design concepts in that domain. If you want to have a useful depiction of the tree size at these scales there is really no alternative to a from-the-top view.

The basic design concept

The basic design idea i have now implemented is using from-the-top view symbols in a line drawing design. If the symbol design is not too dense it allows features below the tree symbol to be visible through the line drawing – while being less distorted in coloring than with the semitransparent fill color of the OSM-Carto design.

Tree depiction using line drawing symbols - left at normal scale (z19), right at double resolution rendering

Tree depiction using line drawing symbols – left at normal scale (z19), right at double resolution rendering

z16 z17 z18 z19 z20
natural=tree broadleaved at z16 with Alternative-colors style natural=tree broadleaved at z17 with Alternative-colors style natural=tree broadleaved at z18 with Alternative-colors style natural=tree broadleaved at z19 with Alternative-colors style natural=tree broadleaved at z20 with Alternative-colors style
natural=tree needleleaved at z16 with Alternative-colors style natural=tree needleleaved at z17 with Alternative-colors style natural=tree needleleaved at z18 with Alternative-colors style natural=tree needleleaved at z19 with Alternative-colors style natural=tree needleleaved at z20 with Alternative-colors style

Zoom level progression of basic natural=tree from z16 to z20

The difficulty is how to deal with symbol overlaps. I definitely wanted to retain the feature of tree symbols being non-blocking because trees frequently exist in dense groups and just rendering one of them in such cases is highly non-ideal. And in areas where there are a lot of POIs (like in dense urban environments) most trees would not be rendered if they were blocked by other symbols.

If you’d naively show line drawing symbols with overlaps you would often end up with a dense, non-discernible mess in areas with dense presence of trees. The technique of applying layer based opacity in OSM-Carto successfully avoids that – but only for a uniform fill color. I considered applying a structure pattern to a polygon aggregated from the buffers of the individual trees (a bit similar to the rendering of unpaved roads). But that would look rather crude for trees and would evidently be a solution just chosen because it is technically simple and not because it is good design.

The samples from architectural drawings and site plans i showed in the first part demonstrate a number of techniques how overlapping top view tree depictions can be handled graphically. None of the automated map rendering tools i know of has any kind of built-in support for anything like that though. What i ultimately ended up with is implementing the whole symbol drawing in PostGIS. This is certainly not very efficient (doing this at the compositioning level of the renderer rather than at the geometry processing stage would be much faster probably) but it is doable.

To make that possible, all the symbols were entered into a distinct database table as PostGIS polygon geometries and are then processed to generate the desired shapes to be then rendered just as if they were mapped geometries from the rendering database. This probably sounds more complex than it is in principle – though the geometry processing is certainly non-trivial. The route to get SVG symbols into a PostGIS database is via conversion to PDF using inkscape and then using OGR basic PDF support (with OGR_PDF_READ_NON_STRUCTURED). As an alternative route of symbol generation for simple abstract shapes i also implemented support for generating them using PostGIS code. Both routes can also be combined to supplement/edit a file based symbol with a PostGIS query. This is in particular useful for dynamically scaled symbols – as i will discuss in more detail below.

Handling of overlaps between tree symbols

Handling of overlaps between tree symbols

Tree type depiction

As you could already see in the examples so far i designed different symbols for different types of trees. This is less easy to do in an intuitive fashion than for profile symbols but it is still manageable at least for the basic leaf_type and leaf_cycle classes.

Tree symbols differentiated by leaf_type/leaf_cycle at z16 to z20

Tree symbols differentiated by leaf_type/leaf_cycle at z16 to z20

At z16 and z17 trees are – like in OSM-Carto before 2017 – rendered with a plain green dot. At z18 trees start being differentiated by leaf_type and from z19 on also by leaf_cycle. The illustration of deciduous leaf_cycle takes the form of a display of the major branches and trunk of the tree (as visible from above in the absence of foliage) or a cleared center area in case of the needleleaved symbol. The lack of a specified leaf_cycle is illustrated with a small circle in the center of the symbol. Palms (genus/taxon=Arecaceae) are shown with a distinct symbol (independent of the leaf_type/leaf_cycle tag).

Tree sizes

The other main feature for individual tree rendering i implemented in addition to the leaf_type/leaf_cycle depiction and the customized handling of symbol overlaps is dynamic scaling of symbols according to recorded information about the tree size.

Trees vary greatly in sizes – what is commonly mapped in OpenStreetMap as natural=tree might have a height of just 2-3m for a small ornamental tree up to more than 50m. Canopy diameters (diameter_crown) can range from a meter or less up to around 50m as well. Such information is not recorded for a large percentage of trees mapped in OpenStreetMap but in absolute numbers it is still quite a few that come with such data. And visualizing tree size information in the map at the higher zoom levels would increase the usefulness of tree rendering quite significantly.

With the top view visualization i have chosen here a natural way to visualize the tree size is by varying the symbol size based on the canopy diameter. This is also the way large scale site plans often differentiate trees – as i have shown in the first part of this blog post series. Just scaling the whole symbol however does not work very well for that. For a line drawing symbol, scaling the line work while maintaining the drawing width would be a possibility – that is what most of the mentioned site plans do. However this still can lead to the problem that smaller symbols would have a too detailed geometry while larger symbols are not detailed enough. I tried and implemented both the approach to design distinct symbols for different rendering sizes and to have parametrized symbol geometries that scale according to non-trivial rules. Here is how this looks like:

Trees with different diameter_crown values at z18 - click to see z19

Trees with different diameter_crown values at z18 – click to see z19

To inspect the symbols in the symbols table i also created a separate map style file (treeinspector.mml) that visualizes the content of the symbols table as is. Here is how this looks like:

The symbols table inspection style

The symbols table inspection style

The symbols for palms exist only for the sizes they have been designed for. The renderer then picks the closest match in size and scales it to the tagged tree size, rounded to the next integer pixel size to ensure pixel alignment as far as possible. For the parametrized symbols (most of the larger ones) a separate symbol is pre-generated by the generate_symbols.py script for every integer symbol size. This is either fully generated by PostGIS SQL code (for the unknown leaf_type symbols on the right) or composed of a scaled SVG based design and additional parametrized modification via PostGIS (for the known leaf_type symbols). As you can see the overall size of the symbols are scaled while the line with of the abstract circles and the geometry of the center symbol is not in the same way (for larger symbols this also gets increased moderately). For the larger symbols with know leaf_cycle the center portion (indicating the leaf_cycle) is rendered at a constant size while the outer rim (which does not change with leaf_cycle) is scaled according to the symbol size. The definition of all these symbol generation rules can be found in symbols.yml.

In the above symbol table only odd sizes (in pixels) are shown above 17 pixels. Only those are used for the individual tree rendering to ensure a uniform pixel aligned rendering at different sizes. For the tree rows (see below), where the symbols are not pixel aligned, also the even sizes are used.

In case diameter_crown is not mapped for the tree an estimate is made from either the height (0.3 times the height) or the trunk diameter (5 times the diameter). This is implemented in the function carto_tree_diameter_mapped(). As with other cases of ground unit rendering i implemented in the alternative-colors style, the symbol size is chosen as the maximum of the default symbol size for that zoom level and the size as tagged or estimated based on tagged height or trunk diameter.

With the shown technique to cut out overlapping tree symbols there is also a possibility to indicate different tree heights separately to some extent by having the height define the drawing order. If the height is not tagged, the drawing diameter of the tree symbol (determined as described above) is used to define the drawing order instead. That of course means that small trees completely under large ones are hidden.

Overlapping tree symbols, ordered by tree height (at z19, with varying diameter_crown as well)

Overlapping tree symbols, ordered by tree height (at z19, with varying diameter_crown as well) click for double resolution rendering

Tree rows

One part of this tree rendering project i struggled quite a lot with are the tree rows. This is to a large extent because natural=tree_row is a fairly ill defined concept in OpenStreetMap. As mentioned in part 2 there is no established tagging to indicate the spacing of trees in a tree row and neither is there a clear agreement what qualifies a set of trees to form a tree row. A tree row might be

  • four trees spaced 50 meters apart but planted evidently in a collinear fashion so forming an abstract row while not in any way forming a continuous object.
  • trees along the edge of a road or stream which might have originally been planted in a systematic fashion but with – at present – irregular intervals between them.
  • a dense arrangement of trees along the side of a agricultural plantation serving as wind protection.
  • a regularly spaced line of trees in a tree plantation for agricultural purposes.

All of that might indiscriminately be modeled with natural=tree_row in OpenStreetMap. Visualizing that in a way that properly represents all of these cases in a decent fashion is next to impossible.

The traditional rendering in OSM-Carto is taking the simplistic approach to simply align the design directly to the data model. By rendering individual trees as a buffered point and rendering tree rows as a buffered line. While that is a reasonable choice to give the mapper very direct feedback to what they have mapped, it does not provide much support or guidance regarding the semantics of that mapping.

The approach i ultimately chose for tree rows is to render them like a line of individual trees with a relatively dense arrangement of individual tree symbols. But the cutting out of the individual tree symbols against each other is done in a different fashion, not prioritizing one symbol over the other but cutting them at the bisector between them. I am not sure if this is the best solution. But it is one that harmonizes quite well with the individual tree depiction and at the same time makes the results easy to distinguish from a line of trees mapped individually.

Tree row rendering with different tagged width values at z18

Tree row rendering with different tagged width values at z18

z16 z17 z18 z19 z20
natural=tree_row broadleaved at z16 with Alternative-colors style natural=tree_row broadleaved at z17 with Alternative-colors style natural=tree_row broadleaved at z18 with Alternative-colors style natural=tree_row broadleaved at z19 with Alternative-colors style natural=tree_row broadleaved at z20 with Alternative-colors style
natural=tree_row needleleaved at z16 with Alternative-colors style natural=tree_row needleleaved at z17 with Alternative-colors style natural=tree_row needleleaved at z18 with Alternative-colors style natural=tree_row needleleaved at z19 with Alternative-colors style natural=tree_row needleleaved at z20 with Alternative-colors style

Zoom level progression of basic natural=tree_row from z16 to z20

For efficiency in rendering – since all of this has to be done in SQL code on the fly – the individual tree symbols of the tree row are only cut against the previous and next point on the line which leads to some overlap artifacts in case of sharp corners on the line.

Tree rows with sharp corners at z19

Some might wonder why i have not used a line pattern to generate this kind of visualization. There are multiple reasons for that. First: line patterns in Mapnik don’t have support for line caps – which would not work very well for this use case. Second: Line patterns are drawn continuously along the length of the line and no special attention is given to corners in the line geometry. What you’d want, however, for tree rows (and especially also hedges – see further below) is that there is a tree symbol positioned at every node of the line to give proper visual feedback on the exact geometry. The additional tree symbols would be distributed between the line nodes until the desired point density is achieved (which can be done with ST_Segmentize() in PostGIS). This has the disadvantage that the symbol density along the line varies and the design needs to be robust enough to deal with that – which however is the case here. The third reason is that line patterns are always oriented along the line so there is no good way to implement a shading effect like i use in the tree symbols here.

Symbol placement on tree rows at z19

Symbol placement on tree rows at z19

Shrubs/bushes and hedges

For rendering shrubs i chose the same design as for trees – but they only get rendered starting at z19 so there is not much risk of confusion. The symbols at z19 are already differentiated by leaf_type – but that difference is very subtle.

natural=shrub rendering at z19natural=shrub rendering at z20

Shrub symbols differentiated by leaf_type at z16 to z20

Hedges are rendered like in OSM-Carto until z19 and get the tree row like differentiated design at z20.

Differentiated rendering of barrier=hedge at z20

Differentiated rendering of barrier=hedge at z20

For hedges i added rendering for the regional specialty in northern Germany called Knick (see the second part of this blog post series) – for which there is a tag in OpenStreetMap: hedge=hedge_bank. This is indicated in a similar fashion as i render implicit embankments on roads:

z16 z17 z18 z19 z20
barrier=hedge at z16 with Alternative-colors style barrier=hedge at z17 with Alternative-colors style barrier=hedge at z18 with Alternative-colors style barrier=hedge at z19 with Alternative-colors style barrier=hedge at z20 with Alternative-colors style
barrier=hedge with hedge=hedge_bank at z16 with Alternative-colors style barrier=hedge with hedge=hedge_bank at z17 with Alternative-colors style barrier=hedge with hedge=hedge_bank at z18 with Alternative-colors style barrier=hedge with hedge=hedge_bank at z19 with Alternative-colors style barrier=hedge with hedge=hedge_bank at z20 with Alternative-colors style

Zoom level progression of basic barrier=hedge from z16 to z20

And here trees, shrubs, tree rows and hedges of various types at z20 for comparison.

Trees, shrubs, tree rows and hedges at z20 all together

Trees, shrubs, tree rows and hedges at z20 all together

Real world examples

And for the end of this part after all the abstract examples illustrating the design in principle, here a few examples based on actual OpenStreetMap data. More samples can be found on the Alternative-colors style sample gallery and on my style comparison page.

Freiburg, Germany, z18

Freiburg, Germany, z18

Freiburg, Germany, z19

Freiburg, Germany, z19

Lago di Garda, Italy, z18

Lago di Garda, Italy, z18

Lago di Garda, Italy, z19

Lago di Garda, Italy, z19

Lago di Garda, Italy, z20

Lago di Garda, Italy, z20

Strasbourg. France, z18

Strasbourg. France, z18

Strasbourg. France, z19

Strasbourg. France, z19

Aukrug, Germany, z19

Aukrug, Germany, z19

Bumwali, Uganda, z18

Bumwali, Uganda, z18

Schallstadt, Germany, z18

Schallstadt, Germany, z18

The fourth and last part of this series of blog post is going to provide some discussion and conclusions on the subject of individual tree rendering in digital maps.

Trees in OpenStreetMap

June 1, 2022
by chris
1 Comment

Trees in OpenStreetMap

This is part two of a four part series of blog posts on the depiction of trees in maps – see part one for a discussion of tree rendering in traditional maps and plans.

Tree mapping in OpenStreetMap

In OpenStreetMap trees are mapped with the tag natural=tree. This is among the ten most common tags in the OSM database with more than 18 million objects. It is also the third most commonly mapped singular feature type (after buildings and addresses – the other more common primary tags are road tags).

Consensus among mappers in OpenStreetMap is that ordinary trees in a forest/wood are not mapped individually. However, any other trees are (independent of any measure of notability – which would be hard to define in a verifiable form anyway) and mapping trees is popular among mappers, in particular in urban environments. Apart from individual trees there is also the concept of tree rows in OpenStreetMap – which essentially refers to trees planted in any way in a linear arrangement. Those are mapped with a way tagged natural=tree_row. There is no established tagging to indicate the spacing of trees in a tree row though. It is also generally accepted that trees are either to be mapped individually or in the form of a tree row object – but not both.

Bushes or shrubs are, on the other hand, so far only rather rarely mapped individually in OpenStreetMap. There is a clearly established tag for that (natural=shrub) – but with quite minimal use.

What is mapped again more frequently in OpenStreetMap is hedges – both in urban and in rural environments. Those are tagged with barrier=hedge on a linear way typically. There is also a practice to use this tag on polygons. However, due to the generally widespread habit of using barrier tags (barrier=wall, barrier=fence – and also barrier=hedge) as an add-on tag on polygons to indicate this polygon is enclosed by such a barrier, such use of barrier=hedge is inherently ambiguous and by many mappers nowadays not considered advisable tagging.

There are a number of secondary tags commonly added to more precisely characterize the tree/shrub/tree row or hedge:

  • denotation – this is an attempt to classify the role a tree has or for what purpose it was planted. In many cases, especially in case of the most common value urban, this is a non-verifiable classification. Still some data users use it as an importance rating. It is tagged on about 9 percent of trees.
  • leaf_type/leaf_cycle – these are the classic classification tags for trees as they are also applied for forests/woods (where i discussed the rendering previously). For individual trees/shrubs there obviously is no mixed type but for tree rows and hedges there can be. The percentages for use on natural=tree are:
    tag percentage of trees tagged
    leaf_type=broadleaved 15.5
    leaf_type=needleleaved 1.28
    leaf_cycle=deciduous 8
    leaf_cycle=evergreen 2.15
  • diameter_crown – this tag is used to map the approximate diameter of the crown of the tree, or if you’d want to do it precisely: The diameter of the area equivalent circle to the footprint of the tree crown. This is only tagged on trees. Shrubs, tree rows and hedges use the tag width instead.
  • width – this tag is used to indicate the cross section/width of the foliage of the plants in question on shrubs, tree rows and hedges.
  • diameter – to indicate the diameter of the trunk of the tree.
  • height – to indicate the overall height of the plant/plants including foliage.
  • species/genus/family/taxon/... – a more precise taxonomical classification beyond leaf_type/leaf_cycle is tagged on quite a few trees. There are different tags used for that, their percentages of use on natural=tree are:
    tag percentage of trees tagged percentage with language tag only
    species 6.8
    genus 3.9
    species:de 1 0.0038
    taxon 1
    species:en 1 0.13
    genus:en 0.9 0.024
    species:it 0.54 0.0011
    genus:it 0.46 0.00036
    genus:de 0.36 0.007
    species:ro 0.35 0
    taxon:species 0.26
    species:es 0.19 0.003
    taxon:family 0.17
    species:fr 0.16 0.012
    taxon:pt 0.15 0
    taxon:cultivar 0.15
    genus:fr 0.14 0.001

    As can be seen, species is the most popular tag, genus and taxon are also popular but often used in addition to species. There are however also a significant number of cases (1.1 percent) where genus is tagged without species (which is understandable when mappers cannot precisely determine the species although they know the genus – in particular in cases where this is difficult like for example Pinus and Salix). The non-Latin tags with the language suffix are commonly used in addition to the main Latin tags – the percentage of trees with only the non-Latin tagging is indicated in the table – by far the highest percentage exists for species:en.

  • name – about 0.3 percent of all trees have a name tagged.

The overall usage of the four primary tags discussed as well as the percentage with various secondary tags can be found in the following table. The taxonomy column indicates how many of the features have any of the common Latin taxonomic tags applied.

tag uses leaf_type leaf_cycle taxonomy dc/width height
natural=tree 18.8M 17% 10% 8.4% 1.3% 4.1%
natural=tree_row 1.14M 17% 10% 0.66% 0.55% <0.1%
natural=shrub 1020k 2.7% 2.5% 9.5% 4.4% 1.9%
barrier=hedge 2.4M 1.3% 0.57% 0.16% 1.8% 0.23%

Tree rendering in OSM-Carto

To say it right away – most maps based on OpenStreetMap data do not render trees. Apart from OSM-Carto and derivatives (almost all of them render trees in some form) there are just a few publicly available online maps that show trees – like Thunderforest Landscape and Geofabrik Topo. OSMAnd also includes trees in its mobile offline maps (with a profile symbol).

Trees in (from left to right): Thunderforest Landscape, Geofabrik Topo and OsmAnd

Trees in (from left to right): Thunderforest Landscape, Geofabrik Topo and OsmAnd

Notable tree rendering exists in Map Machine – which uses profile symbols and differentiates leaf_type and leaf_cycle as well as a few genera.

Map Machine rendering of trees

Map Machine rendering of trees

OSM-Carto has rendered trees since the beginning. Originally that was in the form of small green dots – which were based on a PNG image with a white background – therefore that looked a bit awkward on darker colors. In 2014 we adopted a new design for trees from the French OSM-Carto fork that shows a simple dot at z16 first and at higher zoom levels turns into a top view symbol of a tree.

This design uses transparency on the symbol to allow features underneath to be visible to some extent. The tree symbols are also unique as a point symbol in the style because they are non-blocking, i.e. they can overlap with each other and with other features or text. The transparency is implemented on the layer level instead of the feature level so overlapping trees are shown in a uniform, continuous coloring without the opacity of the individual trees accumulating. Tree rows are rendered in the same manner with a semitransparent line of constant width. There was discussion on changing that but nothing came out of that so far.

Current tree rendering on OSM-Carto with the handling of overlaps between features

Current tree rendering on OSM-Carto with the handling of overlaps between features

In 2017, tree symbol rendering on z16 and z17 was removed but without adjusting either the zoom level progression or the label rendering of the tree name (which was introduced in 2014 as well). This led to the tree rendering becoming more like a static POI symbol which starts appearing suddenly at a certain zoom level (z18) and then remains more or less as this (just getting a bit larger at z19).

z16 z17 z18 z19
tree v1 natural=tree at z16 with OSM-Carto v1 natural=tree at z17 with OSM-Carto v1 natural=tree at z18 with OSM-Carto v1 natural=tree at z19 with OSM-Carto v1
tree v4.2.0 natural=tree at z16 with OSM-Carto v4.2.0 natural=tree at z17 with OSM-Carto v4.2.0 natural=tree at z18 with OSM-Carto v4.2.0 natural=tree at z19 with OSM-Carto v4.2.0
tree current natural=tree at z16 with OSM-Carto natural=tree at z17 with OSM-Carto natural=tree at z18 with OSM-Carto natural=tree at z19 with OSM-Carto
tree_row current natural=tree_row at z16 with OSM-Carto natural=tree_row at z17 with OSM-Carto natural=tree_row at z18 with OSM-Carto natural=tree_row at z19 with OSM-Carto
hedge current barrier=hedge at z16 with OSM-Carto barrier=hedge at z17 with OSM-Carto barrier=hedge at z18 with OSM-Carto barrier=hedge at z19 with OSM-Carto

Apart from that the main problem with this design is the transparency. To make trees well visible on all kinds of backgrounds (both the bright map background color and the white fill color of roads as well as the dark green of forests) the color has to be fairly dark, moderated through the use of low opacity. But that also means that the color in rendering will be very different depending on the background – leading to a very non-uniform appearance of trees in different contexts. And it leads to weird and confusing color mixing with the colors below. The main idea – keeping the features below the tree symbol visible while still showing the tree in a symbol of reasonable size relative to the typical size of a tree canopy – is only poorly facilitated by this.

Problems resulting from the use of transparency in the OSM-Carto tree rendering

Problems resulting from the use of transparency in the OSM-Carto tree rendering

Also the color of trees and tree rows above the plain map background is close to the rendering color of wood/forest polygons. This makes them prone to be mistaken as a mapped polygon geometry – especially since small grained mapping of individual tree canopies or narrow strips of trees with wood polygons is not uncommon in OpenStreetMap.

Tree and tree rows in comparison to wood/forest polygon fill

Tree and tree rows in comparison to wood/forest polygon fill

The third part of this series of blog post will show and explain a new concept for rendering trees, tree rows, shrubs and hedges that i have developed recently.

Tree depiction in traditional maps and plans

May 30, 2022
by chris
1 Comment

Tree depiction in traditional maps and plans

I have discussed depiction of forests and woods in maps on this blog in the past. Here i want to cover a related topic: The depiction of individual trees in maps – as well as bushes/shrubs, tree rows and hedges. I will do so in four parts – with this first part having a look at the depiction of these in traditional maps and plans.

Individual trees are much less commonly shown in maps than forests and woods. Most maps which show individual trees do so only for trees of some kind of significance as a landmark or otherwise, or for isolated trees in an otherwise treeless environment. The dominant depiction of trees of some significance is with a profile view symbol. Here there are a few examples of such designs from maps in the scale range of 1:5k to 1:50k.

Left: German topographic maps of different age and scale, right: From top to bottom: Soviet map individual tree symbols, Finnish, Italian, Slovakian and Spanish

Left: German topographic maps of different age and scale, right: From top to bottom: Soviet map individual tree symbols, Finnish, Italian, Slovakian and Spanish

The profile depiction has the huge advantage that it easily allows to transport additional information on the type of tree in a fairly intuitive fashion – which in the above examples is done in some cases, primarily for the distinction between broad-leaved and needle-leaved trees. A much more detailed differentiation can be found in Soviet large scale (1:1000/1:500) mapping, as illustrated below.

Individual tree symbols in Soviet large scale maps - from left to right: (1) broad-leaved (oak, beech, maple, hornbeam, linden, ash, elm, etc.), (2) broad-leaved (birch, willow, aspen, alder, poplar, etc.), (3) fruit trees, (4) palms, (5) spruces/firs, (6) pines/cedars, (7) larches, (8) cypresses

Individual tree symbols in Soviet large scale maps – from left to right: (1) broad-leaved (oak, beech, maple, hornbeam, linden, ash, elm, etc.), (2) broad-leaved (birch, willow, aspen, alder, poplar, etc.), (3) fruit trees, (4) palms, (5) spruces/firs, (6) pines/cedars, (7) larches, (8) cypresses

In maps of scales between 1:5k to 1:50k the depiction of individual trees or bushes of no particular significance is more rare and, if it is done, that happens usually in the form of a simple dot or circle. Rendering of individual bushes and shrubs is even less common – but there is a specific design of depiction that has been coined in Soviet topographic mapping and is used there both as a point symbol and as an area pattern symbol, and that continues to be used in eastern European mapping traditions. Otherwise, bushes are typically rendered either similar to trees – with simple dots/circles – or with a smaller and simplified version of abstract tree symbols showing a profile depiction.

The iconic bush/shrub symbol from Soviet topographic maps

The iconic bush/shrub symbol from Soviet topographic maps

More common than depiction of individual trees is the aggregate rendering of rows of trees and hedges – either in isolation or at the side of a road. In the German topographic mapping tradition hedges play a strong role and are typically differentiated into normal hedges and elevated hedges on earth dams called ‘Knick’ – which are a common landscape feature in northern Germany. Different versions of depicting those in German topographic maps are shown here. Note in particular the hedge depiction from the traditional DGK5, where circle symbols are combined with abstract profile tree symbols allowing to illustrate the tree type.

Hedge (left) and elevated hedge (Knick, right) depiction in German topographic maps of various ages and scales.  Symbols shown in the middle are used for both.

Hedge (left) and elevated hedge (Knick, right) depiction in German topographic maps of various ages and scales. Symbols shown in the middle are used for both.

Examples for signatures of hedges and tree rows in maps from other countries can be found in the following. Note the Slovakian design, featuring a tree type differentiated from-the-top view rendering with alternating symbols for mixed tree type rows.

Hedges and tree rows in various maps Left: Hedges from top to bottom: Finnish, Danish. Swiss. Estonian, Latvian 2k; right: From top to bottom: Finnish, Slovakian, Latvian 2k, German TK25 BY/NRW

Hedges and tree rows in various maps Left: Hedges from top to bottom: Finnish, Danish. Swiss. Estonian, Latvian 2k; right: From top to bottom: Finnish, Slovakian (three variants), Latvian 2k, German TK25 (BY and NRW)

Use of symbols resembling a view of a tree from the top is much more widespread as a means to display individual trees in larger scale architectural drawings and site plans. In contrast to maps, systematic publication of such plans is quite rare world wide. The most extensive collection in that direction i could find was in the US Library of Congress as part of the HABS/HAER/HALS. These are plans largely drawn for documenting the state of these sites rather than plans drawn by architects according to what is planned to be built (which is a purpose very different from that of maps, obviously). Here are a few examples from those:

Most of these symbols (except for the first row) are not rendered at a constant symbol size but to scale – with the size of the symbol representing the crown diameter of the tree. Generally speaking, the design of such plans follows conventions very different from those in cartography. I won’t go into further detail on that here – that would be a subject for a separate blog post. Below you can see just a few practical examples of tree depiction in plans at different scales and from different times.

The second part of this series of blog posts will deal with how trees are mapped in OpenStreetMap and how these are shown in OSM based maps.

OpenStreetMap-Carto – an update on the project

May 20, 2022
by chris
13 Comments

OpenStreetMap-Carto – an update on the project

It has been more than 4.5 years since i wrote more in depth about OpenStreetMap-Carto, the map style behind the map on openstreetmap.org and in many ways the face of OpenStreetMap.

I have been contemplating writing an update on this for more than a year now, but i have been hesitant because the conclusions i am going to draw in the end are pretty dire and i had still hoped for a turn to the better. And writing a blog post here with fairly negative conclusions seemed to me to be something that could further contribute to seal the fate of the project.

The history of the project

I will start by roughly sketching the history of OpenStreetMap-Carto. For further illustration of that i also produced a comparison gallery of how different versions of OSM-Carto render the current OSM data in various places around the world. The project began in 2012 with Andy Allan porting the old Mapnik XML based style, that had powered the main map of OpenStreetMap so far, to CartoCSS. CartoCSS was one of the most significant innovations in rule-based digital map design as it was the first (and – depending on your perspective – might still be the only) format for describing rule-based map design that was developed with primarily the map designer in mind and not the comfort and liking of the tool developers (as it was the case for Mapnik XML and is for example for JSON based formats that became popular later).

Rendering sample from version 1 of OSM-Carto (December 2012)

Rendering sample from version 1 of OSM-Carto (December 2012)

Andy’s contribution was not just the hand-work of formally porting the style, he also did the important ground work to determine how the CartoCSS based framework would need to be used to scale to the level of complexity the style already had back then.

The port to CartoCSS and the availability of TileMill as a fairly easy to use tool to develop CartoCSS styles encouraged quite a few people to newly get into map style development. Most of the initial work was technical restructuring. Technically the style gained its current shape, in particular with the structure of the roads layers, during these first years. Feature additions were also made to some extent, but the overall appearance and design initially stayed very close to the Mapnik XML style.

Originally Andy Allan managed everything on the project himself, but after some time Matthijs Melissen, Paul Norman and Mateusz Konieczny were added as maintainers who could also merge changes and the system of consensus decisions of the maintainers was established. Later, in 2016, Daniel Koć, myself and Michael Glanznig were appointed as additional maintainers, in 2017 also Lukas Sommer and in 2019 Joseph Eisenberg. Of these only Michael Glanznig has formally retired since, but both Andy Allan and Matthijs Melissen have completely withdrawn from active participation in the project now – leaving five more or less active maintainers.

2015 and 2016 were kind of the golden years of the project. With the interest raised and the better accessibility of style development through CartoCSS, quite a few people by that time had gained enough experience to tackle bigger design challenges the project’s development and the needs of the OSM community created. This in particular refers to the style OpenStreetMap-Carto was based on being strongly connected design wise to the British origin of the project. During the time from 2015 to 2016 the style essentially emancipated itself from this relatively narrow cultural origin and became a truly international project. Components of this emancipation were in particular:

  • Refactoring of urban landuse and building coloring as the first step into systematic color design in OSM-Carto.
  • The roads rendering redesign developed by Mateusz as part of a GSOC project – departing from the traditional British color scheme and adjusting road rendering for good readability in a wide variety of settings world wide.
  • Moving to a standardized design of POI symbols for most features rendered with point symbols – work that has in particular been pursued by Michael.
  • Introducing algorithm generated relaxed random periodic patterns for depicting a larger variety of landcover types beyond what is possible with just fill colors – paired with systematic and holistic design of the area fill color scheme in the style. This in particular facilitated the introduction of a significant number of new landcover types that are rare in areas where historically OpenStreetMap first became popular but widespread in other parts of the world – bare ground landcovers, various wetland types and reefs.

These accomplishments in making the style more international and serving the potential global community in a more balanced fashion were made possible through various technical and design innovations. For this short period OpenStreetMap-Carto was truly avant-garde in the development of rule-based map design.

Rendering sample from version 3 of OSM-Carto (December 2016)

Rendering sample from version 3 of OSM-Carto (December 2016)

What i especially think we have reason to be proud of is that we agreed back then on a document outlining uniquely progressive fundamental goals of the project. In particular the idea that we aim to create a map style for the potential global map user and no special consideration is given to the current geographic distribution of actual map use is a fairly revolutionary concept that has, to my knowledge, never be formulated in such clarity anywhere else. And it was not only a goal on paper, we seriously tried to work towards this goal as i hope the list of changes above illustrates somewhat.

But this quite amazing development came at a price:

  • we were not perfect and while we introduced many significant improvements, there were also things that got lost on the way. Noteworthy is in particular that the depiction of paths for non-automobile transport was significantly better overall in the old Mapnik XML style. The look back at what was lost after changes have been made is something that was and still is done much too rarely.
  • we outgrew the possibilities of the technical framework we were using. While initially what could be and what was done in OSM-Carto was mostly limited by our collective limits in knowledge and experience of how the possibilities our toolchain offered could be efficiently used to create a better map, developers soon mastered these possibilities and reached the limit of what was easily possible. At the same time active development of Mapnik and Carto slowed and ultimately mostly stopped. The options offered by CartoCSS in combination with PostGIS are still manifold but with no higher level abstractions in the tools using those amounts to adding significant levels of complexity on the style level or in form of additional tools and working with script generated CartoCSS/SQL code to integrate those into the rendering framework.
  • the move to become a more global and internationally focused map style was not really that popular. Quite a few people in the OSM community who – coming from regions traditionally influential in OSM, in particular urban mappers from the UK, Europe and North America – were naturally expecting OSM-Carto to primarily cater their interests. Some of them were disappointed by OSM-Carto having a different focus and not quickly supporting the latest trends in urban mapping when they started becoming popular in these regions. Others disliked the fact that OSM-Carto was so distinctly and ostentatiously different from mainstream commercial maps like those from Google and Mapbox and would have very much preferred us to align map design more to what they were used to from these maps.
  • most importantly: This development meant that the skills required to productively work on improving the style were gradually increasing. That in particular meant that for new contributors it became more and more difficult to make substantial changes to the style without first becoming familiar with the design principles of the project.

These developments led to increasing divergence in the maintainer team about the future direction of the project. I discussed this part of the history of OSM-Carto in more depth in my previous blog posts on the project here. In April 2017 the decision was made to give the individual maintainers more autonomy and we maintained this modus of operations until late 2018. Joseph summarized this nicely here.

The hope was that by giving maintainers autonomy in making decisions we could avoid the impasse we had reached on many strategic questions and get to a sustainable dynamic development of the style again. Unfortunately that did not happen. What happened instead was that development got dominated by a relatively small group of people who developed primarily feature additions and design adjustments of interest from an urban European perspective that were often at odds with the overall design paradigm and the goals of the project as they had been pursued before. At the same time, none of the big challenges the style faced, many of which were visible already in 2016, were actually addressed. Many of the developers and maintainers active before who would have been qualified to work on bigger changes, withdrew from active development of OSM-Carto. We have discussed these dynamics on the issue tracker and those were among the main reasons why we moved back to the consensus principle after about 1.5 years.

During this time the style moved significantly away from a consensus position that all active maintainers could support. This created the massive challenge to find a way back to a consensus among the project maintainers about the strategic direction of the style once the consensus principle was re-established. And despite significant efforts, in particular by Joseph, we did not manage to do so. To this date there is no strategic consensus among the maintainers and design wise development is mostly stalled. We have managed to make a number of technical changes – including big ones again but design wise no agreement can be found on most larger changes and even reverting changes that were made without consensus does not find agreement.

This continues to be the situation to this date. Some maintainers (including myself) continue to review changes that are being suggested but releases have become very sporadic and usually contain little of substance in terms of visible changes. I try to – even when i disapprove of a change – indicate that i would be fine with the other maintainers accepting the change none the less if they agree on it, this way actively supporting a more robust form of consensus. But that is no substitute for an actual common goal and a strategic direction we agree on. And most capable style developers have stopped contributing to the project because the lack of a clear overall direction does not provide for a supportive environment to work on good map design.

Rendering sample from the current master branch of OSM-Carto

Rendering sample from the current master branch of OSM-Carto

What went wrong?

It would, of course, be easy to now point back to the decision to abolish the consensus principle in 2017 and say: This is the wrong decision from which on everything went wrong. And i could even say i warned about the potential side effect of the project loosing the common sense for the strategic direction back then. But i don’t think this would be a complete assessment of the situation.

The three main strategic directions map styles can lean towards and at the same time the three necessary components of any sustainable map design endeavor

The three main strategic directions map styles can lean towards and at the same time the three necessary components of any sustainable map design endeavor

From early in its history OSM-Carto was burdened with the fact that it was the only community map design project in the OSM community with a global scope and that did not have a specialized thematic focus. As a result, a lot of people projected their ideas, interests and needs onto the project. If you try to find some structure in these diverse influences you can in my eyes identify three main strategic directions that the style has been pushed in:

  • The populist direction. During the 1.5 years when consensus requirements were lifted and maintainers could make decisions autonomously in OSM-Carto, we pretty much followed this approach in purity. It means focus in development is on small, atomic changes, made and decided on without a larger overall design strategy and essentially allowing anyone who would like to change something to do so, at least as long as it has substantial popular support from people with a loud voice. This has the tendency to lead towards what i, in my previous blog post about OSM-Carto, called a naive art like style. We have already seen that in the long term this is non-sustainable because it fails to consider the need for larger and systematic changes to maintain the technical viability of the project. And it also fails to acknowledge that the usability of a map depends on an overall design concept and strategy that is being consistently followed. But that does not mean it is not an attractive short term model for developing a map style. In case of an OSM map style influential on mappers, there is a strong risk that over time mappers will start and will need to start compensating for the deficits of such a direction by essentially hand painting the map through their mapping work. However, in moderation (meaning in particular to mitigate the imbalance resulting from the loud voices appearing to represent the popular opinion) popular involvement is an essential component both for evaluating design and for recruiting talented and qualified people in development, in particular in case of a cooperative community project like OSM-Carto.
  • The technocratic direction. In purity this means essentially treating style development as software development. The style is treated as a software to process the map data into a graphical representation. You try to do so efficiently – both in terms of run time performance and w.r.t. code complexity. But you pay no significant attention to the non-technical requirements there are for the resulting map. Many digital map design projects work close to this model, they are managed by software engineers with little background and little ambition in actual map design. This model has one major flaw: Map style development is not software development. As a result, such maps at best play a bit clumsily with design possibilities. And the resulting map is usually imbalanced, overly abstract and detached from the geographic reality it tries to represent. On the other hand, since digital maps are powered by technology, the technical expertise of software developers is of course a crucial component for the long term sustainability of any map style.
  • The artistic direction. This direction means focusing on the design component of the map and being guided by the pursuit of excellence in this field. Artistic here is not meant in the sense of pure art but to contrast map design work based on an understanding of map viewing situation and a competent use of colors and visual structures in light of this with the purely technical approach. There are very few map styles in the field of automated rule-based digital cartography so far that put a strong and sustained emphasis on this direction, largely because map designers with artistic ambition still predominantly design maps through manual processing of concrete data. For me this has always been the most interesting direction, exactly because it is a field in its nascence with lots of unexplored territory. In purity the pursuit of this direction would likely lead to highly sophisticated maps which are technically inefficient, difficult to maintain and which constantly struggle with the technical limitations of the map design frameworks they use. But this is also the direction where there is clearly the most unused potential, both in terms of actual map design, as well as in terms of talented and capable designers who could be recruited.

These three paradigms are fundamentally contradicting each other and it seems that map development projects have the tendency to concentrate on one or two of these focus directions and neglect the other(s), because balancing all three directions is difficult and takes a lot of effort. At the same time, as i tried to explain, neglecting one or two of these directions will, in the long term, not be sustainable because all three components are essential.

The core of the problem

Until 2016 we essentially managed to unite these three principal directions in a common style, because we made use of the significant options the technical framework we used and our own expanding capabilities and skills in map design gave us. This reservoir of capabilities made it possible to bridge the gaps between these, otherwise, largely incompatible ideas. But we were already reaching the limits of this development in 2016 as hinted above – both in terms of the technical framework, the capabilities of which we started to outgrow, and in terms of the human design skills as a factor (which made it increasingly difficult to find and recruit talented and capable people as contributors and to educate them in the specific needs of the style).

What we would have needed back in 2016 and also now (though with the baggage of the history of the past years this seems rather unrealistic) is a new effort of consolidation similar to what has happened at the beginning of OSM-Carto. That would in particular mean tapping and developing new technical possibilities, because – as mentioned above – we already outgrew the possibilities of the technical framework we were using back in 2016. This is challenging because most of the components of the framework OSM-Carto uses are not actively maintained any more and there is no open source rendering system on the horizon that could replace it. Therefore i have suggested for quite some time that it is essential for the OSM community to strategically invest into map rendering systems that can produce high rendering quality. Rendering systems suitable in terms of the map styling interfaces, to be used in cooperative community projects, and that scale well towards higher sophistication in map design. This would be paramount to ensure the project’s ability to actively work on and substantially participate in the discourse on graphical representation of OpenStreetMap data in maps.

Where things might go in the future

Back in 2017 when OSM-Carto moved away from consensus based decision making i started the alternative colors style as a fork of OSM-Carto. Initially it was meant purely to demonstrate how the design principles we had developed in OSM-Carto until 2016 could continue to serve as a basis for a balanced map design for a truly global audience. But over the years it became increasingly clear to me that OSM-Carto would likely not find its way back to a balance between the three main strategic directions. And it also became clear that beyond OSM-Carto, community cartography in OpenStreetMap would most likely focus on the technocratic and populist directions. Businesses and non-commercial organizations around OSM produce maps with a predominantly technocratic direction. The OpenStreetmap Foundation has, for some time now, indicated an interest to actively bootstrap new community map design projects – and the OSMF is dominated by people with technical backgrounds. The OSMF board as the main decision making body even consists exclusively of people with a technical IT background. To mitigate this deficit they will likely try to give strong voices from the mapper community a bit of a say in these projects. But no one active in the OSMF has in the past ever indicated to value competence and experience in the artistic aspects of map design.

Long story short – in light of it being highly likely that future OSM community map design will predominantly be negotiated between the populist and technocratic direction, i increasingly started to try to explore the potential of the artistic direction. I don’t have the capacity or the skills to really consolidate the technical basis of map development and to develop map rendering technology that would give map design truly the options to move to the next level as i sketched it to be the desirable future for OSM-Carto above. Hence, much of what i implemented in the alternative colors style is technically somewhat awkward – most ostentatiously visible in the >1500 lines long SQL query for the new roads rendering system. But working on such case studies in advanced map design is an important and necessary step i think to determine what capabilities a future map design framework would need to have and how it would best be structured to truly provide a suitable environment for a balanced map style to flourish in a similar way as CartoCSS and Mapnik did for OSM-Carto in 2015 and 2016.

Rendering sample from the alternative-colors style

Rendering sample from the alternative-colors style

If the OSM community will be in any way actively involved in shaping a future of map design along these lines and in a balance of all the three main directions i described above, is an open question, but as already indicated i consider the chances for this to be quite small. OSM-Carto is still the most likely basis on which this might happen because it already provides a huge amount of preexisting ground work for this. Starting a new project from scratch would have its benefits as well, but the amount of basic, yet careful and considerate design work that would be necessary to bring such an endeavor off the ground would be quite massive.

The other question is if the future of cooperative digital rule-based map design will be based on free open source software or not. Considering how dominant open source is in automatically rendered digital map production these days, this might seem a silly question. But as mentioned above, map design with significant artistic ambition is still predominantly based on manual processing of concrete data and not designing rules for processing generic data these days and, in that field, open source software only plays a marginal role so far. If the FOSS community and the OSM community do not strategically invest in this direction, it is very likely that commercial software companies and service providers will – based on the good standing they already have with map designers with a more artistic and less technical orientation – successfully try to dominate that field.

Reflections on the social limits of community map design

In my previous blog posts on OSM-Carto i discussed quite a bit about the social dynamics of cooperative community map design and pointed out that OSM-Carto is also a social experiment testing if cooperative map design on this scale is possible and sustainable. It becomes increasingly clear now that this experiment has failed. But does this mean that a cooperative map design endeavor that is not either based on a centralized authoritarian leadership or on a small, culturally homogeneous team of people with common interests cannot work? That is a possibility. But not the only one i think. Above i listed what i identified as a number of factors that had led in 2016 to an increasing divergence in the maintainer team about the future direction of the project. None of these factors is however inevitably making such a cooperative effort unfeasible.

The key points that i would identify to be essential for a future experiment in a similar direction, to create a balanced general purpose OpenStreetMap map style for a truly global audience through open community cooperation, to work would be:

  • Bootstraping such a project is a step of paramount importance. In OSM-Carto we had a reasonably diverse group of developers at least roughly covering all the three main directions i consider essential for a well balanced and sustainable map. We managed to codify the key ideas and values of the project into a guiding document but the agreement on these common ideas and values did not last long enough for the project to assemble a developer community invested enough into these values under the more adverse conditions the project was going to meet in the coming years.
  • Such a project needs long term strategic support from an active software development community invested in supporting map design not only as a technocratic endeavor but that also values and supports specifically the artistic direction. If a map design project outgrows the technical framework it is based on, like OSM-Carto increasingly did in 2016, this is a huge problem. Not only because it technically makes it harder to implement design wise necessary or desired changes, but also because it makes contributing increasingly unattractive for the most qualified designers who strive for excellence in design. A map design project where the appearance of the map is not primarily determined by the conscious decision of designers about how they want the map to look, but by the limitations of the software used, is destined for mediocrity and is never going to be a balanced map well serving its design goals.
  • Significant effort needs to be put into writing down practical guidance and providing educative materials and training regarding the practical design rules and principles followed by the project. This is especially important as the field of automated rule based map design is so young and so little material discussing this topic is available at all.

I would consider it likely that if these points can be addressed and followed diligently from the start, a map design project like OSM-Carto can work in the long term. But it is definitely hard and failure is likely. One of the key challenges OSM-Carto struggled with and that any project with similar ambitions will struggle with in the near future, is that it is essentially in uncharted waters not only as a social endeavor, like i discussed previously, but also in terms of map design itself. This is both a huge chance and a burden.

March 30, 2022
by chris
13 Comments

OSM related group communication channels and platforms – revisited

The OpenStreetMap foundation has rolled out their new web based communication platform – which has long been anticipated and which i have mentioned in passing in my yearly OSMF review already.

I want to take this opportunity to update my previous review of OSM related group communication channels and platforms and add a few subjective impressions and comments afterwards.

After having essentially stopped using OSMF communication infrastructure actively in response to the OSMF board establishing a highly questionable behavior control regime and indicating the intention to extent this to all OSMF controlled communication channels, i view this pretty much from an outside perspective now.

But as said – lets start with the basics. Here is the extended table of the different communication platforms and their evaluation according to the different criteria i introduced in the first part.

OSM related group communication channels

I differentiated the open platforms and communication protocols category further into:

  • Free open source software: This documents the basic question if the software used is formally licensed under an open license.
  • Open protocols: The question here is if the platform can be interfaced with from the outside (both technically and legally) to allow exchange of content with other platforms or alternative interfaces.

I think it is important to separate these two – the latter is also possible without the platform itself being open source and a communication platform based on open source software does not automatically provide open protocols that can be used from the outside to access or to enter communication content (which is what seems to be the situation with the new OSMF platform – see below).

Here the updated footnotes for the table:

  1. help.openstreetmap.org has limited support for structured communication.
  2. The OSM wiki in principle allows structured communication and on talk pages it is established practice to thread messages. But you have to do this fully by hand and it only works if everyone does so diligently. No builtin support for that.
  3. On Usenet server operators used to expire messages so there was practically no permanent public record in the system itself. However everyone running a Usenet server could maintain a record and in the late days of Usenet Google practically served as a reference record.
  4. The forum indicates when a post has been modified after initial posting but no record of the original message is retained. Also admins on the forum can (and do) remove messages from the record (with varying frequency, depending on the channel).
  5. The diaries themselves can be edited without the history being visible but the diary comments cannot.
  6. On the wiki the edit history is openly accessible so it serves as an immutable record (though it can be difficult to use as such practically).
  7. Github has a fairly clever system of allowing edits of communication but still allowing users to transparently see if there have been changes and what has been changed. Github however also allows completely removing messages from the public record in some cases (but it is indicated at least that a message has been removed i think).
  8. While there is no alternative interface to posting changeset comments for reading there is the popular tool by Pascal Neis.
  9. Openness in participation in principle applies to most OSM related mailing lists and forums. There are a few exceptions – specifically osmf-talk is only open for posting to OSMF members. There are thematic and regional lists and forums with culture specific rulesets you need to accept for active participation which are usually developed through consensus of those active on the individual channel (like diversity-talk, regional forums). Beyond that the OSMF has for all communication channels and platforms they operate indicated the intention to impose a behavior control system that is designed to allow banning participants if they misbehave in the eyes of the moderators. To what extent that practically limits the de facto openness is not clear so far.
  10. See 9 – plus Discourse seems to implement an automatically managed hierarchy of users based on their formal activity history on the platform. In addition there seems to be a not publicly documented number of people selected from the top so to speak who have special privileges in editing/removing communication content of others. So the system is essentially open only on the base level and has an elaborate hierarchy built on top of that. If people are also manually promoted/demoted within the – on the lower ranks – by default automatically managed hierarchy is not clear to me yet.
  11. Discourse formally seems to record communication structure in the sense that it records which message a message is in reply to if it is explicitly made in reply to one. This is however not consistently displayed. I have seen messages being shown both as a reply to another message as well as being shown a second time as a flat comment to a topic. In general the whole interface seems to strongly discourage replying to the messages of someone else in the sense of structured communication and to nudge people to preferably add comments in a linear fashion to a topic.
  12. Discourse seems mostly like the forum in this regard (see 4. above) but in addition seems to allow you to see the edit history of messages. There seem to be also automated edits of messages made by the software – which are displayed just like human edits. It is unclear to me so far if message removals by moderators are transparently visible or not (i.e. if there is an indication that here a message by X has been removed). Update: It seems that moderator action is not generally publicly visible as such.
  13. From what i have seen Discourse has an API but that is not openly accessible – not even for read-only purposes – but requires an API key from the platform operators. The OSMF has so far not indicated if they are willing to provide access to that to anyone. I have not seen any external clients that interface with discourse in some form based on that API or otherwise. In particular there does not seem to be any workable mechanism for bulk access to the communication content. There are secondary channels (RSS feeds and the so called mailing list mode) that allow limited access to some of the communication content but they seem to be there mostly pro forma (to be able to advertise those as features) and are practically essentially unusable (Update: more details in comment below).
  14. There seem to be some very limited filtering options (like blocking individual users) for logged in users w.r.t. notifications. The message reading interface however does not seem to have any way for users to filter things (although the platform itself performs filtering on its own – deciding based on unclear criteria which messages are shown and which are hidden by default).

Another remarkable data point by the way: The starting page of commmunity.openstreetmap.org loads an impressive 5.4 MB of content to then display what amounts to less than a hundred words. For comparison: The OpenStreetMap forum starting page (which is much more information rich of course) is less than 100 kB.

Personal impressions

When i am in need to choose a software or platform for a certain purpose my primary consideration is usually: Does it allow me to do the things i want to in the way i want to do them without the need for me to invest time and resources in things that are not of interest for me. To give an example from the field of group communication: In case of image content as part of communication and graphical formatting of text: I would want the communication tool to show me the fact that my communication partner has communicated such content but i don’t want it to show me such stuff without me explicitly choosing to see it. In other words: I want to be able to make granulated decisions on how much attention i invest in a specific communication.

The central paradigm of Discourse seems to be pretty much the opposite of that: Active management of the user’s attention. Like:

  • Suggested Topics – who is suggesting that to me and why? I have not asked for suggestions of any kind.
  • Views, likes etc. – apparently someone thinks it is of tremendous importance for me to know how many other people have read or indicate to have liked what i am reading so they rub that into my face prominently and this way distract from the actual communication content i am reading. Too bad that i don’t care about this kind of data.
  • It seems it is also of high importance that i always associate the identity of my communication partner with some picture of their choosing. Tough luck if memorizing mini-pictures is not really your prime talent – because you then have to deal with the between one and three verbal identifiers which are displayed in various tones of gray and weight in a way that is barely discernible from the communication content.
  • Hello! Looks like you’re enjoying the discussion, but you haven’t signed up for an account yet. Seriously? I am just waiting for clippy coming around the corner now…

I could go on with a long list of similar things i find appalling when trying out the new platform but i will cut it short. Discourse is evidently not for me. My impression is that the target user group of that platform is people that have grown up in the attention economy, using twitter, facebook and other social media and feel comfortable with those.

Conclusions

And i think that is fine. As indicated in the past (and others see this similarly) i am convinced that a diverse project like OpenStreetMap needs diverse communication platforms. People who are used to twitter and facebook and like the communication style there should have the opportunity to communicate in a similar style about OpenStreetMap and preferably using open source tools and without the need for people to participate in that communication to sign up with some big corporation and agreeing on their data being sold for profits.

What i see critically – and i have said that in the past as well – is, that there are strong voices in the OSMF that do not just want to offer this new communication platform as an option for those who like this style of platform but who want to unify and to culturally homogenize all OpenStreetMap related communication under this. Kind of the wild dream of all community management. It is very likely that this would fail of course but there is a significant risk that the failure would not be visible from the inside perspective and that at some point in the future people on community.openstreetmap.org might predominantly believe that they are the community (or at least that they are representative for the whole OSM community in all their diversity).

Discourse belongs into the category that i have described in the previous post on group communication channels as stuff developed around some fancy web interface meant to serve the fashion of the day in UI design. It is very likely that this will – like countless other similar platforms in the past – run out of fashion relatively soon again (in like 5-10 years) and will, as a software, likely become unmaintained soon after that. In other words: It could well be that the OSMF will in 10-15 years, possibly sooner, be in more or less exactly the same situation it is right now with the forum and help.openstreetmap.org running old and unmaintained software – just with the software then being discourse. The irony could well be that mailing lists still exist then…

We have a saying in German: Die Zeiten sind hart, aber modern.

March 25, 2022
by chris
0 comments

The Comprehensive Optical Mosaic of the Antarctic (COMA)

I am happy to announce my newest satellite image product – The Comprehensive Optical Mosaic of the Antarctic (COMA).

The Comprehensive Optical Mosaic of the Antarctic (COMA)

The Comprehensive Optical Mosaic of the Antarctic (COMA) – large version shows blended ocean depiction from Green Marble 2.1 (Sentinel-3 data)

The COMA is the result of an evolution of the satellite image mosaic production techniques i have been improving for over a decade now with various images in the resolution range from 10m to 15m having been published over the years – many of which are still the highest quality images available for the areas in question. For this mosaic i worked in particular on the scalability of the whole process towards a larger number of source images and a larger size of the assembled mosaic.

I chose the attribute comprehensive to indicate that this image does not only cover all land and permanent ice of the Antarctic but that it also uses a large fraction of the more recent open data imagery available to ensure the highest quality coverage in the Antarctic interior. Most images of the Antarctic you can see on map services and otherwise these days are still based on the >20 year old LIMA – which, due to the limitations of Landsat 7 imagery (from which LIMA is created), due to the low volume of source data and because the assembly process used is not really suitable for very high quality visualizations. Some image users have patched this up locally with newer images – which then however typically results in color inconsistencies.

Antarctic Peninsula

Antarctic Peninsula

Koettlitz Glacier and Royal Society Range

Koettlitz Glacier and Royal Society Range

The COMA is the first fully new visual color image of the whole continent created after LIMA and a massive step up in terms of quality in every aspect. It is predominantly based on Landsat 8 data, supplemented by ASTER and MODIS images. And it is essentially only now, about 8 years after the launch of Landsat 8, that a sufficient volume of image data is available to produce a mosaic of the whole continent at this quality level.

I’d like to emphasize in particular that the image mosaic is on the same high quality level regarding the lack of clouds as my other high resolution mosaics with – conservatively estimated – less than between one in 10k and one in 100k pixels being significantly cloud affected. This is way better than LIMA or any other Antarctic mosaic available today. The Antarctic is not a particularly cloudy region but it is fairly nasty in that regard because of the nature of the clouds common in the Antarctic and because clouds over snow are notoriously hard to detect reliably in general.

Snow drift and thin clouds on the polar plateau in a strongly contrast emphasized depiction

Snow drift and thin clouds on the polar plateau in a strongly contrast emphasized depiction – from raw image, not in mosaic

It is also worth mentioning that clouds are not the only transient atmospheric phenomenon that affects clarity of satellite images in the Antarctic. Wind blown snow is also a frequent problem. You could even say that on the Antarctic plateau the boundary between ground and atmosphere tends to be somewhat blurry and this makes recording a consistent image of the surface a challenge. In the production of COMA significant care was taken to analyze images and make sure to preferably use those which offer a clear view of the surface free of both clouds and snow drift.

Ice structure visible after strong contrast expansion

Ice structure visible after strong contrast expansion

Some might wonder why i have not used any Sentinel-2 data in this mosaic. The reason is that Sentinel-2 coverage of the Antarctic is very incomplete in terms of spatial coverage and also very infrequent almost everywhere. Using the limited data that is available would have required me in many cases to choose between Sentinel-2 data (for the slightly higher resolution) and Landsat data (for better quality or more recent coverage).

You can have a look at the detailed product description and the sample images over at services.imagico.de. Those interested in using the mosaic are welcome to contact me for details.

The Sør Rondane Mountains

The Sør Rondane Mountains

Denman Glacier, high contrast tone mapping

Denman Glacier, high contrast tone mapping

James Ross Island and Prince Gustav Channel

James Ross Island and Prince Gustav Channel, local contrast enhanced tone mapping

TagDoc logo

March 7, 2022
by chris
11 Comments

Introducing the TagDoc project

This post is also published on TagDoc – you can read it there as well if you prefer.

To many, even to some experienced OSM community members, the free tagging system of OpenStreetMap appears chaotic, unorganized and inefficient. And it is to some extent. However, this system is at the same time also highly functional and fulfills a central role within the project and the OSM community. The alternative to the free, de-centrally developed tagging system we have, would be a centralized, authoritative system of attributes and rules, developed and imposed by some centralized bureaucracy, which would have to be a culturally fairly homogeneous group of people to function as such. In other words: The alternative to the open tagging system, would be for OpenStreetMap to give up the idea of creating and maintaining a map by the people, for the people, based on local knowledge and egalitarian self determined cooperation, and to adopt exactly the kind of methodology centralized mapping authorities use – whose dominance and flaws OpenStreetMap was created to overcome.

One big reason why despite this an increasing number of both data users and mappers are becoming more and more skeptical of free and open tagging, and frequently push with – for example – organized/automated edits or by introducing seemingly authoritative formulations into tagging proposal processes for a more centralized and more authoritative approach to tagging decisions, seems to be the highly dissatisfying status of documentation of the free tagging system we have in OpenStreetMap.

The traditional platform for documenting tagging practice in OpenStreetMap is the OpenStreetMap Wiki. The basic idea is that mappers are free to use any tags they want to use, and they are encouraged to document how they use the tags on the OSM wiki. This has, over the years, led to a highly valuable body of documentation of tags and their use. But – with the tagging documentation on the wiki growing both in size and in importance for the OSM community – it has also become an attractive platform for people who are not just interested in documenting how they use the tags on an equal level with everyone else documenting their tag use, but for the pursuit of various subjective ideas on how the meaning of tags should be presented, irrespective of how the tags are de facto used in the database. As a result, nowadays there is no consensus within the OSM community if the tagging documentation on the wiki is to be descriptive (documenting how tags are practically used) or prescriptive (documenting how tags should be used according to some subjective opinion or some perceived authority). There is not even agreement that these two aspects should be separated. And while – as said – there is a lot of valuable information on the wiki about tags, there is at the same time also a lot of nonsense put there because it suits someone’s agenda. Additionally, the two are often indiscernible and you can only differentiate reliably between them if you have significant up-front knowledge about the tag in question – and in that case you don’t really need the wiki that much.

For data users and software developers, but also for mappers, this is a huge problem because they – due to the lack of alternatives – rely on the wiki as a source of information, a source which, however, is notoriously unreliable. And this is a self emphasizing problem because the more people rely on the wiki, the more attractive it becomes as a vehicle for people with an interest to influence data users, software developers or mappers to introduce ideas into the wiki documentation that do not factually document the de facto meaning of tags, but communicate some subjective ideas on how things should be mapped or how data should be interpreted.

Of course this, in principle, is not a new problem and for a long time people active in OpenStreetMap have discussed various ideas on how to address it. But so far nothing of substance has come out of that. I have also for a long time shied away from working on this – partly due to the sheer size of the task, partly however also because i see a lot of value in the idea of free tagging without centralized rules being combined with documentation of said tagging being developed through equally open cooperation. But more recently i came to realize that the direction the tagging documentation on the OSM wiki is heading, and the dysfunctional social dynamics you can widely observe there, are likely more damaging than supportive for the core idea of OpenStreetMap, especially in light of the fact that the tagging documentation on the OSM wiki is, both de facto and in the perception of people, without alternative. And it has become clear to me that what works for mapping itself – the egalitarian cooperation across language and culture barriers by communicating through the act of mapping itself – does not work likewise for developing a language based documentation of mapping practice.

The idea of TagDoc

What is the solution to this problem? I don’t really know. There might not even be a real solution and OpenStreetMap will need to deal with that. But given how little of substance has been tried to address the problem of reliable and competent documentation of the use of tags in OpenStreetMap, there are ways to substantially improve on that compared to the status quo. The difficulty is how to approach this problem in a way that has a good chance for success. What i have come up with as a concept, after quite a lot of contemplation, is the following:

  • Limiting the scope to documenting the de facto meaning of tags, that is describing how they are actually used in the OSM database. If there is a need for a prescriptive tagging documentation it is probably something that could be discussed but it is beyond doubt, i think, that any serious attempt at developing prescriptive tagging instructions (be that as text or in the form of tagging presets in editors for example) would have as a prerequisite solid knowledge of the de facto meaning of tags as they are used so far. Much of the problem of the current situation stems from the fact that this does not exist.
  • Writing with the aim to be useful and of value for the data user. The reason for that is twofold – First: Because i think data users are in most serious need of a trustworthy and competent documentation of tags in OpenStreetMap and where the dysfunctional nature of the tagging documentation on the OSM wiki creates the biggest issues. Second: While i hope that TagDoc is also of interest for mappers, my idea is that the agreement among mappers on the use of tags should continue to primarily develop through the cooperative mapping process itself (see also What Tags are). The influence of any single other factor on mappers can, if overly large, lead to imbalance and become a problem over time.
  • Written and curated by proven and independent experts, open to scrutiny by the whole community. I know that the idea of meritocracy has fallen out of fashion in significant parts of the OSM community because it is considered to be unjust. And, as indicated, i have myself for a long time valued the open collaboration as a principle for documenting tagging. Over the last years this has however proven not to be able to produce the qualified and reliable documentation needed, at least not in the current and foreseeable future social environment of OpenStreetMap. And the alternative – a committee of political appointees – does not, to put it mildly, have a very good track record in curating intellectual writing.

The plan

As written on the starting page at the moment with me alone writing for TagDoc in my spare time is not in any way sustainable. There are mainly two options how this could be solved:

  • If people (in particular data users) are willing to support this idea and see value in the project as i sketched it above and are therefore willing to finance me working on it, i could extend the amount of time i can invest in this work. Of course this opens up the question of in how far my financiers might influence the content of TagDoc and this way we might solve the problem of the unreliable information on tags from the OSM wiki being the only source of information available, but only with the other source of information being curated by financial interests to their liking. What i can say to that is that (a) i have shown in the past i think on plenty of occasions that with my public writing i tend to not pay much regard to what views are economically opportune, (b) that i would be transparent about where i receive funding for writing and editing for TagDoc from and (c) that any financier of my work on TagDoc would be aware up-front about the basic premises of the project as outlined above which are fundamentally at conflict with the idea of pushing certain subjective views on tagging. What could happen of course is that financiers influence what tags i write about and analyze with priority and in particular detail. That however is already the case right now – what tags i know most about and write about with priority is evidently not independent of what kind of OSM data i work on as part of my paid work. I am not offering this option because i really need additional paid work per se but because i see a strong need for this project in the OSM community and from data users in particular, and i would be willing to reduce other parts of my paid work in favor of this in case there is interest in financing that.
  • If other independent experts are interested in contributing to the project under the premises described above, i would be willing to open the project to other authors. So far i have not put the content of TagDoc under an open license but i would be open to such a step and this evidently would be kind of a prerequisite for turning it into a larger cooperative endeavor of several authors. This could work on various levels – from authors contributing analysis or documentation of a single tag to writing and editing whole thematic segments. Note that i would still want to remain the overall curator and proprietor of TagDoc at least until i can be sure that an alternative form of governance would sustainably protect and develop the premises of the project in a responsible way.

If you are a data user (or otherwise want to invest in OpenStreetMap beyond software development and paid mapping) and interested in contributing to financing writing for and curating this project, and thereby help making it more sustainable, or if you are an independent expert in the de facto meaning of tags in OpenStreetMap data with proven experience in analyzing tag use in the OSM database, English language writing skills and knowledge about the diversity of world wide geography and are interested in contributing to this project as an author, then you are welcome to get in touch with me about contributing.

About me, the proprietor and curator of TagDoc

So, some might ask, what qualifies me to run and curate a project like this?

The main incentive for me contemplating the problem of meaningful and reliable tagging documentation and ultimately starting TagDoc came from my work as a maintainer of OSM-Carto. Forming a qualified opinion on requests to add or change the rendering of certain features in the map under the goals of the project always requires a solid knowledge of the de facto use of the tags involved in OpenStreetMap. Same applies for developing features for my own OSM-Carto derived map style. Doing research on well more than a hundred such cases, involving a large bandwidth of tags, helped me acquire a broad knowledge background in practical use of tags in OpenStreetMap, a lot of practical experience in analyzing how tags are used in OpenStreetMap world wide and what the quirks and inconsistencies in that use are, as well as a good sense of the quality problems of the tagging documentation on the OpenStreetMap Wiki. Combine that with the experience with using OSM data on global scale, i have gained as part of my paid work and what i researched over the years regarding practical use of tags out of curiosity and as part of writing about OpenStreetMap in general on my blog and elsewhere, i have probably broader background knowledge about the de facto meaning of tags in OpenStreetMap than most involved in OpenStreetMap. But the key, ultimately, is combining that broad background about OpenStreetMap with a solid knowledge and experience with the geographic diversity of the planet. Most OpenStreetMap contributors in their on-the-ground mapping work acquire extensive knowledge of the geography of the area they map in but very few have a solid understanding of the full range of geographic diversity of Earth which OpenStreetMap aims to document.

Of course this is a valuable qualification for doing consulting work regarding use of OpenStreetMap data and it has helped me many times in giving competent advice to customers. But ultimately it is kind of dissatisfying to not being able to make this knowledge available systematically to everyone who would find it useful and value it because of the lack of an economic basis to do so. I know this is a problem i am not alone with. The economic ecosystem around OpenStreetMap is traditionally heavily biased towards technical work. Software development and paid mapping are the most valued and therefore the dominant paid activities around OpenStreetMap and intellectual work of all kind is seriously under-appreciated. It would be important for the future of OpenStreetMap to change that and having quite a bit of experience in doing consulting work at the edge between technical work (data processing) and intellectual work (map design) and being one of the most prolific public writers around OpenStreetMap i think i am in a good position to lobby a bit for such change and to attempt demonstrating the importance and value of intellectual work using TagDoc as an example.

And finally one other important thing that i think qualifies me as a curator of tagging documentation is that i am largely independent of OpenStreetMap economically so i can provide a honest and open assessment without being constrained by my own or others’ economic interests. While i do paid work with OpenStreetMap data, most of my income these days is based on working with satellite data and its visualization and therefore does not depend on OpenStreetMap.

Conclusion

Ultimately all of this is just an offer from my side. I hope it finds the resonance and support it needs to become sustainable in the way i outlined and this way becomes a useful and valued source of knowledge about OpenStreetMap data and the tags used in it to record local knowledge about the geography of Earth. If it does, but especially also if it does not, i hope my endeavor to start TagDoc incentivizes others to think and talk about the importance of creating and maintaining a reliable and competent documentation of the de facto meaning of tags in OpenStreetMap, not unduly affected by subjective ideas of their ought-to-be meaning and the best way to develop and maintain such. This kind of project, like any other intellectual work, can only thrive and be excellent if it receives intellectual resonance and critical feedback.

TagDoc logo