Imagico.de | blog

November 16, 2023
by chris
0 comments

No OSMF analysis this year

The annual general meeting of the OpenStreetMap Foundation is approaching and some readers probably expect me to write about the last year in the OSMF and present an outlook on future developments. I do not plan to do so this year though and i want to explain a bit the background of this.

It is not that nothing i could write about happened in the OSMF in the past year – on the contrary, there were plenty of remarkable decisions made that would deserve a more elaborate discussion. But as the year went on and my collection of material on the matter grew i noticed that – while the way the OSMF is developing as an organization and sociologically remained fascinating – the perspective to write publicly about it in autumn in the form of an overall yearly review and lookout before the AGM seemed increasingly unattractive. This has led me to reflect a bit on why i write about these things.

I will start with a concrete example: In last year’s analysis i described an incident were a money spending decision was made in explicit and ostentatious disregard of the conflict of interest of one board member and i was struggling to understand how this happened without any of the other board members raising concerns. Of particular interest was the question if the board members were collectively unaware of the problem or if they noticed it but rationalized for themselves that there is no need to act on that problem. As a result of that i carefully observed similar cases this year of money spending decisions being made with board members subject to conflicts of interest. And based on that i now think i have a much better understanding about the social dynamics and the mindset of board members in such situations.

Me writing about these kind of observations here on the blog would probably quite significantly increase awareness of these matters from just a handful of people actively observing the OSMF to maybe a few hundred. But that is not enough to affect meaningful change in the OSMF – even if all of these would roughly share my concerns and ideas for improvements. And this is not a matter of limited reach but a matter of lack of interest.

Generally speaking, i do not mind writing for a small audience – if i did you would not see posts about map design and other things here. The difference is that these things have real world significance way beyond the circle of readers they reach at the moment. Commentary on the organizational developments and governance of the OSMF, however, only has relevance within the small world of the OSMF – unless of course you are looking at it from a organizational sociology standpoint with the OSMF as a case study. But i am not a sociologist. While intellectual curiosity about the social dynamics in an organization like the OSMF is one of the things that motivates me to observe the OSMF, i do not have the ambition to systematically publish my observations.

This does not mean i am going to completely stop writing about the OSMF. But as i previously have moved from real time commentary on acute OSMF matters to aggregate yearly analysis a few years back i am probably going to further move to focus on discussing long term developments with a multi-year horizon on a case-by-case basis. With that i would also pretty much adjust to the mode of discussion prevalent in the OSM community at large regarding the OSMF these days. Discussion on acute OSMF topics on channels with a public record has essentially ceased, so has meaningful pre-election discussion before the AGM. But there have been valuable irregular comments from others who – like me – keep an eye on and analyze OSMF politics. Like, for example, recently the insightful comments from Ilya and Simon.

This is the level on which meaningful discourse in the OSM community on OSMF matters tends to happen these days and where a valuable exchange of different views and perspectives is possible.

One other decision i have made in context of this is that this year i applied for active contributor membership in the OSMF. I had been struggling for quite some time with the moral implications of contributing to the OSMF finances – even if only on a very small scale – in light of the OSMF money spending habits and oversight mechanisms being massively at odds with what i deem minimally necessary. The active contributor membership gives me a way out of that dilemma. I am aware that this of course does not absolve me from the responsibility resulting from legitimizing the OSMF’s actions simply by being a member.

But i don’t want to create the impression here that everything is bad about the OSMF these days – no, there are both positive and negative developments in the past and there is potential for positive and negative developments in the future. And as far as bad decisions are concerned – these are not exclusively the fault of those making them originally, they are just as much a problem of the failure of the OSMF members to exercise responsible oversight and control.

To not leave the readers without anything of substance actually on the OSMF – here a short comment to wrap up this post.

Ilya in the comments linked to above diagnoses a communicative disconnect between the OSMF and the Overture Consortium and i would say a similar, probably even more fundamental, disconnect exists between the OSMF on one hand and the OSMF membership and the OSM community on the other. Like in the OSMF-OMF case this disconnect cannot be fully blamed on one side, it is a two-sided lack of understanding of each other and a lack awareness and acknowledgement of their problems and interdependencies.

But the key point here is that the big corporations the Overture consortium is composed of and the OSM community at large can much better afford to ignore other actors in the field communication wise. For big corporations this is simply the way they are used to. And for the OSM community being self sufficient was, back in the days when OpenStreetMap started, simply a necessity to emancipate itself from traditional geodata production. The OSMF on the other hand absolutely cannot afford a communicative disconnect with either the OSM community or their commercial environment (or even misunderstanding economic developments in its neighborhood – like the OMF or HOT). Yet, what the OSMF largely seems to be doing for the last years is digging itself in more and more with the small circle of people whose work we know and enjoy. And even though some people in the OSMF seem to start to realize how disastrous this is, the ideas to address this problem seem to go towards more corporate style PR and community management (i.e. trying to actively shape how others see them) rather than trying to understand their larger social and economic environment as it is.

Coastal geomorphology for mappers in OpenStreetMap

October 24, 2023
by chris
6 Comments

Coastal geography for mappers in OpenStreetMap

I have written about the mapping of the physical geography of coasts in OpenStreetMap before several times. However, when i see people discussing coastal mapping these days and editing the OpenStreetMap wiki i still frequently get the impression that – while discussion contributions are usually well intended – they unfortunately all too frequently seem to be – well – fairly uninformed. Discussions often lack relevant background equally in terms of present day mapping practice (a matter which i also discussed more in depth in broader terms), the history of mapping in OpenStreetMap and concerning actual geography.

Which is why i feel like it might be a good idea to provide a solid background on the very basics of coastal physical geography mapping in OpenStreetMap – which i am going to attempt doing here.

Current coastal mapping conventions in OpenStreetMap

I will start with describing what the currently most widespread mapping practice is in OpenStreetMap for mapping and tagging basic coastal features. And i will use the rendering from the Alternative-Colors style here for illustration because it is more descriptive for coastal features. You can see the OSM-Carto variant when clicking on the images. Of course there is a large variety of different coastal settings with dozens of different tags, in particular if you include vegetated coasts. So i will only cover the very basics here. First, lets look at a setting with a relatively low tidal range, strong waves and a relatively steep slope of the coast:

Wave dominated sandy coastal setting

As most mappers will know, the coastline is placed at the high water mark (that is the highest level of water regularly reached during the tidal cycle). In the map examples shown this is the line between the bluish parts on the left and the yellow/gray areas on the right. The low water mark (lowest level of water regularly reached during the tidal cycle) is not necessarily explicitly mapped because it is practically not well verifiable – but for clarity i marked it here in the illustrations with an administrative boundary (which in many jurisdictions happens to be located at the low water mark).

What we have here is a wave dominated setting and a sandy beach. This is mapped in OpenStreetMap with natural=beach and surface=sand – which does not imply recreational use, it is purely a physical geography characterization. What is tagged natural=beach is the whole range covered by loose material that is shaped primarily by waves. In such a setting that range extends from slightly below the low water mark to slightly above the high water mark. Any area of loose sand above the high water mark that is not primarily shaped by water waves but by wind is by current conventions in OpenStreetMap not tagged natural=beach but natural=sand. Some mappers split the mapping of the beach at the coastline and tag the lower part in addition with tidal=yes. This does not provide any additional information, but it does not hurt either.

Practically, when mappers map remotely based on aerial/satellite imagery, the beach is often mapped downward only to the waterline of the moment as depicted in the image – and in case mapping is coarse or the image quality is insufficient for more precise placement, the coastline is also placed at this (typically too low) location. Still current mapper consensus about how things should be mapped ideally is clearly as described.

Sand areas below the low water mark that are permanently covered by water are typically tagged (as far as mappers have knowledge of that) with natural=reef + reef=sand.

The same scenario is of course also possible with the beach composed of coarser material – like surface=gravel or surface=shingle.

How do things look like in a more tidally dominated setting (more gentle slope of the coast overall, stronger tides and less strong waves)?

Tide dominated sandy coastal setting

In this scenario you typically still have a beach at the upper end of the tidal range – but it tends to be fairly narrow – or even can be fully absent in cases where waves are negligible or the coast is vegetated. Below that, in the tidal range, you can typically find a tidal flat, an area with very gentle slope consisting of fine grained loose material (mud or sand) that is shaped primarily by the tidal currents rather than waves. This area is by current convention tagged natural=wetland + wetland=tidalflat, sometimes with an additional surface tag (surface=sand or surface=mud).

In case of rocky coasts there is no distinction between above and below the high water mark in terms of primary tagging – natural=bare_rock is used in the tidal range just like on land.

Rocky coastal setting

History of coastal tagging

What i have described above is the currently most widespread practice of mapping coastal physical geography in OpenStreetMap. Most mappers evidently find this suitable for the most part, but there is quite a bit of variation and edits on the OSM wiki and comments made in various settings also indicate that some mappers find this practice undesirable or confusing. To understand this it is helpful to look at the history of tagging in OSM – because what i sketched above has not always been how coastal land forms are mapped in OpenStreetMap.

As most people know, OpenStreetMap started in the United Kingdom and in the UK the traditional cartographic conventions for representation of coastal forms distinguish not between different geo-morphological settings but exclusively between different surface materials. Tidal areas are classified as either sand or mud – as you can see here in the map keys of two different Ordnance Survey maps:

Legend from Ordnance Survey map 1:25k from 1966

Legend of contemporary Ordnance Survey map 1:50k

This concept was reproduced in the very early mapping practice in OpenStreetMap with natural=beach and natural=mud. Of the tags discussed these were the most widespread in use until about 2011. These two tags were, for use in the tidal range, in particular also promoted by the standard map style, which showed them both with a regular dot grid pattern.

natural=beach/natural=mud rendering in OSM-Carto version 1

Use of other tags, in particular natural=wetland + wetland=tidalflat and natural=sand became popular only somewhat later. Coastal mapping outside the UK almost exclusively used natural=wetland + wetland=tidalflat for both sandy and muddy tidal flats and natural=beach was not adopted significantly as a general tag for sandy areas in the tidal range outside the UK but was more strictly limited to beaches in the geo-morphological sense (as well as artificially created sand stretches at the shorelines of lakes and the sea). In the UK, use of natural=beach for larger flat areas in the tidal range was disputed quite early as well with some mappers preferring to adopt the continental European practice of using natural=wetland + wetland=tidalflat in general while others pushed for maintaining the mud-sand duality from Ordnance Survey maps on the level of primary tagging and to use natural=wetland only for muddy tidal flats while using natural=sand for sandy flat areas in the tidal range.

The use of natural=sand in the tidal range stayed a niche exotic use outside the UK so far despite some wiki-fiddlers trying to push this as the universal ought-to-be tagging. It gained some adoptions in regions with a mixture of sandy and rocky tidal areas, because – as described above – on rocky coasts mapping consensus is to use natural=bare_rock both in the tidal range and for dry rock above the high water mark and it can seem plausible to use natural=sand in analogy. Most mappers, however, seem to consider it better not to use the same primary tag for the geo-morphologically fairly contrasting settings of coastal dunes and sandy tidal flats and consider natural=wetland + wetland=tidalflat a suitable tagging for sandy tidal flats – with the option to specify that via surface=sand.

Practical examples

So far, i have discussed coastal mapping here on a fairly abstract level. So i will provide a few practical examples to illustrate what i explained above and to show in how far practical mapping in OSM consistently follows these conventions and where it sometimes diverges.

All examples are shown with a photo – linked to a larger version on an external site – and a map screenshot from OSM-Carto showing the current mapping in OpenStreetMap and linked to osm.org where you can inspect it in more detail. Be aware that the direction of view in the photos is not universally northwards so the photo and the map typically have different orientation.

First a classical wave dominated beach:

by Bin im Garten – CC-BY-SA 3.0

This example is a bit special because the sand is artificially groomed for recreational use, making it difficult to identify the upper end of the beach and the high water mark. The mapping correctly represents the existence of wind shaped stretch of loose sand above the beach. It does not extend the beach mapping below the high water mark, which is quite common though, as already mentioned above.

Next example is equally a wave dominated beach:

by dronepicr – CC-BY 2.0

Here mapping and tagging are based on imported data and therefore do not match manual mapping conventions in OpenStreetMap that well. The natural=beach mapping includes the non-vegetated parts of the coastal dunes. On the other side the beach ends at the high water line and the tidal part of the beach is mapped with natural=wetland + wetland=tidalflat – though the image shows quite clearly a wave formed beach, at least down to the water level shown in the photo.

The next two examples are more edge cases between a wave shaped beach and a tidal flat shaped by tidal currents.

by UK Environment Agency – OGL 3.0

Both are mapped with natural=beach – above and below the high water line. A good argument can be made though that this is more fitting for natural=wetland + wetland=tidalflat + surface=sand as you can see substantial structuring of the surface due to tidal currents. In addition, the coastline in the second of these two samples is placed too low, significantly below the high water line, probably because it was mapped from images based on the water level of the moment as shown in those. In the first case the mapping of the beach reaches too far upwards into the vegetated areas.

Much more clearly a tidal flat setting can be found here:

by UK Environment Agency – OGL 3.0

which mapping accurately represents with natural=wetland + wetland=tidalflat + surface=sand.

An even clearer case is this example

by Bundesanstalt für Wasserbau – CC-BY 2.0

which directly matches the tidally dominated setting sketched above – though the mapping here conflates the lower end of the beach and the beginning of the tidal flat with the high water mark. The photo, however, clearly shows that the transit from the wave shaped beach to the tidally shaped flat is significantly below the high water line.

Another somewhat different example is here – where the (unmapped) beach is very narrow and cut short at the upper end by a constructed dyke:

by Bermicourt – CC-BY-SA 3.0

So far these were ground level or oblique aerial photos. I want to close with an orthophoto example, providing some hints how the delineation between a wave shaped beach and a tide shaped tidal flat can be practically made when mapping from imagery.

by IGN France

The beach here is currently not mapped but can be well seen on the image. Typically, waves create structures parallel to the coast while tidal currents produce structures orthogonal to the coast roughly in direction of the coastal slope. Also beaches with their stronger slope often drain water faster at low tide and do not remain waterlogged as much as tidal flats at low tide – which will often lead to different coloring on images. The most reliable source of information is, of course, always observation on the ground.

Conclusions

The obvious question some will probably ask is why does the OSM community not adopt the Ordnance Survey convention to distinguish by surface material only and not by some fairly specialized geo-morphological classification? There are several reasons that probably play a role here:

Most mappers consider the distinction between different geo-morphological forms (beaches, tidal flats, coastal dunes etc.) as more meaningful than the distinction between different surface materials.
In remote mapping from aerial/satellite imagery, the distinction between beaches and tidal flats as geo-morphological forms can often be made with careful observation of the images while the distinction by surface material cannot.
In most regions on Earth, sand and a mixture of sand and finer material are more common as surface material of tidal flats than mud, making the geo-morphological classification simply the only meaningful distinction that can be made.

The more relevant point i am trying to make here is, however, that this example illustrates well how developing globally uniform tagging conventions in OpenStreetMap is hard, in particular because of pre-existing diverging cartographic conventions in different parts of the world. Although OpenStreetMap has a free form tagging system and deliberately is not based on institutional data models and classifications, those still tend to color tagging ideas in the project. I hope this example shows how not sticking to a specific cartographic tradition but opening up to more diverse mapping and tagging ideas from all over the world, developed by people based on how they perceive the local geography, helps to develop a more differentiated way to see and document the diverse global geography.

And i also hope that the desire for simplicity and the prioritization of data volume over data depth, that is understandably felt and expressed by many mappers these days, does not lead to a rolling back of these substantial achievements in the name of ‘modernization’ – as we have unfortunately seen all too often already in OpenStreetMap.

Augmented symbols for bus stops with shelters

October 9, 2023
by chris
2 Comments

Symbols and labels #3 – augmenting symbols

In the previous post i discussed how a relatively simple feature addition to Mapnik called anchors, which allows conditionally connecting the rendering of some visual elements to that of other elements in the map in a freely customizable way, can help solving the basic dilemma of combined symbol and label visualizations of point features.

But these anchors are useful for more than that – among other things for augmenting symbols.

I have already demonstrated augmented symbols in the past in a very rudimentary fashion for water features with a drinking water attribute. The technique as shown there was rather limited and essentially this was just a basic proof of concept for the visual design idea.

rendering of water sources introduced back in 2018

Augmenting symbols with anchors

The idea of symbol augmentation is that you have a pictorial/geometric symbol in the map visualizing a certain point of interest and that you add a supplementary design element to that symbol to illustrate either additional attributes of the feature in question or to provide information about other data in the geometric context.

The difference to a symbol variation, where you vary the symbol used from a common base design – like it is shown for example in the example above for intermittent and hot springs – is, that symbol augmentations are optional, they are rendered only when there is space for them and the non-augmented symbol, while providing less information, still illustrates the fundamental nature of the feature in question.

The idea with the water POIs is to indicate that a spring/well/fountain provides drinking water with a supplemental beaker symbol, as it is used also for the generic primary tag amenity=drinking_water (see above).

In many cases the design of base symbol and augmentation allow for a number of different relative placements – which can be prioritized based on where other symbols block space – as shown here:

or based on other geometric context – like in case of connected springs:

Augmentation of springs connected to streams at z19

Augmentation of springs connected to streams at z18

Augmentation of springs connected to streams at z17

Augmentation of springs connected to streams at z16

Augmentation of springs connected to streams at z15

Augmentation of springs connected to streams at z14

Technically, a number of things are necessary to make this work well and be flexible in terms of symbol design:

Priority of the augmentation needs to be adjustable independent of the base symbol.
Overlap of the augmentation and base symbol needs to be possible without blocking.
It needs to be possible to specify the relative placement of the augmentation in all the variants precisely.
It needs to be possible to use different symbol designs in different relative placements.

Existing possibilities in Mapnik do not provide the means for this. How anchors can be used to accomplish that i will show in the following.

Like with the symbols and labels rendering, the base symbol is rendered with

marker-anchor-set: '[osm_id]';

The rendering of the beaker symbol in a separate lower priority layer naturally in the first placement variant gets the already known

marker-anchor-cond: '[osm_id]';

but in addition also

marker-allow-overlap-anchor: '[osm_id]';
marker-anchor-set: "'dw_'+[osm_id]";

marker-allow-overlap-anchor is a selective allow-overlap, meaning that rendering allows overlaps of this symbol with any already rendered symbols with the specified anchor. The second line sets another anchor for the augmentation symbol. This is then used as a condition in subsequent placement variants:

dir2/marker-anchor-cond: "[osm_id]+',!dw_'+[osm_id]";
dir2/marker-allow-overlap-anchor: '[osm_id]';
dir2/marker-anchor-set: "'dw_'+[osm_id]";

This means that rendering of this second placement variant is conditional to (a) the successful rendering of the base symbol and (b) the first placement variant of the augmentation symbol not having been successfully placed.

In addition, for connected spring rendering, the SQL query sorts the different placement variants in priority based on the direction of the connecting waterway – resulting in the behavior illustrated above.

Here a few real data examples illustrating how this looks practically:

Well at z19

Well at z18

Connected spring at z19

Fountain at z19

Aggregating features with augmented symbols

That was how symbol augmentation can be implemented with anchors in the very simple case that the augmentation visualizes exclusively a certain secondary tagging on the features in question (amenity=drinking_water or drinking_water=yes on natural=spring/man_made=water_well/amenity=fountain). I will now show and explain how this concept can be extended to aggregate different close-by and semantically connected features.

The case i want to demonstrate this on is bus stops and shelters. Bus stops in OpenStreetMap are tagged with highway=bus_stop. If the bus stop has a shelter for protection of waiting passengers against the weather this can be tagged with

a separate feature (node or polygon) representing the shelter tagged amenity=shelter (optionally with shelter_type=public_transport)
or a secondary tag shelter=yes on the highway=bus_stop node
or both

In many cases the bus stop and the shelter are very close to each other so there is no room at the lower zoom levels to render both with a dedicated symbol without displacement of either. In OSM-Carto in addition amenity=shelter has priority over highway=bus_stop in rendering so you see a lot of shelters but not a lot of bus stops at z16/z17.

A viable solution is again to use symbol augmentation. Here is how this looks like at z16:

From left to right: (1) amenity=shelter (2) amenity=shelter + shelter_type=public_transport (3) highway=bus_stop (4) highway=bus_stop with either shelter=yes or separate amenity=shelter nearby

and at z17 and above:

Bus stop rendering with augmented symbols at z17 and above – see text for detailed explanation

From left to right this shows:

amenity=shelter
amenity=shelter + shelter_type=public_transport
highway=bus_stop
highway=bus_stop with separate node tagged amenity=shelter nearby
highway=bus_stop with separate node tagged amenity=shelter + shelter_type=public_transport nearby
highway=bus_stop with separate node tagged amenity=shelter + shelter_type=public_transport nearby and nearby other symbol blocking default placement of symbol augmentation
highway=bus_stop with secondary tag shelter=yes and with separate node tagged amenity=shelter + shelter_type=public_transport nearby

The last four are shown on top with the shelter node so close that there is no space for both symbols, hence resulting in the augmented symbol rendering, and on bottom with the shelter node far enough away for a separate rendering. In the last variant on the right this version shows the non-augmented bus stop symbol because the shelter is close enough to be recognized as the shelter of this bus stop explicitly representing the implicitly tagged shelter=yes – while in the third variant below, the shelter node is so far away that it is no more recognized as belonging to the bus stop – hence the shelter=yes leads to the augmentation.

Implementing this with anchors in Mapnik requires the query for the add-on symbols to provide both the osm_id of the bus stop and that of the closest nearby shelter (as osm_id_shelter). This is then used for the first placement variant of the augmentation symbol like

marker-anchor-cond: "[osm_id]+',!'+[osm_id_shelter]";
marker-allow-overlap-anchor: '[osm_id]';
marker-anchor-set: "[osm_id]+'_'+[osm_id_shelter]";

meaning to

only render the augmentation symbol if the base symbol has been successfully rendered and if the standalone symbol for the shelter has not. For only implicitly mapped shelters, osm_id_shelter is set to zero so this second condition is always met in those cases if no explicitly mapped shelter is nearby.
Allow the augmentation symbol to overlap with the base symbol.
Set a new anchor concatenating the bus stop id and the shelter id.

For the second placement variant (augmentation on the right) the anchor condition is then obviously extended to

alt/marker-anchor-cond: 
  "[osm_id]+',!'+[osm_id_shelter]+',!'+[osm_id]+'_'+[osm_id_shelter]";

meaning that this marker is only rendered if

the corresponding base symbol has been successfully rendered,
the shelter has not been successfully rendered as a standalone symbol,
the previous placement variant(s) have been non-successful

Why do we include both the bus stop id and the shelter id in the anchor? Because we want the same shelter to potentially serve for several bus stops and they need to have different anchors in both augmentations for that to work. For bus stops this is practically not a common arrangement but for tram stops and railway halts this is quite common when both directions of transport share a common platform with a single shelter (the 2+1 variant below).

Augmented symbols for railway halts and tram stops

And here finally a few example of how this looks like with real world data.

Bus and tram stops in Strasbourg at z16

Bus and tram stops in Strasbourg at z17

Bus stops in Strasbourg at z18

Conclusion

What i showed here is how symbol augmentation can be used as a technique to transport additional information in a map in an intuitive way in cases where space is scarce and there is not enough room to depict everything through atomic and static point symbols. Doing this in a proper way requires support from the rendering tools and i demonstrated how the low level feature of anchors i added to Mapnik can help with that.

As mentioned in the previous post already these anchors implement only part of point 5 in my list of features map design would like to have from the tools it uses. At the same time what i showed in this and the previous blog post barely scratches the surface of what can be done with this relatively minor additional feature. This should give you a bit of an idea how we are in automated rule based map design – figuratively speaking – barely at the threshold towards the Neolithic.

October 7, 2023
by chris
0 comments

Symbols and labels #2 – making a connection

I could not be at the Karlsruhe Hack Weekend this time, but i did some hacking work during that and the following week i wanted to write about here. Back in May i discussed in depth the problem of visualizing points of interest in automated rule based maps with symbols and labels in combination. I want to revisit that problem now.

In a nutshell – what i discussed back then was, that you can approach the problem primarily in two ways:

separately rendering the symbols and labels, typically with the symbols having higher priority than the labels.
connecting the label and the symbol for every feature and render them with the same priority (typically via shield symbolizer in Mapnik)

Both approaches have advantages and disadvantages – as discussed and demonstrated in that post. The bottom line is – quoting what i wrote back then:

What we would really like instead is rendering the symbols as in the first approach but then render the labels only for those features for which a symbol has been successfully rendered. This, however, is not possible with the current tools.

I have tried to work on this dilemma now by extending Mapnik with the possibility to connect two different independently rendered elements in the map – making the rendering of an element (like a label or a marker) conditional in some form on if other elements have been (or have not been) successfully placed before.

Introducing anchors

The reason why this kind of functionality is not available in Mapnik so far is probably because Mapnik mostly aims to provide higher level features to connect different elements in the form of what is called group and shield symbolizers (to tie several elements to be either rendered together or not at all) and placements (to try several alternative labeling/symbol placement options). But those are practically severely limited in the level of control they provide to the map designer and they do not allow connecting elements independent of their priorities.

Mapnik’s internal design is fairly complicated and very sparsely documented so what i implemented is surely not very elegant and in many ways not how it should be done. Be that as it may – i will try to explain the feature and how it works in its CartoCSS syntax. For setting up Carto with the new feature you also need an updated mapnik-reference.

The feature is called an anchor – which is a string you can assign to any blocking element you render (point, marker and text symbolizers) – either statically or based on data properties via expressions. With an osm2pgsql PostGIS database the most straight away attribute to use is the osm_id.

So with the separately rendered POI symbols and labels the symbols get something like

marker-anchor-set: '[osm_id]';

which means: When this marker is successfully placed (i.e nothing previously rendered is blocking it) record its osm_id attribute in the list of elements successfully placed. osm_id of course has to be added to the SQL query for the layer.

When you later, after having placed all the symbols, want to place the matching offset labels as far as there is space left and want to avoid placing labels for features for which no symbol has been successfully placed you use

text-anchor-cond: '[osm_id]';

which means: Only place this label if its osm_id attribute is in the list of anchors of elements successfully placed before. If you want to – in case no symbol could be placed – try to render a centered non-offset label for the feature in question you use

textonly/text-anchor-cond: "'!'+[osm_id]";

which means: Only place this label if its osm_id attribute is not in the list of anchors successfully placed before.

Practical results

As you can see this is pretty simple to use – and it solved our symbol and label rendering dilemma quite nicely. For reference here the two variants offered by the AC-style so far – first the symbol and label separate version (similar to OSM-Carto, but with consistent priorities):

AC-Style with separately rendered symbols and labels

and in the combined version:

AC-Style with combined symbol and label placement

which has the main disadvantage that the labels of high priority features block the symbols for lower priority features – hence no shelters on the right side in the second sample image.

With the help of the new Mapnik feature the separate version combines the benefits of both:

separately rendered symbols and labels connected via anchors

On the very right you can see that the higher priority wilderness hut does not prevent the shelter symbol from showing because its label is not rendered together with the symbol but placed later with lower priority. At the same time it avoids the problem of stray offset labels without a corresponding symbol as it happened previously when rendering symbols and labels separately.

That’s it essentially – a relatively simple extension of Mapnik solves the dilemma of combined point symbols and labels that bugged OSM-Carto for almost exactly ten years now.

In terms of the list of desirable features of map design tools i published recently this implements parts of point 5 by the way.

But of course the concept of anchors in Mapnik and making the rendering of elements conditional on one another in a freely definable way can be used for other things as well. More on that in the next post.

September 25, 2023
by chris
5 Comments

Map style licensing – another comment

Earlier this year i wrote a bit about map style licensing. The OSMF has just added a new featured tile layer on the openstreetmap.org website – and i wanted to quickly point out an interesting quirk in that regard here.

First – the tile layer added is an interesting mashup of OSM-Carto and OpenTopoMap – and individual styling ideas integrated from other map styles and from map development discussion on public channels. If i had to specify a unique and interesting aspect of the style in terms of the guidelines for new tile layers that would be the mashup aspect. It is not the first style that combines design elements developed independently into a distinctly and deliberately heterogeneous design, but it is definitely a noteworthy example of that.

However, the style is not open source and as you might know (i mentioned it in January) OpenTopoMap is licensed under CC-BY-SA. So unless Tracestrack has obtained an explicit license for non-open use from the OpenTopoMap developers we are having a clear license violation here.

The freedom to recombine different design elements and to create mashups is an important and attractive aspect of Open Source map design. But you still have to respect the licenses.

September 16, 2023
by chris
8 Comments

What rule based map design requires from the tools it uses

I have in the past explained on several occasions that the free and open-source software (FOSS) tools available to map designers developing rules for automatically rendered maps are severely lacking and that it would be of high importance for the OSM community to invest strategically in that field because commercial map providers develop tools for fairly narrow short term needs and focus very strongly on technical aspects and have little interest in substantial innovation in actual map design.

But i have never actually explained more in detail what the concrete needs of state-of-the-art rule based map design are regarding tools to be able to produce innovative and high quality maps. The main reason for that is that this is not something you can put into a simple list of requirements a software developer can work through and then ends up with a product that meets the requirements. Most of the needs i will list below are matters that require an in depth discussion between designers and software engineers with a solid background in computer graphics and an open mind towards the work of map designers. Me trying to provide a concise summary of some of the main requirements below is not a substitute for such discussion – although i will try to provide some hints to examples that can help initiating a more in depth contemplation.

The other reason i have been hesitant in discussing this in public is that the most likely practical beneficiaries of such discussion are proprietary software developers. Professional FOSS development in the field of map rendering seems unfortunately very focused on the already mentioned commercial map providers (understandably, because that is what at the moment largely pays for their work) and is fairly detached from the world of actual map design. The vast majority of actual innovation in FOSS tools for map design of the last 5-10 years has happened not by professional software developers but by map designers trying to address their specific needs on their own. Most of these attempts are fairly awkward, too specialized and of low technical quality because most map designers of course lack the software development skills to produce something more solid.

I like to emphasize that this list of requirements is not meant to be complete in any way. Map design is a highly diverse field and my personal areas of focus in map design work of course influence the priorities i see. Other map designers will have additional needs that i do not discuss here. At the same time i tried to keep this limited to clear needs and left out out stuff that would be just nice to have or things that would be worth exploring as potential technical innovations without there being a clear practical necessity visible so far.

I also left out any specific higher level feature requirements for the renderer. I did that because (a) such a list would be highly specific to the design use cases you have in mind and (b) higher level feature can be built directly into the renderer or they could be implemented by the users based on more fundamental drawing functions and a flexible and efficient framework for defining rendering rules. Bottom line: High level rendering features are not strictly required at all. They can, but they do not have to be, an integrated part of a well usable map rendering framework.

I also like to emphasize that this only covers map design in a strict sense. Many map providers these days regard their business as providing an overall map viewing experience with interactivity and integration with other services in the foreground. This is often the most important selling point, not the quality of the map design used. This whole domain i deliberately do not cover.

So here is the list of what – from my perspective – rule based map design requires from the tools it uses to be innovative and produce high quality maps:

A rendering engine capable of precision rendering. In other words: The rendering engine should faithfully execute the instructions of the map designer on how to draw things. I had previously analyzed a number of rendering engines in that regard in a rudimentary fashion. So far Mapnik (and maybe Mapserver) are the only practically usable map rendering engines that come close to meeting this requirement.
A map design centered language/file format allowing the map designer to implement their map rendering rules in a way that matches their work and thought process and that scales well with design complexity. The interpreters of this language/file format should support the full feature set of the rendering engine. CartoCSS was fairly revolutionary w.r.t. being map design centered. At least in the form supported by Carto it has, however, only very rudimentary options for the parametrization of design rules and only very limited modularization and code reuse options and therefore is not that well suitable for more complex map design work. And Carto does not support the full feature set of Mapnik. Examples exist in the broader field of graphics design that can provide valuable cues – i for example like to point at POV-SDL, which supports both geometric modeling and defining styling together in a design centered language (for the most part – user defined functions, which were a late addition, are not particularly well usable for designers).
Practically usable means to define scale dependent precision geometry processing. Practical use cases for this can be found in the AC-Style (like trees, water barriers, fords). Support for this is excellent in the OSM-Carto toolchain but completely lacking in any of the client side rendered frameworks. Even more: This is practically not feasible to add there because the inevitable lossy geometry data compression makes client side precision geometry processing impossible and scale dependent geometry processing on the server side would lead to massive increases of the data volume.
The ability to define drawing order independent of the way the data is structured. This is currently not possible with Mapnik without unifying all the involved data into a single layer (as done in the AC-Style for the road layers). This is one of the main hard limits of all Mapnik based toolchains.
The design rules should have access to detailed information on what has been drawn in the map so far. Currently this is limited to simple blocking in most map rendering frameworks, meaning that you can draw something independent of what has been drawn before or you only allow it to be drawn if no other blocking feature has been drawn before at the same location.
The ability to define drawing order independent of the prioritization of competing elements. Classic example where this is needed is contour line labeling: Contour line labels typically have the lowest priority of all labels and are not to be placed overlapping most of the other features in a map (including line work like roads, water lines etc.) At the same time contour line labels are traditionally cut out from the contour lines with a buffer for good readability and harmonic integration into the map. And the contour lines are drawn below almost everything else in the map. Doing this (drawing the contour lines early but cutting out the low priority labels) is not possible with most automated map rendering frameworks available these days so you will practically not see contour line labels with cutouts in such maps (with very few exceptions).
Flexible raster compositioning options in the rendering engine. Mapnik has rudimentary support for that (feature and layer based compositioning of the current feature/layer relative to what has been drawn so far). More flexibility here would be very important, in particular regarding rendering performance, because it would reduce the need for expensive geometry processing and for drawing features several times and it would practically allow developing solutions for problems that are currently not solvable in an efficient way (like the road line/polygon layering issue). Cues for this could be taken from common raster processing frameworks like ImageMagick and G’MIC.
A data processing chain that provides access to the full data model of the source data to the map designer. This has so far not been the case for the OSM-Carto tool chain for OSM data because osm2pgsql only supported multipolygon and route relations and only in a rudimentary way (without exposing relation membership information). The osm2pgsql flex backend improved this a lot though.
A text rendering engine that provides information on the would-be geometry of labels in the design rules without actually placing the label in the map.
The practical possibility to do automated integration testing of the full rendering tool chain (from the raw source data to the final raster image).

The list is roughly sorted by importance with the most important requirements on top. The first three points i regard as essential. Without meeting these points at least in a rudimentary fashion the map design work i do is practically impossible. That is the main reason why so far i have very little interest in working with any of the client side map rendering frameworks that are popular these days – because they substantially lack in all three of these points without a clear perspective of improvement in any of them.

September 14, 2023
by chris
0 comments

Flooding in eastern Libya

Supplementing the previous post about the extreme rain and resulting flooding in the eastern Mediterranean i also prepared an image from the other side of the sea at the north African coast in Libya, where the same storm that caused the flooding in Greece also lead to extraordinarily intense rainfall and flooding a few days later.

Most of the damage and fatalities due to the flooding happened at the coast in the north in the steep and narrow valleys, indirectly visible in this image mostly due to the sediment carried by the streams in these valleys into the sea and coloring it.

More impressive in this picture is, however, the extensive flooding in the desert further south. In general the flatland regions of much of northern Africa have been very dry during the past few thousand years compared to the long term average before. The more humid long term history of northern Africa is still well visible in many places in the form of valleys shaped by water flow and extensive lakes which are permanently dry in more recent history. In a way the flooding we can observe now in the desert of northeastern Libya gives us a bit of an idea of how the conditions might have been in the Sahara desert thousands of years ago.

September 12, 2023
by chris
0 comments

Flooding in the Thessalian plain

A few days ago a storm has brought extraordinarily intense rain to some parts of the Eastern Mediterranean, leading to substantial flooding in some regions. I prepared a rendering of a satellite image recorded on September the 10th of the Thessalian plain in Greece, showing the flooding a few days after the event around the course of the Pineios River that i wanted to show here.

You can see both the primary flooding in the western part of the plain around Trikala and the secondary flooding further downstream to the east caused by the huge amounts of water raising the level of the Pineios River, right up to the coast were an impressive sediment plume can be observed in the Mediterranean Sea.

I don’t know how useful that is – but i put up the image in my images for mapping in case it is useful to update OpenStreetMap in the area (like indicating impassable roads). Note, however, that mapping temporary water cover itself in OSM is not a good idea.

September 3, 2023
by chris
3 Comments

A note on OSM-Carto

I like to make another note on OSM-Carto here. Background is that there is another wave of negativity (aiming to find a relatively neutral, yet at least a bit descriptive term here) being expressed by some people from the OSM community on some channels regarding the project.

It is not the first such articulation and it was pretty much to be expected that this would become a recurring pattern. The background i already explained at length back in May 2022. Back around that time i made the decision (and i indicated that in the text) that i would continue trying to do my best to support the idea of OSM-Carto, to be an openly cooperative map design project, governed by consensus of a diverse group of maintainers.

What i do is I provide guidance to contributors how to make changes to the style in a way that complies with the documented goals of the project as i understand them and with the unwritten conventions of the style. I explain the historic context how OSM-Carto came to render things the way it does today, explain how things connect design wise creating a harmonic overall result and provide ideas what possibilities exist to resolve the various technical and design problems the style is faced with. And i listen to arguments that challenge the views i articulate and, when arguments are convincing, re-evaluate these views. At the same time i explicitly avoid standing in the way of consensus forming among the other maintainers even if i disagree with those ideas. And i merge changes when they are suitable for the style in my opinion. What i don’t do is approving changes that i don’t stand behind merely because of pressure from others to have them in the style.

I took this approach despite knowing that the chances of OSM-Carto finding its way back to a sustainable strategic consensus among the maintainers would be very small. The alternative would have been to try essentially staging a coup and to attempt single-handedly reshaping the governance of the project in a way that i would consider more sustainable. Or i could have abandoned the project. This would either have killed the project or it would at least have lead to an end of the diversity in governance. And i was not willing to do either of these things because that would have clearly betrayed the idea of OSM-Carto as described. What i did instead was summarizing some of the lessons learned in terms of governance and social dynamics and some key points that should – from my perspective – be taken into account in case someone would want to bootstrap a new cooperative map design project with a similar paradigm as OSM-Carto. It was unclear if something along these line was going to happen of course (and it still is) or if the OSM-Community is going abandon the idea of consensus based cooperative map design and move back to the more common, more traditional single benevolent dictator paradigm or to something governed by a homogeneous group of like-minded people with common interests. But from today’s perspective i stand firmly by my decision to continue holding up the ideals of OSM-Carto but decidedly not trying to actively shape where OSM community map design is going beyond OSM-Carto while continuing to share my experience and thoughts on the matter with everyone willing to listen – both within OSM-Carto development discussion and on this blog.

The recent wave of negativity towards OSM-Carto is remarkable in the level of resentment that is articulated towards the values of the project, its maintainers and me specifically as the person most outspoken about the project and its problems. I have read quite a bit about and also have seen the effects of people radicalizing themselves in closed groups, reaffirming each other in their preconceptions and simplified friend-foe images. But i had previously never seen this process in a setting that is on public record (mostly – at least one of the most vile comments has been silently removed – not before the author received cheers from his peers though).

The whole thing is also telling in context of the bold claims of the OSMF that the channel in question is moderated with the aim that everyone feels safe and confident to take part. I criticized that idea on ethical grounds in the past but it is visible here that this claim also simply fails to live up to its promise. The level of resentment, bad faith insinuations as well as evidently and ostentatiously false claims (including the title of the topic) is remarkable. Personally i don’t really care – it is not the first time people have articulated this kind of spiteful thoughts about me. But it well demonstrates that the whole top-down moderation idea of the OSMF is an empty promise. And there are plenty of people who – when seeing this kind of conversation happening unchallenged in the OSM community – will think twice before they engage actively in this kind of social context. Note this is not me projecting, other people have articulated that very sentiment to me on quite a few occasions. And the problem here is not the lack of moderators sanctioning people breaking the rules (or weaseling around them as it is also openly done here – mit Ansage as we say in German). The problem is that there is apparently no one left on that channel having the courage to push back and calling out the insults, disrespect and plainly false claims, leading to the mentioned radicalization and reaffirmation of preconceptions in a closed group.

I like to emphasize that the sentiment of people, that when things develop in a bad direction they are trying to identify the reason and who is responsible for it, is understandable and often helpful – but also misguided here as far as assigning blame is concerned. And i think i made it clear back in May 2022 already and i will do so again here: The OSM-Carto maintainers having failed to find back to a consensus direction back in 2018 and the years after – that is our failure and we have to collectively accept responsibility for that. Now you could try to assign individual blame to individual maintainers based on how you assess their efforts to achieve consensus within the group, if you like their ideas and preferences regarding map design work or whether you approve of their style of communication or simply who you can relate to better on a personal level. I would not do that. Keep in mind that the OSM-Carto maintainer team was deliberately put together as a heterogeneous group of people, selected not by like-mindedness and common interests but because of their abilities and the diverse viewpoints and ideas they are able to contribute. This diverse composition of the maintainer team is the very essence of the project that made it unique and was largely responsible for its success. Everyone within this heterogeneous group did what they could within their unique abilities for the success of the project. So we own this collective failure as a group.

What is getting lost – and that is the key point – is that the ultimate problem is not OSM-Carto, it is the lack of anything other than OSM-Carto in the field of openly cooperative map design with an ambition of a balanced global representation. I have been pointing out that this is a problem for many years, practically every time i write about OSM-Carto. I have also pointed out that strategically investing in development of software that facilitates openly cooperative map design on that level is imperative for the OSM community. The most recent plans of the OSMF in the context of map rendering are highly disappointing in that regard. Not a single word, let alone a single cent, is invested there into the question what the needs and requirements of community map design are regarding tools.

September 1, 2023
by chris
0 comments

Map style modularization

I have finally gotten around completing the modularization of my Alternative Colors map style and i want to introduce that here.

I already discussed this briefly in context of the new symbol and label rendering system. Introducing that change depended on splitting project.mml into separate files – one for each layer – which then get combined into a single file for use by carto with a python script. I already sketched the idea that this system could also be used to have different style variants and make some more computationally expensive features of the style optional. But this was still incomplete back then.

What was missing was in particular the option to split and further modularize the lengthy SQL query of the roads layer. I added this now and also implemented the option to turn off

entrance cutouts in line barriers
implicit sidewalks on roads
sophisticated rendering of junctions of roads in ground units
contextualized rendering of fords and mountain passes
preprocesed low zoom waterbody rendering
sophisticated tree rendering
contextualized water barrier rendering

How does it work? layers.yaml defines tags for each of the layers configured and when running assemble_project.py you can select which tags you want to use. For example running

scripts/assemble_project.py -t ac

generates the standard AC-Style setup with all the sophisticated and expensive layers while

scripts/assemble_project.py -t simple

leaves out all the more complex stuff or replaces it with a simplified version.

Within SQL code different variants can be implemented with specific directives. Like:

@@if(ac)
  @@include(roads/sidewalks.sql)
@@else
  @@include(roads/sidewalks-dummy.sql)
@@end

A few notes on the purpose of the AC-Style

Since people are sometimes confused about what the purpose of the AC-Style is and why there is no interactive map based on the style available i wanted to add a few notes on that.

In my last update on the OpenStreetMap-Carto project i explained a bit the history of the AC-Style. After starting as a simple demonstrator of how a more holistic and systematic color design in the style can lead to a more harmonic, better readable and better maintainable map, it changed over time into a much broader test bed for case studies in advanced map design.

What has not changed since the beginning though is that i decidedly do not invest into producing an interactive map based on the style. I would support anyone who is interested in deploying a map based on the AC-Style with advice and if necessary adjustments to the style. But i don’t want to spend my time on doing something that i am not good at and administering a map deployment is not my field of expertise.

Why do i not actively try to get any of the developments from the AC-Style into OSM-Carto? Because for many of the core design changes there is no consensus support among the OSM-Carto maintainers. Joseph Eisenberg invested quite a lot of time and energy to get some of the fundamental color changes i developed into OSM-Carto – but without success. And many of the other changes in the AC-Style design wise depend on these changes. There are still some improvements and feature additions that could be integrated and some actually are – at times with modifications, like the allotments rendering or the ridge/arete patterns.

Do i think it would be good to replace OSM-Carto in deployments with the AC-Style? Hell no. OSM-Carto is foremost a cooperative community project where decisions are made through consensus of a fairly diverse team of maintainers. As i have pointed out in the previously mentioned update on OSM-Carto the project has been struggling with the practical difficulties of this approach for some time (for reasons i discussed there in more depth). But any project run by a single person – no matter how benevolent they are – is not a suitable substitute.

As i have pointed out in the past the OSM-Community could very much use a larger variety of independently developed map styles. Quite a bit of progress has been made on that front in the past years, but unfortunately exclusively on styles targeting the needs and wishes of specific local communities and mostly under control of a single decision maker. Nothing is in sight so far even remotely aiming to create a map style for a broad global audience in a similar fashion as OSM-Carto though.

Would it in my eyes makes sense if someone was to start off a new cooperative map design project with such a scope to base it on the AC-Style rather than OSM-Carto? Design wise a strong argument could be made for that but i would still advise against it because of the technical complexity and the massive hurdle that would pose to potential contributors. What might be worth considering though is to have some of the fundamental design changes from the AC-Style, that – as explained – do not have consensus support in OSM-Carto right now, from the start in such a project because it would make further design work so much easier.