Imagico.de

blog

OpenStreetMap-Carto – an update on the project

May 20, 2022
by chris
13 Comments

OpenStreetMap-Carto – an update on the project

It has been more than 4.5 years since i wrote more in depth about OpenStreetMap-Carto, the map style behind the map on openstreetmap.org and in many ways the face of OpenStreetMap.

I have been contemplating writing an update on this for more than a year now, but i have been hesitant because the conclusions i am going to draw in the end are pretty dire and i had still hoped for a turn to the better. And writing a blog post here with fairly negative conclusions seemed to me to be something that could further contribute to seal the fate of the project.

The history of the project

I will start by roughly sketching the history of OpenStreetMap-Carto. For further illustration of that i also produced a comparison gallery of how different versions of OSM-Carto render the current OSM data in various places around the world. The project began in 2012 with Andy Allan porting the old Mapnik XML based style, that had powered the main map of OpenStreetMap so far, to CartoCSS. CartoCSS was one of the most significant innovations in rule-based digital map design as it was the first (and – depending on your perspective – might still be the only) format for describing rule-based map design that was developed with primarily the map designer in mind and not the comfort and liking of the tool developers (as it was the case for Mapnik XML and is for example for JSON based formats that became popular later).

Rendering sample from version 1 of OSM-Carto (December 2012)

Rendering sample from version 1 of OSM-Carto (December 2012)

Andy’s contribution was not just the hand-work of formally porting the style, he also did the important ground work to determine how the CartoCSS based framework would need to be used to scale to the level of complexity the style already had back then.

The port to CartoCSS and the availability of TileMill as a fairly easy to use tool to develop CartoCSS styles encouraged quite a few people to newly get into map style development. Most of the initial work was technical restructuring. Technically the style gained its current shape, in particular with the structure of the roads layers, during these first years. Feature additions were also made to some extent, but the overall appearance and design initially stayed very close to the Mapnik XML style.

Originally Andy Allan managed everything on the project himself, but after some time Matthijs Melissen, Paul Norman and Mateusz Konieczny were added as maintainers who could also merge changes and the system of consensus decisions of the maintainers was established. Later, in 2016, Daniel Koć, myself and Michael Glanznig were appointed as additional maintainers, in 2017 also Lukas Sommer and in 2019 Joseph Eisenberg. Of these only Michael Glanznig has formally retired since, but both Andy Allan and Matthijs Melissen have completely withdrawn from active participation in the project now – leaving five more or less active maintainers.

2015 and 2016 were kind of the golden years of the project. With the interest raised and the better accessibility of style development through CartoCSS, quite a few people by that time had gained enough experience to tackle bigger design challenges the project’s development and the needs of the OSM community created. This in particular refers to the style OpenStreetMap-Carto was based on being strongly connected design wise to the British origin of the project. During the time from 2015 to 2016 the style essentially emancipated itself from this relatively narrow cultural origin and became a truly international project. Components of this emancipation were in particular:

  • Refactoring of urban landuse and building coloring as the first step into systematic color design in OSM-Carto.
  • The roads rendering redesign developed by Mateusz as part of a GSOC project – departing from the traditional British color scheme and adjusting road rendering for good readability in a wide variety of settings world wide.
  • Moving to a standardized design of POI symbols for most features rendered with point symbols – work that has in particular been pursued by Michael.
  • Introducing algorithm generated relaxed random periodic patterns for depicting a larger variety of landcover types beyond what is possible with just fill colors – paired with systematic and holistic design of the area fill color scheme in the style. This in particular facilitated the introduction of a significant number of new landcover types that are rare in areas where historically OpenStreetMap first became popular but widespread in other parts of the world – bare ground landcovers, various wetland types and reefs.

These accomplishments in making the style more international and serving the potential global community in a more balanced fashion were made possible through various technical and design innovations. For this short period OpenStreetMap-Carto was truly avant-garde in the development of rule-based map design.

Rendering sample from version 3 of OSM-Carto (December 2016)

Rendering sample from version 3 of OSM-Carto (December 2016)

What i especially think we have reason to be proud of is that we agreed back then on a document outlining uniquely progressive fundamental goals of the project. In particular the idea that we aim to create a map style for the potential global map user and no special consideration is given to the current geographic distribution of actual map use is a fairly revolutionary concept that has, to my knowledge, never be formulated in such clarity anywhere else. And it was not only a goal on paper, we seriously tried to work towards this goal as i hope the list of changes above illustrates somewhat.

But this quite amazing development came at a price:

  • we were not perfect and while we introduced many significant improvements, there were also things that got lost on the way. Noteworthy is in particular that the depiction of paths for non-automobile transport was significantly better overall in the old Mapnik XML style. The look back at what was lost after changes have been made is something that was and still is done much too rarely.
  • we outgrew the possibilities of the technical framework we were using. While initially what could be and what was done in OSM-Carto was mostly limited by our collective limits in knowledge and experience of how the possibilities our toolchain offered could be efficiently used to create a better map, developers soon mastered these possibilities and reached the limit of what was easily possible. At the same time active development of Mapnik and Carto slowed and ultimately mostly stopped. The options offered by CartoCSS in combination with PostGIS are still manifold but with no higher level abstractions in the tools using those amounts to adding significant levels of complexity on the style level or in form of additional tools and working with script generated CartoCSS/SQL code to integrate those into the rendering framework.
  • the move to become a more global and internationally focused map style was not really that popular. Quite a few people in the OSM community who – coming from regions traditionally influential in OSM, in particular urban mappers from the UK, Europe and North America – were naturally expecting OSM-Carto to primarily cater their interests. Some of them were disappointed by OSM-Carto having a different focus and not quickly supporting the latest trends in urban mapping when they started becoming popular in these regions. Others disliked the fact that OSM-Carto was so distinctly and ostentatiously different from mainstream commercial maps like those from Google and Mapbox and would have very much preferred us to align map design more to what they were used to from these maps.
  • most importantly: This development meant that the skills required to productively work on improving the style were gradually increasing. That in particular meant that for new contributors it became more and more difficult to make substantial changes to the style without first becoming familiar with the design principles of the project.

These developments led to increasing divergence in the maintainer team about the future direction of the project. I discussed this part of the history of OSM-Carto in more depth in my previous blog posts on the project here. In April 2017 the decision was made to give the individual maintainers more autonomy and we maintained this modus of operations until late 2018. Joseph summarized this nicely here.

The hope was that by giving maintainers autonomy in making decisions we could avoid the impasse we had reached on many strategic questions and get to a sustainable dynamic development of the style again. Unfortunately that did not happen. What happened instead was that development got dominated by a relatively small group of people who developed primarily feature additions and design adjustments of interest from an urban European perspective that were often at odds with the overall design paradigm and the goals of the project as they had been pursued before. At the same time, none of the big challenges the style faced, many of which were visible already in 2016, were actually addressed. Many of the developers and maintainers active before who would have been qualified to work on bigger changes, withdrew from active development of OSM-Carto. We have discussed these dynamics on the issue tracker and those were among the main reasons why we moved back to the consensus principle after about 1.5 years.

During this time the style moved significantly away from a consensus position that all active maintainers could support. This created the massive challenge to find a way back to a consensus among the project maintainers about the strategic direction of the style once the consensus principle was re-established. And despite significant efforts, in particular by Joseph, we did not manage to do so. To this date there is no strategic consensus among the maintainers and design wise development is mostly stalled. We have managed to make a number of technical changes – including big ones again but design wise no agreement can be found on most larger changes and even reverting changes that were made without consensus does not find agreement.

This continues to be the situation to this date. Some maintainers (including myself) continue to review changes that are being suggested but releases have become very sporadic and usually contain little of substance in terms of visible changes. I try to – even when i disapprove of a change – indicate that i would be fine with the other maintainers accepting the change none the less if they agree on it, this way actively supporting a more robust form of consensus. But that is no substitute for an actual common goal and a strategic direction we agree on. And most capable style developers have stopped contributing to the project because the lack of a clear overall direction does not provide for a supportive environment to work on good map design.

Rendering sample from the current master branch of OSM-Carto

Rendering sample from the current master branch of OSM-Carto

What went wrong?

It would, of course, be easy to now point back to the decision to abolish the consensus principle in 2017 and say: This is the wrong decision from which on everything went wrong. And i could even say i warned about the potential side effect of the project loosing the common sense for the strategic direction back then. But i don’t think this would be a complete assessment of the situation.

The three main strategic directions map styles can lean towards and at the same time the three necessary components of any sustainable map design endeavor

The three main strategic directions map styles can lean towards and at the same time the three necessary components of any sustainable map design endeavor

From early in its history OSM-Carto was burdened with the fact that it was the only community map design project in the OSM community with a global scope and that did not have a specialized thematic focus. As a result, a lot of people projected their ideas, interests and needs onto the project. If you try to find some structure in these diverse influences you can in my eyes identify three main strategic directions that the style has been pushed in:

  • The populist direction. During the 1.5 years when consensus requirements were lifted and maintainers could make decisions autonomously in OSM-Carto, we pretty much followed this approach in purity. It means focus in development is on small, atomic changes, made and decided on without a larger overall design strategy and essentially allowing anyone who would like to change something to do so, at least as long as it has substantial popular support from people with a loud voice. This has the tendency to lead towards what i, in my previous blog post about OSM-Carto, called a naive art like style. We have already seen that in the long term this is non-sustainable because it fails to consider the need for larger and systematic changes to maintain the technical viability of the project. And it also fails to acknowledge that the usability of a map depends on an overall design concept and strategy that is being consistently followed. But that does not mean it is not an attractive short term model for developing a map style. In case of an OSM map style influential on mappers, there is a strong risk that over time mappers will start and will need to start compensating for the deficits of such a direction by essentially hand painting the map through their mapping work. However, in moderation (meaning in particular to mitigate the imbalance resulting from the loud voices appearing to represent the popular opinion) popular involvement is an essential component both for evaluating design and for recruiting talented and qualified people in development, in particular in case of a cooperative community project like OSM-Carto.
  • The technocratic direction. In purity this means essentially treating style development as software development. The style is treated as a software to process the map data into a graphical representation. You try to do so efficiently – both in terms of run time performance and w.r.t. code complexity. But you pay no significant attention to the non-technical requirements there are for the resulting map. Many digital map design projects work close to this model, they are managed by software engineers with little background and little ambition in actual map design. This model has one major flaw: Map style development is not software development. As a result, such maps at best play a bit clumsily with design possibilities. And the resulting map is usually imbalanced, overly abstract and detached from the geographic reality it tries to represent. On the other hand, since digital maps are powered by technology, the technical expertise of software developers is of course a crucial component for the long term sustainability of any map style.
  • The artistic direction. This direction means focusing on the design component of the map and being guided by the pursuit of excellence in this field. Artistic here is not meant in the sense of pure art but to contrast map design work based on an understanding of map viewing situation and a competent use of colors and visual structures in light of this with the purely technical approach. There are very few map styles in the field of automated rule-based digital cartography so far that put a strong and sustained emphasis on this direction, largely because map designers with artistic ambition still predominantly design maps through manual processing of concrete data. For me this has always been the most interesting direction, exactly because it is a field in its nascence with lots of unexplored territory. In purity the pursuit of this direction would likely lead to highly sophisticated maps which are technically inefficient, difficult to maintain and which constantly struggle with the technical limitations of the map design frameworks they use. But this is also the direction where there is clearly the most unused potential, both in terms of actual map design, as well as in terms of talented and capable designers who could be recruited.

These three paradigms are fundamentally contradicting each other and it seems that map development projects have the tendency to concentrate on one or two of these focus directions and neglect the other(s), because balancing all three directions is difficult and takes a lot of effort. At the same time, as i tried to explain, neglecting one or two of these directions will, in the long term, not be sustainable because all three components are essential.

The core of the problem

Until 2016 we essentially managed to unite these three principal directions in a common style, because we made use of the significant options the technical framework we used and our own expanding capabilities and skills in map design gave us. This reservoir of capabilities made it possible to bridge the gaps between these, otherwise, largely incompatible ideas. But we were already reaching the limits of this development in 2016 as hinted above – both in terms of the technical framework, the capabilities of which we started to outgrow, and in terms of the human design skills as a factor (which made it increasingly difficult to find and recruit talented and capable people as contributors and to educate them in the specific needs of the style).

What we would have needed back in 2016 and also now (though with the baggage of the history of the past years this seems rather unrealistic) is a new effort of consolidation similar to what has happened at the beginning of OSM-Carto. That would in particular mean tapping and developing new technical possibilities, because – as mentioned above – we already outgrew the possibilities of the technical framework we were using back in 2016. This is challenging because most of the components of the framework OSM-Carto uses are not actively maintained any more and there is no open source rendering system on the horizon that could replace it. Therefore i have suggested for quite some time that it is essential for the OSM community to strategically invest into map rendering systems that can produce high rendering quality. Rendering systems suitable in terms of the map styling interfaces, to be used in cooperative community projects, and that scale well towards higher sophistication in map design. This would be paramount to ensure the project’s ability to actively work on and substantially participate in the discourse on graphical representation of OpenStreetMap data in maps.

Where things might go in the future

Back in 2017 when OSM-Carto moved away from consensus based decision making i started the alternative colors style as a fork of OSM-Carto. Initially it was meant purely to demonstrate how the design principles we had developed in OSM-Carto until 2016 could continue to serve as a basis for a balanced map design for a truly global audience. But over the years it became increasingly clear to me that OSM-Carto would likely not find its way back to a balance between the three main strategic directions. And it also became clear that beyond OSM-Carto, community cartography in OpenStreetMap would most likely focus on the technocratic and populist directions. Businesses and non-commercial organizations around OSM produce maps with a predominantly technocratic direction. The OpenStreetmap Foundation has, for some time now, indicated an interest to actively bootstrap new community map design projects – and the OSMF is dominated by people with technical backgrounds. The OSMF board as the main decision making body even consists exclusively of people with a technical IT background. To mitigate this deficit they will likely try to give strong voices from the mapper community a bit of a say in these projects. But no one active in the OSMF has in the past ever indicated to value competence and experience in the artistic aspects of map design.

Long story short – in light of it being highly likely that future OSM community map design will predominantly be negotiated between the populist and technocratic direction, i increasingly started to try to explore the potential of the artistic direction. I don’t have the capacity or the skills to really consolidate the technical basis of map development and to develop map rendering technology that would give map design truly the options to move to the next level as i sketched it to be the desirable future for OSM-Carto above. Hence, much of what i implemented in the alternative colors style is technically somewhat awkward – most ostentatiously visible in the >1500 lines long SQL query for the new roads rendering system. But working on such case studies in advanced map design is an important and necessary step i think to determine what capabilities a future map design framework would need to have and how it would best be structured to truly provide a suitable environment for a balanced map style to flourish in a similar way as CartoCSS and Mapnik did for OSM-Carto in 2015 and 2016.

Rendering sample from the alternative-colors style

Rendering sample from the alternative-colors style

If the OSM community will be in any way actively involved in shaping a future of map design along these lines and in a balance of all the three main directions i described above, is an open question, but as already indicated i consider the chances for this to be quite small. OSM-Carto is still the most likely basis on which this might happen because it already provides a huge amount of preexisting ground work for this. Starting a new project from scratch would have its benefits as well, but the amount of basic, yet careful and considerate design work that would be necessary to bring such an endeavor off the ground would be quite massive.

The other question is if the future of cooperative digital rule-based map design will be based on free open source software or not. Considering how dominant open source is in automatically rendered digital map production these days, this might seem a silly question. But as mentioned above, map design with significant artistic ambition is still predominantly based on manual processing of concrete data and not designing rules for processing generic data these days and, in that field, open source software only plays a marginal role so far. If the FOSS community and the OSM community do not strategically invest in this direction, it is very likely that commercial software companies and service providers will – based on the good standing they already have with map designers with a more artistic and less technical orientation – successfully try to dominate that field.

Reflections on the social limits of community map design

In my previous blog posts on OSM-Carto i discussed quite a bit about the social dynamics of cooperative community map design and pointed out that OSM-Carto is also a social experiment testing if cooperative map design on this scale is possible and sustainable. It becomes increasingly clear now that this experiment has failed. But does this mean that a cooperative map design endeavor that is not either based on a centralized authoritarian leadership or on a small, culturally homogeneous team of people with common interests cannot work? That is a possibility. But not the only one i think. Above i listed what i identified as a number of factors that had led in 2016 to an increasing divergence in the maintainer team about the future direction of the project. None of these factors is however inevitably making such a cooperative effort unfeasible.

The key points that i would identify to be essential for a future experiment in a similar direction, to create a balanced general purpose OpenStreetMap map style for a truly global audience through open community cooperation, to work would be:

  • Bootstraping such a project is a step of paramount importance. In OSM-Carto we had a reasonably diverse group of developers at least roughly covering all the three main directions i consider essential for a well balanced and sustainable map. We managed to codify the key ideas and values of the project into a guiding document but the agreement on these common ideas and values did not last long enough for the project to assemble a developer community invested enough into these values under the more adverse conditions the project was going to meet in the coming years.
  • Such a project needs long term strategic support from an active software development community invested in supporting map design not only as a technocratic endeavor but that also values and supports specifically the artistic direction. If a map design project outgrows the technical framework it is based on, like OSM-Carto increasingly did in 2016, this is a huge problem. Not only because it technically makes it harder to implement design wise necessary or desired changes, but also because it makes contributing increasingly unattractive for the most qualified designers who strive for excellence in design. A map design project where the appearance of the map is not primarily determined by the conscious decision of designers about how they want the map to look, but by the limitations of the software used, is destined for mediocrity and is never going to be a balanced map well serving its design goals.
  • Significant effort needs to be put into writing down practical guidance and providing educative materials and training regarding the practical design rules and principles followed by the project. This is especially important as the field of automated rule based map design is so young and so little material discussing this topic is available at all.

I would consider it likely that if these points can be addressed and followed diligently from the start, a map design project like OSM-Carto can work in the long term. But it is definitely hard and failure is likely. One of the key challenges OSM-Carto struggled with and that any project with similar ambitions will struggle with in the near future, is that it is essentially in uncharted waters not only as a social endeavor, like i discussed previously, but also in terms of map design itself. This is both a huge chance and a burden.

March 30, 2022
by chris
12 Comments

OSM related group communication channels and platforms – revisited

The OpenStreetMap foundation has rolled out their new web based communication platform – which has long been anticipated and which i have mentioned in passing in my yearly OSMF review already.

I want to take this opportunity to update my previous review of OSM related group communication channels and platforms and add a few subjective impressions and comments afterwards.

After having essentially stopped using OSMF communication infrastructure actively in response to the OSMF board establishing a highly questionable behavior control regime and indicating the intention to extent this to all OSMF controlled communication channels, i view this pretty much from an outside perspective now.

But as said – lets start with the basics. Here is the extended table of the different communication platforms and their evaluation according to the different criteria i introduced in the first part.

OSM related group communication channels

I differentiated the open platforms and communication protocols category further into:

  • Free open source software: This documents the basic question if the software used is formally licensed under an open license.
  • Open protocols: The question here is if the platform can be interfaced with from the outside (both technically and legally) to allow exchange of content with other platforms or alternative interfaces.

I think it is important to separate these two – the latter is also possible without the platform itself being open source and a communication platform based on open source software does not automatically provide open protocols that can be used from the outside to access or to enter communication content (which is what seems to be the situation with the new OSMF platform – see below).

Here the updated footnotes for the table:

  1. help.openstreetmap.org has limited support for structured communication.
  2. The OSM wiki in principle allows structured communication and on talk pages it is established practice to thread messages. But you have to do this fully by hand and it only works if everyone does so diligently. No builtin support for that.
  3. On Usenet server operators used to expire messages so there was practically no permanent public record in the system itself. However everyone running a Usenet server could maintain a record and in the late days of Usenet Google practically served as a reference record.
  4. The forum indicates when a post has been modified after initial posting but no record of the original message is retained. Also admins on the forum can (and do) remove messages from the record (with varying frequency, depending on the channel).
  5. The diaries themselves can be edited without the history being visible but the diary comments cannot.
  6. On the wiki the edit history is openly accessible so it serves as an immutable record (though it can be difficult to use as such practically).
  7. Github has a fairly clever system of allowing edits of communication but still allowing users to transparently see if there have been changes and what has been changed. Github however also allows completely removing messages from the public record in some cases (but it is indicated at least that a message has been removed i think).
  8. While there is no alternative interface to posting changeset comments for reading there is the popular tool by Pascal Neis.
  9. Openness in participation in principle applies to most OSM related mailing lists and forums. There are a few exceptions – specifically osmf-talk is only open for posting to OSMF members. There are thematic and regional lists and forums with culture specific rulesets you need to accept for active participation which are usually developed through consensus of those active on the individual channel (like diversity-talk, regional forums). Beyond that the OSMF has for all communication channels and platforms they operate indicated the intention to impose a behavior control system that is designed to allow banning participants if they misbehave in the eyes of the moderators. To what extent that practically limits the de facto openness is not clear so far.
  10. See 9 – plus Discourse seems to implement an automatically managed hierarchy of users based on their formal activity history on the platform. In addition there seems to be a not publicly documented number of people selected from the top so to speak who have special privileges in editing/removing communication content of others. So the system is essentially open only on the base level and has an elaborate hierarchy built on top of that. If people are also manually promoted/demoted within the – on the lower ranks – by default automatically managed hierarchy is not clear to me yet.
  11. Discourse formally seems to record communication structure in the sense that it records which message a message is in reply to if it is explicitly made in reply to one. This is however not consistently displayed. I have seen messages being shown both as a reply to another message as well as being shown a second time as a flat comment to a topic. In general the whole interface seems to strongly discourage replying to the messages of someone else in the sense of structured communication and to nudge people to preferably add comments in a linear fashion to a topic.
  12. Discourse seems mostly like the forum in this regard (see 4. above) but in addition seems to allow you to see the edit history of messages. There seem to be also automated edits of messages made by the software – which are displayed just like human edits. It is unclear to me so far if message removals by moderators are transparently visible or not (i.e. if there is an indication that here a message by X has been removed). Update: It seems that moderator action is not generally publicly visible as such.
  13. From what i have seen Discourse has an API but that is not openly accessible – not even for read-only purposes – but requires an API key from the platform operators. The OSMF has so far not indicated if they are willing to provide access to that to anyone. I have not seen any external clients that interface with discourse in some form based on that API or otherwise. In particular there does not seem to be any workable mechanism for bulk access to the communication content. There are secondary channels (RSS feeds and the so called mailing list mode) that allow limited access to some of the communication content but they seem to be there mostly pro forma (to be able to advertise those as features) and are practically essentially unusable (Update: more details in comment below).
  14. There seem to be some very limited filtering options (like blocking individual users) for logged in users w.r.t. notifications. The message reading interface however does not seem to have any way for users to filter things (although the platform itself performs filtering on its own – deciding based on unclear criteria which messages are shown and which are hidden by default).

Another remarkable data point by the way: The starting page of commmunity.openstreetmap.org loads an impressive 5.4 MB of content to then display what amounts to less than a hundred words. For comparison: The OpenStreetMap forum starting page (which is much more information rich of course) is less than 100 kB.

Personal impressions

When i am in need to choose a software or platform for a certain purpose my primary consideration is usually: Does it allow me to do the things i want to in the way i want to do them without the need for me to invest time and resources in things that are not of interest for me. To give an example from the field of group communication: In case of image content as part of communication and graphical formatting of text: I would want the communication tool to show me the fact that my communication partner has communicated such content but i don’t want it to show me such stuff without me explicitly choosing to see it. In other words: I want to be able to make granulated decisions on how much attention i invest in a specific communication.

The central paradigm of Discourse seems to be pretty much the opposite of that: Active management of the user’s attention. Like:

  • Suggested Topics – who is suggesting that to me and why? I have not asked for suggestions of any kind.
  • Views, likes etc. – apparently someone thinks it is of tremendous importance for me to know how many other people have read or indicate to have liked what i am reading so they rub that into my face prominently and this way distract from the actual communication content i am reading. Too bad that i don’t care about this kind of data.
  • It seems it is also of high importance that i always associate the identity of my communication partner with some picture of their choosing. Tough luck if memorizing mini-pictures is not really your prime talent – because you then have to deal with the between one and three verbal identifiers which are displayed in various tones of gray and weight in a way that is barely discernible from the communication content.
  • Hello! Looks like you’re enjoying the discussion, but you haven’t signed up for an account yet. Seriously? I am just waiting for clippy coming around the corner now…

I could go on with a long list of similar things i find appalling when trying out the new platform but i will cut it short. Discourse is evidently not for me. My impression is that the target user group of that platform is people that have grown up in the attention economy, using twitter, facebook and other social media and feel comfortable with those.

Conclusions

And i think that is fine. As indicated in the past (and others see this similarly) i am convinced that a diverse project like OpenStreetMap needs diverse communication platforms. People who are used to twitter and facebook and like the communication style there should have the opportunity to communicate in a similar style about OpenStreetMap and preferably using open source tools and without the need for people to participate in that communication to sign up with some big corporation and agreeing on their data being sold for profits.

What i see critically – and i have said that in the past as well – is, that there are strong voices in the OSMF that do not just want to offer this new communication platform as an option for those who like this style of platform but who want to unify and to culturally homogenize all OpenStreetMap related communication under this. Kind of the wild dream of all community management. It is very likely that this would fail of course but there is a significant risk that the failure would not be visible from the inside perspective and that at some point in the future people on community.openstreetmap.org might predominantly believe that they are the community (or at least that they are representative for the whole OSM community in all their diversity).

Discourse belongs into the category that i have described in the previous post on group communication channels as stuff developed around some fancy web interface meant to serve the fashion of the day in UI design. It is very likely that this will – like countless other similar platforms in the past – run out of fashion relatively soon again (in like 5-10 years) and will, as a software, likely become unmaintained soon after that. In other words: It could well be that the OSMF will in 10-15 years, possibly sooner, be in more or less exactly the same situation it is right now with the forum and help.openstreetmap.org running old and unmaintained software – just with the software then being discourse. The irony could well be that mailing lists still exist then…

We have a saying in German: Die Zeiten sind hart, aber modern.

March 25, 2022
by chris
0 comments

The Comprehensive Optical Mosaic of the Antarctic (COMA)

I am happy to announce my newest satellite image product – The Comprehensive Optical Mosaic of the Antarctic (COMA).

The Comprehensive Optical Mosaic of the Antarctic (COMA)

The Comprehensive Optical Mosaic of the Antarctic (COMA) – large version shows blended ocean depiction from Green Marble 2.1 (Sentinel-3 data)

The COMA is the result of an evolution of the satellite image mosaic production techniques i have been improving for over a decade now with various images in the resolution range from 10m to 15m having been published over the years – many of which are still the highest quality images available for the areas in question. For this mosaic i worked in particular on the scalability of the whole process towards a larger number of source images and a larger size of the assembled mosaic.

I chose the attribute comprehensive to indicate that this image does not only cover all land and permanent ice of the Antarctic but that it also uses a large fraction of the more recent open data imagery available to ensure the highest quality coverage in the Antarctic interior. Most images of the Antarctic you can see on map services and otherwise these days are still based on the >20 year old LIMA – which, due to the limitations of Landsat 7 imagery (from which LIMA is created), due to the low volume of source data and because the assembly process used is not really suitable for very high quality visualizations. Some image users have patched this up locally with newer images – which then however typically results in color inconsistencies.

Antarctic Peninsula

Antarctic Peninsula

Koettlitz Glacier and Royal Society Range

Koettlitz Glacier and Royal Society Range

The COMA is the first fully new visual color image of the whole continent created after LIMA and a massive step up in terms of quality in every aspect. It is predominantly based on Landsat 8 data, supplemented by ASTER and MODIS images. And it is essentially only now, about 8 years after the launch of Landsat 8, that a sufficient volume of image data is available to produce a mosaic of the whole continent at this quality level.

I’d like to emphasize in particular that the image mosaic is on the same high quality level regarding the lack of clouds as my other high resolution mosaics with – conservatively estimated – less than between one in 10k and one in 100k pixels being significantly cloud affected. This is way better than LIMA or any other Antarctic mosaic available today. The Antarctic is not a particularly cloudy region but it is fairly nasty in that regard because of the nature of the clouds common in the Antarctic and because clouds over snow are notoriously hard to detect reliably in general.

Snow drift and thin clouds on the polar plateau in a strongly contrast emphasized depiction

Snow drift and thin clouds on the polar plateau in a strongly contrast emphasized depiction – from raw image, not in mosaic

It is also worth mentioning that clouds are not the only transient atmospheric phenomenon that affects clarity of satellite images in the Antarctic. Wind blown snow is also a frequent problem. You could even say that on the Antarctic plateau the boundary between ground and atmosphere tends to be somewhat blurry and this makes recording a consistent image of the surface a challenge. In the production of COMA significant care was taken to analyze images and make sure to preferably use those which offer a clear view of the surface free of both clouds and snow drift.

Ice structure visible after strong contrast expansion

Ice structure visible after strong contrast expansion

Some might wonder why i have not used any Sentinel-2 data in this mosaic. The reason is that Sentinel-2 coverage of the Antarctic is very incomplete in terms of spatial coverage and also very infrequent almost everywhere. Using the limited data that is available would have required me in many cases to choose between Sentinel-2 data (for the slightly higher resolution) and Landsat data (for better quality or more recent coverage).

You can have a look at the detailed product description and the sample images over at services.imagico.de. Those interested in using the mosaic are welcome to contact me for details.

The Sør Rondane Mountains

The Sør Rondane Mountains

Denman Glacier, high contrast tone mapping

Denman Glacier, high contrast tone mapping

James Ross Island and Prince Gustav Channel

James Ross Island and Prince Gustav Channel, local contrast enhanced tone mapping

TagDoc logo

March 7, 2022
by chris
11 Comments

Introducing the TagDoc project

This post is also published on TagDoc – you can read it there as well if you prefer.

To many, even to some experienced OSM community members, the free tagging system of OpenStreetMap appears chaotic, unorganized and inefficient. And it is to some extent. However, this system is at the same time also highly functional and fulfills a central role within the project and the OSM community. The alternative to the free, de-centrally developed tagging system we have, would be a centralized, authoritative system of attributes and rules, developed and imposed by some centralized bureaucracy, which would have to be a culturally fairly homogeneous group of people to function as such. In other words: The alternative to the open tagging system, would be for OpenStreetMap to give up the idea of creating and maintaining a map by the people, for the people, based on local knowledge and egalitarian self determined cooperation, and to adopt exactly the kind of methodology centralized mapping authorities use – whose dominance and flaws OpenStreetMap was created to overcome.

One big reason why despite this an increasing number of both data users and mappers are becoming more and more skeptical of free and open tagging, and frequently push with – for example – organized/automated edits or by introducing seemingly authoritative formulations into tagging proposal processes for a more centralized and more authoritative approach to tagging decisions, seems to be the highly dissatisfying status of documentation of the free tagging system we have in OpenStreetMap.

The traditional platform for documenting tagging practice in OpenStreetMap is the OpenStreetMap Wiki. The basic idea is that mappers are free to use any tags they want to use, and they are encouraged to document how they use the tags on the OSM wiki. This has, over the years, led to a highly valuable body of documentation of tags and their use. But – with the tagging documentation on the wiki growing both in size and in importance for the OSM community – it has also become an attractive platform for people who are not just interested in documenting how they use the tags on an equal level with everyone else documenting their tag use, but for the pursuit of various subjective ideas on how the meaning of tags should be presented, irrespective of how the tags are de facto used in the database. As a result, nowadays there is no consensus within the OSM community if the tagging documentation on the wiki is to be descriptive (documenting how tags are practically used) or prescriptive (documenting how tags should be used according to some subjective opinion or some perceived authority). There is not even agreement that these two aspects should be separated. And while – as said – there is a lot of valuable information on the wiki about tags, there is at the same time also a lot of nonsense put there because it suits someone’s agenda. Additionally, the two are often indiscernible and you can only differentiate reliably between them if you have significant up-front knowledge about the tag in question – and in that case you don’t really need the wiki that much.

For data users and software developers, but also for mappers, this is a huge problem because they – due to the lack of alternatives – rely on the wiki as a source of information, a source which, however, is notoriously unreliable. And this is a self emphasizing problem because the more people rely on the wiki, the more attractive it becomes as a vehicle for people with an interest to influence data users, software developers or mappers to introduce ideas into the wiki documentation that do not factually document the de facto meaning of tags, but communicate some subjective ideas on how things should be mapped or how data should be interpreted.

Of course this, in principle, is not a new problem and for a long time people active in OpenStreetMap have discussed various ideas on how to address it. But so far nothing of substance has come out of that. I have also for a long time shied away from working on this – partly due to the sheer size of the task, partly however also because i see a lot of value in the idea of free tagging without centralized rules being combined with documentation of said tagging being developed through equally open cooperation. But more recently i came to realize that the direction the tagging documentation on the OSM wiki is heading, and the dysfunctional social dynamics you can widely observe there, are likely more damaging than supportive for the core idea of OpenStreetMap, especially in light of the fact that the tagging documentation on the OSM wiki is, both de facto and in the perception of people, without alternative. And it has become clear to me that what works for mapping itself – the egalitarian cooperation across language and culture barriers by communicating through the act of mapping itself – does not work likewise for developing a language based documentation of mapping practice.

The idea of TagDoc

What is the solution to this problem? I don’t really know. There might not even be a real solution and OpenStreetMap will need to deal with that. But given how little of substance has been tried to address the problem of reliable and competent documentation of the use of tags in OpenStreetMap, there are ways to substantially improve on that compared to the status quo. The difficulty is how to approach this problem in a way that has a good chance for success. What i have come up with as a concept, after quite a lot of contemplation, is the following:

  • Limiting the scope to documenting the de facto meaning of tags, that is describing how they are actually used in the OSM database. If there is a need for a prescriptive tagging documentation it is probably something that could be discussed but it is beyond doubt, i think, that any serious attempt at developing prescriptive tagging instructions (be that as text or in the form of tagging presets in editors for example) would have as a prerequisite solid knowledge of the de facto meaning of tags as they are used so far. Much of the problem of the current situation stems from the fact that this does not exist.
  • Writing with the aim to be useful and of value for the data user. The reason for that is twofold – First: Because i think data users are in most serious need of a trustworthy and competent documentation of tags in OpenStreetMap and where the dysfunctional nature of the tagging documentation on the OSM wiki creates the biggest issues. Second: While i hope that TagDoc is also of interest for mappers, my idea is that the agreement among mappers on the use of tags should continue to primarily develop through the cooperative mapping process itself (see also What Tags are). The influence of any single other factor on mappers can, if overly large, lead to imbalance and become a problem over time.
  • Written and curated by proven and independent experts, open to scrutiny by the whole community. I know that the idea of meritocracy has fallen out of fashion in significant parts of the OSM community because it is considered to be unjust. And, as indicated, i have myself for a long time valued the open collaboration as a principle for documenting tagging. Over the last years this has however proven not to be able to produce the qualified and reliable documentation needed, at least not in the current and foreseeable future social environment of OpenStreetMap. And the alternative – a committee of political appointees – does not, to put it mildly, have a very good track record in curating intellectual writing.

The plan

As written on the starting page at the moment with me alone writing for TagDoc in my spare time is not in any way sustainable. There are mainly two options how this could be solved:

  • If people (in particular data users) are willing to support this idea and see value in the project as i sketched it above and are therefore willing to finance me working on it, i could extend the amount of time i can invest in this work. Of course this opens up the question of in how far my financiers might influence the content of TagDoc and this way we might solve the problem of the unreliable information on tags from the OSM wiki being the only source of information available, but only with the other source of information being curated by financial interests to their liking. What i can say to that is that (a) i have shown in the past i think on plenty of occasions that with my public writing i tend to not pay much regard to what views are economically opportune, (b) that i would be transparent about where i receive funding for writing and editing for TagDoc from and (c) that any financier of my work on TagDoc would be aware up-front about the basic premises of the project as outlined above which are fundamentally at conflict with the idea of pushing certain subjective views on tagging. What could happen of course is that financiers influence what tags i write about and analyze with priority and in particular detail. That however is already the case right now – what tags i know most about and write about with priority is evidently not independent of what kind of OSM data i work on as part of my paid work. I am not offering this option because i really need additional paid work per se but because i see a strong need for this project in the OSM community and from data users in particular, and i would be willing to reduce other parts of my paid work in favor of this in case there is interest in financing that.
  • If other independent experts are interested in contributing to the project under the premises described above, i would be willing to open the project to other authors. So far i have not put the content of TagDoc under an open license but i would be open to such a step and this evidently would be kind of a prerequisite for turning it into a larger cooperative endeavor of several authors. This could work on various levels – from authors contributing analysis or documentation of a single tag to writing and editing whole thematic segments. Note that i would still want to remain the overall curator and proprietor of TagDoc at least until i can be sure that an alternative form of governance would sustainably protect and develop the premises of the project in a responsible way.

If you are a data user (or otherwise want to invest in OpenStreetMap beyond software development and paid mapping) and interested in contributing to financing writing for and curating this project, and thereby help making it more sustainable, or if you are an independent expert in the de facto meaning of tags in OpenStreetMap data with proven experience in analyzing tag use in the OSM database, English language writing skills and knowledge about the diversity of world wide geography and are interested in contributing to this project as an author, then you are welcome to get in touch with me about contributing.

About me, the proprietor and curator of TagDoc

So, some might ask, what qualifies me to run and curate a project like this?

The main incentive for me contemplating the problem of meaningful and reliable tagging documentation and ultimately starting TagDoc came from my work as a maintainer of OSM-Carto. Forming a qualified opinion on requests to add or change the rendering of certain features in the map under the goals of the project always requires a solid knowledge of the de facto use of the tags involved in OpenStreetMap. Same applies for developing features for my own OSM-Carto derived map style. Doing research on well more than a hundred such cases, involving a large bandwidth of tags, helped me acquire a broad knowledge background in practical use of tags in OpenStreetMap, a lot of practical experience in analyzing how tags are used in OpenStreetMap world wide and what the quirks and inconsistencies in that use are, as well as a good sense of the quality problems of the tagging documentation on the OpenStreetMap Wiki. Combine that with the experience with using OSM data on global scale, i have gained as part of my paid work and what i researched over the years regarding practical use of tags out of curiosity and as part of writing about OpenStreetMap in general on my blog and elsewhere, i have probably broader background knowledge about the de facto meaning of tags in OpenStreetMap than most involved in OpenStreetMap. But the key, ultimately, is combining that broad background about OpenStreetMap with a solid knowledge and experience with the geographic diversity of the planet. Most OpenStreetMap contributors in their on-the-ground mapping work acquire extensive knowledge of the geography of the area they map in but very few have a solid understanding of the full range of geographic diversity of Earth which OpenStreetMap aims to document.

Of course this is a valuable qualification for doing consulting work regarding use of OpenStreetMap data and it has helped me many times in giving competent advice to customers. But ultimately it is kind of dissatisfying to not being able to make this knowledge available systematically to everyone who would find it useful and value it because of the lack of an economic basis to do so. I know this is a problem i am not alone with. The economic ecosystem around OpenStreetMap is traditionally heavily biased towards technical work. Software development and paid mapping are the most valued and therefore the dominant paid activities around OpenStreetMap and intellectual work of all kind is seriously under-appreciated. It would be important for the future of OpenStreetMap to change that and having quite a bit of experience in doing consulting work at the edge between technical work (data processing) and intellectual work (map design) and being one of the most prolific public writers around OpenStreetMap i think i am in a good position to lobby a bit for such change and to attempt demonstrating the importance and value of intellectual work using TagDoc as an example.

And finally one other important thing that i think qualifies me as a curator of tagging documentation is that i am largely independent of OpenStreetMap economically so i can provide a honest and open assessment without being constrained by my own or others’ economic interests. While i do paid work with OpenStreetMap data, most of my income these days is based on working with satellite data and its visualization and therefore does not depend on OpenStreetMap.

Conclusion

Ultimately all of this is just an offer from my side. I hope it finds the resonance and support it needs to become sustainable in the way i outlined and this way becomes a useful and valued source of knowledge about OpenStreetMap data and the tags used in it to record local knowledge about the geography of Earth. If it does, but especially also if it does not, i hope my endeavor to start TagDoc incentivizes others to think and talk about the importance of creating and maintaining a reliable and competent documentation of the de facto meaning of tags in OpenStreetMap, not unduly affected by subjective ideas of their ought-to-be meaning and the best way to develop and maintain such. This kind of project, like any other intellectual work, can only thrive and be excellent if it receives intellectual resonance and critical feedback.

TagDoc logo

Mapping in the Antarctic

January 6, 2022
by chris
0 comments

Antarctic mapping in OpenStreetMap – a critical look

About nine years ago i did the first import of up-to-date data for the Antarctic followed by another with more detailed data a short time later.

The intention of those was to provide a solid basis – both in terms of main physical geography features as well as for establishing basic mapping conventions – for a continent that until then had been mostly unmapped in OpenStreetMap. Essentially back then there were a few dozen nodes of research stations and individual POIs as well as some named islands and a crude overall coastline which also formed one giant glacier polygon. Mappers did not really have any context to start mapping locally (either from on-the-ground observations or imagery) what they might be interested in mapping. My hope was that providing such context in at least a rudimentary form would encourage both mapping human infrastructure (buildings, POIs), characterizing and structuring the physical geography in more detail as well as improving and updating/replacing the imported features based on newer and more accurate imagery or to take into account the movements of the ice that constantly reshapes the continent.

This initial import work was also the basis to start some attempts in displaying the Antarctic on OSM data based maps in a meaningful way. This evidently in particular refers to interpreting the implicit mapping of land ice that was established with the imports. But it also included adding support for rendering bare ground (natural=bare_rock and natural=scree) in OSM-Carto – which until then was not shown and which evidently forms the vast majority of ground not covered by ice in the Antarctic.

The state of OpenStreetMap in the Antarctic

So how have these attempts turned out in terms of community mapping in the Antarctic? The result is rather sobering. On the positive side genuine local knowledge based mapping has established itself in the Antarctic quite a bit in the form of mapping of research stations – presumably by people working there or visiting. This involves mapping of buildings, roads and paths around these as well as other human infrastructure in the surroundings of research stations. More than 1000 building and about 500 road lines of various kinds have been mapped so far. This kind of mapping has the potential to be self sustaining in the future.

Research station mapping at the Antarctic coast

Research station mapping at the Antarctic coast


Research station mapping in the Antarctic interior

Research station mapping in the Antarctic interior

However this only covers a minimal part of a vast continent. Outside the immediate vicinity of the research stations it looks much more bleak. Over the past 8-9 years almost the whole continent of the Antarctic was potentially a highly rewarding place to map physical geography, especially since 2013 when Landsat 8 was launched and began providing up-to-date high quality imagery for most of the continent. Essentially you could have taken any not fully cloud covered Landsat scene and have started mapping based on that and in most cases immediately would have produced the most accurate mapping of the area in question in existence. While this is also true for some other parts of the planet, nowhere outside the Antarctic is it that much ensured universally. Some time ago i demonstrated this on two smaller areas – one in the Antarctic interior, the other at the coast of the Antarctic peninsula, based on recent Landsat and Sentinel-2 images as well as other open data sources:

Mapping of the Belgica Mountains

Mapping of the Belgica Mountains


Mapping of Cape Calmette

Mapping of Cape Calmette

However i am nearly alone with such localized mapping of physical geography. This became even more obvious when recently someone made a well meaning but misguided change to the Antarctic coastline disrupting the routine coastline processing by moving it to the location they saw on what is unfortunately 20 year old image data. I had updated the major ice shelf edges where the ice moves most rapidly over larger sections two times since the 2013 import already and i have updated it to the state of 2021 again now (which includes taking into account the major iceberg calvings of D-28 in 2019 on the Amery Ice Shelf and of A-76 in 2021 on the Filchner–Ronne Ice Shelf). But with the exception of the mentioned ill-advised edit i was the only one who has ever worked on these ice edges during the nine years and made sure that they are not off too far. That is definitely not sustainable. And the rest of the Antarctic coastline – the majority of which is moving ice as well – has not been updated for at least nine years now (much of it for longer because the imported data was already older). This has led to inaccuracies exceeding 20km in some places.

Amery Ice Shelf edge history in OSM

Amery Ice Shelf edge history in OSM


Mismatch of OpenStreetMap ice edge data and present day situation in Lützow-Holm Bay

Mismatch of OpenStreetMap ice edge data and present day situation in Lützow-Holm Bay

At the same time the Antarctic has become kind of a playing field for imports and other mass additions of data as well as drawing of abstract labeling polygons with no connection to the observable geography. The almost complete absence of invested local mappers who could object to such edits makes it attractive for such activities.

Inconsiderate import bulldozing over existing data

Inconsiderate import bulldozing over existing data


Example for labeling polygon

Example for labeling polygon

The lack of invested mappers with interest and knowledge

The core of the problem is that OpenStreetMap in the way it functions is based on the collection of local knowledge. In the Antarctic this local knowledge is primarily in the minds of scientists and support personnel working there – not necessarily physically on the continent but on subjects related to it. And while – as pointed out – as far as this knowledge relates to human infrastructure and features significant in everyday life of people, OpenStreetMap has quite successfully recruited people with such knowledge to contribute, this does not extend beyond those narrow thematic fields.

In particular it is noticeable that OpenStreetMap has universally not managed to attract institutional scientists working on Antarctic topics to contribute to OpenStreetMap as part of their professional work. While OpenStreetMap would in principle be well suitable as a platform for scientists to exchange geographic knowledge about the continent, scientists seem to universally shy away from that. Primary reason is probably the very different work cultures and the concerns about everyone being able to edit the data in OSM and that being disruptive for any serious work (which is likely reaffirmed by many not very well informed editing activities like what i discussed before). Another reason could be that countries investing a lot of money into Antarctic research do so to support potential claims of sovereignty on the continent. A component in that is often maintaining a national cartography of the Antarctic or parts of it. And that obviously needs to be clearly attributed to the country to function as such – if mapping work would dissolve into a common international database like OSM right when being made that function would be less well served.

OpenStreetMap maintains the claim that it wants to create and maintain the best map of the world. For the Antarctic that goal has – despite some noticeable progress in the mapping of human infrastructure – become overall more distant and less achievable over the past nine years because – with very few exceptions – OpenStreetMap’s representation of the physical geography of the continent has remained static over that time period while the continent has continued to rapidly change and at the same time much better and more precise data sources have become available (in particular better and more frequent open data imagery) that increases the gap between what we have in OpenStreetMap and what would be available for mapping (and what is available to other mapping projects that compete to be the best map of the Antarctic – or parts of it).

Contributing factors

As already mentioned the Antarctic is different from other parts of the world in particular due to the absence of a larger local population from which mappers can be recruited. There are however also other circumstances that make mapping in the Antarctic particularly difficult.

Imagery is one of the main problems. Not because suitable images are not available – i already mentioned that a wide spectrum of recent open data images meanwhile exists covering most of the continent. The problem is that mappers are used to finding decent images they can map from in the main global image layers like from Bing and Maxar. But for the Antarctic that is predominantly not the case and mappers need to rethink their image choices. All global image layers use the 20 year old LIMA as base imagery for the majority of the Antarctic. And no one should map from that these days – it would be best if editors would actually prevent that.

I have made more recent images available for some quite extensive parts of the continent that mappers can use as a starting point for mapping. You can also use image layers featuring the latest data from low resolution open data satellites like the ones from GIBS. JOSM has meanwhile support for the time parameter of WMS layers – allowing you to choose the date of image recording in those. It does not work for TMS layers though and it is not very comfortable to find a day with no clouds in the area you want to map. Beyond that you are bound to manually looking for recent open data images from Sentinel-2 and Landsat and process them yourself. That might seem like a high hurdle to some – but compared to the time you also will need to invest in familiarizing yourself with the area you want to map this is actually not terribly much work.

In general OpenStreetMap’s focus on the Mercator projection also creates tons of additional practical hurdles when mapping at high latitudes like in the Antarctic. Most map styles are basically useless for feedback because you have to zoom in way too far for things to turn up. And for mappers the download area size limitations of the OSM API are annoying as well.

What i have been asking myself – and where the title of this blog post also aims at – is if the data imports back in 2013 have helped or hurt manual mapping efforts. That is not easy to answer. What we can try to do to get an idea is using the northern polar region as a point of comparison. Here different regions have seen different intensities of base mapping and imports (in Canada) in the past. It does not seem that localized mapping contribution vary very systematically between such settings. Overall manual mapping contributions are rare and usually small, independent of if there is significant pre-existing data or not. Still my observation is that because mappers of human infrastructure seem to shy away from also mapping the physical geography, providing a basic context of that can help with supporting manual mapping.

Physical geography context of station mapping

Physical geography context of station mapping

That the presence of the imported data has in some cases discouraged mappers from engaging in physical geography mapping is a possibility. Some mappers definitely find it more appealing to map on a fully clean slate instead of dealing with pre-existing data. But the data from the import is rudimentary in detail so even when mapping from Landsat or Sentinel-2 images it is clear that you can in most cases simply get rid of the pre-existing data in your area of mapping and start with nothing if you want to. So i doubt this potential influence is that significant. And there are definitely also a lot of remote mappers who like working on pre-existing geometries and simply adding detail to existing ways. This can be frequently observed in other parts of the world.

Possibly the largest influence the Antarctic imports had were the establishment of the Antarctic mapping convention of all land being implicitly ice covered unless tagged otherwise. The problem with that decision is that it was technically without alternative because plastering the whole continent with glacier polygons (or with one big glacier polygon with tens of thousands of holes) would have been unfeasible. Yet this might have been a significant negative impact factor on mapping activities because mappers receive feedback from the map with delay and mapping in this inverse fashion is not always that intuitive, especially near the coast. And among map styles interpretation of these Antarctic mapping conventions is almost fully absent beyond OSM-Carto and derivatives.

Also with the two experimental localized mapping projects i have shown above i realized that there is a significant gap in properly established and documented mapping conventions for polar regions and glacial features. A mapper working in the Antarctic who wants to properly document the reality they see on images or on the ground will often need to think about inventing new tags and mapping rules to represent this. And inventing such specific mapping rules can be very frustrating, especially if you realize that because of the exotic nature of these features the chances that data users will actually make use of them at some time are rather small.

And while the combination of all these challenges is unique globally, each of them individually can also be found elsewhere on Earth. That also means the Antarctic with its fast changing physical features might give us a glimpse into what kind of problems we can expect in other parts of the world in terms of data maintenance in OpenStreetMap also elsewhere in the future.

Conclusions

Mapping of the Antarctic – which constitutes around ten percent of the Earth land surface – represents a significant challenge for mapping in OpenStreetMap. As the data of the continent in the OSM database ages while the geography changes rapidly and increasingly more and better data becomes available that provides information on the region, this challenge will become more pressing in the future. If the OpenStreetMap community wants to maintain its goal to become the best map of the world, it needs to invest systematically in mapping of this continent and in improving the conditions for mapping it in the technical and social framework of mapping in OSM to attract more competent volunteer to contribute in that region.

There might be voices in the OSM community that suggest essentially giving up on the goal of creating the best map w.r.t. the Antarctic and focus on the naturally populated parts of the world instead. That however would create a slippery slope because the universality of a global map – aiming to map the whole planet and not just select parts of it – is a significant part of the attraction and the usefulness of OpenStreetMap for the map and the data user.

But this raises also an important further reaching question – if and to what extent the success of OpenStreetMap depends on and is tied to a functional hegemony of the project in its domain. An interesting topic for future discussion.

January 2, 2022
by chris
0 comments

La Palma eruption – before and after

The volcanic eruption that strongly reshaped the island of La Palma since it began in September 2021 seems to have ended during the last days of the last year. Here is a before-after comparison based on Sentinel-2 image data comparing an image taken on December 29, 2021 with one from January – so approximately a year ago with similar lighting conditions and seasonal appearance.


La Palma in 2021 (large: before/after)

Apart from the area directly affected by the lava flow you can see changes in color over a larger area affected by the fall of volcanic ash. In particular large parts of the pine forest in the mountains close to the eruption have changed in color from green to yellow.


La Palma in 2021 detail (large: before/after)

Here for reference a ground picture i took in 2011 from the area just above the site of the new eruption.

La Palma in 2011

The ground in this image is formed by volcanic ash probably quite similar in color and structure to many areas newly covered by ashfall from the recent eruption now so it could be indicative of how the wider area in parts might develop over the next decades.

December 24, 2021
by chris
0 comments

Snow in the North American West

Extensive snowfall is pretty common on the plateaus of the North American West in Winter. The white of the snow, the reddish colors of the rocks in the region and the gray-green of the winterly coniferous forests make for a remarkable color combination as it can be observed in this mid December Sentinel-2 image.

Snow in the American West

The area of this view spans much of southwestern Utah and northern Arizona, including three fairly well known National Parks which i show in the following crops.

Zion Canyon

Zion Canyon

Bryce Canyon

Bryce Canyon

Grand Canyon

Grand Canyon

December 22, 2021
by chris
4 Comments

Rethinking road access restrictions rendering

In my previous blog post on road rendering in OpenStreetMap based maps i focused on better and more consistent layering of road feature and depiction of additional physical characteristics of roads, in particular the ground unit with rendering, visualization of the number of lanes, differentiation of paved and unpaved roads as well as implicitly mapped sidewalks. I did not much look at the basic classification system for roads and other non-physical characteristics of roads depicted in the map.

Traditionally in OSM-Carto and derivative map styles roads rendering depicts roads primarily according to the road classes, that is the highway=* values assigned by mappers. This system of road classes is mostly a classification system based on primary mode of transport and intensity and purpose of road use.

In addition OSM-Carto traditionally depicts another non-physical aspect of roads – the access restrictions. Access restrictions are traditionally rendered in OSM-Carto in a very basic form – access=no and access=private are depicted with a broad and rounded gray dashing on top of the road line while access=destination is shown with a similarly styled dotting. For the narrow road classes (track, path, footway etc.) only access=no/access=private is depicted with a lighter/de-saturated line color.

Towards an intuitive interpretation of access restriction tags

This system is widely (and for a long time) considered a too simplistic interpretation of access restriction tagging, which is one of the strong points of OpenStreetMap since it allows for a very differentiated mapping of all kinds of access conditions a road might have. Picking just three very undifferentiated types of access restriction tagging to depict sends the wrong message to mappers wrongly incentivizing them to simplify a complex and differentiated reality to just these values already on the tagging level. And it is also of fairly little value for the map user because they really cannot tell much from this simplistic depiction regarding if a certain road can be factually used for a certain purpose in a lot of cases.

The starting point for my work on access restrictions rendering i am going to talk about here was that after introducing differentiation of paved and unpaved roads for all major road classes (see the previous road rendering post for that) readability of access restrictions rendering was kind of poor in some cases. This has led to considerations in OSM-Carto that it might be difficult to depict access restrictions rendering and paved/unpaved differentiation at the same time and considering the problems described in the previous paragraph it might be worth considering to simply drop the access restriction rendering in favor of the paved/unpaved differentiation. I reviewed the colors adjusting the gray tone used for the unpaved dashing individually for the different road backgrounds for a uniform contrast and readability and i think the results are fairly decent.

Access restrictions rendering on major road classes

Access restrictions rendering with adjusted colors of dashing for better readability (click to also show the unpaved version)

The main step towards an intuitive interpretation of access restriction tags in a map is in my eyes to focus on what restrictions actually apply to the main mode of transport of the road class in question. That requires interpreting a lot of different tags because of the way access restriction tagging in OpenStreetMap works.

Traditionally in OSM-Carto we try to have a relatively simple and easy to understand relationship between tagging and rendering results. We believe that this is essential for mappers to be able to use the map as a feedback instrument for mapping. If the mapper does not understand how their tagging results in a certain appearance in the map they cannot derive any hints from the map if they mapped things correctly or not. However for access restriction tagging a basic understanding of how the system of tags in OSM for that works is a fundamental prerequisite for correctly mapping things. The UI of editors can help with that of course. The map style however cannot. And once we as style designers accept that such basic understanding of access tagging on the side of the mapper is a prerequisite for us providing useful mapper feedback it quickly becomes clear that we can best support mappers and non-mapping users by interpreting the tagging in total for what it means regarding the primary transport mode of the road class in question.

The primary transport modes i assume as well as hierarchy fall-backs are as follows:

road class primary transport mode fallback access tags
main road classes motorcar motor_vehicle, vehicle, access
highway=track motorcar motor_vehicle, vehicle, access
highway=cycleway bicycle vehicle, access
highway=bridleway horse access
highway=path foot access
highway=footway foot access
highway=steps foot access

In terms of access values i interpret

  • destination, customers and delivery as light (previously destination)
  • no and private as no
  • yes, designated and permissive as yes – for highway=track also agricultural, forestry and agricultural;forestry
  • all other values are ignored

This logic is implemented in a Postgresql function you can find in roads.sql. Here is how this looks like in principle.

De facto access restriction rendering based on primary mode of transport

De facto access restriction rendering based on primary mode of transport

So far we have been treating access restriction as a simple binary yes/no or ternary yes/no/light classification regarding a single mode of transport considered the primary one. This is often insufficient to decently depict the practical reality of the road use and its restrictions and permissions. This is why i also looked into situations with something like access=no, but… – in other words: Roads that are closed to all forms of use with a single exception. The practically most relevant cases of this type for the normal road classes are access=no + bus=yes for bus only roads and access=no + bicycle=yes for cycling roads. These i tried to indicate by coloring the access restriction dashing in a pale version of the colors of the respective roads.

Access restrictions with exceptions

Access restrictions with exceptions

Implicit access restrictions

Experienced mappers among the readers will probably have noticed that so far i have left out two pretty common road classes in my discussion and illustrations. Those are highway=pedestrian and highway=living_street. In contrast to most of the other major or wide road classes (those drawn with a fill and casing as opposed to the narrow classes drawn with a single line and a bright halo) these are defined mostly by implicit access restrictions. highway=pedestrian is for roads that could be used with a car but that are not allowed to be used with a car and reserved for pedestrians. They essentially have the implied access tags access=no + foot=yes. The exact meaning of highway=living_street regarding use permissions is a bit more vague and varies from country to country – mostly it means pedestrians have precedence over vehicle traffic here while vehicles are still allowed, often with some restrictions.

There are other road classes with similar implicit access restrictions suggested and partly used by mappers in OSM, specifically highway=busway and highway=bus_guideway. The suggestion to add support for highway=busway to OSM-Carto and the difficulty to do so in a consistent way was one of the factors that inspired me to work on this topic here.

There are two possible ways to integrate rendering of such road classes with implicit access restrictions into a map style. The first is to render them like a normal road with the implied restrictions tagged. That would – for highway=pedestrian and with the above scheme for rendering access restrictions – mean like highway=residential with red-orange access dashing. The advantage is that this would be intuitively readable for the map user who knows about the system of access restrictions rendering without learning specifically about the highway=pedestrian road class. The main disadvantage would be that highway=pedestrian roads often form fairly dense networks in town or city centers and all of the red dashing would create quite a lot of noise on the map. And by displaying the implicit access restriction you essentially have no room left to show any explicit tagging of access restrictions and permissions. The other – and commonly used way of rendering – is to treat them as a distinct road class. This is the approach OSM-Carto takes and that i took for the alternative colors style so far. And i unified highway=pedestrian and highway=living_street to avoid having too many distinct road colors. This was not quite satisfying though because the practical meaning of the two is quite different.

The situation with highway=busway is yet another difficulty because those mappers who introduced this tag want it to be used for some – but not for all – bus exclusive roads. Hence the aim would need to be to distinguish between highway=busway and normal roads tagged access=no + bus=yes in a meaningful and intuitive way or to unify them in design. And to not be confusing the approach would need to match that for highway=pedestrian of course. As an additional quirk we have in OSM-Carto for a long time rendering of highway=bus_guideway in a railway like fashion – which might make some sense from an engineering perspective but makes very little sense from the practical map user perspective.

To demonstrate how all of these issues can be addressed together i went for a base design as shown in the following with a violet color for highway=busway/highway=bus_guideway (which works better in the ac-style than in OSM-Carto because of the violet color for transport related symbols) and the previous light gray for highway=pedestrian and highway=living_street. In contrast to OSM-Carto which does not render access restrictions on highway=pedestrian (likely because that would clash with the implicit restrictions of this road type) i do so – based on the paradigm that the access dashing represents the de facto restrictions for the primary mode of transport. This is not that common practically for highway=pedestrian but there are cases where this applies – for example for pedestrian roads in military complexes or non-public parks.

Road classes with implicit access restrictions

Road classes with implicit access restrictions

But there remains a serious question with roads with implicit access restrictions: How to depict exceptions from the restrictions like i do with the colored dashes for explicitly access restricted roads? We can’t use the colored dashing because we have made the choice to not use that for the implicit restrictions. Yet this implicit restriction with exception situation is a fairly common case – in particular for pedestrian roads where bicycles are allowed. The solution that i chose and that i think works quite well is to use a continuous colored center-line for additional permissions compared to the primary mode of transport. This also nicely allows for a general differentiation between highway=pedestrian and highway=living_street – by indicating highway=living_street to be like a pedestrian road with additional permission for vehicles. How this looks like can be seen here:

Additional permissions on road classes with implicit restrictions

Additional permissions on road classes with implicit restrictions

Two final points – first: If you followed my previous post on roads rendering you might notice that the use of a solid center-line signature kind of interferes with using the same context to visualize lanes. Yes, it does, but the two still work decently together with some limitations to readability and also this is a rare combination – lanes tags on highway=pedestrian and highway=living_street are really rare. And second: Yes, on busways the center-line rendering is fairly poor in contrast and can be hard to reliably read, especially in the unpaved version. Again – this is a very unlikely combination practically and i included it here mostly to demonstrate the design and tag interpretation concept in principle, not because it is such a practically relevant combination. The colored center-line rendering could equally be dropped on the busways.

And i am not even sure if at this time a distinct rendering of highway=busway is advisable because there is not a clear and well established semantic difference between that tag and normal roads with access=no + bus=yes independent of local culture specific conventions. I included it here mostly as a demonstrator to have another type of implicitly access restricted roads in the mix and show that this rendering concept allows for such extensions.

Practical examples

Here a few practical examples of this new rendering using real world data.

Access restrictions rendering example

Access restrictions rendering example

Access restrictions rendering example

Access restrictions rendering example

Access restrictions rendering example

Access restrictions rendering example

Note in particular how – with the colored center-lines and colored access restriction dashing – you can pretty well spot continuous routes for cycling in a network of predominantly pedestrian or otherwise access restricted roads for example.

What these examples also show in some places is that both the changes introduced here and the previous road rendering related changes in the ac-style interpret a lot of tags that are not that frequently interpreted by maps elsewhere. This makes the map fairly difficult to read in some situations because of errors and inconsistencies in the data. Even just the mapping of certain aspects being incomplete and patchy can add a lot of noise to the map and make it rather difficult to read. But ultimately that mainly means that maps providing visual feedback on these features to mappers are badly needed.

Conclusions

So what did i show here? I presented a new system of interpreting and displaying access restrictions and permissions on roads that is close to the tagging system of OpenStreetMap by following the main road classification system but applies a considerably higher level of tag interpretation on the access tags to generate a meaningful and intuitive rendering.

Of course this adds quite a lot of design elements to the map that a map user needs to get acquainted with. In particular the added use of color makes the map fairly colorful in some settings – even though the color semantics are consistent across the whole domain of roads making it quite intuitive overall. Still it stretches the limits of what you can reasonably claim to be a map that can be understood without a map key.

Overall i think this is a viable demonstration how access restrictions like they are mapped in OpenStreetMap can be depicted in a manner that follows a consistent inner logic in terms of the symbology and presents the restrictions and permissions of roads in a meaningful way to the map user, significantly increasing the practical usefulness of the map compared to OSM-Carto. And as i discussed above i think this also significantly helps mapper feedback compared to the selective depiction of access=no/private/destination known from OSM-Carto. While the mapper cannot directly see the tagging in the map display any more with the new system and need to understand how the system of access restriction tagging in OSM works up front to make sense of the rendering, i think this practically is more useful and more valuable.

Most of the complexity of the tag interpretation logic is in two Postgresql functions which are generic in nature and not specific to the visual design used. Technically the same could be implemented with a lookup table for the different tag combinations which some developers might consider more suitable depending on individual preferences and taste.

December 20, 2021
by chris
6 Comments

Why it is essential for the OpenStreetMap community to actively pursue map design innovation

This is kind of a note on the general matter of map design in OpenStreetMap based maps – on which i am going to write more specifically in the following blog post. Like in various previous posts on map design matters i am going to write about new ideas and cartographic techniques to display the information mappers record in OpenStreetMap in a rich – yet hopefully intuitively readable – map, suitable for a large bandwidth of geographic settings. Unfortunately i am fairly alone in publicly writing about this topic in the context of OpenStreetMap based automatically rendered interactive maps aimed at global use. Even outside the OpenStreetMap world in depth discussion about rule based map design is rare and most of what is written in the wider context of map rendering focuses on purely technical aspects and improving rendering efficiency.

You can get a bit of an idea how this lack of innovation in map design in the OpenStreetMap community is damaging for the project and its public image from an article in Cartographic Perspectives published recently. In a nutshell this article’s failure is conflating OpenStreetMap with the mediocre OSM data based maps produced by a commercial map service provider tuned for cost efficiency in providing the map tiles rather than quality and information content of the map. That this is foremost a failure of the peer review of the article is obvious, but this is not my topic here. What this however also illustrates is that the vast majority of OSM data based maps do neither show ambition nor ability to aim higher than the products of Google, Bing etc. in terms of innovation and quality of map design. Roughly 80 percent of OSM based maps are the basic garden variety styles combining basic rendering of roads, buildings and static POIs and labels, sometimes with some purely decorative landcover depiction. The other 20 percent are specialty maps for specific narrow use cases – sometimes with considerable innovative ideas but always limited to a very narrow thematic field. With that background it is somewhat understandable if conservative cartographers unfamiliar with OpenStreetMap think it is identical data-wise to the poor Google Maps imitations commercial map providers have created using OSM data.

If the OpenStreetMap community wants to stay avant-garde in cartographic data collection and actively shape the future of that domain rather than swimming with the stream and becoming a mere data provider serving the cartographic data needs of data users with big pockets, it needs to be able to shape the map design depicting and presenting the data the mapper community collects. The few efforts from within the OSM community in that direction that we still have are hampered by the map design tools and rendering tools for automated rule based cartography available – tools which for many years now have been designed and developed almost exclusively for the needs of commercial map service providers like i discussed above. So the challenge for the OSM community is two-fold – nurturing and valuing innovative community map design work inspired by the values of the project rather than short term external economic interests as well as developing and supporting the software needed for these map design efforts.

When looking at comments and contributions on the OSM-Carto issue tracker and following map rendering discussions in the OSM community elsewhere i am frequently quite frightened by the lack of effort and more in depth interest in the topic. To put it bluntly – the attitude of most OSM community members w.r.t. map design seems to be one of two: (a) That maps based on OSM data will just happen to appear somewhere and develop themselves based on needs and opportunities without requiring any specific talents, experience or education or (b) that map design is essentially nothing more than occasionally adding rendering support for a new tag with some randomly picked new line or polygon fill color or adding yet another static poi symbol type using an icon more or less related to the tag in question.

This needs to change – significantly and rather urgently. I have been pointing this out in various forms for quite a long time already and i do here again with more emphasis. What i am occasionally discussing here with my OSM map design related blog posts is just one of many possible approaches to innovation in OSM related map design. All of them deserve more and more ambitious interest and more support from within the OSM community.

Although this post might in parts suggest something different – it is not my intention to assign blame here regarding who is responsible for the developments of the past. What is important is the realization that there is a need to act here and that an own and diverse innovative map design capability from within the OSM community that is not just piggy-bagged on the work of third parties outside the project but that is capable and willing to guide and shape design in its own direction, is essential for the long term success of OpenStreetMap. And that the technical foundations for this in the form of software that enables such innovative map design with flexibility, likewise need to be developed and shaped from within the project and can equally not just be attached to external endeavors which follow completely different economic goals.

If that does not happen OpenStreetMap would get in trouble and loose significance rather quickly. The idea that OpenStreetMap could compete on metrics like those of the Cartographic Perspectives article i linked to with proprietary cartographic data sources is unrealistic – those will be massively expanding on machine generated data for machine generated maps and OpenStreetMap will never be able to compete in that domain. What OpenStreetMap is good at and where the proprietary competition has no chance against it is producing a map based on local knowledge, by the people, for the people – both in the data collection and mapping part and in the actual map design and production.