Imagico.de

blog

Trends for the future of map design in OpenStreetMap
Trends for the future of map design in OpenStreetMap

Trends for the future of map design in OpenStreetMap

| 5 Comments

After having looked at the history of digital map design in general in the first part of this short series and at the current situation of map design in OpenStreetMap in the second part, i am now, in this last part, going to specifically look at what the OpenStreetMap Foundation seems to be pursuing in that field at the moment and in what direction that is likely going to go.

The OSMF’s history in providing maps

The OSMF has for some time followed a quite erratic course with regards to rendering and providing maps for the OSM community. Historically, a significant percentage of the OSMFs IT resources have gone into rendering the standard map layer on openstreetmap.org. And while there has been discussion and ultimately a policy was developed on who is allowed to use that tile service under what conditions, the OSMF has never really stated a clear purpose for that tile service or the conditions for choosing OpenStreetMap Carto as the style for it. There is a set of requirements for tile layers to be included on osm.org – which are mostly hosted externally – but nothing to this date defines what qualifies OSM-Carto to be the style the OSMF invests substantial resource in to render as a map. I have, as an OSM-Carto maintainer, repeatedly criticized this omission, because it also means the OSMF takes no responsibility for the decision to do that.

Back in 2021 a poll the OSMF board made of the OSM community included the question about the future of map services provided by the OSMF, for the first time indicating to the OSM community that they are contemplating making changes in that field. The question was very vague and ill-defined (which i criticized back then) and the answers were accordingly inconclusive. The term vector tiles had already become a magic word back then without much of a well defined meaning but where everyone seemed to project their hopes and dreams into.

Later that year the OSMF announced that they would – on the operational level – do some testing in that field. But even upon inquiry no overall strategy was mentioned this initiative would be part of and it became clear that this was an initiative from operations and not part of a larger strategy of the OSMF at that time. The interesting part here is that, while this announcement was only about a relatively small operational investment into testing with nothing in terms of money being involved at all, there were quite immediate reactions from economically interested parties substantially lobbying for their pet frameworks being considered by the OSMF as well.

Long story short: Back in 2021 it was clear that (a) the OSMF had no substantial strategy for the future of the map services provided by the OSMF, (b) there were already a lot of people projecting their hopes into the vague concept of vector tiles and loudly calling for the OSMF to make these hopes to magically come true while the whole matter became further political because economically interested parties began to see the chance to profit and started lobbying the OSMF to adjust plans in their favor – and (c) that actual map design was not going to play even the most marginal role in the whole process. After that there was not much publicly visible progress for some time. The strategic plan decided on in July 2021 included only a development platform for vector tiles by volunteers – which seems to reflect the board’s reading of the survey results back at that time to leave that to the community to pursue.

Something changed at some point in 2022. In November there was a short mentioning of the matter in one of the meeting minutes. From then on a fairly remarkable timeline in discussions and decisions can be reconstructed from the minutes that paints quite a rich tapestry of the inner workings of the OSMF. Although this is of little relevance for the topic of this blog post i highly recommend taking the time to read through all of that.

The meaningful information w.r.t. the topic of this post is that, when the OSMF contracted Paul Norman to do the work that these days seems to be coming to an end, they still apparently did not actually have a strategy on their own regarding the OSMF’s map services. And it is unclear if they have de facto adopted what Paul sketched in his proposal (published here, which is rather vague in terms of strategic goals) as the OSMF’s de facto strategy or if they have meanwhile developed some other kind of plan. What is, however, quite self evident is that they will certainly want to roll out the results of spending EUR 42k in some form. And this is what i will base the following considerations on, i am not making any further assumptions beyond that.

The map serving process

What i am going to do here is discussing what that likely means for the OSM community. To explain that, here first how the current setup of the standard tile layer on openstreetmap.org is run (based on OSM-Carto):

The practical map rendering process and mapper feedback loop with OSM-Carto (or similar styles) - link goes to larger version

The practical map rendering process and mapper feedback loop with OSM-Carto (or similar styles) – link goes to larger version

I illustrated which parts of this setup are controlled by who: The gray rectangle titled Map design is controlled by the OSM-Carto map designers and maintainers. The Mappers using the map decide which parts to view and what to map and the OSMF (Operations) is in control of the OSM database and the map rendering chain until the rendered tiles are delivered to the map user. Some might be irritated because i split up the rendering process with the tile data (internal only) to make the process better comparable to the vector tiles one shown below. This split is not visible to operations, it exists only internally in Mapnik. It is, however, visible to the map designer who separately designs the query to generate the tile data and the rules to draw the tile from this data as formulated in the CartoCSS code.

The important thing to note is that the rendering database in case of OSM-Carto is deliberately kept generic. This allows you to render several completely different styles from the same database, to make almost any style changes without reloading the rendering database and it also simplifies systematically testing the style as a whole because the database import does not need to be part of this testing.

In case of OSM-Carto the processes within the map design domain are relatively simple. For comparison: Here is how this looks like with the Alternative Colors style.

Differences with a more complex style like the AC-Style - link goes to larger version

Differences with a more complex style like the AC-Style – link goes to larger version

Like a lot of other map styles the AC-Style extends the rendering database with some style specific data. In my case this is some SQL functions, mostly for tag interpretation and geometric processing, and the tree symbology (which needs to be in the database to work around limitations in Mapnik – see the blog post discussing tree rendering). But this is static data, it does not change when the main database content changes.

Here is now how the whole process would change if Paul’s work would be rolled out as developed (based on my understanding from what has been communicated – i have not actually tested this setup myself).

The expected map rendering process and mapper feedback loop with minutely updated vector tiles (as currently pursued by the OSMF) - link goes to larger version

The expected map rendering process and mapper feedback loop with minutely updated vector tiles (as currently pursued by the OSMF) – link goes to larger version

From the operational side this means – in a nutshell – that the last part of the traditional rendering chain as shown for OSM-Carto above is cut off and outsourced to the map user. This substantially reduces the resources required for serving the tiles and at the same time allows customizing the actual rendering of the tiles without this requiring additional resources.

That is – in short – the whole magic of vector tiles. In reality it is of course not all that simple – for multiple reasons.

First: For the outsourcing of rendering to the map viewer to work you need to massively reduce the volume of data transferred and involved in rendering. In the diagrams shown i indicated high volume data with the wide red arrows. And the tile data internally processed by Mapnik is typically even substantially larger volume than what is in the rendering database (which tends to be many hundred GB for a full planet import) because the queries do geometry processing that frequently adds data volume. In case of the client side rendered vector tiles this volume needs to be massively boiled down with the help of lossy data compression. And despite these efforts (and their negative side effects) the bandwidth requirements for viewing one of these styles is usually much higher than for a traditional raster tile based map.

Second: The rendering database is not generic any more. That is not inevitable but practically i am not aware of any client side rendered vector tile setup that uses a generic rendering database.

Third: Map design as a domain in control of parts of the whole process is gone. That requires a more detailed discussion of course.

Above, i explained that one of the main promises of client side rendered vector tiles is, that the actual rendering can be customized for every map viewer. But the ability to customize the actual rendering does not mean that you can fully customize the map design. In reality, most of the map design decisions are implemented in the vector tile generation, partly because of the need to keep the tile sizes small. This means that everything that is not necessary for the specific map design chosen needs to be stripped from the data. Or seen the other way round: Trying to introduce any level of map design genericity into client side rendered vector tiles will often have massive impact on bandwidth requirements.

Practically, all client side rendered maps i have seen so far have a division between the people/organization in control of the vector tile generation and those developing the client side part of the map design (which i labeled style design in the diagram). And the vector tile generation furthermore tends to be under control of software developers and not map designers – hence i labeled that part software development. In the setup Paul has developed this part is further split into multiple components under control of different people.

What this means is that map designers would, in such a setup, get marginalized to mere style designers who can play a bit with colors and line width selection and designing point symbols. The real control over map design would sit with the software developers in control of schema and vector tile production, whose main concern is not going to be map design but operational efficiency and keeping the size of the vector tiles small – the technocratic direction as i described it in my discussion of OSM-Carto. Theoretically, you could argue, you could put the style designers hierarchically on top of the software developers in control of the vector tile schema and require the latter to adhere to the wishes of the former. But in today’s OSMF this has zero chance to happen as evidenced by the fact that map design aspects played no role at all in the whole process of developing this setup. As i wrote in the discussion of OSM-Carto, the only way from my perspective for sustainable progress in map design would be to bring the technocratic and the artistic direction together in a single cooperative project with clear common goals everyone subscribes to.

Beyond the magic

So what remains of the magic of vector tiles and its promises in light of this? The lower resource requirements for operating a tile server are indeed there. But only if you look at it purely from the micro-economic perspective of the tile server operator. From a macro-economic perspective the resource requirements are actually higher in sum because the actual rendering is done separately for every map user. Client side vector tiles is not Tiles@home.

But don’t get me wrong, there are of course benefits of client side rendered maps. One major one is the ability to implement more interactive maps based on the semantic data transferred to the client. And if you invest the bandwidth to include additional data in the tiles, customizing the map can be a real benefit. But there is no magic here, this all comes with costs.

The real question, however, is, what this means for map design. If i put aside the organizational division between style design and vector tiles generation for the moment what remains is what changes if you design for a client side renderer compared to a Mapnik based rendering chain. There are still several really big issues i am afraid. I already explained those in more detail in my post on what rule based map design requires from the tools it uses.

But now comes the twist some readers probably don’t expect: All of this does not mean the OSMF did something wrong by investing into implementing a toolchain for minutely updated vector tiles. Of course apart from the board epically failing in the procurement process for this on so many different levels. If i ignore that for the moment then investing in this is a completely sound decision. Client side rendered vector tiles are – independent of their practical relevance for the OSM community – currently the main technique for producing interactive web maps used by commercial map providers. And these commercial actors, while having invested substantially in the toolchains for this style of map production, have very little interest in minutely updates. These are a relatively unique interest of the OSM community. Hence it is a reasonable idea that the OSMF should chime in to serve this specific need.

If the OSMF now feels that this reasoning is not sufficient to justify the project and feels instead the need to roll out a cheaply thrown together postmodern OpenMapTiles/Mapbox clone with minutely updates as the main map on openstreetmap.org that would be regrettable. But it would not really mean much in the grand scheme of things. It would surely not solve the structural problems in practical map design in and around the OSM community that i explained in the previous post.

5 Comments

  1. Hi!

    Thanks for writing this. I am an avid user of maps, software developer by training and trade, very sporadic OSM contributor. I once found myself attempting to style a map to make it legible (and consistently disappointed at my results). I found your post on HN and went on to read the whole series.

    I take that you’re passionate about map style design (that’s what made me continue reading) and are attempting to bring this aspect to the attention of the OSM community and foundation. Thank you for doing this!

    Now on with bits of unsolicited feedback as an internet stranger

    – When describing style differences, consider adding visual examples, like maps of the same area rendered differently. Same when describing bits of design that can be done when editing a concrete map but are not possible (or practical) with state of the art rule-based styling: what are we loosing?

    – When referring to other posts, consider also including a short summary of the relevant from the other post. While I see how you may not want to repeat yourself, this makes the post harder to stand by itself, which may make it less useful: someone less determined than me may miss the point if it’s spread across many long-form posts. Maybe there’s room for a summarized version, possibly phrased as an open letter to OSMF?

    • Thanks for the comment.

      Providing useful examples is often a difficult thing to do. Because providing an example can be equally helpful (because it practically illustrates a point) and misleading (because it is non-representative and can lead to readers completely misunderstanding a point). The limitations of tools for map rendering i discussed more in depth here. But you will also not find concrete examples there, because it is inherently hard to give examples for something that does not exist. Examples of things that is difficult with existing tools can be found in many of my discussions of concrete map design work (some of which are linked to in that post). For example the work i did on trees is something that is very hard to do with a Mapnik based framework (as demonstrated) but impossible to do with any of the client side rendered vector tiles frameworks.

      Thanks also for your suggestions regarding style of writing. But please note i am writing these posts primarily for readers with an in-depth interest, not for a twitter-like attention span. I am not trying to maximize the number of clicks on my posts, i am interested in communicating my thoughts and reasoning and discuss these with others in a non-superficial way. That certain matters might be discussed more in depth and more elaborately than you’d want is something that you have to deal with. Linking to other posts is meant to point readers to further reading on specific matters. If you want summaries of these regarding certain points that specifically interest you there are tools to generate such.

      And i am not trying to communicate with the OSMF here. If OSMF members or members of the OSM community are convinced by my arguments and reasoning and want to suggest certain measures to the OSMF in light of this that would be great. But i have no ambition to lobby the OSMF myself. As i have pointed out in the past the OSMF these days is almost exclusively about the negotiation of interests and public discourse based on arguments and reasoning does not play a substantial role any more. I am not going to try to compete in lobbying against multi-billion Euro interests.

  2. Hello
    Interesting series to read, as always – thank you.

    Some notes about vector tiles.

    TILE SIZE: From experiments with *.openmap.lt series of maps I could tell, that vector tiles are generally smaller than corresponding raster tiles. Exception is medium scale tiles in busy areas: if you take raw data and simply shovel it to vector tiles – you can get 400K-800K tiles (and maybe even more). For example this scale and position would be pretty heavy: https://topo.openmap.lt/#t/12.2/54.69212/25.32491/0/0 There are two options in such cases: 1. perform proper generalisation (and that is not only buildings even they are most visible, but also landuse, roads and pois) 2. omit the data altogether – most (all?) commercial vtiles providers have chosen option 2 – therefore you have mostly empty maps at those scales.

    STYLING: you’re right that choices of styling in vtiles are limited, because you cannot change the data (choose/alter/order!). But it is not that bad 🙂 There is an option of splitting the data into parts and sometimes having alternative versions of several parts. This is what is being done in *.openmap.lt series – there is a “base” tile layer and then several specific ones – details, craftbeer, whitewater, admin boundaries – all combined for specific maps. It of course adds some complexity as well as additional potential requirements for future vtile servers (like being able to combine several layers into one vtile on the fly).

    • Thank you for your comment.

      Regarding tile size – this is hard to compare in an objectively balanced fashion because it is evidently apples and peaches. There are a few statements though that can be made

      • it depends a lot on what design elements dominate your map style. If you have – as an extreme example – only point symbols and point labels that will evidently be much smaller in a vector representation than in raster images. If – on the other end of the scale – you have plain color polygons – those will be hard to get in size anywhere near a (lossless compression) raster representation (see discussion here) without it degrading to a useless pile of lossy compression noise.
      • if you render a real life vector tile set, massively optimized to minimize data volume, with the corresponding map style, the resulting raster tiles will often be larger than the vector representation. That is, however, not a really meaningful comparison because
      • if you take a map style developed purely based on map design concerns with complete disregard to data volume and then try to encode that into vector tiles without a substantial loss in visual quality you almost always will end up a with massively larger data volume than in a raster representation.

      My comment in the blog post was referring primarily to the last point – since my focus here is map design, not operational efficiency.

      The term generalization has to be used very carefully these days because in the vast majority of cases it is used as a framing/code for lossy data compression (which is a bit like calling JPEG compression a noise reduction technique). Actual generalization in the traditional sense of the word does not generally reduce data volume, especially not in a polygon/linestring representation. There are some generalization operations (like aggregation) that will reduce data volume while even the removal of high spatial frequency components from geometries does not generally reduce the data volume (especially not at larger scales).

      Regarding styling – splitting the data into several parts and processing them individually is not a new feature of vector tiles, this has already been practiced with raster tiles for a long time. Practically this is mostly done with label/symbol overlays, for example to facilitate localization. But it is also possible with other map elements as demonstrated here. In all cases this is based on the premise that the different components in the map are completely independent of each other so they can be processed separately. In a well designed map that is often not the case (displacements, knock-outs). Also notice that the issue of map design decisions being largely already implemented in the vector tiles is only one of the current issues with current client side rendering frameworks. Other problems are often at least as severe. See here. The map you linked to for example nicely demonstrates the serious limitations in precision rendering in Mapbox GL/Maplibre GL (specifically what is discussed here)

  3. Pingback: weeklyOSM 718 – weekly – semanario – hebdo – 週刊 – týdeník – Wochennotiz – 주간 – tygodnik

Leave a Reply

Required fields are marked *.



By submitting your comment you agree to the privacy policy and agree to the information you provide (except for the email address) to be published on this blog.